Decentralized evaluation for evidence-based decision making Decentralized Evaluation FINAL EVALUATION of WFP Haiti’s FY 16 McGovern￾Dole Food for Education and Child Nutrition Programme (2016-2019) Final Evaluation Report October, 2019 WFP Haiti Evaluation Manager, Claudia Schwarze Agreement: FFE-521-2016/016-00 Prepared by Louise Mailloux, Team Leader Dean Palllen, Deputy Team Leader Eunide Louis, Gender Specialist Thomas De Brouwer, Evaluator/Data Anaylst Fredly Antenor, Community Development Specialist Final Report i | P a g e Acknowledgements Our thanks go to the World Food Programme (WFP) Monitoring and Evaluation Unit in Haiti for their constant support throughout this evaluation. The evaluation team (ET) is very grateful for all the assistance provided by Claudia Schwarze (Head of M&E Unit), Willienne Thélémaque (Beneficiary Feedback Coordinator). The ET would also like to thank Ivan Touza (Regional Evaluation Officer) and Ana Urgoiti (Evaluation and Learning Consultant) of the Regional Bureau in Panama for their support. The evaluation team (ET) would especially like to thank Ronald Tran Ba Huy (Representative/Country Director), Raphael Chuinard (Deputy Country Director), Ilaria Martinatto (Head of Programme), Viviana Sandoval (Head of the School Feeding Programme), Damieta Mendes (McGovern-Dole Grant Coordinator) and other staff of WFP Haiti country office (CO) and in the field for facilitating, supporting and contributing to the evaluation. Without their valuable and timely support, the mission would not have been so well informed, nor conducted so efficiently and effectively according to schedule. Our gratitude also goes to the many respondents from the various government authorities, the donor community, the United Nations (UN) system, non-governmental organizations (NGOs) and civil society who afforded the time to participate in some very useful and informative discussions and contributed to the findings of this evaluation. The contributions of beneficiaries and non-beneficiaries of different ages and gender in Haiti are also very much appreciated. Disclaimer The opinions expressed in this report are those of the Evaluation Team, and do not necessarily reflect those of the World Food Programme. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov Final Report ii | P a g e Table of Contents Executive Summary ................................................................................................................ v 1. Introduction...................................................................................................................... 1 1.1. Subject of the Evaluation................................................................................................................2 1.2. Context .............................................................................................................................................4 1.3. Evaluation Approach, Methodology and Limitations .................................................................5 2. Evaluation Findings.......................................................................................................... 7 2.1. EQ 1: Relevant of the programme in light of the needs of school children and families, and education and school meals policies and strategies of national government.................................7 2.1.1. Alignment with the National School Feeding Policy and Strategy.......................................7 2.1.2. Geographic targeting, food security, nutrition needs, vulnerability and access to education...............................................................................................................................................8 2.2. EQ 2: Has the programme achieved the expected outputs and outcomes at school level and which factors are explaining those results? ................................................................................11 2.2.1. Achievement of Outputs .........................................................................................................11 2.2.2. Other Output Indicators of Interest ......................................................................................12 2.2.3. Achievement of Outcomes .....................................................................................................15 2.2.4. Outcomes based on gender, type of school, location, model or complementary activities ...............................................................................................................................................18 2.3. EQ 3: How community involvement in schools contribute to outcomes and sustainability 20 2.3.1. Principals, Committees and Parents’ Contribution to Outcomes .....................................20 2.3.2. Male and female participation...............................................................................................22 2.3.3. Sustainability ............................................................................................................................23 2.3.4. Potential to create income generation opportunities ........................................................24 2.4. EQ 4: Are the key institutions and governance structures able to effectively deliver, implement, sustain and/or scale up school meal interventions in the long-term?.......................25 2.4.1. Policy Framework and Governance Structure .....................................................................25 2.5. EQ 5 Differences between the McGovern-Dole model and other approaches, and how adapted it to increase effectiveness, sustainability, efficiency and effects ....................................28 2.5.1. Advantages and Disadvantages of McGovern-Dole compared to other models...........28 2.5.2. Improving community involvement and sustainability ......................................................30 2.5.3. Opportunities to add or augment a nutrition sensitive approach....................................35 3. Conclusions and Recommendations............................................................................ 36 Final Report iii | P a g e 3.1. Conclusions.....................................................................................................................................36 2.6.1. Question 1 - Relevance............................................................................................................36 2.6.2. Question 2 - Effectiveness ......................................................................................................37 2.6.3. Questions 3 and 4 - Effectiveness, Efficiency, Sustainability..............................................38 2.6.4. Question 5 - Effectiveness, Efficiency, Sustainability...........................................................39 2.6.5. Gender Equality and Environmental Sustainability.............................................................39 3.2. Recommendations .........................................................................................................................40 Annexes Annex 1: Terms of Reference (without the annexes).....................................................................43 Annex 2: Evaluation Matrix................................................................................................................63 Annex 3: Country Context .................................................................................................................71 Annex 4: Detailed description of subject of the evaluation..........................................................80 Annex 5: Evaluation Methodology....................................................................................................92 Annex 6: Documents Reviewed .....................................................................................................115 Annex 7: Stakeholders Interviewed................................................................................................122 Annex 8: Data Collection Instruments ...........................................................................................125 Annex 9: Additional Evidence for EQ 1 ..........................................................................................133 Annex 10: Additional Evidence for Question 2 .............................................................................143 Annex 11: Findings, Conclusions and Recommendations .........................................................157 Annex 12: Evaluation Team............................................................Error! Bookmark not defined. List of Acronyms ...............................................................................................................................171 List of Figures Figure 1: Number of school children provided a school meal, planned vs. actual ............................11 Figure 2: Number of actual meals served vs. planned per semester ..................................................12 Figure 3: Number of participants by training activities, men and women..........................................14 Figure 4: Assessment of Haiti national capacity, following SABER dimensions in 2015....................25 Figure 5: Map of intervention: North, North-East, Gonaives/St-Marc and Port-au-Prince corridors ......................................................................................................................................................................81 Figure 6: Distribution of McGovern-Dole supported schools by location, 2018-19...........................82 Figure 7: Number of McGovern-Dole Schools, school year 2018-19 ...................................................82 Figure 8: Number of schools by activity, school year 2018-19 .............................................................85 Figure 9: Graphic representation of the Theory of Change ..................................................................88 Figure 10: WFP initial McGovern-Dole result framework - School meals programming in Haiti .....89 Figure 11: Revised McGovern-Dole results framework .........................................................................90 Figure 12: Revised McGovern-Dole results framework .......................................................................91 Figure 13: Map of Chronic Food Insecurity by Livelihood Area, IPC 2015.........................................138 Figure 15: Percentage change in enrollment rate, year-per-year (WFP national level) ...................146 Figure 16: Pass rate in WFP assisted schools (WFP national level).....................................................147 Figure 17: Retention rate WFP assisted primary schools (WFP national level).................................147 Figure 18: WFP monitoring data of the McGovern-Dole Programme...............................................148 Final Report iv | P a g e List of Tables Table 1: Number of food distribution days as a total percentage of planned distribution days .....13 Table 3: Enrolment rate for boys and girls between 2015 and 2018 in WFP assisted schools ........15 Table 4: Retention rate in McGovern-Dole assisted primary schools, 2015-2018. ............................17 Table 5: Pass rate in McGovern-Dole assisted primary schools, 2015-2018. .....................................18 Table 6: Budget awarded and total expenditure, by year .....................................................................80 Table 7: Stakeholder analysis and mapping ...........................................................................................99 Table 8: Summary of site map for field data collection.......................................................................102 Table 9: List of Schools visited by region and type...............................................................................104 Table 10: Research techniques by category of stakeholders, and totals ..........................................106 Table 11: Summary of Evaluation Activities and Timelines.................................................................113 Table 12: Overall evaluation activities and timelines...........................................................................114 Table 13: Comparison between PNSAS and McGovern-Dole programme objectives.....................133 Table 14: Vulnerability ranking in McGovern-Dole-assisted districts ................................................139 Table 15: List of Suspension Criteria......................................................................................................142 Final Report v | P a g e Executive Summary 1. This report provides the findings, conclusions and recommendations from the Final Evaluation of WFP Haiti’s Food for Education and Child Nutrition Programme from two grants from USDA McGovern Dole International Food for Education and Child Nutrition Program (McGovern-Dole/USDA) totalling USD 24 million. WFP’s Haiti Country Office (CO) commissioned this decentralized outcome evaluation, which covers the period from September 2016 to June 2019 and the entirety of activities covered by the grants in the development corridors of Cap Haïtien, Port-au Prince and Saint Marc. The primary beneficiaries of the programming are the children attending the targeted schools, directors, teachers and parents. Other beneficiaries include communities, local authorities and the Haitian government, including decentralized entities. The goals of WFP’s school feeding activities are to enhance overall student performance and build capacity to manage school feeding in Haiti. 2. The purpose and objectives of this outcome evaluation were to assess whether the programme achieved expected results, as outlined in the results framework and identify any other outcomes beyond this framework (accountability), and to provide evidence￾based findings to inform operational and strategic decision-making for future similar or alternate programming (learning). The evaluation also assessed the relevance (including its alignment with Haiti’s policy and strategies on school feeding), its efficiency and the sustainability of the McGovern-Dole model. The users of the evaluation are the WFP Haiti CO, USDA, the Ministry of Education, other ministries involved in food production and health activities, as well international development partners. Context and Programme Description 3. In 2017, Haiti’s ranked low on the Human Development Index (168 out of 189 countries) and on gender equality (144 out of 160 countries). Illiteracy and low educational achievement remain major concerns. Primary school children tend to be overage, with some entering primary education at age 12 and half of children will not reach the 6th grade. School attendance and performance is roughly the same for boys and girls but slightly higher for girls. Regarding food consumption and nutrition, food insecurity is higher in rural areas (49.6 percent of households) than in urban areas (37.3 of households on average). Iodine deficiency affects 72 percent of children aged 6-12 in rural areas and 52 percent in urban areas and 32 percent of Haitian children suffer from vitamin deficiencies. 4. The key activities of the McGovern-Dole programme are to provide primary school children, mostly in the public-school system, a daily hot meal made of bulgur, pulses, fortified vegetable oil, provided in-kind by McGovern-Dole, and purchased iodised salt. Micronutrients powder and peanuts were distributed in 2017. Since July 2017, it piloted various activities to improve reading and writing skills in 42 public schools. Awareness raising activities were conducted on hygiene practices across all schools since 2018. Of these, 110 received water chlorine purifying kits and 144 schools with the lowest water Final Report vi | P a g e quality received water purifying tablets. Deworming tablets were distributed to over 111,000 children (51% girls). WFP also carried out capacity building activities at the national, regional and local level to strengthen government management capacity of school feeding programming. Evaluation Questions and Methodology 5. The evaluation questions (EQ) were: - EQ 1: Are the programme activities relevant in light of the needs of school children and their families, and education and school meals policies and strategies of national government? - EQ 2: Has the programme achieved the expected outputs and outcomes at school level and which factors are explaining those results? - EQ 3: To what extent does the community involvement in schools (general, parents especially via PTAs, SFCs, directors, etc.) contribute to outcomes and sustainability? - EQ 4: To what extent are the key institutions (national, provincial/district and local stakeholders; international and national implementing partners and NGOs; international donors and multilateral agencies working on school meals in Haiti) and governance structures able to effectively deliver, implement, sustain and/or scale up school meal interventions in the long-term? - EQ 5: What are the differences and advantages/disadvantages between the McGovern-Dole model and other approaches, focusing on nutritional and educational outcomes and community engagement, and how could it be adapted to increase effectiveness, sustainability, efficiency and long-term effects? 6. The evaluation used a theory-based and non-experimental design with a mixed method approach and purposeful sampling. It collected primary qualitative data combined with a review of secondary data. It entailed a review of literature, a review of programme documents, 50 key informant interviews at the national, regional and school level, and visits to 35 schools to conduct interviews with principals (n=35), cooks (n=35) and focus group discussions with girls and boys and school committees, involving over 350 people. School infrastructures were observed at all 35 schools to determine whether they met the programme’s standards. Due to security concerns, certain schools were deemed too dangerous to visit. To mitigate this, the evaluation team ensured that the sample was representative of the different types of schools in the programme and conducted phone interviews where it was not possible to conduct interviews in person. Key Findings 7. EQ 1: The programme is highly relevant to food insecure and vulnerable school children and families. However, the programme does not systematically consider vulnerability as a suspension criterion and risks excluding the most vulnerable children. The programme is generally aligned with the National Policy and Strategy on School Feeding. It also found that vulnerable children risk being excluded from the programme due to weak school Final Report vii | P a g e management, or by providing food to 3rd cycle students or overage children, which the programme prohibits. 8. EQ 2: The programme has been effective, achieving most planned outputs and outcomes. Key outputs achieved include the number of children attended, meals served, committees trained in canteen management, nutrition and hygiene, as well as number of teachers trained and certified. It also achieved most outcomes, with significant improvements in the enrollment, attendance and pass rate over the baseline, with similar results for girls and boys. However, the baseline did not show a significant difference between beneficiary and non-beneficiary children. Without an end line comparing beneficiary and non￾beneficiary schools, it is not possible to make a direct connection between school feeding activities and academic performance. The evaluation found no significant differences between school type, location, model (traditional, homegrown) or complementary activities such as WASH, school gardens. However, schools which benefited from the literacy component, children exhibit better reading abilities. 9. EQ 3: The involvement of principals, parents (women and men), and school feeding committees contributed to the achievements of outputs and outcomes but, in many cases, the cash or in-kind contribution of parents is insufficient to meet the canteen daily needs and ensure long-term sustainability. Support to local purchases and other homegrown models, such as in Nippes, may help improve community engagement and support the local economy, including women. Women contribute disproportionally more, particularly the cooks who work six or seven hours a day in difficult conditions, including the use of wood, which is detrimental to the environment and their health. 10. EQ 4: Support to the Government of Haiti (GoH) at the national, decentralized government structures and school level has increased capacity to manage school feeding. While important policy framework is in place, the Haitian Government does not have the institutional or financial capacity to manage the programme independently, even partially, until crucial governance issues are resolved at the national level. 11. EQ 5: Local purchases benefit both school children and local producers, including women. Financial mechanisms used by other development actors needs to be explored. As such, the experience of other Caribbean countries with similar climatic issues as Haiti provide examples of initiatives that create learning and economic opportunities at school and community level. Overall Conclusions (see full conclusions in section 3a) 12. EQ1/relevance: The programme is highly relevant to school children and their families, as it is implemented in food insecure areas, particularly rural areas and public schools but vulnerability should be more central as a criterion when WFP is considering suspending schools. 13. EQ2/effectiveness: The programme has been effective in achieving outputs and outcomes benefitting equally girls and boys, regardless of the type of schools, except those with a literacy component, which has the potential to enhance learning outcomes if expanded. Final Report viii | P a g e 14. EQ3/ effectiveness, efficiency, sustainability, impact: The contribution of women and men, girls and boys has been essential to the functioning of the canteen and the achievement of results. However, there is a need to develop a strategy to address gender inequalities. A community mobilisation strategy, focusing on women’s leadership and empowerment is needed. 15. EQ4/ effectiveness, efficiency, sustainability: The evaluation concluded the CO should continue to support capacity building efforts at all levels of government. 16. EQ5/ effectiveness, efficiency, sustainability: The local purchases approach, such as in Nippes, provides economic benefits to communities, including women. Exploring innovative gender sensitive strategies focusing on protected agriculture and climate change could contribute to learning at school level and economically at community level. Recommendations 17. Based on the findings and conclusions of this evaluation, the evaluation team make the following seven recommendations to WFP’s Haiti CO. Strategic recommendations 18. Recommendation 1: continue to support the validation of normative documents and governance structures and assist with their adoption and application, based on the Ten Year Plan for Education and Training (2018-2028)1 . At the same time, emphasis should be put on capacity building efforts at decentralized government structures, including awareness raising and training on gender equality. WFP could support a partnership between partner with the Ministère à la Condition Feminine2 et aux Droits des Femmes to develop material to that effect. 19. Recommendation 2: Develop a gender-transformative strategy for community engagement, using proven approaches that employ social mobilisation within a community development approach to be implemented by NGOs. The suitability of financing tools such as Village Savings and Loans should be investigated in terms of their potential application in support of school feeding and income generation activities. This could also include private sector actors such as SogeBank or Digicel or one of their affiliates such as Mon Cash. 20. Recommendation 3: Consider increasing local purchases and support local producer organizations, especially women using gender transformative approaches. In addition, WFP should seek to promote complementary activities related to nutrition and food growing activities to provide an opportunity for children and their families to learn agricultural practices, and in particular protected agriculture such as greenhouses, better suited to current climatic challenges. WFP’s section in charge of climate change should 1 Plan Décennal de l’Éducation et de la Formation (PDEF) 2 Ministry for the Status of Women and Women's Rights. Final Report ix | P a g e reach out to the Innovation Department of MARNDR and IADB both who expressed an openness to collaborate with WFP in this area. WFP should explore the possibility of turning schools into community hubs through partnerships with community-level service providers. 21. Recommendation 4: Establish strategic partnerships in the field of education when implementing school feeding so that schools served by WFP can also be supported by programmes to strengthen the quality of education. Operational recommendations 22. Recommendation 5: A comprehensive cook stove strategy should be established. This strategy should aim to work with available circumstances to use the most appropriate and practically feasible cooking stove set ups. This should include the elimination of all occupational health and safety risk for women cooks such as those related to cooking on open fires in closed spaces, burning of wood in urban areas where for example, propane stoves could be used and the overall reduction of unwanted environmental impacts. To this end, a partnership with foundations, NGOs and the private sector should be actively explored. 23. Recommendation 6: Include vulnerability as a key criterion in school suspension processes. Provide additional support (e.g. more training) to schools struggling to comply with programme rules. WFP has to establish a strategy to integrate the three school cycles. The practice of sanctioning schools for feeding older students needs to be reviewed. In the event that McGovern-Dole will not fund feeding activities for students in the 3rd cycle, WFP Haiti should seek alternative funding. 24. Recommendation 7: Due to the frequency of emergencies in Haiti, an analysis should be carried out together with donor and Haitian authorities on the use of the McGovern-Dole school feeding platform as a response mechanism to slow and rapid-onset emergencies, such as droughts, earthquakes and hurricanes and political and economic instability. WFP should take the opportunity to ensure that its approach is gender-responsive and promote gender equality. Final Report 1 | P a g e 1. Introduction 1. This report presents the findings of the summative decentralized evaluation of the World Food Programme (WFP)’s Haiti’s Food for Education and Child Nutrition Programme (2016-2019), commissioned by the WFP Haiti Country Office (CO). The evaluation covers two grants of USDA McGovern Dole International Food for Education and Child Nutrition programme: FFE￾521-2015-020-00 (USD 10,000,000) and FFE-521-2016/016-00 (USD 14,000,000), totalling USD 24,000,000. Hence the programme budget was $10M for year 1 and $7M each for year 2 and 3. 2. The purpose of the evaluation was to assess whether the project achieved expected results as outlined in the results framework and identify any other outcomes beyond this framework. It sought to assess its relevance and alignment with the Politique et Stratégie nationales d’Alimentation Scolaire (PSNAS).3 The dual objectives of the evaluation were accountability, i.e. assess performance / achievement of results and learning, i.e. identify why results occurred and use evidence-based findings to inform operational and strategic decision￾making for future similar or alternate programmes. 3. The primary users of the evaluation will be the WFP Haiti Country Office, USDA, the Ministère de l’Éducation national et Formation Professionnelle (MENFP) and the Programme National des Cantines Scolaires (PNCS). WFP will use the evaluation to create awareness among key stakeholders on project activities.4 It will also be used as a platform for discussions to better align the McGovern-Dole programme with national policies and strategies, and to better support GoH’s effort to strengthen social protection and monitoring and evaluation systems. WFP will also use the findings and recommendations of the evaluation to inform the implementation of WFP’s Haiti Country Strategic Plan (CSP), started in July 2019. 4. To this end, the evaluation provides evidence-based findings and recommendations in line with UNEG and WFP standards for evaluation. The evaluation questions were the following: - EQ 1: Are the programme activities relevant in light of the needs of school children and their families, and education and school meals policies and strategies of national government? - EQ 2: Has the programme achieved the expected outputs and outcomes at school level and which factors are explaining those results? - EQ 3: To what extent does the community involvement in schools (general, parents especially via PTAs, SFCs, directors, etc.) contribute to outcomes and sustainability? - EQ 4: To what extent are the key institutions (national, provincial/district and local stakeholders; international and national implementing partners and NGOs; international donors and multilateral agencies working on school meals in Haiti) and governance 3 National School Feeding Policy. 4 For further details on key stakeholders identified in the evaluation see Annex 5, c. Stakeholder Analysis. Final Report 2 | P a g e structures able to effectively deliver, implement, sustain and/or scale up school meal interventions in the long-term? - EQ 5: What are the differences and advantages/disadvantages between the McGovern-Dole model and other approaches, focusing on nutritional and educational outcomes and community engagement, and how could it be adapted to increase effectiveness, sustainability, efficiency and long-term effects? 5. For a more detailed description of the purpose, objectives and scope of the evaluation and the programme, see the terms of reference (ToR) in Annex 1. The evaluation matrix is located in Annex 2. Annex 3 provides a detailed overview of the context to which the programme responds to with a summary presented under sub-section 2b below. 1.1. Subject of the Evaluation 6. The scope of the evaluation was the entirety of activities covered by the grants in all three McGovern-Dole corridors across five departments (North, North-East, Artibonite, Centre and West) in which WFP has been implementing the programme. This section provides a brief overview of the programme. A detailed description of the programme is at Annex 4, including the Theory of Change, programme objectives and activities. 7. During the period reviewed by the evaluation, the McGovern-Dole Schools Meals Programme supported up to 600 schools. About half of the supported schools (49 percent) are located in the Cap-Haïtien, North and North East development corridor; 32 percent are located in the Gonaives and St-Marc, Artibonite corridor and 19 percent are in the Port-au-Prince, West and Centre corridors (see map in Annex 4). 8. The theory of change (ToC) for WFP’s McGovern-Dole funded school meals programme is captured in the Logic model and Results Framework for McGovern- Dole programming in Haiti provided in Annex 4. The ToC asserts that providing a nutritious hot meal to children early in the day will help children learn better, based on various studies on learning and nutrition. 5 According to the World Health Organization (WHO), evidence indicates a causal effect between improved nutrition and learning ability, leading to stronger academic performance.6 Other studies also suggest that low literacy levels in early grades leads to a range of poor academic and health outcomes later. 7 Lastly, numerous studies have found evidence of positive 5 Just, David (2014), https://articles.extension.org/pages/68774/3-ways-nutrition-influences-student-learning-potential-and￾school-performance#_ednref8 Cornell Center for Behavioral Economics in Child Nutrition Programs: https://articles.extension.org/pages/68774/3-ways-nutrition-influences-student-learning-potential-and-school￾performance#_ednref8. 6 World Health Organisation: http://www.euro.who.int/__data/assets/pdf_file/0019/152218/E89501.pdf 7 Gray, Abigail; Goldsworthy, Heather; May, Henry; and Sirinides, Philip. (2017). Evidence for Early Literacy Intervention: The Impacts of Reading Recovery. CPRE Policy Briefs: https://repository.upenn.edu/cgi/viewcontent.cgi?article=1083&context=cpre_policybriefs Final Report 3 | P a g e outcomes related to reductions in diarrhoeal disease from water, sanitation and health (WASH) interventions in schools.8 9. From 2016 to 2019, the key programme activity has been the provision of a daily hot meal to school children between the ages of five to twelve (grade 1 to 6 or 1st and 2nd cycle in Haiti) to ensure regular attendance and improve educational outcomes. The daily hot meal consists of 120 grams of bulgur wheat, 30 grams of green peas, 10 grams of vegetable oil, provided in￾kind by USDA, and 3 grams of iodized salt per child, purchased by WFP on the national market through other sources of funding. Other activities, as agreed upon in the original proposal, were government capacity building, distribution of fuel-efficient stoves, cooking and eating utensils, and various trainings to improve school meals management and quality. 10. The McGovern-Dole programme is implemented by WFP, supported by five implementing partners. During the 2016-17 school year (grant 1), the programme assisted nearly 190,000 primary school children (instead of the agreed to 176,700) across 595 schools (instead of the agreed 650). This is due to a fluctuation in the number of children per school. Following this first grant, WFP began the current programme (September 2017 to June 2019). With this second grant, WFP intended to support approximately 100,000 students across approximately 380 primary schools. However, the number of children fluctuated between 186,889 during the 2016-2017 school year (94,294 girls / 92,595 boys) and 116,735 (56,938 girls / 59,797 boys) during the 2018-2019 school year. 11. The programme delivers the same food rations to both boys and girls and also aims to advance gender equality through female participation and leadership in the school committees as well as through involvement in the management of school meals. The WFP Haiti Country Office (CO) is collecting various data of interest disaggregated by gender. However, during the period under review, the programme did not have a strategy or plan of action to promote gender equality. In March 2019, the CO hired a gender specialist to undertake a comprehensive gender analysis the programme, as recommended in the 2017 review commissioned by Canada. 12. Distribution of a daily bag of roasted peanuts (28g) was done in all schools in 2017 using other sources of funding. Micronutrient powders (MNP) , also from other sources were added to meals in two development corridors: one bag of 8 g per 20 children but discontinued the same year. The programme also conducted various training and awareness raising activities designed to improve literacy, health and hygiene. Since July 2017, activities to improve reading and writing skills were carried out in 42 public schools as a pilot project. Awareness raising activities were conducted on hygiene practices and soap distributed across all schools (401) since 2018. Of these, 110 received water chlorine purifiers kits and some 144 schools received water purifying tablets with replaceable batteries and solar panel battery chargers. WFP also carried out capacity building activities at the national, regional and local level to strengthen 8 McMichael, C, (2019). Water, Sanitation and Hygiene (WASH) in Schools in Low-Income Countries: A Review of Evidence of Impact, International Journal of Environmental Research and Public Health: https://www.mdpi.com/1660-4601/16/3/359/html. Final Report 4 | P a g e government management capacity of school feeding. During the 2017-2018 school year, Micronutrient Powders (MNPs) were used to enrich school meals in 204 selected schools for 63,000 boys and girls, in the MDG development corridors of Saint Marc and Port-au-Prince. 13. Partners involved in the intervention include: the Ministère de l’Education Nationale et de la Formation Professionnelle (MENFP) and the Programme National des Cantines Scolaires (PNCS) at the national government level; five implementing civil society partners at the field level; various school-connected partners at the community level; private transportation companies and finally, UNICEF and the World Bank as key technical partners. An analysis of stakeholders and their specific interest in the evaluation is provided in Annex 4 (see par. 21 and Table 8). 1.2. Context 14. Haiti has a population of approximately 11.24 million, with 42 percent of its population under the age of 18.9 Haiti is also the poorest country in the Americas, with 59 percent of its population living below the national poverty line of USD 2.41 per day, with 50% of the population considered undernourished. In 2017, Haiti’s Human Development Index (HDI) value stood at 0.498 placing it at the lower end of the scale, 168 out of 189 countries. Furthermore, political instability over the past three decades has been accompanied by underperformance of government institutions, and poor governance and accountability. In addition, the country’s fiscal deficit has resulted in low spending in key areas such as the education, and health sectors. Furthermore, while only 60 percent of rural households (hh) have access to safe drinking water compared to 95 in urban areas and only 17 percent of hh in rural areas have electricity compared with 76 percent in urban areas.10 15. Haiti’s Gender Inequality Index (GII) is high at 0.601, ranking the country 144 out of 160 in 2017. This ranking is due to various inequalities between men and women and issues such as gender-based violence, high teenage pregnancy rates, and low participation of women in the labour market. In 2014, Haiti adopted its first 20-year National Gender Equality Policy: “Ensuring gender equality" is a central axis of this Policy and, in principle, guides the actions of all sectors. 16. The agricultural sector in Haiti, though remaining the anchor of the economy, has been functioning under duress and suffered consecutive years of inconsistent agricultural production. In 2017, at the peak of the spring harvest, 3.6 million people were considered food insecure, 1.5 million severely so. Severe risks of food shortage were also forecasted for 2018 and 2019. Climate change and ongoing environmental degradation have further altered the agricultural-food security equation in Haiti. Large scale environmental disasters, primarily the earthquake in 2010, Hurricane Sandy in 2015 and Matthew in 2016, caused widespread damage, leaving the country continually rehabilitating its agriculture sector. These major 9 This section presents a summary of relevant background for the evaluation. For more information on the context please refer to the ToR in Annex 1 and Annex 3, based on a literature review that the ET conducted in the inception phase. 10 EMMUS VI. Final Report 5 | P a g e disasters exacerbated the seemingly endless cycle of droughts, storms, and floods that have become commonplace in increasingly unpredictable climatic conditions. 17. While all children in Haiti have the right to free and quality education, illiteracy and low educational achievement remain of major concern. According to EMMUS VI (2016-17), 13 percent of women and 9 percent of men have never attended school, while a UNICEF report estimates that half of children will not reach the 6th grade. The situation is even direr in rural areas where attendance decreases sharply after the 4th grade. Iodine deficiency affects 72 percent of children aged 6-12 in rural areas and 52 percent in urban areas and 32 percent of Haitian children suffer from vitamin deficiencies. 18. The MENFP has tried to improve student performance through a national training policy for teachers and subsidizing school fees since 2007. Recently, however, subsidies were suspended, shifting the costs back onto families. Consequently, challenges faced by the public education system resulted in non-state actors increasingly filling gaps. Non-state actors now operate almost all pre-schools in the country, more than 80 percent of primary schools, and more than 70 percent of secondary schools. 19. Haiti has a long history, starting in the 1980s, of school feeding initiatives. The PNCS was created in 1997 to administer and regulate the national school meals programme. Following the 2010 earthquake, a GoH report called for a national school feeding strategy, which led to the creation of the PSNAS in 2016. At the time of its development, it was hoped that PSNAS would lead to additional support, notably from MENFP, to ensure the success of the PNCS. However, despite past and recent efforts, including an evaluation and a review of WFP’s support to national school feeding,11 school meals day-to-day operations are still almost entirely dependent on donor contributions and implementing partners (e.g. WFP and non￾governmental organisations (NGOs). 1.3. Evaluation Approach, Methodology and Limitations 20. This outcome evaluation, 12 assessed the relevance, effectiveness, efficiency and sustainability of the McGovern-Dole school feeding model in conformance with WFP and UNEG ethical standards and respected the norms and standards of the United Nations Evaluation Group (UNEG).13 21. Given that school-feeding has been well tested over time and for practical reasons, e.g. cost of quasi-experimental designs, the evaluation used a theory-based, non experimental design with a mixed method approach14, which included the collection of primary qualitative data 11 Aa evaluation of WFP’s Support to the National School Feeding Programme and a Review of Canada funded programming we conducted in 2014 and 2017 respectively. 12 As opposed to an impact evaluation. The ToRs (See Annex 1 section 4.4 Methodology) indicate that the evaluation is expected to “assess to what degree and why the outcomes have changed between the baseline and end line data collection rounds. 13 UNEG (2005) Norms for Evaluation in the UN System. Towards a UN system better serving the peoples of the world; overcoming weaknesses and building on strengths from a strong evidence base. United Nations Evaluation Group 14 See Annex 5, par 4 for a justification of this approach. Final Report 6 | P a g e combined with a review of secondary quantitative monitoring data collected by WFP. WFP data contained the universe of schools in the programme, which the evaluation team could not have replicated within the evaluation’s scope and budget. 22. The methodology entailed a review of literature, a review of programme documents and WFP monitoring data, 50 key informant interviews at the national, regional and school level, and visits to 35 schools to conduct interviews with principals (n=35), cooks (n=35) and focus group discussions with girls and boys and school committees, involving over 350 people. School infrastructures were observed at all 35 schools to determine whether they met the programme’s standards. Due to security concerns, certain schools were deemed too dangerous to visit. To mitigate this, the evaluation team ensured that the sample was representative of the different types of schools in the programme. Team members also conducted phone interviews where in-person was not possible. Further details of the methodology are in Annex 5. Annex 6 provides a detailed list of documents reviewed, Annex 7 a list of stakeholders interviewed by gender, and Annex 8 the data collection instruments. 23. The evaluation used a purposive sampling approach but also integrated elements of randomness into its design. A sample of 35 schools across the three programming corridors were visited and the representativeness of these schools was validated by WFP central and field offices. The evaluation team (ET) reviewed secondary data and visited schools that were representative of three different scenarios: 1) school meals alone; 2) school meals with the literacy component; and, 3) school meals and the WASH component to assess whether results for boys and girls differed by type of schools. 24. Gender was also an integral part of the evaluation. The data collection approach, using secondary and primary data that was sex-disaggregated, ensured that the ‘voice’ of women, men, girls and boys were equally reflected and analysed in light of different gender considerations. The evaluation also benefited from the findings of a gender study15 conducted concurrently to complement and confirm its own findings. See Annex 10, section d. for a summary of the gender study findings and suggestions to improve gender equality through feeding programming in Haiti. 25. Data were analysed and triangulated across different methods and sources to generate findings for each evaluation question and validating these with WFP and the evaluation reference group (RG). Sufficient valid and reliable data was captured to answer all of the evaluation questions. The programme had up-to-date monitoring data up to March 2019, which the ET used to assess results against the baseline. 26. The main limitation was the difficulty in obtaining quantitative data at the national level to compare results of the programme with other schools, as well as for McGovern- Dole schools specifically. Despite several attempts, national data was not communicated by MENFP. WFP annual reports provide data for all its areas but not differentiated between McGovern-Dole 15 Nora Astor, Nora Astor, Gender Analysis of WFP School Feeding Program in Haiti, September 2019. Final Report 7 | P a g e and non-McGovern-Dole schools. Data triangulation was used as a key measure to mitigate these limitations. 27. Finally, quality assurance was ensured by pre-testing all data collection tools, as well as ensuring that team members had a common understanding of the evaluation questions and indicators. Team meetings were frequently held during data collection to uncover, discuss and resolve issues. All data collected was stored in a common site accessible by all team members and discussed among the team, thus allowing the data to be peer reviewed and findings validated among the evaluation team before presenting them to WFP and the RG. 2. Evaluation Findings 2.1. EQ 1: Relevant of the programme in light of the needs of school children and families, and education and school meals policies and strategies of national government 2.1.1. Alignment with the National School Feeding Policy and Strategy Finding: The programme activities are generally aligned with the National Policy and Strategy on School Feeding (PSNAS) in terms of educational benefit, nutrition and health but less so in some areas, such as support to the local economy, environmental safeguards, and meeting recommended nutritional standards for school feeding. The programme is also not sufficiently shock responsive. 28. The Politique et Stratégie Nationale d’Alimentation Scolaire (PSNAS), approved in 2016, sets the cornerstones of the national school feeding programme such as implementation modalities, the roles of respective actors and financing mechanisms. It consists of three General Objectives (GO) and 12 Specific Objectives (SO), guided by key principles that define the national SF programme’s implementation framework. 29. The comparative analysis of the McGovern-Dole Results Framework and the objectives of the PSNAS shows that the programme results are generally consistent with PSNAS objectives (see Table 13, Annex 9). Programme results contribute to educational benefits (SO 1.1 of the PSNAS), as well as nutrition and health (SO 1.2). The programme aims to strengthen national institutions (SO 3.1), develop sustainable funding strategies (SO 3.2) and has accountability at his core (SO3.4), as well community involvement (SO 3.5). 30. However, the programme does not actively support local economy and agriculture (GO 2). 16It also cannot be considered as ‘shock-responsive’ (SO1.3) and does not have environmental safeguards (SO 1.4). No mention is made in McGovern- Dole programme documents of the use of cooking fuel coming from sustainable, non-polluting sources or of any means to reduce unwanted environmental impacts, even though, according to the baseline study, 78 percent 16 This issue is treated extensively in the EG 5. Final Report 8 | P a g e of schools use dried wood to cook and 22.7 percent use coal, while only 4.7 percent use propane gas. 31. Shock-responsive safety nets are becoming more common in the region but deepening or extending the McGovern- Dole programme as a safety net during emergencies has not been considered in the aftermath of Hurricane Matthew in 2016, according to WFP staff. WFP staff indicated that they used schools as an entry point to identify families affected by slow onset disasters, such as drought in late 2018 and early 2019, by providing short-term assistance, with schools used as food distribution points and using some of them as shelters. However, these activities fall short of a 'shock-responsive’ extension of the school feeding programme, especially as they are not funded by the McGovern- Dole programme. 32. In terms of nutritional impact, although the programme implements specific health activities, such as the distribution of a food basket composed of three products, and provides training on nutrition, they do not allow for food diversity and the daily food ration has an energetic value inferior to PSNAS target (601 Kcal on average versus 1040 Kcal). The PSNAS also recommends a morning ‘snack’ to be given to children when they arrive at school to help their concentration. However, the programme does not include this important feature; the peanut snack, distributed in 2017, was discontinued. 2.1.2. Geographic targeting, food security, nutrition needs, vulnerability and access to education Finding: The programme has a stronger presence in food insecure regions, rural areas and public schools, where needs are greater. However, some of the most vulnerable children risk being excluded from the programme due to inadequate school feeding management at school level and the exclusion of 3 rd cycle students from school feeding. 33. The evaluation examined the extent that WFP used geographic targeting and other needs assessment to determine which schools would be included in the programme. It found that, overall, the targeted schools are serving vulnerable children and their families. 34. Interviews with WFP staff indicate that most schools included in the initial selection in these corridors were part of a legacy programme funded by Canada. Other schools were added based on assessments conducted by implementing partners to replace schools excluded due to poor school feeding management. In other instances, the programme targeted vulnerable communities with no schools in the programme, giving priority to public schools, which generally serve economically deprived populations. 35. A review of the Integrated Food Security Phase Classification (IPC), a tool ranking livelihood areas and based on the percentage of food insecure people, shows that overall programme schools are located in food insecure areas. The last IPC for chronic malnutrition in Haiti developed in 2015 (see Figure 13, Annex 9), shows that four out of five departments where the programme is implemented were classified as severely food insecure (North-East, Centre) or moderately food insecure (Artibonite, North). The only exception is Port-au-Prince region where only 5 percent of the population is severely insecure. Final Report 9 | P a g e 36. In 2018, the Coordination Nationale sur la Sécurité Alimentaire (CNSA) collected data and ranked 43 districts on food insecurity. The ranking shows programme coverage included the district with the highest level of food insecurity in Haiti (Gros Morne) but also the three districts with the lowest incidence of food insecurity (Dessalines, Saint-Mart, Borgne). WFP staff interviewed noted that food insecurity is present in every department and district and that the programme sought to identify vulnerable schools in those districts. 37. With regards to vulnerability, the Agreement signed between WFP and McGovern-Dole (FY 2016) states that ‘public schools in rural areas were to be prioritized.’ The programme indeed focuses on public and rural schools, which serve some of the most vulnerable children in Haiti. WFP monitoring data shows that among the 400 schools receiving food during the 2018-19 school year, 72 percent are public and 74 percent are located in rural areas. 38. The vulnerability of children in rural areas is also highlighted in the UNICEF Out of School Children report. It describes the situation of school drop-outs, exclusions, late entrance and early exits from the education system, there is a significant variance between urban and rural areas. Three times more children from rural areas are not going to school compared to urban areas, and children are more likely to leave school early in rural areas: 22 vs. 14 percent for primary school children 17 , and 11 vs 7 percent for children of the 3 rd cycle. The study also notes that socio-economic indicators are worse in rural areas compared to urban areas, with poverty rates of 74.9 percent vs. 40.6 percent. This is explained in part by the fact that Haitians living in urban areas have more access to private financial transfers such as remittances, employment opportunities and essential goods and services. 39. Interviews and focus groups discussions at school level confirmed that that providing a school meal is very relevant to the children and their families. The evaluation team (ET) heard countless times how meals at school helps children concentrate, especially as many schools start at 7 am and end at 1 pm. The ET also found that it reduces the overall cost of sending children to school, considering high percentage of female headed households in Haiti and the economic barriers they face, especially in rural areas (See Context in Annex 3). 18 Indeed, several school principals and committee members noted that, for some children, this meal is the only meal of the day. Some stakeholders noted that in urban areas, many mothers work as street vendors, leaving the house early and returning late. The school canteen helps those families by providing a meal that would otherwise be difficult to come by. In some urban schools, principals and committee members noted that some children living in domesticity do not receive sufficient attention from their guardians on their daily dietary needs. Some stakeholders also pointed out that school feeding is the largest safety net in Haiti. 40. However, while programme targets and assists vulnerable children, the standards and rules that schools must abide by, or face suspension, risk excluding very vulnerable children. The 17 Gender disaggregated data was not provided in the study. 18 Charles Boisseau (2018) University of Florida, citing FAO, The State of Food and Agriculture, Closing the Gender Gap (2010-11) Final Report 10 | P a g e minimum structural and personnel requirements make it difficult for some public schools to qualify for the programme. To overcome this challenge, WFP has used other sources of funding to help some of these schools meet the standards. Stakeholder interviews and school visits provided solid evidence that WFP has supported the building or improvement of infrastructure such as warehouses, kitchens and wells to ensure that some of the most vulnerable children have access to a school canteen. WFP also supported principals and management committees by providing training and monitoring tools to back the day-to-day management of the canteen. As a result, all schools, even the most vulnerable, have access to the programme, which in turn supports access to education. 41. Monitoring reports, interviews and FGDs with stakeholders highlighted the fact that WFP is diligent in its monitoring to ensure that the food reaches the intended beneficiaries. It undertakes strict attendance monitoring, validating reported attendance against random monthly physical counts and adjusting food deliveries accordingly. WFP also suspends schools unable to hold classes as per the national school calendar or other irregularities due to inadequate management of the canteen. During the reporting period, WFP reports show that out of 400 schools assisted by the McGovern-Dole programme, two schools were definitively suspended from the programme and another 25 temporarily suspended for the reasons mentioned above. 42. One criterion for suspension is feeding children from the 3rd cycle, which the programme prohibits. Most schools visited considered this problematic given the difficulty of implementing this when the three cycles are in class at the same time, not to mention challenging on moral grounds to deny food to children who are also hungry. Furthermore, in rural areas, 72 percent of first-grade students are overaged for their grade.19 The evaluation team saw children as old as 16 years old in cycle 2 and heard of 12-year old children in the 3 rd cycle not eligible for a school meal. According to interviews, in some cases this situation has generated a violent reaction. 43. The absence of school canteens for the 3rd cycle students is perceived by many principals and committee members as an injustice and a serious grievance. On many occasions, the evaluation team heard that it is morally untenable in Haiti to be eating in front of others who are also hungry. Furthermore, the PSNAS encourages feeding children of the 3 rd cycle, while giving priority to children from the 1st and 2nd cycles. Furthermore, some argued that nutrition studies have also shown that many adolescent girls suffer from anaemia, 20 and would benefit nutritionally from receiving a school meal. 44. Most principals interviewed consider WFP rules for school suspension to be clear and fair. WFP clearly communicated suspensions criteria and school directors signed a contract with WFP local Partner NGOs and MENFP representatives. They are aware that the conditions 19 https://www.unicef.org/haiti/french/overview_16366.html 20 WHO (2011). Prevention of iron deficiency anaemia in adolescents. Role of Weekly Iron and Folic Acid Supplementation. Final Report 11 | P a g e needed to remain in the programme. Local authorities are also informed promptly of any wrongdoing and are asked to take action. However, some interviewed expressed concerns about some schools being excluded due to bad management, the responsibility of school principals, while those who suffer are the children. They suggested being informed more regularly about any misconduct so that they can transfer problematic principals to schools that do not have canteens, which reportedly happened on one occasion to date. WFP staff also reported that in Artibonite, the Direction Départementale de l'Éducation (DDE) called a meeting with school principals and warned them that they would suffer the consequences of not abiding by the rules of the programme. 2.2. EQ 2: To what extent has the programme achieved the expected outputs and outcomes at school level and which factors are explaining those results? 2.2.1. Achievement of Outputs Finding: The programme has achieved most planned outputs, such as the number of children attended, meals served, committees trained in canteen management, nutrition and hygiene, as well as number of teachers trained and certified. 45. This section summarizes the outputs achieved, based on the Performance Monitoring Plan (PMP) indicators and targets, which follow the structure of the Programme Results Framework, articulated around both WFP Strategic Objectives. (see Table 14, Annex 10). The full compilation of WFP monitoring indicators is presented in Figure 19, Annex 10. 46. Number of individuals benefiting from McGovern-Dole-funded interventions: Target achieved. Targets were exceeded for each school year but the number of children covered decreased as the funding was not sufficient to assist as many children as planned. Figure 1 shows the planned vs. actual number of children provides with a school meal by year. Overall, the programme has served meals to an equal number of boys and girls, with variations on a yearly basis based on attendance, as shown in Figure 1. See Figure 2 for the number of meals served vs planned per semester. Figure 1: Number of school children provided a school meal, planned vs. actual 94,294 77,905 56,938 92,595 81,071 59,797 School year 2016-2017 School year 2017-2018 School year 2018-2019 Male Female Final Report 12 | P a g e Source: WFP McGovern-Dole semi-annual reports 21 47. Number of daily school meals provided as a result of McGovern-Dole assistance: Target not achieved for FY 2015 and FY 2016. The evaluation found several reasons for this. For instance, documents show that for the first project, the contract was signed in December 15, 2015 but commodities only reached Haiti between April to June 2016. McGovern-Dole and the CO agreed to delay distribution to the following school year. In addition, Hurricane Matthew, which hit Haiti in September 2016 and Irma in August 2017 had immediate and medium-term destabilizing effects, including severe water and fuel shortages. With McGovern-Dole agreement, food for the school feeding programme was used to respond to these emergencies. Other events such as strikes (e.g. school closures due to unpaid salaries), the absence of cooks and inaccessible roads reduced the number of school canteen days during that period. Figure 2 provides an overview of the meals served during the period under review. Figure 2: Number of actual meals served vs. planned per semester Source: WFP semi-annual reports 2.2.2. Other Output Indicators of Interest22 48. Percentage of days of food distribution as a total of planned distribution days. WFP monitoring data show that the programme covered two-thirds of planned distribution days during the 2016-17 school year, increasing to 80 percent and 92 percent for subsequent years. WFP found the actual number of days with a functioning canteen was around 120 days per year versus planned distribution of 180 days. For instance, there is a tendency for some schools, especially in rural areas, to remain closed a few additional days after major public holidays. In addition, various sources indicate that the political disturbances (demonstrations, riots, etc.) in November 2018 and February 2019 led to many schools remaining closed. 49. In terms of food deliveries to schools, various sources, including school directors, confirmed that delays have been minimal. The exception was found in the St-Marc corridor, where the 21 Figures for the last semester will be updated taking into consideration latest meals distributions. 22 These indicators are not part of the McGovern-Dole PMP but have been monitored by WFP Haiti. 0 5,000,000 10,000,000 15,000,000 20,000,000 25,000,000 30,000,000 35,000,000 01/10/2016 to 31/03/2017 01/04/2017 to 30/09/2017 01/10/2017 to 31/03/2018 01/04/2018 to 30/09/2018 01/10/2018 to 31/03/2019 01/04/2019 to 30/09/2019 School year 2016-2017 School year 2017-2018 School year 2018-2019 Planned/ Target Actual/ Achieved Final Report 13 | P a g e canteen started a month later than planned due to delays in signing the contract with the Bureau de Nutrition et Développement (BND). Documents show that in 2019, WFP anticipated a gap of 997 metric tons to cover the remaining school year, due to increased tariffs on the US–Caribbean trade route. 23 WFP covered the gap with 622 MT of emergency stocks (529 tons of rice, 47 MT of peas and 46 MT of vegetable oil) from other donor’s stocks in order to keep the programme running to the end of the school year without interruption. Table 1: Number of food distribution days as a total percentage of planned distribution days Source: WFP monitoring data 50. Food and meals diversity. WFP monitoring data collected during the 2018-19 school year with a sample of 980 children show that nearly three quarters of pupils (73 percent) consider that meals were monotonous. This frustration was expressed unfailingly by parents, school principals and children, regardless of gender, in all schools visited. The evaluation team heard consistently that meals based on bulgur and pulses were extremely monotonous day in and day out throughout the school year and that more diverse meal options should be provided. Only a portion of the schools visited, mostly those run by religious congregations, were able to complement the offering with vegetable and fish (‘la sauce”)24 on a regular basis. 51. In addition, WFP monitoring data showed that 53 percent of boys and girls think the portions served are too small, especially for older children attending primary school. In line with the planned caloric value for the McGovern-Dole programme, the daily ration of 163 grams represents about 40 percent of the energy needs for students aged from five to ten years of age (the daily need is 1,640 kilocalories). However, this only covers 25 percent of the nutritional needs of students over ten years old, many of whom are attending the 2nd cycle. 25 52. Nutrition-sensitive activities. In addition to the food provided, three types of complementary activities were completed during the evaluation period: ▪ BND created 128 school gardens in the Saint Marc corridor. This complementary activity focused on the distribution and cultivation of moringa plants, in alignment with the PSNAS. According to WFP, the plants were delivered, and follow-up was done for a year. However, while the ET saw plants laying about in school yards in several locations, they did not see a functioning garden. 23 Haiti FY 2016 Amendment Request Letter, 25 July 2018. 24 Sauce is the word used in Haiti to complement the basic meal of grain and pulse. 25 C. Dabone (2019). McGovern Dole special study in Haiti – Final Report. 2016-17 2017-18 2018-19 # of calendar days 186 194 194 # of planned distribution days 180 100% 150 100% 130 100% # of actual school days 165 92% 150 100% 148 113% # of days with functioning canteen 119 66% 120 80% 113 86% Final Report 14 | P a g e ▪ During the school year 2018-2019, deworming tablets were distributed to a total of 111,051 children (51percent girls vs 49 percent boys). According to WFP staff, this activity is conducted every six months, per the Ministère de la Santé Publique et de la Population (MSPP)26 guidelines. ▪ In 2017, micronutrient powders (MNPs) were used to enrich school meals in 204 selected schools (63,137 children) in the Saint Marc and Port-au-Prince corridors. The overall objective was to reduce the micronutrient deficiencies, particularly anaemia. The impact study conducted demonstrated the effectiveness of the MNPs. However, according to interviews, this activity ended due to a request from the GoH to improve nutrition through food rather than powder. 53. Number of teachers/educators/teaching assistants trained or certified as a result of McGovern￾Dole assistance: Target almost achieved. All the indicators related to literacy have been achieved or were close to being achieved through Catholic Relief Services (CRS) activities. Starting in September 2017, CRS distributed 61,000 Creole and French reading kits and supplies for first and second grades to 42 public schools in the Port-au-Prince corridor, reaching 8,596 students (4,270 boys and 4,326 girls) and 116 teachers (39 males and 77 females). The kits include textbooks, workbooks and teaching materials for teachers. CRS also trained teachers and school directors on the Literacy Curriculum, Methodology and Classroom Management, called “Ann Ale” in Creole. The school directors, teachers and students interviewed all reported being very happy with the materials. Interviews directors and teachers also indicated they would like materials and training for other grades as well. 54. Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of McGovern-Dole assistance: 27 Target achieved. All schools taking part in the programme are requested to have functioning ‘supervision’ and ‘management’ committees in charge of programme implementation. These are composed of the principal, teachers, students, parents, and members of the personnel. According to the baseline study, 93.3 percent of the schools already had a committee at the beginning of the programme. Based on WFP monitoring data, by 2018 all schools in the programme had a functioning committee. Figure 3 shows the number of PTA trained by semester over the period under review. Figure 3: Number of participants by training activities, men and women. 26 Ministry of Public Health and Population. 27 More information on the role of men and women in committees will be given in Question 3. Final Report 15 | P a g e Source: WFP Outputs matrix 55. Based on group discussions with committees in various schools, women participate more than before in decision-making as a result of the programme’s requirements. Women state being less afraid and more motivated to exercise their rights by running in different elective positions on committees. WFP data shows that the number of women occupying positions of responsibility has increased during the evaluation period. 2.2.3. Achievement of Outcomes Finding: Overall, the programme has achieved planned outcomes, with targets nearly achieved for enrollment, attendance and retention and surpassed for pass rate, with similar results for girls and boys. 56. As noted in the methodology section, a key limitation of outcomes indicators is that the Ministry of Education did not provide data for comparison purposes. WFP reports national level data but does not differentiate between McGovern-Dole and non-McGovern-Dole schools. However, the evaluation found that it could use data of other WFP school feeding programming in Haiti and data from the literature to assess results, as the programmes are very similar, except for rice being the staple instead of bulgur. 57. Enrolment Rate28 . The rate of enrollment for WFP assisted schools increased overall from the baseline but below the annual target of 6.0. Enrollment grew in 2016 and 2017 over the baseline but experienced a marked decrease in 2018. In 2016, the rate of increase in enrollment was estimated at 5.10, with a rate of 4.80 for boys and 5.40 for girls. The overall rate of in 2017 was 4.95, with 5.62 for boys and 4.29 for girls. Enrollment decreased inexplicably in 2018 for boys and girls to minus 2.8 overall, a huge drop from 4.95 in 2017, with rates of - 3.2 for boys and -2.4 for girls. Table 3 provides an overview of enrollment rates between 2015 (baseline) and 2018. Table 2: Enrolment rate for boys and girls between 2015 and 2018 in WFP assisted schools Outcome Indicator Gender Annual Target Increase Baseline 2015 2016 2017 2018 Male >6.00 0.30 4.80 5.62 -3.2 28 Sources : WFP SPR; UNICEF ‘Out-of-School’ report; Cambridge Education (2019). Analyse sectorielle détaillée. 95,008 217 497 217 95,939 94,165 576 994 576 96,311 0% 50% 100% Child health and nutrition Food preparation and storage School canteen management Supply chain management Total Women Men Final Report 16 | P a g e Outcome Indicator Gender Annual Target Increase Baseline 2015 2016 2017 2018 Average annual change in number of children enrolled in WFP￾assisted primary schools Female >6.00 -1,1 5.40 4.29 -2,4 Overall >6.00 -0,7 5.10 4.95 -2,8 Sources: SPR 2016 and 2017; ACR 2018 58. Attendance Rate.29 The output indicator was Number of students regularly (80%) attending McGovern-Dole supported classrooms/schools. The evaluation found that WFP monitored attendance rates were not reported systematically for the McGovern- Dole programme or at national level through WFP’s standard programme reports (SPR). However, based on semi￾annual reports, attendance compared to enrolled students varied between 75 percent and 97 percent. Between October 2016 and March 2017, the core months of schooling, was 97 percent. The difference between the overall attendance rate for 2016-17 is due to the overall reduction in children in the programme. Between October 2017 and March 2018, the reported attendance rate was 75 percent and remained the same between October 2018 and March 2019, falling slightly short of the yearly increase targets but well above the attendance target of 60%. 59. While there was no clear explanation for the decrease from 97 percent in 2016-17 to 75 percent in the other school years. According to WFP staff, the distance of the school from home, security issues along the routes, especially for girls, and the economic support that children must provide to their parents such as working in the fields (more for boys) or at the market (more for girls) all play a role. 60. The attendance rate achieved in WFP assisted schools is similar with the UNICEF estimated national average, based on the ‘Out-of-school children’ study published by UNICEF30 in 2019. The study indicates that an average of 18 percent of children aged 6 to 11 are not going to school, establishing an average attendance rate of 82 percent for children in this age category. According to the study, this exclusion is more significant in rural areas and highly dependent on the economic status of the children and their families.31 61. While the causal link between attendance could not be verified (through a control group for instance). The attendance by grade level and gender observed in the classroom of the beneficiary and non-beneficiary schools during the baseline suggests such a casual effect, with significant difference in beneficiary and non-beneficiary schools for both girls and boys: 11,123 vs 4,875 and 11,430 vs 4,895 respectively. While the evaluation could repeat this survey, stakeholder interviews, and FGD at school level consistently highlighted the positive 29 Defined as the number of students attending school on a specific day as a percentage of students enrolled. 30 UNICEF, Etude sur les enfants hors de l’école en Haïti, Rapport Final, 2018. 31 PSNAS, 2016. Final Report 17 | P a g e impact of school feeding programme on attendance. Focus group discussions with committees and children and interviews with school principals confirmed that school meals are a significant incentive to attend classes, especially for children coming from extremely poor families, and more so in rural areas where food production has become more difficult. The evaluation team was often told that at the start of the day some students first visit the kitchen to check if the meal is being prepared. If not, some decide to go home. Attendance therefore drops when meals are not served, even more so when the meals are not served during a sustained period of time. According to interviews with canteen management committee members and principals in the West corridor, children living in domesticity do not receive the same support from guardians for attendance and learning than children living with their own parents. These stakeholders also strongly believe that school feeding improves children’s performance. However, the evaluation did not find a direct causal effect between school feeding and pass rates. The baseline showed relatively similar pass rates for children in beneficiary and non-beneficiary schools, but slightly higher for beneficiary schools. Without an endline survey comparing beneficiary and non beneficiary schools, the effect cannot be established with certainty. 62. Retention Rate.32 Retention rates have been relatively stable but decreased slightly over the programme’s lifespan for both boys and girls, from 95.5 percent on average in 2015 to 91 percent in 2018, slightly under the project target of 95.5 percent. 33 According to the baseline study, most of the drop-outs can be attributed to economic hardship, parents’ migration out of the country, and the disruptions that the programme endured during political unrest. The Analyse sectorielle détaillée34 data shows that drop-out rates at national level for primary schools is lower in schools that provide daily meals. Data also indicate that boys and girls have similar retention rates in primary schools, since there are no social barriers to girls’ participation, based on a variety of sources, including a meta analysis conducted by USDA. 35 Table 3: Retention rate in McGovern-Dole assisted primary schools, 2015-2018. Outcome Indicator Gender Project End Target Base Value % 2015 % 2016 % 2017 % 2018 % Retention rate in WFP￾assisted primary schools Male >96.00 96 93 92,4 92 91 Female >95.00 95 92 92 91.6 91 Overall >95.50 95.5 92.5 92.2 91.8 91 32 Indicator that measures the extent to which a school meals program has contributed to keeping girls and boys in school, calculated as the percentage of students enrolled at the beginning of the year, that pursue their study the next year. As such, it is the complement of the drop-out rate. 33 The UNICEF ‘out-of-school’ report provides data of an average drop-out rate of 5% for the first two cycles, at the national level, which means a retention rate of 95%. 34 Cambridge Education (2019). Analyse sectorielle détaillée, PDEF 2018-2028 Haïti, p. 27. 35 The McGovern-Dole International Food for Education and Child Nutrition Programme School Feeding and Educational Outcomes in Developing Countries: A Systematic Review and Meta-Analysis. Final Report 18 | P a g e Sources: SPR 2016, 2017; ACR 2018 63. Pass rates for programme assisted students have been more or less stable over the three years, at more than 70 percent, 36 a significant increase from the baseline and well above the 60 percent target, with a small advantage for girls. Table 4: Pass rate in McGovern-Dole assisted primary schools, 2015-2018. Outcome Indicator Gender Project End Target Base Value 2015 2016 2017 2018 Pass rate in WFP￾assisted primary schools Male >60.00 50 77.77 79.2 72.40 72 Female >60.00 50 78.85 81.1 74.00 74 Overall >60.00 50 77.85 80.2 73.10 73 Sources: WFP SPR 2016 and 2017; ACR 2018 2.2.4. Differences in outcomes based on gender, type of school, location, model or complementary activities Finding: The evaluation did not find significant differences based on gender, type of school (public, private), geographic location, model (traditional, homegrown) or complementary activities (WASH, school gardens), except for schools which benefited from the literacy component. 64. With regards to gender equality, the programme helped promote the rights of boys and girls to education. For instance, in the proposed legislation on school feeding and nutrition, Article 5 deals with the Right to Food and School Nutrition: "Every child, girl or boy, has the right to proper nutrition, given its age, for its growth and development and the State guarantees without discrimination the progressive realization of this right in all schools of the country." By acting on school feeding, the programme reduces a major constraint in the realization of this right. The programme equitably gives girls and boys the same opportunities to flourish in education and through education. By increasing school attendance and retention rates, the programme also contributes to the realization of this right. 65. The evaluation did not find solid comparative data regarding the impact of the McGovern￾Dole programme on different types of schools. However, school visits indicate that schools managed by religious congregations have a better capacity to obtain parental contributions and compensate cooks more regularly, and with higher amounts. Religious congregations can make up for shortages in parental contributions, according to interviewees. This results in school canteens functioning better and parents contributing more financially. Teachers are 36 Data collected for 2018 in McGovern-Dole (in 68 schools) and other assisted schools (53) do not show any significant difference. Final Report 19 | P a g e paid and trained regularly, so that the impact of the school meals on attendance is lower than in public schools, particularly in rural areas. 66. Furthermore, the data shows that in private schools, the number of students is controlled, which influences learning and educational achievements. By contrast, public schools have to live with irregular funding, overcrowded classrooms, and delayed payments of wages for teachers or teachers not on the payroll. In some cases, teachers look for replacements and the substitutes are paid a fraction of the teacher’s salary. All these elements together mean that in public schools the quality of education is lower and less valued and that school feeding plays a stronger role in improving attendance, particularly in rural areas. 67. In most schools visited, principals, committee members and children mentioned that WASH activities improved children’s health, with children falling sick less often, increasing attendance. A more in-depth investigation would be required to validate these statements. However, the link between WASH and better nutritional outcomes has been established. 37 68. The evaluation found that the Literacy Component made a significant difference in schools where it was implemented, based on an impact study of project results and testimonies of principals and teachers, who expressed considerable satisfaction with the overall support from CRS, particularly the training and materials received. 69. Records show that after a year of implementation, a standard test (Early Grade Reading Assessment - EGRA) was given to first-year students. A second assessment was carried out six months later with second-year students, most of whom were the same as in the first test. The first EGRA test showed that only 6.2 percent of first grade students could read at least 30 words of Creole per minute. The second EGRA test showed a 12 percent improvement in Creole, a significant increase over the baseline, with girls obtaining substantially better results than boys. For Creole, girls improved by 23 percent vs 15.3 for boys. For French, the improvement was 14.15 percent vs 9.48 percent respectively. The CRS evaluation also shows that results are twice as good in participating schools compared to proxy schools.38 70. However, the CRS evaluation found that the improvement was much lower than expected (12 percent versus the target of 60 percent). The low performance of students discouraged teachers and principals. As a remedy, CRS has provided teachers and directors additional support through training on methodology, class management, etc., personalized follow-up by supervisors or group sessions allowing them to share experiences. The evaluation identified several factors negatively affecting students’ achievements, namely, improperly trained teachers and, in some cases, replaced by unqualified assistants; lack of support from school 37 See for instance: WHO, United Nations Children’s Fund, United States Agency for International Development (2015). Improving nutrition outcomes with better water, sanitation and hygiene: Practical solutions for policy and programmes. Retrieved from https://www.who.int/water_sanitation_health/publications/washandnutrition/en/ 38 Reading for instance an average of 15.4 familiar words in French Creole, versus 6.8 in pilot schools Final Report 20 | P a g e principals.39 The lack of support at home is also a key negative factor. Classroom capacity, estimated at 40 students per grade, is far higher in public schools due in part to the lack of teachers. In some schools, principals are required to teach classroom lessons to help address the problem of overcapacity. CRS documents also indicate that studies in other jurisdictions show that it can take up to five years to obtain significant changes. 2.3. EQ 3: Does the community involvement in schools (general, parents especially via PTAs, SFCs, directors, etc.) contribute to outcomes and sustainability? Finding: The involvement of parents, principals and committees have contributed to the achievements of outputs and outcomes but, in many cases, the contribution of parents is insufficient to meet the canteen needs on a daily basis or for school feeding to become sustainable over time. 2.3.1. Principals, Committees and Parents’ Contribution to Outcomes 71. School principals, committees (school, supervision and canteen management) and parent participation are a key feature of school feeding programmes in Haiti. Indeed, the PSNAS states that community engagement is a crucial component of school feeding.40 Furthermore, the most robust and sustainable school feeding programmes from around the world incorporate some form of parental or community contribution, be they cash payments or in￾kind (e.g. through donated food, fuel or labour).41 72. The evaluation found that principals, teachers, students, parents, canteen management and oversight committees all contribute in varying degrees to the functioning of the school canteens. The data also shows that schools have a canteen management and a supervision committee in place. Many of the school principals the evaluation team interviewed were clearly supportive and pro-active in their involvement in the canteens. The programme monitoring data, interviews and FGDs show that, in most cases, parents make enough cash or in-kind contributions to the canteens to allow them to function at a minimum level on a regular basis. This means that children are at least getting a meal of bulgur and peas on a regular basis. 73. However, stakeholder interviews at the national, regional and school level highlighted that parent and community engagement remains a major challenge. All the principals interviewed indicated that they try to raise awareness among parents but these efforts have met with varying degrees of success. By and large, school principals, teachers and a few dedicated community members, most often women, e.g. volunteer cooks and canteen supervisors, have ensured that canteens operate on a daily basis. 39 According to CRS, women directors seems also to have more interest in the methodology than men but they represent only a fraction of the school directors (9 out of 42). 40 GoH, Politique et stratégie nationales d’alimentation scolaire, 2016, p. 38. 41 WFP, Global School Feeding Sourcebook, p. 51. Final Report 21 | P a g e 74. Parents clearly recognize the importance of school feeding activities for their children. In focus groups, parents acknowledged that the hot meal provided through the McGovern-Dole programme, as criticized as it might be for nutritional, taste and variation limitations, is the most important meal of the day for many children. However, this understanding and appreciation is not translating into active engagement of parents, and communities as a whole. 75. As a result, most schools struggle to complement the daily fare of bulgur and pulses. The ET often heard that for many of them, the complement of fish or vegetables occurs once in a while. Others, more fortunate or successful, are able to complement meals more often. Based on the school visits, it appears that schools run by religious congregations have less difficulty obtaining contributions from parents. Several of the religious schools visited said that they often offer vegetables and/or fish. However, they also noted having difficulty getting the full amount requested. For example, one principal indicated that about one third of parents do not pay the full amount and that overall the total contributions amount to less than two-thirds of the total requested. 76. A few reasons were given to explain why parents fail to contribute to the school canteen. Some stakeholders said that a presidential decree circulated by the Ministry of Education in 2015 regarding public school education being free and insisting that parents do not have to give anything to the school has not helped efforts to build stronger financial parental engagement for the canteen. 77. Another reason given is a lack of financial means. All schools reported that a significant number of households struggle enormously financially, particularly with drought causing extreme hardship in several regions. It was explained that this makes it very difficult for some rural households in the North and West to meet their daily needs. On several occasions during the school visits it was stated that there is a willingness to pay but not the means. At the same time, most principals admitted that parents could be contributing more. To remedy the situation, WFP sent notices to schools and asked implementing partners to increase awareness raising activities with parents on the importance of contributions. Reportedly, the awareness strategy developed by BND in the fall of 2018 helped increase contributions in the schools that they are responsible for. 78. Other possible factors mentioned by committees and principals that may have an impact on canteen operations or undermine community/parent confidence are irregularities present in several schools: appointed teachers who do not come to class; teachers who work but were never officially appointed; and, children living in domesticity. It is also possible that parents do not entirely trust the school principals. For example, when asked why religious schools were more successful at convincing parents, some advised that parents have more trust in their management. It is difficult to determine to which degree such conditions may contribute to lack of parental engagement without further investigation. 79. In addition, while few, the ET did find examples of schools in poor areas that managed to complement the daily fare of bulgur and pulses on a regular basis. The ET visited two such Final Report 22 | P a g e schools in Artibonite where families have developed a sense of pride in contributing additional food and firewood. While these examples were the exception, it suggests nonetheless that poverty is not an insurmountable barrier for households to contribute to canteens. See par. 112-116 for additional findings on this issue. 2.3.2. Male and female participation Finding: While both men and women contribute to the functioning of school canteens and participate in decision-making through committees, women contribute disproportionally more, particularly cooks. 80. The evaluation found that aside from school principals who are mostly men, women/mothers and girls are more involved than men/fathers and boys in supporting the daily activities of the school canteen as supervisors, cooks and serving food. However, both men and women, as well as boys and girls, participate in committees. This being said, as discussed earlier, the programme has allowed women to participate in the management and operation of the school canteen and, on a smaller scale, in supplying vegetables and spices against payment. 81. The involvement of the community and especially parents through different committees (parents, supervision, management) has provided a space where women along with men have participated in decision-making on various school issues, including the school canteen. FGD and interviews revealed that the number of women in these committees has gradually increased and provided opportunities for women, on a small scale, to exercise their leadership. Based on the baseline and WFP monitoring data, the number of women in decision-making roles in committees has progressively increased in the three years of programming, from 40.8 percent (baseline); to 41 percent in 2016-2017; to 45.45 percent in 2017-2018. However, the evaluation also found that positions such as the chairmanship of the committees, are occupied mainly by men, except in cases where the school director is a woman. The baseline study found that about 80 percent of chairpersons were male, compared to 20 percent female. This data is similar to what is confirmed by this evaluation through field visits and what was reported in the gender study of WFP school feeding programming in Haiti (See Annex 10, section d. for a summary of the gender study findings). 82. Awareness of gender issues was expressed in FGD with some stereotypes being questioned by women, men, girls and boys. For example, on several occasions, teachers and students, regardless of gender, stated that leadership can no longer be left to men and issues such as hygiene and cleanliness (environment, body, food) are not only women’s responsibility. However, gender roles are well entrenched. Other than the participation of women in the supervisory and management committees and the training offered on canteen management and nutrition, the evaluation did not find other tangible measures in the design of the programme to promote greater equality. 83. Women involved in the canteen described their role as overseeing the content and flavour of the meals, care in the preparation, as well as respect of hygiene standards and programme rules. They advise and even give punctual support to the cooks if needed. Girls (or female Final Report 23 | P a g e teachers) are almost exclusively the ones who serve the food to students. As for cooks who play a central role in the canteen, the burden is heavy: their tasks are arduous, tiring, risky and undervalued. Typically, they start food preparation very early and work an average of almost six hours a day. In the majority of cases, they use wood or coal in confined spaces that give off intense heat and they must handle and lift large pots of hot food. 84. Several people, including some cooks, pointed out that previously, cooks were employees of the State. Many committee members, principals and cooks also noted that they used to receive a dry ration, which helped compensate them but that WFP no longer allows this practice. The cooks do not question the voluntary aspect of their contribution but all those interviewed felt that a small gesture to recognize their work would make a sizable difference in their situation. 85. At the same time of this evaluation, a gender analysis of the canteen programme was commissioned by WFP. It arrived to similar findings as this evaluation regarding gender roles and the unacceptable conditions that the canteen cooks endure to provide an essential component of the canteen programme. This also runs contrary to WFP's Strategic Plan (2017- 2021), and more specifically WFP's Policy on Gender Equality (2015-2020), which recognizes the importance of gender equality and women's empowerment approach in food aid programmes and policies and states that WFP is committed to "working for transformative gender outcomes with its beneficiaries". It would appear that WFP fulfills its commitment to equitable access to school for girls and boys but working for transformative gender outcomes does not extend to adults to the same extent. 2.3.3. Sustainability Finding: The current level of involvement of parents and community is insufficient to ensure long term sustainability of school feeding at community level. The lack of a strategic approach to social mobilisation may be the explanation. 86. As discussed earlier, the contribution of parents and the community to school feeding activities is not sufficient to ensure the sustainability of the programme in the long term. Overall, the analysis of the data indicates that the relative success to mobilize funds and operate the canteen depends largely on the leadership skills of school principals. While committees clearly cared and wanted the school canteen to continue, they were at a loss when asked how they could mobilise more funds for the school canteen. For instance, in some FGDs, when asked what they could do to further mobilize parents or the community, the conversation often led to committee members asking why WFP could not give more varied food and revert back to giving milk, cheese and fish, as it did in the past. 87. The data provides strong evidence from development partners in Haiti of the benefits of investing in social mobilization to achieve programme objectives. As noted earlier, BND, one of the implementing partners, came to this conclusion and developed a communication strategy and trained its monitoring officers so that they could be more effective in convincing parents to be more active in the canteen. However, in general, messaging from WFP, PNCS Final Report 24 | P a g e and monitoring officers has been limited to practical issues such as procurement, warehousing, handling and preparation of food and monitoring rations. 88. The evaluation team found that other programmes in Haiti such as the American Red Cross have made social mobilization a priority to support the achievement of outputs and outcomes and invested heavily in it. This greatly enhanced the possibility of programming success. 42 The ET found several examples of NGOs in Haiti using community involvement strategies to improve project outcomes, including WV, Plan, CARE, and CECI. For example, CARE works with stakeholders to operationalize inactive Executive Committees in schools, following the Partnership Guide developed by the Direction d'appui à l'enseignement privé et du partenariat (DAEP)43 . In the first year, the committees are heavily supported and as they become more autonomous and democratic, support is reduced until desired changes are institutionalized. 89. Increasing accountability to beneficiaries also generates trust from parents. One feature shared by McGovern-Dole and other programmes is the use of Accountability to Beneficiaries (AtB) in the form of a telephone number that could be called to express concern about school feeding activities. In the case of McGovern-Dole, it appears that the hotline is not always functional. On a few occasions, committee members told the ET that calls to the WFP hotline would go unanswered. In response, WFP staff told the ET that it returns missed calls as soon as possible and that the hotline had been recently complemented with a whatsapp account to receive messages. The programme’s monitoring officers (MO) also provide an external accountability mechanism but, as discussed earlier, the ET did not find that they interact much with committees or the community. 2.3.4. Potential to create income generation opportunities Finding: Overall, the contribution of the McGovern-Dole programme to income generation and the local economy has been minor. However, WFP programming to support local purchase and other ‘homegrown’ models provide substantial benefits to local communities and producers, including women. 90. Based on interviews with various stakeholders, including WFP, operating partners and producer organizations in Gonaives and Nippes, it is clear that there is scope to generate more income generation for farmers, including women. Several stakeholders mentioned that, if there was a portion of the McGovern-Dole programme budget that allowed for more local purchases, it would make a more substantial contribution to income generation at the local level. 91. For example, the visit to the Nippes local purchase initiative brought into focus the economic potential of local purchases. Interviews with various stakeholders, including schools, show 42 Baastel (2018) Kanaan Final Evaluation Report. Le Groupe-conseil Baastel ltée for the American Red Cross, Dec 19, 2018. 43 Private Education and Partnership Support Branch. Final Report 25 | P a g e that the informal sector benefits from purchases of vegetables and spices in local markets and the volunteer cooks receive a certain amount of compensation through growing and selling produce to schools (with a lot of variation between the schools visited). The evaluation team (ET) also heard that in some McGovern-Dole schools, cooks prepare ‘treats’ that they sell to children but also heard that the programme prohibits this. 92. At the national level, WFP purchases salt on the national market as part of the programme with McGovern-Dole funding. One implementing partner, AMURT, works with local producers through associations to improve salt production. The project ensures that women benefit both in terms of the technologies introduced and in terms of profits generated. Interviewees thought it not inconceivable that these producers could supply salt to local schools one day. 2.4. EQ 4: Are the key institutions and governance structures able to effectively deliver, implement, sustain and/or scale up school meal interventions in the long￾term? Finding: While some important elements policy framework for school feeding are in place or underway at the national level, the Haitian Government lacks the institutional or financial capacity to manage the programme independently, even partially, in the short term. Stakeholders agree that building capacity of decentralized government structures, as well as promoting local production can bolster capacity until crucial governance issues are resolved at the national level. 2.4.1. Policy Framework and Governance Structure 93. The data from the document review and interviews with a range of stakeholders at the national, regional and local levels show that while there are important elements in place or being developed, such as the development of the school feeding policy framework and legislation, other critical elements outlined in the PSNAS have not advanced. Crucially, the MENFP and PNCS have not taken on a leadership role as expected. The following paragraphs provide an overview of the status of key elements necessary to move forward. They were identified in a workshop held in 2015 to assess national capacities and elaborate an action plan for the government to manage the school feeding programme independently. It used a methodology called Systems Approach for Better Education Results (SABER). 94. The SABER methodology states that a functioning national programme requires a series of key elements such as a strong political and legal framework, robust institutions, the capacity to coordinate actions with all stakeholders, stable funding, sound planning, and tangible community ownership. The findings of SABER are summarized in Figure 4, followed the evaluation findings on progress made to date. Figure 4: Assessment of Haiti national capacity, following SABER dimensions in 2015 Final Report 26 | P a g e Dimension / Status Assessment of national capacities Policy Framework / Emerging Need to finalize, validate and disseminate the content of the national policy, to elaborate a school feeding law and to ratify it. Financial Capacity and Coordination / Emerging Need to strengthen, stabilize and diversify the funding of school canteens, and a need to strengthen the governance of funds transferred for school canteens. Institutional Capacity / Latent Need to strengthen the capacity of the PNCS; a need for synergy with sectoral departments and national agencies/structures for food security and health / nutrition; a need for coordination with structures involved in school health, hygiene and sanitation and NGO partners; Weak collaboration between the Territorial Communities and the deconcentrated services; and a need to energize consultation frameworks. Design and Implementation / Latent Need to define, at the national and local levels, procedures for targeting schools; a need to have standard qualities of rations, taking into account the nutritional aspects and eating habits; a weakness in the staffing, procurement and food management process; low participation of small producers; weakness in the M&E system, lack of information and data, and weakness in reporting. Roles of Communities / Emerging Need for training and building capacities of local committees as well as need for greater community involvement in decision￾making and implementation of the program Source: Rapport de l’atelier SABER sur les capacités nationales et d’élaboration d’un plan d’action sur l’alimentation scolaires en Haïti, 25 et 26 Août 2015. 95. The development and adoption of a national policy of school feeding (PNSAS) in 2016, with the support of donors and other international organizations such as WFP, was an important milestone in providing a solid governance structure for school feeding programming in Haiti. As discussed under question 1, the PSNAS is oriented around three strategic objectives: 1) the provision of a daily snack and hot meal at school; 2) promoting the local economy and local food production by requiring local foods be provided; and 3) the development of national capacities necessary for the good management of school feeding actions. 96. Following the adoption of the PSNAS, a draft version of the National School Feeding Operational Manual (Manuel d’Opération du Programme National de Cantine Scolaire) was submitted to the Minister of Education in October 2017 for revision and to advise on the next steps of the validation process. 44 (). However, at the time of this evaluation, the document had not yet been approved by national authorities. Importantly, there are still disagreements within the GoH as to the role of the PNCS. Many stakeholders maintain that to move forward the PNCS requires institutional strengthening and needs to better play its role of regulator and standard setting, and coordination of partners. 44 McGovern-Dole semi-annual narrative report. Final Report 27 | P a g e 97. In 2018, legislation was developed to legally frame school feeding in Haiti. With the support of the FAO, parliamentarians held consultations in various regions of the country, in which WFP participated. At the time of the evaluation, the Parliament of Haiti was still studying the draft legislation. 98. While capacity building efforts at national level have been delayed, interviews and WFP reports revealed that WFP continued building capacity of the DDE and PNCS at regional level. For example, WFP McGovern-Dole semi-annual reports indicate that the DDE and PNCS authorities continued to participate in joint field visits to monitor the implementation of the McGovern-Dole school feeding programme through their decentralized structures located in different departments. Through these joint activities, these regional authorities are able to better understand school feeding issues at the local level and help identify and address problems. Interviews with these institutions and WFP confirmed that these joint monitoring visits have been mutually beneficial. 99. As for coordination, stakeholders agree that the PNCS has not played the role it was assigned in the PSNAS. While roles and responsibilities of all the key stakeholders in school feeding activities are outlined in the PSNAS for all levels, e.g. ministries (MARNDR, MENFP, MSPP, MAST), donors, UN agencies, NGOs and Territorial Communities, the PNCS has not convened the sectoral table on school feeding since the PSNAS was approved in 2016. Furthermore, according to the PSNAS, school feeding is centered on communities. As such, they should be supported by Municipal Education Commissions but this structure did not exist at the time of the evaluation. 100. In terms of financing a national school feeding programme, the PSNAS outlines a gradual transition from dependence on international donors, from 100 percent in 2016 to 20 percent in 2030. According to the PSNAS, during Phase 1 (2016 to 2020), a multi-donor account is to be created. Concurrently, the PNCS is expected to strengthen its administrative and financial capacities so that by 2020, it would be able to manage international financing. During Phase 2 (2020-2025), the system is to be further strengthened and by 2025, 50 percent of the funding for school feeding would come from national coffers. At the time of the evaluation (Phase 1), no visible progress had been made. 101. So while some key policy frameworks are being put in place, much remains to be done to allow for even the partial take over of SF programming by the GoH. Most stakeholders interviewed agreed that international partners should continue working on the three objectives of the PSNAS where possible. Key informants indicated that capacity building activities and involving the DDE and PNCS at regional level and joint monitoring visits were useful for these entities to get first hand knowledge of the benefit of the programme and be able to take action when WFP reports problems that they can help solve. Final Report 28 | P a g e 2.5. EQ 5 Differences and advantages/disadvantages between the McGovern-Dole model and other approaches, and how could it be adapted to increase effectiveness, sustainability, efficiency and long-term effects? Finding: The key advantage of the McGovern-Dole model is that the cost of the basic meal provided to schools is much lower than models relying on local purchases. The key disadvantage is that it contributes almost nothing to the local economy. Other models, such as the local purchase model in Nippes, that include local purchases demonstrate that more dynamic and collaborative actions can be nurtured through school feeding with higher nutritional and economic benefits achieved for local communities. 2.5.1. Advantages and Disadvantages of McGovern-Dole compared to other models 102. The McGovern-Dole model provides a good point of comparison to understand the strengths and weaknesses of different school feeding programmes in Haiti. The advantages of the McGovern-Dole approach are seen to be its dependability guided by effective administrative oversight. Only a small number of schools reported gaps in supply of any length of time and none were considered by the evaluation team to be particularly worrisome. At the level of the schools, canteens that were visited operated more or less effectively. The McGovern-Dole schools consistently provided a solid nutritional base for school children and the importance of this was demonstrated earlier in this report. 103. Unfortunately, the McGovern-Dole approach never really moved beyond creating the most basic sustenance. The appreciation for McGovern-Dole is tempered by the disappointment regarding the lack of variety and quantity of food. Apart from a few schools, households and communities were unable to consistently complement the basic food provided. This meant McGovern-Dole was never able to reach the nutritional standards that programmes such as the World Bank Providing an Education of Quality in Haiti (PEQH) initiative provides a combined snack and hot meal. It was designed to ensure children have a minimum of 1,200 calories a day with vitamin enriched food, following the PSNAS guidelines. The PEQH model is also tied to promoting academic improvements through teacher training and the provision of schoolbooks, deworming activities, local purchases and community involvement through small grants for school improvement projects. There are clear nutritional and taste satisfaction advantages with the WB model. It is also more closely aligned with the objectives and strategy outlined in the PNSAS. 104. The McGovern-Dole meal is less costly compared to other SF programmes. The WB model and others, such as CARE’s ’s catered meal approach, that provide a snack and a more complete meal and also promotes the local economy, have higher cost implications. While the WB programme operates in the difficult Haitian context, it focuses largely on private schools while the McGovern-Dole programme is implemented in the more challenging context of Haitian public schools. The question is what is the cost-benefit of these programmes that achieve better nutritional targets and food quality but do so at a higher cost but support a smaller number of schools. This is an issue that WFP Haiti CO has also been Final Report 29 | P a g e grappling with. Interviews with MENFP representatives made it clear that the GoH does not want the number of assisted schools reduced. 105. Nevertheless, the local purchases or procurement models employed by WFP in the department of Nippes with Canadian and previously Brazilian funding and piloted in Artibonite with funding from France, have several advantages, notably in securing local employment for both men and women, and of course raising nutritional standards through a more balanced supply of food for children attending school. 106. With BND as its local partner, the Nippes local purchases model creates linkages between local food producers and local schools. WFP and BND have established a well-organized and managed supply chain. Fresh and dry food products purchased in local markets are brought to a centre for inspection and sorting before being packaged and delivered to schools. Each school receives approximately six products. Based on interviews and documents reviewed, there were 23 schools involved during the 2015-16 school year. Currently, the programme serves 142 schools, with 28,000 beneficiaries. A tracking system was established to ensure product quality and to promote accountability from producers. The evaluation team was told that some of the parents are also producers, which according to stakeholders, increased the engagement of the community and parents. 107. While school children were not interviewed extensively in Nippes, it appears there is a higher level of satisfaction of children eating meals that are enriched through the local purchase system compared with the McGovern-Dole model. From a sustainability standpoint, the local purchases model has many positive aspects. It enables local producers, WFP and BND staff and schools to demonstrate how a comprehensive school feeding programme can be integrated into the local agricultural economy. However, as with the WB and McGovern￾Dole models, the ET did not see signs that the ‘homegrown’ approach could transition in the foreseeable future into a locally controlled and funded enterprise, largely independent from external donors, as is currently the case. 108. The evaluation team reviewed another variation on the local purchase model: CARE’s School Food Vendors Project (SFVP) catering project. The project started in the 2015-16 school year as a pilot initiative with financial support from USAID. Today, six communes and 21 schools are involved in Artibonite and the Southeast. It involves 137 caterers, of which only four are men. These are mostly women who also sell meals in the streets. For USD 0.75 (USD 0.20 for the snack and USD 0.55 for the hot meal)45 , children have access to a diversified and balanced diet and a different menu every day. 109. One key advantage of the model is that eliminates transportation costs, which are assumed by the caterers providing the meals. Another advantage, is that school principals and teachers are liberated from any managerial responsibility of operating a canteen. The main duty 45 These prices are based on currency exchanges of July 2019. There amount was closer to 1 dollar US before the devaluation of the gourdes in recent months. CARE noted that the devaluation of the gourde has presented some challenges for the project. Final Report 30 | P a g e schools have is to ensure the distribution of the snack and hot meal. This model implemented by CARE promotes women's businesses, who also employ other women and support local production through local purchases. As in the Nippes model, there is a strong multiplier effect. A risk, or disadvantage, of this model is that the purchases are made on a small scale by individuals. It is more affected by rapid price hikes, as seen in 2019. The McGovern-Dole , or even Nippes model, can be more economical through their ability to purchase larger quantities and store dry food for longer periods of time. As with all the school feeding models reviewed, it relies heavily on outside funding. 110. There is certainly an opportunity for more local purchases through the McGovern-Dole funded programme. For instance, the evaluation found it implements the Local and Regional Food Aid Procurement (LRP) programme in at least 6 countries,46 providing complementary mechanisms for the procurement of eligible commodities for food assistance programming.47 In those countries, its objectives include: ‘to strengthen the ability of local and regional farmers, community farmer groups, farmer cooperatives and associations, processors, and agribusinesses to provide high-quality commodities (and for organizations and governments to procure such commodities) in support of school feeding programmes’. This is directly congruent with the objectives and guidelines of the PNSAS. 2.5.2. Improving community involvement and sustainability Finding: Other countries, particularly in the Caribbean region, which face similar conditions as Haiti have developed initiatives that integrate innovative climate adaptation measures to enhance community involvement, as well as create local economic opportunities. 111. One of the more perceptive comments made about the McGovern-Dole programme came from a school director whose school had been suspended from the programme for having fed 3rd cycle children. According to the director, if there was more food and variety overall, feeding older children wouldn’t be an issue. His point was there is room in the overall context of school meals and school canteens to evolve into something more vibrant, especially in terms of nurturing sustainability, greater food variety and quantity and stronger community involvement. 112. As seen under Question 3, the evaluation identified several organizations successfully and proactively involving individuals and communities in actions intended for their benefit, which can serve as models for WFP. Naturally, strategies for community engagement and sustainability include income generation and financial mechanisms to support economic activities. 113. School meals programming once had strong political commitment in Haiti and stronger government financial support in general. This is currently not the case. With the collapse of 46 Benin, Kenya, Laos, Mozambique, Rwanda, Tanzania. 47 See McGovern-Dole, LRP Factsheet Final Report 31 | P a g e national funding, little attention has been paid to understanding options at the local level for establishing local financial mechanisms, including income generation for vulnerable households. As the programme is implemented largely in rural areas, the evaluation team identified opportunities with national partners such as with the MARNDR’s Innovation Unit and the Inter-American Development Bank (IADB) to develop partnerships to promote innovative ideas in the area of food production, the main source of income for rural households. 114. However, in examining strategies that could improve the functioning of the McGovern-Dole school canteens and other WFP efforts in Haiti, stakeholders and studies have pointed out that it is important to keep in mind the significant development challenges Haiti faces and will continue to face in the future, particularly those related to climate change. The problems Haitian schoolchildren and adolescents will eventually face are unlike what their parents and grandparents had to face. Any suggested strategies or innovations should be conscious of this and have to be introduced in a controlled manner and tested prior to rolling them out on a larger scale. 115. For example, traditionally, with two main harvests a year, there would be a strong possibility for rural households to contribute to the school feeding through in-kind food donations. However, this option is diminishing, and as discussed, many are increasingly relying on schools to feed their children.48 116. While some agricultural experts interviewed felt that this is a temporary situation and eventually people would return to farming, the contributory reasons are many and complex, including the country’s agricultural policies. However, many people interviewed for this evaluation and the review of literature indicate that the key contributing factor is climate change. The North, for example, has suffered three consecutive years of drought since 2017. Rainfed agriculture for small scale producers as a way to make a living and feed a family has become increasingly unreliable and all studies indicate that climate change will continue to plague agriculture production in Haiti. 117. This situation is common to countries in the Caribbean. For example, an evaluation of the Rural Enterprise Development Initiative (REDI) in Jamaica clearly showed the extent of challenges facing small scale farmers from a climatic standpoint.49 REDI introduced a range of climate adaptation measures to assist farmers to adjust to this new reality. REDI promoted, among other practices, the use of Climate Smart Agriculture, protected agriculture, and the use of renewable energy. These practices were introduced at a time when, similar to Haiti, droughts, extreme heat and flooding were occurring across Jamaica. These practices were put to the test, with protected agriculture being the only practice that clearly withstood the 48 Ironically, more food is now available in many urban areas than is found in the countryside.. See for example, World Food Programme and Conseil National pour la Sécurité Alimentaire (2016) Haïti Urban Food Security Assessment, October 2016. 49 GGI (2017) Independent End-of-Project Evaluation for the Rural Economic Development Initiative (REDI). Submitted by Goss Gilroy Incorporated for the World Bank and the Jamaican Social Investment Fund. Final Report 32 | P a g e challenge. Based on the REDI evaluation, which GGI conducted, and the various studies of the Haitian context produced by the FAO, and others, it is clear that the situation in Haiti is worse than other Caribbean countries, especially from the standpoint of the local capacity to respond.50 118. The REDI project in Jamaica promoted protected agriculture mostly in the form of greenhouse operations. Over the course of the project, about 100 greenhouses were built to be operated by small-scale producers to produce a variety of food types in secure circumstances. REDI success resulted in other projects being implemented by the Jamaican Social Investment Fund, deciding to promote greenhouses. This includes an education project that has invested in building 27 greenhouses for primary schools. The objective of the school greenhouses is part of the “Eat What You Grow, Grow What You Eat initiative.” It is important to note that one of the key developments of the REDI project was the interest of young people in learning how to operate green houses and stimulating their interest in agriculture. 119. WFP has been increasingly experimenting with greenhouses as a means to produce food for school meals. WFP programmes identified supporting protected agriculture include Algeria51 Ecuador, Kenya52 , Laos, Lebanon53, and Mauritania.54 This should be an important source of information for the WFP CO. Additionally, WFP partners in the Caribbean region have also taken action in this area. In the Barbados, IADB is supporting the Agricultural Development and Management Cooperation to work with the school system to introduce shade/greenhouses to install hydroponic systems in schools to produce vegetables and fish. The project is designed to enable participating students to obtain Caribbean Vocational Qualifications (CVQs) certification in crop production. 120. Experts point out that school gardens of yesterday have to be brought into a new context that recognizes climate change. Protected agriculture in the form of greenhouses is part of this new scenario.55 However, demonstrating robust rainfed agricultural systems such as agroforestry that can be carried out in a context of climate induced duress should also be considered. There are other modern techniques coming into focus that explore and exploit space differently. Some examples include planters on wheels and rooftop gardens. These technologies offer promising teaching opportunities and are viable options for growing food in different contexts that can contribute to nutritious meals and developing the local economy through entrepreneurship. WFP can also find inspiration within its own organization through the Innovation Accelerator, which has shown, with the H2Grow initiative, that food can be grown hydroponically by families in the desert slums of Lima, among other unlikely places.56 50 FAO and WFP. 2017. Special Report: FAO/WFP Crop AND Food Security Assessment Mission to Haiti. Available at: https://reliefweb.int/sites/reliefweb.int/files/resources/a-i8279e.pdf 51 https://insight.wfp.org/growing-food-in-the-algerian-desert-28dc89219a9a 52 https://insight.wfp.org/growing-food-anywhere-b88507cdca02 53 https://insight.wfp.org/planting-hope-in-lebanons-bekaa-valley-649e3921b320 54 https://insight.wfp.org/plants-vs-sand-f304fd219e89 55 GGI (2017) Independent End-of-Project Evaluation for the Rural Economic Development Initiative (REDI). 56 WFP: https://innovation.wfp.org/project/h2grow-hydroponics/how-grow-food-desert-slums-lima. Final Report 33 | P a g e 121. These approaches are at the technological forefront and are openings for considerable funding opportunities. WFP as an organization is familiar with the funding possibilities, having for example secured financing through the Green Climate Fund in Tajikistan.57 The question is determining how school feeding activities can be put in a context where such funding becomes accessible. Since 2017, WFP has a climate change policy. This policy defines how WFP is expected to contribute to preventing climate change and climate-related shocks from worsening existing vulnerabilities and risks and undermining progress towards ending hunger and malnutrition. The policy provides WFP staff with guiding principles and programmatic options for integrating activities to address climate change into their work, with a focus on supporting adaptation and reducing loss and damage from climate extremes.58 Despite the challenges Haiti is facing as a country, there is little evidence that this policy is being actively embraced by the WFP Haiti CO. 122. Too often in the past, the social and community dynamics surrounding growing food at schools has not been well thought out. WFP is now aware that asking children and teachers to grow food on a production scale can be construed as being manipulative and an unthoughtful demand on the time of parents especially mothers. Furthermore, growing food in school yards cannot be grown in sufficient quantity and sustainably. There are also concerns of finding a place for food growing within the overall educational aspirations of schools, not to mention the logistical challenges that food growing creates, particularly with children who are less than 12 years of age. 59 123. The agricultural experts interviewed noted that growing activity in Haiti can be boosted by a greater mastering of the science and practices that can enable agriculture in this current context of climatic distress. As shown elsewhere, the evaluation found that there is an opportunity for children and parents to learn collectively and develop applicable skills, and build momentum towards stimulating greater interest and support for school feeding activity. The Barbadian example of associating vocational certification opens up all sorts of opportunities for connecting school feeding with educational opportunity. There are motivated institutional partners such as IADB, MARNDR, and in particular, the Ministry’s Innovation Unit and MENFP need to help facilitate interest and experimentation. Finding: Financial and credit mechanisms to support income generation and households expenditures, such as Village Savings of Loans, Savings and Internal Lending Community and schemes as ‘Mon Cash’ and ‘Sogesol’ and other more formal financial vehicles, are increasingly accessible in Haiti. WFP has also used mechanism such as cash transfers and food for work to build community assets. 57 https://www1.wfp.org/news/green-climate-fund-approves-wfp-led-project-climate-change-adaptation-tajikistan 58 WFP (2017) World Food Programme Climate Change Policy. World Food Programme (March 2017) https://docs.wfp.org/api/documents/WFP-0000023971/download/?_ga=2.40692770.1462598447.1560876601- 1124090005.1559294908 59 WFP (2017) Smart School Meals Nutrition-Sensitive National Programmes in Latin America and the Caribbean: A Review of 16 Countries. World Food Programme July 2017. Final Report 34 | P a g e 124. As mentioned earlier, there are financial mechanisms for income generation that development organizations have been implementing with success in Haiti and in other countries. Haitians, including the very poor, have a good track record of participating in financial arrangements to improve their situation or their local communities. Since the earthquake, micro credit and mechanisms such as Village Savings of Loans (VSLA) have a foothold in the country. In 2013, USAID Haiti commissioned an external qualitative assessment of its four-year Savings and Internal Lending Community (SILC) programme. USAID wanted to know whether SILC met members’ needs in terms of positively impacting individual, child and household well-being.60 Respondents in the USAID evaluation reported that SILC group activities met member needs and out of the many economic and social reasons for using SILC loans, members cited paying the school fees of children. A key factor ensuring success and sustainability seemed to be groups’ sense of ownership of their money. This assessment along with the general openness of Haitian to alternative forms of small￾scale financing, indicates that there may be alternatives models for having parents contribute financially to school meals.61 125. It is also important to point out that schemes such as ‘Mon Cash’ and ‘Sogesol’ and other more formal financial vehicles are increasingly accessible in Haiti by phone and other means, and can be used to structure financial arrangements.62 At a national level, El Salvador created a fully financed national school feeding programme through a Trust Fund, replacing the reliance on WFP. Perhaps exploring different local level financial approaches to funding may eventually lead to a similarly inventive strategy that is potentially applicable on a national level. Partnering with the private sector to increase operational capacities to deliver school meals in Haiti should be seriously considered and not simply from the standpoint of producing food. For example, the American Red Cross, successfully partnered with Mon Cash and a Sogexpress branch, a division of SogeBank as an integral part of an overall risk reduction and physical renewal initiative in the settlement of Canaan, known as the Canaan Integrated Programme.63 126. Lastly, WFP has a long history of using WFP instruments in the form of food-for-work and cash/voucher-for-work to contribute to broader development objectives. In the Andean and Amazonian region, WFP is making this type of contribution as part of broader efforts to support climate change adaptation.64 60 Lisa Parker, Kednel Francois, Olbeg Desinor, Toni Cela & Karen G. Fleischman Foreit (2017) A qualitative analysis of savings and internal lending communities in Haiti – do they make a difference? Vulnerable Children and Youth Studies, 12:1, 81-89, DOI: 10.1080/17450128.2016.1263773. 61 Lisa Parker, Kednel Francois, Olbeg Desinor, Toni Cela & Karen G. Fleischman Foreit (2017) A qualitative analysis of savings and internal lending communities in Haiti – do they make a difference? Vulnerable Children and Youth Studies, 12:1, 81-89, DOI: 10.1080/17450128.2016.1263773. 62 https://www.sogebank.com/accueil/sogesol/ 63 Baastel (2018) 64 https://www.wfp.org/node/3446/4278/639350 Final Report 35 | P a g e 127. Beyond climate change, Haiti presents other environmental challenges including of energy suitable for use in school canteens. The evaluation team acknowledges the efforts to promote improved woodstoves and the use of cleaner energy in the form of propane. However, it was not difficult to find propane stoves going unused because propane was not available in rural areas or to visit schools in urban areas such as Gonaives where open fires in closed spaces were used to prepare food. The occupational and health safety risks associated with open fire cooking in closed spaces are well documented. 65 128. In 2019, such circumstances are not acceptable in a programme supported by a major donor. According to industry representatives interviewed for this evaluation, improvements are being made in terms of energy infrastructure and the availability of products such as propane cooking stoves. WFP has made efforts with partners such as Canada to eliminate situations where the health of women and children could be jeopardized or where there are undesirable environmental impacts. Canada has recently, announced CAD 20 million to reduce the use of coal in Haiti. 129. Other solutions were discussed during interviews with private sector providers and NGOs. For instance, the two providers interviewed noted receiving funding from a Foundation, World Central Kitchen (WCK), created by a famous American Chef to convert coal fuelled stoves to propane. One mentioned that WCK provides funding for 50 schools each year, and there is an opportunity for WFP to partner with WCK or other similar foundations to help reduce the negative effects of using dry wood and coal in programme schools. BND also mentioned that when possible, it buys improved stoves for some of the schools it monitors. 2.5.3. Opportunities to add or augment a nutrition sensitive approach Finding: Developing partnerships between the education, health and agricultural sectors and related organizations at the community level has the potential to achieve better educational outcomes for vulnerable children. 130. As seen earlier in this report, health related activities such as water purification, handwashing and providing deworming tablets are part of school feeding in McGovern-Dole programme schools. The link between these activities and learning outcomes is well known. Interviews with stakeholders and a review of documents highlighted the fact that there are further health and educational benefits to be gained by developing stronger partnerships between the education and health sector. For example, ‘Optimizing Education Outcomes: High-Return Investments in School Health for Increased Participation and Learning’66 published by the World Bank provides evidence of the impact of investing in health activities in schools. 65 https://www.wfp.org/climate-change/innovations/stoves 66 World Bank Group (2018), Child and Adolescent Health and Development Optimizing Education Outcomes: High-Return Investments in School Health for Increased Participation and Learning, Volume 8. Disease Control Priorities Third Edition : https://www.globalpartnership.org/sites/default/files/2018-04-gpe-optimizing-education-outcomes.pdf. Final Report 36 | P a g e 131. In fact, it urges investing in a package of essential health interventions in schools (including those mentioned above) and provides evidence or impacts of these investments and partnerships around the world. It also recommends close collaboration between ministries of education and ministries of health to make the case to fund school health investments jointly to ministries of finance. It also recommends ‘to fully exploit the experience, commitment, and contributions of the many partners outside of government (for example, civil society organizations, international nongovernmental organizations, and the private sector) and external funders (for example, the United Nations, bilateral and multilateral aid agencies, philanthropists and foundations, and the private sector)’. 132. Discussions at school level with principals, committee members and children indicate that they would welcome gaining more knowledge on nutrition and other health related issues. The PSNAS encourages and expects such partnerships. Other stakeholders, including representatives from MSPP and WFP staff pointed out that schools could be a hub for nutritional and other health related activities in communities, thereby contributing to community involvement in schools. 3. Conclusions and Recommendations 133. Based on the findings presented in the previous section, an overall assessment that responds to the evaluation questions is provided below. This is followed by six recommendations for WFP, McGovern-Dole, and other actors such as the GoH to take action to build on the lessons learned. 3.1. Conclusions 3.1.1. Question 1 - Relevance 134. The evaluation concludes that the McGovern-Dole programme is highly relevant for the children it serves who receive a basic nutritious meal daily. School feeding acts as a major safety net for vulnerable households, given the high levels of food insecurity in the areas where the programme is implemented. Despite its lack of variety and local produce, this daily meal is valued by parents, schools and children alike. For some children, this is the major meal of the day, sometimes the only meal. The school canteen was found to be very important in rural areas, where poverty rates are almost twice those of urban areas, and vital for regions that have been hit by drought and failing crops in recent years. While the programme targets food insecure areas, and the public-school system, where the most economically and socially disadvantaged children go to learn, the targeting system could be more systematic to ensure that the most vulnerable children and schools benefit from the programme. 135. The evaluation also found that the monitoring system WFP has established to ensure proper management of the food supply chain and management of food stocks at school level can undermine programme goals by risking excluding some of the most vulnerable children, due to mismanagement at school level. The programme could provide more support to the Final Report 37 | P a g e most vulnerable schools to prevent their exclusion from the programme when they cannot comply with school feeding management requirements. Another issue highlighted in the evaluation is the exclusion of students of the higher grades 3rd cycle from school feeding which raises moral and ethical issue in the Haitian context, where many girls and boys in primary school are overaged. WFP needs to explore sources of funding to include children of the 3rd cycle in school feeding where they are attending alongside cycle 1 and 2. 136. While the programme is generally aligned with the national policy and strategies for school feeding (PNSAS), the limited use of local purchases and support for local agriculture, while present in WFP school feeding programming funded by other donors, should be seen as a notable gap in the McGovern-Dole funded programme. The evidence shows that local purchases are highly valued by the communities and schools where they are implemented, providing benefits for children and the local economy that merit pursuing in the future. Local purchases not only help make the basic meals more interesting for children but also help to address micronutrient and vitamin deficiencies. While it was demonstrated micronutrient powders were effective in reducing these deficiencies, the consumption of MNP went largely counter to the general objectives of the PNSAS (consuming locally grown food) and local culture. Local purchases also benefit local producers who are members of the communities where the schools are located, which is another value added. 137. Other shortcomings in terms of alignment with the PNSAS, include the failure to address the extensive use of firewood and charcoal to prepare meals, in light of the extensive level of deforestation in Haiti, notwithstanding the health hazards borne mainly by the women who more often than not toil in inadequately aired kitchen. The evaluators consider that given the recurrent extreme and devastating climatic events in Haiti and its rank as the 4th most affected country in the world, there is an opportunity to make future school feeding programming more shock responsive. The absence of emergency mechanism in a context of high vulnerability warrants a review of the programme, ensuring that the strategy that adopted is also gender responsive. 3.1.2. Question 2 - Effectiveness 138. The programme has been effective in outputs and outcomes. The evidence strongly suggests that providing a daily meal and other complementary activities such as WASH bolsters attendance, allowing a greater number of boys and girls to acquire a certain level of education by maintaining them in school longer despite dire economic circumstances. However, the data did not show conclusively that school feeding has a strong effect on students’ academic performance. As discussed, the baseline data did not show a significant difference in pass rates between beneficiaries and non-beneficiary children. This suggests that other factors such as competency of teachers (as noted in the Annex 3, only 17% of teacher are qualified to teach), lack of school materials, over crowding, teacher absenteeism and lack of training play a greater role academic performance. 139. The literacy component piloted in 42 schools has helped improve student performance in reading Creole, even though results were much lower than expected. The short Final Report 38 | P a g e implementation period was given as the explanation for those modest results. However, the evaluation of the pilot showed that beneficiaries had EGRA results twice as high as those of children from control groups. This indicates that, in addition to school meals, which help bolster attendance, there is a need to continue investing in teacher training, and adequate teaching materials. While structural issues still need to be addressed at national level, the programme can provide tangible benefits to the children it serves. 3.1.3. Questions 3 and 4 - Effectiveness, Efficiency, Sustainability 140. The day-to-day operations and management of school feeding and its sustainability depends on the participation and leadership of a continuum of actors at all levels, from the schools and local communities to the national level, e.g. MENFP, PNCS and other ministries and organizations with a stake in education, food security, nutrition, economic development, climate change etc. School principals, parents (women and men) and teachers have played an important role in programme implementation, thanks to the solid norms supporting parent committee roles and responsibilities at local level. However, these measures and the committees’ activities have not generated sufficient cash or in-kind contribution for a fully functioning school feeding programme, let alone ensuring sustainability at the community level. Women’s leadership in committees also needs to be strengthened, 141. Lack of resources from parents, especially in light of rapid price increases recently and in vulnerable areas affected by drought, are factors that have to be considered in the lack of contributions from parents. There is clearly a need for a better community mobilization strategy, as the efforts to date through monitoring officers alone have not yielded expected results. Support for revenue generation at the local level, which other programmes have found useful to achieve better outputs and outcomes. 142. There is also a need to devise strategies to ensure that leadership is exercised more equally at school level, while taking into consideration that a large proportion of households are headed by women, both in rural and urban areas, with all the economic and social barriers that this entails. Approaches such as Village Savings and Loans that allow small groups of people to come together to finance social and economic related ambitions could be valuable in the context of school feeding, particularly to support income generation activities and should be actively explored. 143. Furthermore, based on the findings of the evaluation, there is reason to believe that support of local purchase and other homegrown models, may foster greater community engagement and support for the school canteens. Any strategy developed will need to be gender-transformative in essence and seek to change perceptions and behaviours on appropriate gender roles among women and men, boys and girls. 144. Ultimately, sustainability rests on the ability of the MENFP to mobilize resources to take charge of the school feeding programme. The findings of the evaluation show that despite the progress that has been made over time (PNSA), legislation, governance structures, operations manual, etc., are insufficient for even partial transfer of the programme to the GoH at this Final Report 39 | P a g e juncture. The SABER exercise provided ample evidence of this and there is consensus among international partners that the GoH needs to budget for SF, resolve crucial governance issues and develop the capacity and strong financial systems to adequately manage the resources. The findings point to the need for WFP to continue supporting the capacity building process at national level, while focusing efforts on decentralized structures and community level. This will help develop capacity at all levels of government for an eventual transfer. 3.1.4. Question 5 - Effectiveness, Efficiency, Sustainability 145. With regards to strategies to promote the generation of revenue, evidence points to the experience in Nippes as a suitable model and also contribute to better nutritional outcomes. At any rate, intersectoral collaboration could support these objectives through innovative approaches. As discussed, there is a strong case for increasing local purchases, with numerous advantages which, in the longer term outweigh the higher initial costs. In recent years, several donors have demonstrated their interest in promoting local purchases, e.g. Canada, France, the USA, Brazil. It is also clear from the findings that climate change and gender will need to be part of strategies. 146. Regarding gender, as shown, women face numerous barriers in the agricultural sector, related to land tenure, which limits their access to productive resources, including credit. Strategies should be designed to help overcome these barriers. 147. Regarding climate change, the experience from other countries, including from WFP, has shown that schools can provide learning for children and economic opportunities for youth and women, through climate adapted food growing, using greenhouses and hydroponics. The experience of other WFP COs promoting greenhouses and hydroponic gardening (see Innovation Accelerator) should be studied, but perhaps as importantly, the experiences of other Caribbean countries including Jamaica and Barbados. The evaluation found that there was interest in piloting these innovative strategies from the Innovation Unit of the MARNDR and IADB. 3.1.5. Gender Equality and Environmental Sustainability 148. As discussed above, two particular gender issues need to be addressed as priorities. One is the issue of compensation for cooks, which is not commensurate with their daily input; essentially, they are working fulltime in hard conditions as volunteers, which virtually all stakeholders at school lamented. The second issue concerns both the health and working conditions of the cooks. WFP needs to give serious consideration to helping provide less environmentally damaging and more healthy ways of cooking the food provided to the children on a daily basis, particularly through a cookstove strategy. There are opportunities for WFP to provide improved cookstoves adapted to the local circumstances by partnering with foundations (e.g. World Kitchen), implementing partners (e.g. BND and others) and the private sector. Final Report 40 | P a g e 3.2. Recommendations 149. Based on the findings and conclusions of this evaluation, the evaluation team makes seven recommendations to WFP’s Haiti CO. The recommendations are labeled as Strategic or Operational and ordered by importance. In terms of the timing of their implementation, it is important to start implementing them as soon as possible. This is largely because a good part of these recommendations depend on developing new practices and such matters invariably take time. Several of the recommendations will require considerable reflection on the part of WFP Haiti. Management and relevant sections need to work together to devise strategies and define roles and responsibilities for their implementation. 3.2.1. Strategic 150.Recommendation 1: To enhance sustainability of the national school feeding programme, the Haiti CO should continue to support the validation of normative documents and governance structures and assist with their adoption and application, based on the Ten Year Plan for Education and Training (2018-2028). At the same time, emphasis should be put on capacity building efforts at decentralized government structures, including awareness raising and training on gender equality. WFP could support a partnership between the Ministère à la Condition Feminine et aux Droits des Femmes67 and the MENFP to develop material to that effect. 151.Recommendation 2: Develop a gender-transformative strategy for community engagement, using proven approaches that employ social mobilisation within a community development approach implemented by NGOs. The suitability of financing tools such as Village Savings and Loans should be investigated in terms of their potential application in support of school feeding and income generation activities. This could also include private sector actors such as SogeBank or Digicel or one of their affiliates such as Mon Cash. 152.Recommendation 3: Consider increasing local purchases and support local producer organizations, especially women using gender transformative approaches. In addition, WFP should seek to promote complementary activities related to nutrition and food growing activities to provide an opportunity for children and their families to learn agricultural practices, and in particular protected agriculture such as greenhouses, better suited to current climatic challenges. WFP’s section in charge of climate change should reach out to the Innovation Department of MARNDR and IADB both who expressed an openness to collaborate with WFP in this area. WFP should explore the possibility of turning schools into community hubs through partnerships with community-level service providers. 153.Recommendation 4: Establish strategic partnerships in the field of education when implementing school feeding so that schools served by WFP can also be supported by programmes to strengthen the quality of education. 67 Ministry for the Status of Women and Women's Rights. Final Report 41 | P a g e 3.2.2. Operational 154.Recommendation 5: A comprehensive cook stove strategy should be established. This strategy should aim to work with available circumstances to use the most appropriate and practically feasible cooking stove set-ups. This should include the elimination of all occupational health and safety risk for women cooks such as those related to cooking on open fires in closed spaces, burning of wood in urban areas where for example, propane stoves could be used and the overall reduction of unwanted environmental impacts. To this end, a partnership with foundations, NGOs and the private sector should be actively explored. 155.Recommendation 6: Include vulnerability as a criterion (as defined in the PSNAS: education, nutrition, safety nets) in the school suspension processes. Provide additional support (e.g. more training) to schools struggling to comply with programme rules. WFP has to establish a strategy to integrate the three school cycles. The practice of sanctioning schools for feeding older students needs to be reviewed. In the event that McGovern-Dole will not fund feeding activities for students in the 3rd cycle, WFP Haiti should seek alternative funding. 156.Recommendation 7: Due to the frequency of emergencies in Haiti, an analysis should be carried out together with donor and Haitian authorities on the use of the McGovern-Dole school feeding platform as a response mechanism to slow and rapid-onset emergencies, such as droughts, earthquakes and hurricanes and political and economic instability. WFP should take the opportunity to ensure that it is gender-responsive and promote gender equality. Final Report 42 | P a g e Annexes Final Report 43 | P a g e Annex 1: Terms of Reference (without the annexes) FINAL EVALUATION of WFP Haiti’s Food for Education and Child Nutrition Program (2016-2019) McGovern Dole Grant FFE-521-2015-020-00, FFE-521-2016/016-00 Introduction 1. These Terms of Reference (ToR) are for the final evaluation of WFP Haiti’s Food for Education and Child Nutrition Programme 2016-2019 (McGovern Dole (McGovern-Dole) Grant FFE-521- 2015-020-00 and FFE-521-2016/016-00) in Haiti. This WFP’s categorized activity evaluation is commissioned by WFP Haiti and will take place from January 2019 (inception phase) to September 2019 (finalization of dissemination activities). 2. The ToR was prepared by the Monitoring and Evaluation Unit of WFP Haiti Country Office, supported by the Regional Bureau in Panama, based upon the programme’s evaluation plan, initial document review and consultation with stakeholders and following a standard template. The purpose of the ToR is twofold: first, to provide key information to the evaluation team and helps guide them throughout the evaluation process; second, to provide key information to stakeholders about the proposed evaluation. The final evaluation will meet the WFP´s policy and guidance (DEQAS) and the United States Department of Agricultures’ (USDA) Monitoring and Evaluation Policy. 3. Since the beginning of the 2016-2017 and the 2017-2019 grants (FY15 and FY16 grant), the Programme has supported more than 190,000 primary students across almost 600 schools in the departments of North, North-East, Center, West and Artibonite with daily school meals to improve educational outcomes and ensure regular attendance. The programme also currently undertakes activities related to literacy promotion with 42 schools in the West and Center, and water and sanitation support (WASH) in 110 schools in all departments. Another pillar of the programme is the support given to strengthen government capacities. The programme targets boys and girls, who in WFP schools have similar attendance, retention and pass rates. 4. The scope of the final evaluation will be all departments in which WFP implements the McGovern-Dole School Meals Programme. Although it is expected that data and insight collected from other departments and programmes is also considered. 2. Reasons for the evaluations 2.1 Rationale 5. Since 2016, WFP and the Government of Haiti have been implementing a one and a two-year McGovern-Dole school meals project. The purpose of this evaluation is to assess whether the project has achieved the expected results as outlined in the results framework and identify any other outcomes the project has had beyond this framework. The evaluation should provide lessons learned and recommendations for McGovern-Dole, program participants and other key stakeholders for future food assistance and capacity building programs, in the context of a potential new McGovern-Dole or other grants. 6. WFP and its project partners will use the final evaluation to assess the programme’s implementation and achievements; the relevance of the intervention; as well as its alignment to the National School Feeding Policy (PSNAS); and discuss and recommend improvement actions when necessary. Final Report 44 | P a g e 7. WFP will also use the evaluation findings as a platform to engage discussions with the Ministry of Education (MoE) and the School Feeding Agency (PNCS) on central and departmental level on the alignment of the McGovern-Dole programme with the national school feeding policy, the government’s effort to strengthen social protection and the reinforcement of the national school feeding program and monitoring and evaluation system. Following the Systems Approach for Better Education Results) SABER workshop undertaken in 2015, WFP and MoE efforts have been oriented towards the formulation, adoption and implementation of a National School Feeding Policy and an Operations Manual. The National Policy defines the general orientations of the National School feeding programmes while the Manual sets the targets/indicators, the specific roles of various stakeholders, including communities and the implementation procedures of the Programme. The evaluation will look into this aspect to come up with information on progress achieved and underline new strategies to better align the McGovern-Dole programme to the National School Feeding Policy and Operations Manual. 8. Furthermore, WFP will use the final evaluation’s findings to create awareness among key school feeding stakeholders about project activities that could be used to review and prepare Haiti’s national school meals operation manual for nationwide implementation. 9. Findings and recommendations from the final evaluation would inform and feed into the implementation of the WFP Haiti Country Strategic Plan (CSP), which is planned to start in July 2019. To serve this purpose the evaluation should inform on all 3-school feeding related activities (further detail provided in Annex 10, draft CSP Logframe extract). 2.2 Objectives 10. The evaluations will serve the dual and mutually reinforcing objectives of accountability and learning. • Accountability.- The final evaluation will assess on the performance and results of the Programme 2016-2019. • Learning.- The evaluation will determine the reasons why certain results occurred, to draw lessons and derive good practices for learning. It will also provide evidence-based findings to inform operational and strategic decision-making for future similar or alternate programmes. Findings will be actively disseminated and lessons will be incorporated into relevant information-sharing systems. 2.3 Stakeholders and users 11. Several stakeholders both inside and outside of WFP have interests in the results of the evaluation and some of them will be asked to play a role in the evaluation process either as part of the evaluation reference group or as key informants. Table 1 below provides a preliminary stakeholder analysis, which should be deepened by the evaluation team as part of the inception phase, this includes especially identifying potential additional stakeholders (Refer to Annex 3 on Stakeholders’ expected relevance to the evaluation process). 12. Accountability to affected populations is tied to WFP’s commitments to include beneficiaries as key stakeholders in WFP’s work. As such, specific debriefing content and participatory dynamics to support WFP’s tailored sessions to inform beneficiaries representatives and implementing partners on relevant evaluation findings, recommendations and lessons should be developed by the evaluation team as part of the evaluation’s communication strategy. Key groups identified as relevant parties are School Directors, Students both in the programme as well as the older students or from other schools excluded from the programme, teachers, cooks, canteen management and supervisory committee, parents, the wider community and local Final Report 45 | P a g e authorities. A special attention should be given to special communication needs of women and girls. 13. WFP is committed to ensuring gender equality and women’s empowerment (GEEW) in the evaluation process, with participation and consultation in the evaluation by women, men, boys and girls from different groups. Gender equity and women’s empowerment envisaged since the beginning of the present the project, will be confirmed in the present study namely: the long-term effect of girls enrollment and regular attendance in assisted schools, participation of women in food management committees, the long-term effect of nutrition training for cooks, and the potential to integrate local food purchases from women’s associations for provision to schools and their empowerment in literacy and income generation into the programme. Table 1: Preliminary stakeholders’ analysis68 Stakeholders Interest in the evaluation and likely use of evaluation reports for this stakeholder WFP STAKEHOLDERS Country Office (CO) Haiti Responsible for the country level planning and programme implementation, WFP Haiti has a direct stake in the evaluations and an interest in learning from experience to inform decision-making. It is also called upon to account internally as well as to its beneficiaries, partners and donors for performance and results of its programme. Key to WFP is to better understand the different school feeding models, currently in use in Haiti and learn from the experience of other countries, to ensure future projects are designed around the key beneficiaries, addressing their needs and supporting them in a sustainable manner. Regional Bureau (RB) Panama Responsible for both oversight of COs and technical guidance and support, the RB management has an interest in an independent/impartial account of the operational performance as well as in learning from the evaluation findings to apply this learning to other country offices. The Regional Evaluation Officers supports CO/RB management to ensure quality, credible and useful decentralized evaluations. Office of Evaluation (OEV) OEV has a stake in ensuring that decentralized evaluations deliver quality, credible and useful evaluations respecting provisions for impartiality as well as roles and accountabilities of various decentralized evaluation stakeholders as identified in the evaluation policy. WFP Executive Board (EB) The WFP governing body has an interest in being informed about the effectiveness of WFP programmes. These evaluations will not be presented to the Board, but their findings may feed into thematic and/or regional syntheses and corporate learning processes. OTHER STAKEHOLDERS Beneficiaries As the ultimate recipients of food assistance, beneficiaries have a stake in WFP determining whether its assistance is appropriate and effective. Consequently, school directors, students both in the programme as well as the older students or from other schools excluded from the programme, 68 This Table refers to direct or indirect evaluation results users or intended users. Regarding consultations along the evaluation process, see stakeholders’ relevance categorization in Annex 3. Final Report 46 | P a g e teachers, cooks, canteen management and supervisory committee, parents, the wider community and local authorities are considered key stakeholders. The level of consultation of and debriefing to women and men, boys and girls in the evaluation through the most appropriate social research technique and their perspectives will be primarily sought. Data should be disaggregated by sex and age when possible. Specific notice should be taken for vulnerable groups e.g. handicapped children and potentially more vulnerable girls. Government The Government has a direct interest in knowing whether WFP activities in the country are aligned with its priorities, harmonized with the action of other partners and meet the expected results. Issues related to capacity development, handover and sustainability will be of interest. Various Ministries are partners in the design and implementation of WFP activities, including, for this specific project, the Ministry of Education and PNCS. NGO, implementing partners WFP Haiti partners with well-established local Haitian and international NGOs to implement its activities. The results of the evaluation might affect future implementation modalities, strategic orientations and partnerships. WFP reviews its partnerships regularly and excludes partners where necessary. In the last years BND has been a strategic partner in the general implementation, while CRS is a sub-grantee focusing on literacy and World Vision on water and sanitation. Main donor, McGovern-Dole For this evaluation, the main stakeholder is McGovern-Dole. McGovern-Dole funds WFP’s school feeding programme through a McGovern Dole Grant (FY 15 for school year 2016-2017 and FY 16 for school year 2017-2018 and 2018- 2019) and so has a strong interest in knowing whether their funds have been spent efficiently and if WFP’s work has been effective and contributed to the strategies and programmes of McGovern-Dole. McGovern-Dole is also funding this evaluation. Its role is to review, comment and approve the TORs and the Inception and Final evaluation reports after the ERG approval; in addition, it participates in a key informant interviews with the selected evaluator prior to field data collection. Other School Feeding implementors WFP Haiti is implementing a school feeding programme in other departments of Haiti with the support of other donors, mainly Canada and France. The programme is very similar in terms of provision of a daily meal but has a strong local procurement component. While these donors are not stakeholders in the implementation in the abovementioned development corridors they are a stakeholder for the advancement of national policies and often face similar challenges in the day to day implementation. World bank was a major implementor of a school feeding programme in Haiti although they have recently scaled down. Their experiences could be valuable for future programmes and the evaluation could inform their future strategy. The World bank model provided a local 5-day standardized menu via two main implementing partners. UN Country Team Other UN actors in the education sector such as UNICEF and actors related to promoting food security and local production such as FAO can potentially provide useful information and insight. 14. The primary users of the final evaluation will be: Final Report 47 | P a g e − The WFP Haiti Country Office and its partners in decision-making, notably related to programme implementation and/or design, Country Strategy and partnerships. The evaluation should provide an evidence-based, independent assessment of performance of the school feeding project so that WFP and its project partners can adjust course as necessary and in line with the national school feeding policy for the future programme cycles. − McGovern-Dole will use evaluation findings to inform planning and implementation of the Haiti and other McGovern-Dole projects. − Given the core functions of the Regional Bureau (RB), the RB is expected to use the evaluation findings to provide strategic guidance, programme support, and oversight. − WFP Head Quarters (HQ) may use evaluations for wider organizational learning and accountability. − The Office of Evaluation (OEV) may use the evaluation findings, as appropriate, to feed into evaluation syntheses as well as for annual reporting to the Executive Board. 3. Context and subject of the evaluation 3.1 Context 15. Since the 1980s, Haiti’s average annual economic growth has remained below its demographic expansion, contributing to the nation’s ranking among the Least Developed Countries and the poorest nation in the Americas. The combination of inconsistent gross domestic product (GDP) growth, notably due to an underperforming agriculture sector, as well as sharp contractions following natural disasters, have seen poverty affect as many as 59 percent of Haitians, while 24 percent lived in extreme poverty. Moreover, economic growth appears to have most benefited the richest quintile of the population who owns over 64 percent of national wealth – with a Gini of 0.61, Haiti is the second most unequal country in the world. Women are disproportionately affected by this inequality, as they are more likely to be unemployed or employed in the informal sector and their wages are 32 percent lower than men’s. 16. Widespread poverty, economic inequality and governance issues also contribute to overall political instability. In the wake of the cancelled 2015 run-off presidential elections, an interim presidency was installed; new elections were organized in 2016 but were postponed due to population protests and Hurricane Matthew. The elections took place on 20 November 2016; Jovenel Moise was declared the winner with 55.6 percent of the vote and a reported turnout of 21 percent. Since then, the upward price adjustments of petroleum products in 2018, as well as allegations of misuse of Venezuela-sponsored Petrocaribe funds by previous administrations, contributed to rising social instability which culminated in riots and civil unrest the 6-9 July 2018, leading to the resignation of the Prime Minister, Jack Guy Lafontant, on 14 July 2018. Reinstating a new Prime Minister has been a difficult task for the Government. In this context, the departure of the UN mission for the stabilization of Haiti (MINUSTAH) and the expected closure of the UN mission for justice support in Haiti (MINUJUSTH) in 2019 could increase security risks, jeopardize private investments and economic growth, while augmenting political instability. 17. Haiti still faces multiple challenges in relation to representation of women in public office and gender-based violence. Women seeking political office face considerable obstacles, including patriarchal attitudes toward leadership, lack of financial support, and threats of violence and intimidation. However, some progress has been made. In 2012, the Parliament passed an amendment instituting a 30 percent quota for women in all elected and appointed positions at Final Report 48 | P a g e the national level, and the 2015 Electoral Decree added the same quota for local councils and political candidates. Haiti's long-term economic and democratic development rely on prioritizing the protection and empowerment of women. Schools are an important place where girls can start learning to take responsibility. It was observed that often girls are not systematically considered as e.g. Class President or School Representative. 18. Gender based violence is another major challenge. One in three Haitian women, ages 15 to 49, has experienced physical and/or sexual violence. Moreover, poor legal protection, fear of reprisals and the social stigma attached to be a victim of sexual violence contributes to under￾reporting. Among girls, ages 13 to 17 who reported sexual abuse, school was the second most common place for unwanted touching. Working with educators and children in schools from an early age is therefore key to ensure the school is a safe space for girls. 19. Further gender analysis is planned by WFP Haiti in 2019, through a gender specialist that will support the school feeding team in 2019. It will be important for the evaluation team to work in close collaboration with this expert and the implementing gender officer. 20. Another recurring issue that impacts educational outcomes for children, is a necessity to work at least once a week either inside or outside the house. Based on a UNICEF definition 50 percent of children between 5- and 14-years work, in two of the implementation areas of the McGovern￾Dole Programme (North-East and Center) even 64 percent. According to a study conducted in 2006 (EMMUS IV), 87 percent of all children between 5 and 17 years had done work in the week before the survey. This affects girls and boys differently as women and men, traditionally engage in different tasks. 21. For several decades, the recurring problems faced by the education system have been a major obstacle to its development and to the provision of quality educational services. Pointed out by all the strategic and operational documents of the Ministry of Education, these problems mainly relate to the governance of the sector, the access and the provision of education, the internal and external efficiency of the system. 22. In greater detail, the key factors are: (i) the shortage and inequitable distribution of the predominantly private school system; (ii) the poor quality of the provision of education and the weakness of the internal efficiency resulting in very low rates of academic achievement and relatively high rates of repetition and drop-out; (iii) the weakness of the education system governance which prevents the Ministry of Education (MENFP) from effectively performing its major roles of regulating, planning and guiding. The massive destruction of school and administrative infrastructures, the material and human losses caused by the devastating earthquake and hurricane have magnified the situation with the risk of an increase in the number of school-age children outside of the system. (Approximately 500,000). 23. According to the latest school census (2011), only 20 percent of the provision of education comes from the public sector; the remaining being in the hands of the non-public sector most of the time managed without regulation and operating below minimum quality standards. Despite its proven importance, access to activities targeting young children (0-5 years) remains very limited (67 percent of gross enrollment rate for pre-school 3-5 years, MENFP 2011). Poor quality is reflected by average repetition rates of 15 percent and drop-out rates around 13 percent. Combined with late entry, these factors increase the proportion of over-age children in primary school to 65 percent. It is noted that the pass rate in 5th grade of primary school is low (25 percent). This alarming situation is largely due to the high proportion of unskilled teachers (more than 65 percent), unfavorable learning conditions, and the non-application of norms and standards that can guarantee a quality education. Final Report 49 | P a g e 24. Among the children most affected by limited access and lack of quality are those in rural areas, low-income families in slums of large urban centers, children separated from their families (residential centers, children in domestic service, children living in the streets), children with disabilities and displaced children. 25. The social shocks of recent global crises have led to increased demand for school feeding programs in low-income countries. These programs are part of the response to social shocks but are also a sustainable financial investment in human capital as part of the long-term global efforts to achieve Education for All (EFA/EPT) and social protection for the poor. 3.2. Subject of the evaluation 26. The WFP McGovern-Dole programme seeks to contribute to Sustainable Development Goal 2 (SGD2) SDG 2: End hunger, achieve food security and improved nutrition and promote sustainable agriculture and SGD 17: Strengthen the means of implementation and revitalize the global partnership for sustainable development.69 27. The first year McGovern-Dole school feeding project was implemented for school year 2016- 2017. It was possible to support almost 190,000 primary students instead of the agreed 176,700. Given the number of students per school varies 595 schools instead of the agreed 650 were supported (FY15). Activities under FY15 include school meals distribution of a daily hot meal, which consists of 120 grams of bulgur wheat, 30 grams of green peas, 10 grams of vegetable oil, and 3 grams of iodized salt per child. This activity was continued without change for FY16. Distribution of a daily bag of roasted peanuts (28g) was done in all schools in 2017. Micronutrient powders (MNP) were added to meals in 2 development corridors. One bag of 8 g per 20 children. Key other activities, as agreed upon in the original proposal were Government Capacity Building, distribution of fuel-efficient stoves, cooking and eating utensils, and various trainings to improve school meals management and quality. 28. Following this one-year project, WFP began to implement the current programme (September 2017 to June 2019). During the school year 2017-2018 and 2018-2019, it was intended that the McGovern-Dole programme would support around 100,000 students across around 380 primary schools in the three United States Government (USG) development corridors in Haiti: The Cap Haitian corridor; the Saint Marc corridor; and the Port-au-Prince corridor. This two-year project is broader in programmatic scope than the 2016-2017 programme, as it includes, in addition to the activities already mentioned above, activities contributing to improved literacy, health and hygiene. At the same time the delivery of roasted peanuts and MNPs was discontinued in 2017. Both projects are targeting the same beneficiaries, except for those children who have started school during the new period and the beneficiaries that were excluded due to the reduction in size. 69 https://sustainabledevelopment.un.org/ Final Report 50 | P a g e 29. Currently the McGovern-Dole funded school feeding project is embedded in the WFP Haiti Transitional Interim Country Strategic Plan (T-ICSP) January 2018 – June 2019. Additional activities covered by the Country Programme are emergency preparedness, nutrition, resilience and support to smallholder farmers. 30. Overall the McGovern Dole grant for the three -year programme is USD 24 million ($10 M received in FY15 (1 year project) and $14 M received in FY16 (2 year project). 31. To achieve the McGovern-Dole project objectives and results, the following activities are being implemented over the course of 3 years (FY15 and FY16): SO, 1: Improved Literacy of School-Aged Children: a) Provide School Meals; b) Provide Storage and Food Preparation Equipment, Tools and Eating Utensils; c) Capacity Strengthening Strategy: Building the foundations of a National School Feeding Programme; d) Provide Creole/French literacy kit and school supplies (added for FY16); e) Provide training/materials to teachers and administrators on literacy and professional teaching modules (added for FY16) SO2: Increased Use of Health and Dietary Practices: a) Training on Good health and nutrition practices at the school level; b) Training on Supply Chain, Warehouse and Commodity Management for local Cooperating Partners and Government of Haiti (GoH); c) Training on food preparation and storage practices; d) Provide Storage and Food Preparation Equipment, Tools and Eating Utensils; e) Capacity Strengthening Strategy: Building the foundations of a National School Feeding Programme; f) Distribute water purification tablets and soap to schools (added for FY16). 32. The key activity from 2016-2019 is the provision of a daily hot meal to school children between 5-12. This activity is complemented by various trainings on safe food preparation, storage practices and good health and nutrition as well as school feeding management and supervision. In addition, essential items for food storage, preparation and consumption are provided. Targeted activities included WASH and Literacy. Further details on the activities implemented are provided in Annex 10. 33. With the support of other donors WFP is implementing a similar school feeding programme across most of the rest of Haiti (excluding South department). Key difference is that the other programmes have a strong local procurement component, either centrally (major part of the project) or through decentralized home-grown school feeding (30,000 boys and girls). These programmes also focus on smallholder farmer capacity building, training and access to markets. They do not include literacy, which is unique to McGovern-Dole. 34. For these programmes extensive information will be provided in the document library (Annex 4). It is expected that the evaluation team thoroughly reviews this material, meets with key stakeholders and if necessary conducts focus groups in the respective programme areas, to ensure the differences between the programmes can be fully taken into consideration and form an integral part of the evaluation report 4. Evaluation approach 4.1 Scope 35. The scope of the final activity evaluation is the entirety of activities covered by the McGovern￾Dole programme in Haiti (FY 15 and FY 16). The programme started in September 2016 with the goal of supporting school feeding in the three United States Government (USD) development corridors of Cap Haitian, Port-au Prince and Saint Marc and is scheduled to end in June 2019. It is important to note that the WASH and Literacy activities as well as the MNP study were not Final Report 51 | P a g e implemented across all schools. This will need to be taken into consideration for the sampling of these schools. 4.2 Evaluation criteria and questions 36. The evaluation will address the following key questions allied to the evaluation of relevance, effectiveness, efficiency and sustainability criteria (See Table 2). Collectively, the primary evaluation questions aim at highlighting the key lessons and performance of the Programme, which could inform future strategic and operational decisions on school feeding framed by the CSP. 37. The evaluation questions will be further reviewed and developed by the evaluation team during the inception phase.70 If not enough explicitly included in the ToR, Gender Equality and empowerment of women should be mainstreamed throughout the questions and sub￾questions. It will be important to include gender specific results identified so far and adjust the evaluation questions where necessary to accommodate that. Data collected will require disaggregation by gender where possible. Stakeholder interviews should ensure the gender point of view is considered and relevant stakeholders who can inform on this are included. 38. WFP is operating in the complex environment of having to ensure an effective and efficient management of the school feeding programme on the school level, while also assisting the schools that welcome the most vulnerable (usually national schools). It will be key for the evaluation to inform on how to best reconcile these two goals and ideally further refine the geographic targeting. Table 2: Original criteria and evaluation questions Evaluation Questions Criteria 1) To what extent is the programme relevant considering, the needs of school children and their families, national government’s education and school feeding policies and strategies? And how can the programme ensure to identify those most in need through a collaborative approach? Relevance 2) To what extent has (and has not) the programme resulted in the expected outputs and outcomes71? How and why were the observed results achieved? To what extent internal/external factors explain those results? Is there any difference on school feeding outcomes based on gender differences, differences of school types (Public, Private), location (urban, rural) or model Effectiveness and Efficiency 70 As such, primary evaluation questions should be broken down through an operacionalization process, from relevant sub￾questions (for instance: Has the use of health and dietary practices increased? If so, how and why? How and why is this affecting girls and boys?) to indicators and variables to be considered during the design of the most appropriate social research tecniques. This process should be captured into the Evaluation Matrix for each evaluation question/criteria. 71 As mentioned, otucomes are related to nutrition (in general and through nutrition supplements), education (attendance, absence, repetition/pass, dropp out and/or literacy performance) and WASH-related behaviour changes at schools, and technical assistance to Government entities. As such primary evaluation question will require an operacionalization process from relevant sub-questions (for instance: Has the use of health and dietary practices increased? If so, how and why? How and why is this affecting girls and boys?) to indicators and variables to be included in the most appropriate social research tecniques. The whole process should be represented into the Evaluation Matrix. Final Report 52 | P a g e Evaluation Questions Criteria (Traditional, Homegrown), complementary activities (e.g. literacy, WASH, school gardens)? Are there opportunities to add/strengthen a feasible nutrition sensitive approach in any of those models? 3) To what extent are the key institutions (national, provincial/district and local stakeholders; international and national implementing partners and NGOs; international donors and multilateral agencies working on school feeding in Haiti) and governance structures able to effectively deliver, implement, sustain and scale up school meal interventions in the long-term? What can be highlighted as key success factors for a nationwide implementation of this programme? What would need to be in place to allow for a full transition to the government of Haiti? Are there other possible exit strategies for WFP that could be considered? Effectiveness, Efficiency, Sustainability 4) To what extent is the level of community involvement and participation in schools (General Community, Parents especially via PTAs, SFCs, director, cooks, etc.) contributing to the project school feeding results and sustainability? Particularly, what is the level of involvement and participation of women and girls and what are the reasons behind it? Is there any difference of involvement based on differences of school types (Public, Private), location (urban, rural) or model (Traditional, Homegrown)? What are potential levers to increase this involvement? Effectiveness, Efficiency, Sustainability 5) What are the differences and advantages and disadvantages between the McGovern-Dole model and other approaches, focusing on nutritional and educational outcomes and community engagement? How could the Theory of Change be adapted to increase effectiveness, sustainability, efficiency and long-term effects? Effectiveness, Sustainability 6) To what degree factors for sustainability (financial, political, cultural, gender, environment, neutrino, risk of fraud…) were considered and which ones are reinforcing the interventions sustainability and future transition to the Government, especially identifying key stakeholders and milestones? To what extent, and which government capacities have been strengthened so far? As part of this analysis: Is WFP’s capacity to support key line ministries enough to effectively facilitate national ownership? Are the key ministries willing and able to take back the responsibility for school feeding within the originally agreed timeline or what would a more ideal timeline be? What would the cost be for GoH and what could different budget options be? Sustainability, Effectiveness 7) Does the programme have the potential to create sustainable income generation opportunities for communities? Sustainability 4.2 Data availability Final Report 53 | P a g e 39. Due to the short duration and given the discussions on the FY16 Grant were already ongoing, it was decided to only conduct a baseline study for FY15 in late 2016 and do the end line after the full 3 years of implementation of both FY15 and FY16 in 2019. 40. The baseline study conducted between December 2016 and January 2017 focused on indicators that could be measured before project implementation. Some indicators could not be measured because their definition is linked to the implementation of project activities. Baseline values for each indicator measured against its corresponding targets, as per the project document, were summarized in the final baseline evaluation report and informed all indicators requested in the McGovern-Dole Results Framework. The baseline on the other hand, did not dedicate enough resources to qualitative data collection and did not go beyond the PMP. While the qualitative indicators largely confirmed the already available data, it did not answer the key question of why the indicators are the way they are and how the programme can address this, if it is identified as an issue. This is expected from the final evaluation. 41. Information on the challenges in the political and learning (e.g. lack of funds, learning material, qualified teachers) landscape of Haiti’s National Schools should be ideally gathered through stakeholder interviews and site visits. And documents should be included if stakeholders indicate relevant reports. The identification of key stakeholders might be a challenge in some cases, and a certain independence of the evaluation team is expected. 42. It is expected the evaluation team will use the following specific sources of information: − Semi-annual project reports; − Baseline survey; − Project databases inserted monthly into COMET (WFP Programme Management System) and other systems; − The project baseline survey; WFP annual Standard Project Reports (SPR) and other data collected periodically by the project team, including partners. − Trimestral cooperating partner reports − Monthly monitoring data − The 2017 review of the Canada Programme − All other documents mentioned in the ToR, Annex 4 and the overall document library. 43. These documents contain quantitative and qualitative information that will assist the evaluators. Potentially some data and/or information can also be obtained through the decentralized services (Regional Directors) of the Ministry of Education, which contains data on schools that are not assisted by the WFP and which can be used for the comparison of some indicators. Data quality differs though depending on the School District. In the past this has been challenging in Haiti, but WFP Haiti is working on improving relationships with Decentralized Government entities and enhance collaboration. For primary data collection at school level during the field work, May 2019 will be the last month possible as June due to exams not all children will be available in the schools. 44. The evaluation will entail qualitative and quantitative primary data collection that the evaluation team will be responsible for. In Annex 4 is a preliminary list of background data and/or information available for the evaluation team. It is expected that the team will expand this at inception phase, especially to consider context specific factors such as funding of schools and school materials, political considerations and differences between school types and regions. 45. The project was initially developed with its corporate indicators from WFP results framework, but with McGovern-Dole funding, other specific indicators were incorporated. So, at this time, Final Report 54 | P a g e there are two sets of indicators (corporate and specific, developed after McGovern-Dole funding). Most of these indicators are being collected monthly, while some are collected on a yearly basis. All indicators can be easily updated during the evaluation phase if necessary. It is expected that the evaluators review the currently used questionnaires for this purpose and those used for the baseline to ensure, indicators are comparable. Overall since the beginning of the project the key indicators have stayed stable and around or above WFP corporate targets. The WFP Annual Reports (SPRs) for the DEV 200150 are publicly available on the WFP website (www.wfp.org). 46. During the inception phase, the evaluation team will determine whether gaps exist in data availability and quality. The team will especially check if there is gender related data available from other sources than WFP, in how far the planned WFP gender analysis for 2019 can contribute or if there is a need to conduct a gender sensitive evaluation. 47. Due to the multiple obligations of the WFP M&E staff, on-site data collection will have to be done by the evaluation team. This requires that the team has at least 2 French and Créole Speakers, ideally one man and one woman to discuss gender specific subjects in a gender sensitive approach, as the WFP team can give only limited support. 48. The evaluation team responsible for the baseline assessment warned about the availability and usability of certain data sets, the incoming evaluation team will therefore need to assess the usability in cases where it is necessary to use historical data. Following the recommendations of the baseline team, WFP and its partners have improved data collection tools to provide the level of granularity required by the donor and to answer most of the evaluation’s questions. For instance, school records now provide attendance information per individual child and teacher, records are revised monthly and are subject to random checks. However, during the inception phase, the evaluation team will be responsible for controlling the quality and reliability of data sets and formulate alternative strategies to fill potential data gaps. 49. The evaluation team is expected to explore key questions 5, 6 and 7 largely through qualitative data (although some quantitative data on gender parity is also expected). Key question 6 will require an analysis of similar experiences in the country. 50. Concerning the quality of data and information, the evaluation team should: − Assess data availability and reliability as part of the inception phase expanding on the information provided in the ToR. This assessment will inform the data collection; − Systematically check accuracy, consistency and validity of collected data and information and acknowledge any limitations/caveats in drawing conclusions using the data. 4.4 Methodology 51. The final evaluation will assess: (i) the programme’s relevance, effectiveness, efficiency, sustainability and possible long-term contribution; (ii) whether or not the project achieved its expected results; (iii) identify recommendations and lessons learned as per the agreed PMP and additional topics identified during the inception phase. This is not an impact evaluation, but it is expected that the evaluation will assess to what degree and why the outcomes have changed between the baseline and end line data collection rounds. 52. The evaluation will rely on the Baseline Study for baseline data whenever possible, secondary data and primary data collection when necessary. Extensive data collection has taken place over the total duration of the programme, this data will be shared via the document library (Annex 4) and it is expected that the evaluation team works in close collaboration with the M&E Final Report 55 | P a g e team to avoid duplications in quantitative data collection if the quality and amount of data allow. 53. Other key studies to include will be those done on literacy, nutrition and MNPs for the McGovern-Dole grant and those on HGSF, the 2017 Canada review and the WFP lessons learned and reviews of other school feeding programmes. Finally, the past global evaluation of the Haiti Development (DEV) Project 200150 conducted in 2014 should be taken into consideration (Annex 4). 54. The methodology will be designed by the evaluation team during the inception phase. It should: − Confirm and define specific evaluation questions that are answered, and record and operationalize them in the WFP Evaluation Matrix, considering the data availability challenges, the budget and timing constraints. − Make references to the relevant evaluation criteria listed in section 4.2. − Ensure comparability to the data collected as part of the baseline study and during regular monitoring. − Demonstrate impartiality and lack of biases by relying on a cross-section of information sources (stakeholder groups, including beneficiaries, etc.) The selection of field visit sites will also need to demonstrate impartiality. − Using mixed methods to ensure a comprehensive design, and the reasons for the changes in indicators should be explained. This can include triangulation of information through a variety of means, or different evaluation questions being answered through different methods and types of data. The use of mixed methods should be documented in the inception report. A key method should be stakeholder interviews and focus groups, if resources for quantitative data collection are not needed due to the amount and quality of available data. Other qualitative data collection mechanisms should ideally already be mentioned in the tender, to assess the level of innovation and understanding of the evaluation team; − Contain a sampling strategy, including the sampling method, sample size calculations, and power calculations. Ideally this should be random sampling coupled with a more targeted approach in cases where only a limited number of schools fulfil the requirements (WASH interventions), back-up sampling if data is not available and integrate schools already sampled for the baseline − Ensure using mixed methods that women, girls, men and boys from different stakeholder’s groups participate and that their different voices are heard and used; − Mainstream gender equality and women’s empowerment, as above; while findings and conclusions are expected to contain gender analysis and to address GEEW in the approach and recommendations. − Ensure the methodology and evaluation implementation are ethical and conform to the UNEG Ethical Guidelines for Evaluation. 55. The evaluation will also take into consideration the national school feeding programme priorities, especially where they go beyond the McGovern-Dole indicator framework (e.g. local sourcing) and indicators from the MENFP Operation Manual. And try to inform on these as far as possible. 56. The evaluation team must assess the quality of the baseline data and design during inception, to see at what extent it can be used as part of the final evaluation. experimental/quasi experimental in answering the effectiveness-related evaluation questions above. 57. Sub-questions related to outcomes on literacy will be answered using data collected through the early grade reading assessment (EGRA) standard test. Sub-questions related to health and Final Report 56 | P a g e dietary practices are likely to be based on data collected through school and student surveys, direct observation and key informant interviews as well as review of relevant other reports especially studies related to nutrition. 58. WFP anticipates that the consultants will likely include carrying out key informant interviews and focus group discussions. However, bidding companies should also propose a wider variety of methods as relevant to the evaluation questions and the availability of data. 59. The qualitative data collection methods will gather information on gender equality, capacity strengthening and changes in the institutional context. It will include key informant interviews with relevant stakeholders, including: WFP staff, USDA (on the local, regional and Washington level if possible), Regional Directors and representatives of the Ministry of Education (MENFP) and National School Feeding Agency (PNCS). Additionally, community leaders, School Management Committees, Parent Association members, teachers, and cooks will be targeted for focus group discussions. 60. The following mechanisms for independence and impartiality will be employed: an external service provider will be hired to conduct the evaluation; WFP has appointed a dedicated evaluation manager to manage the evaluation process internally; an internal WFP Evaluation Committee (EC), led by CO management, will make key decisions on the evaluation; an Evaluation Reference Group (ERG) (including WFP and external stakeholders) will be set up to steer the evaluation process and further strengthen the independence of the evaluation. All feedback generated by these groups will be shared with the service provider. The service provider will be required to critically review the submissions and provide feedback on actions taken/or not taken as well as the associated rationale. The compositions of the EC and the ERG are provided in the Annexes section. 61. One of the risks associated to the methodology includes a potential difference in the methodological approach used by the service provider for this final evaluation and the one used for the baseline exercise. To mitigate this risk, an in-depth review of the methodological approach and dataset for the baseline study will be needed during the inception phase. The team should identify other potential risks of the approach and mitigation measures. The inception report will be carefully reviewed by WFP and stakeholders to ensure methodology and approach are sound. 4.5 Quality assurance and assessment 62. WFP’s Decentralized Evaluation Quality Assurance System (DEQAS) defines the quality standards expected from this evaluation and sets out processes with in-built steps for Quality Assurance, Templates for evaluation products and Checklists for their review. DEQAS is closely aligned to the WFP’s evaluation quality assurance system (EQAS) and is based on the UNEG norms and standards and good practice of the international evaluation community and aims to ensure that the evaluation process and products conform to best practice. 63. DEQAS will be systematically applied to these evaluations. The WFP Evaluation Manager will be responsible to ensure that the evaluation progresses as per the DEQAS Process Guide and for conducting a rigorous quality control of the evaluations products ahead of their finalization. 64. In particular, the DEQAS is also consistent with the principles and criteria outlined in the USDA’s Food Assistance Division’s Monitoring & Evaluation Policy. The evaluation team will decide to ensure data used in the evaluation report is checked for accuracy and reliability, and the report will clearly indicate limitations to the conclusions that can be drawn from the evidence. Final Report 57 | P a g e 65. WFP has developed a set of Quality Assurance Checklists for its decentralized evaluations. This includes Checklists for feedback on quality for each of the evaluation products. The relevant Checklist will be applied at each stage, to ensure the quality of the evaluation process and outputs. 66. To enhance the quality and credibility of these evaluations, an outsourced quality support (QS) service directly managed by WFP’s Office of Evaluation in Headquarter provides review of the draft inception and evaluation report (in addition to the same provided on draft TOR), and provide: a. systematic feedback from an evaluation perspective, on the quality of the draft inception and evaluation report; b. recommendations on how to improve the quality of the final inception/evaluation report. 67. The evaluation manager will review the feedback and recommendations from QS and share with the team leader, who is expected to use them to finalize the inception/ evaluation report. To ensure transparency and credibility of the process in line with the UNEG norms and standards[1], a rationale should be provided for any recommendations that the team does not consider when finalizing the report. 68. This quality assurance process as outlined above does not interfere with the views and independence of the evaluation team, but ensures the report provides the necessary evidence in a clear and convincing way and draws its conclusions on that basis. 69. The evaluation team will be required to ensure the quality of data (validity, consistency and accuracy) throughout the analytical and reporting phases. The evaluation team should be assured of the accessibility of all relevant documentation within the provisions of the directive on disclosure of information. This is available in WFP’s Directive CP2010/001 on Information Disclosure. 70. All final evaluation reports will be subjected to a post hoc quality assessment by an independent entity through a process that is managed by OEV. The overall rating category of the reports will be made public alongside the evaluation reports. 4. Phases and deliverables 71. The final evaluation process will proceed through five phases. Annex 5 provides more details related to timeline of activities and deliverables. And Annex 9 provides more details on each phase and expected products. The timeline for fieldwork and reporting will be confirmed during inception phase. [1] UNEG Norm #7 states “that transparency is an essential element that establishes trust and builds confidence, enhances stakeholder ownership and increases public accountability” 1. Prepare 2. Inception •Inception Report 3.Collect data •Aide memoire / debriefing PPT 4. Analyze data and Report •Evaluation Report 5.Disseminate and follow-up Final Report 58 | P a g e 72. Preparation phase (June-December 2018): Preparation of the evaluation and contracting of the evaluation company 73. Inception phase (January – April 2019): Desk review and inception field mission by the evaluation team to prepare the inception report detailing how the team intends to conduct the evaluation. 74. Data Collection phase (April – May 2019): The fieldwork will span over 4 to 5 weeks including visits to project sites (schools) and primary and secondary data collection from local stakeholders and be followed by debriefing sessions presenting preliminary findings and conclusions. 75. Reporting phase (April-June 2019): Data analysis and additional consultations with stakeholders will allow the evaluation team to produce a draft of the final evaluation report. The final approved version will form the basis of the WFP management response to the evaluation. 76. Follow-up and dissemination phase (from July 2019 up to September 2019): The CO Management will respond to the evaluation recommendations via the CO’s Management Reponses. The final evaluation report will be published and communicated with all key stakeholders. 77. Both the evaluation report and the evaluation summary brief should include charts, graphs and general data-viz tools and infographics to visualize the data in a clear, easy to read format, accessible to stakeholders from the community level to the government level. The results and summary reports will be shared with project stakeholders. 78. The inception and evaluation reports shall be written in English and follow the DEQAS templates. The evaluation team is expected to produce written work that is of very high standard, evidence-based, and free of errors. The evaluation company is ultimately responsible for the timeliness and quality of the evaluation products. If the expected standards are not met the evaluation company will, at its own expense, make the necessary amendments to bring the evaluation products to required quality level while respecting the timeline as far as possible. 79. The evaluation TOR, evaluation reports and management responses will be public and posted on the WFP External Website (wfp.org/evaluation). The other evaluation products will be kept internal. 5. Organization of the evaluation 6.1 Evaluation conduct and Ethics 80. The evaluation team will conduct the evaluation under the direction of its team leader and in close communication with Claudia Schwarze, WFP Haiti Evaluation Manager. The team will be hired following agreement with WFP on its composition. 81. The evaluation team will not have been involved in the design or implementation of the subject of evaluation or have any other conflicts of interest. Further, they will act impartially and respect the code of conduct of the evaluation profession. 82. The evaluation team should also guarantee the right to provide information in confidence and make participants aware of the scope and limits of confidentiality, while ensuring that sensitive information cannot be traced to its source (right to confidentiality). 83. Evaluators shall act to minimize risks and harms to, and burdens on, those participating in the evaluation, without compromising the integrity of the evaluation findings (avoidance of harm). Final Report 59 | P a g e 84. Data collection must to ensure that all participants are fully informed about the nature and purpose of the evaluation and their involvement. Only participants who have given informed written or verbal consent should be involved. Noting that this evaluation includes possible contact with children, women and other vulnerable groups, recruitment process should assess suitability of all persons involved to work with these groups. 85. The evaluation team is expected to provide a detailed plan on how the above ethical principles72 will be ensured throughout the evaluation process. This should be reflected in the inception report. 6.2 Team composition and competencies 86. The evaluation team is expected to include three to four members, including the team leader and at least one national/resident consultant. At least one team member should be female, ideally this should a Haitian national and Créole speaker to ensure sensitive GEEW related topics can be discussed in a safe space with beneficiaries and stakeholders. To the extent possible, the evaluation will be conducted by a gender-balanced, geographically and culturally diverse team with appropriate skills to assess gender dimensions of the subject as specified in the scope, approach and methodology sections of the ToR. At least one team member should have WFP experience. 87. The team will be multi-disciplinary and include members who together include an appropriate balance of expertise and practical knowledge in the following areas: − School Feeding programmes; Food and Nutrition Security; WASH; Institutional capacity development; − Gender expertise / good knowledge of gender issues; − Sound knowledge of the Haitian or a comparable context; − Familiarity with the USDA Monitoring & Evaluation (M&E) policy; − All team members should have strong analytical and communication skills, evaluation experience and, to the extent possible, familiarity with Haiti and/or western Africa development context; − Oral and written language requirements include proficiency in English and French for at least the team leader, and at least one team member speaking Créole. French across all team members would be an asset. 88. The Team leader should be a senior evaluator with at least 15 years of experience in research and/or evaluation with demonstrated expertise in managing mixed method impact evaluations. S/he will have technical expertise in one of the technical areas listed above as well as expertise in designing methodology and data collection tools and demonstrated experience in leading similar evaluations. S/he will also have leadership, analytical and communication skills, including a track record of excellent English and French writing and presentation skills. 89. Her/his primary responsibilities will be: i) defining the evaluation approach and methodology; ii) guiding and managing the team; iii) leading the evaluation mission and representing the evaluation team; iv) drafting and revising, as required, the inception report, the end of field work (i.e. exit) debriefing presentation and evaluation report in line with DEQAS. 72 (a) Respect for dignity and diversity; (b) Fair representation; (c) Compliance with codes and ethics of research involving young children or vulnerable groups); (d) Redress; (e) Confidentiality; and (f) Avoidance of harm. Final Report 60 | P a g e 90. The team members will bring together a complementary combination of the technical expertise required and have a track record of written work on similar assignments. Team members should ideally have a minimum of 5 years of experience. 91. Team members will: i) contribute to the methodology in their area of expertise based on a document review; ii) conduct field work; iii) participate in team meetings and meetings with stakeholders; iv) contribute to the drafting and revision of the evaluation products in their technical area(s). 6.3 Security considerations 92. Security clearance where required is to be obtained. As an ‘independent supplier’ of evaluation services to WFP, the evaluation company is responsible for ensuring the security of all persons contracted, including adequate arrangements for evacuation for medical or situational reasons. The consultants contracted by the evaluation company do not fall under the UN Department of Safety & Security (UNDSS) system for UN personnel. 93. However, to avoid any security incidents, the WFP Haiti Evaluation Manager is requested to facilitate that: − The WFP CO registers the team members with the Security Officer on arrival in country and arranges a security briefing for them to gain an understanding of the security situation on the ground. − The team members observe applicable UN security rules and regulations – e.g. curfews etc. 6. Roles and responsibilities of stakeholders 94. The WFP Haiti country office: a) The WFP Haiti Country Director (DCD as alternate) will take responsibility to: − Assign an internal Evaluation Manager for the evaluation not directly involved in the implementation of the project: Claudia Schwarze, M&E Officer. − Compose the internal Evaluation Committee and the Evaluation Reference Group (see below). − Approve the final TOR, inception and evaluation reports. − Ensure the independence and impartiality of the evaluation at all stages, including establishment of an Evaluation Committee and of a Reference Group (see below and TN on Independence and Impartiality). − Participate in discussions and debriefings on the evaluation design and subject (Inception report phase), its performance and results (Evaluation report phase) with the Evaluation Manager and the evaluation team. − Oversee dissemination and follow-up processes, including the preparation of a Management Response to the evaluation recommendations. b) The external Evaluation Manager must be provided by the firm and will work closely with the WFP’s internal one on the following tasks: − Manages the evaluation process through all phases including drafting this TOR. − Ensures quality assurance mechanisms are operational. − Consolidates and shares comments on draft ToR, inception and evaluation reports with the evaluation team. − Ensures expected use of quality assurance mechanisms (checklists, quality support). Final Report 61 | P a g e − Ensures that the team has access to all documentation and information necessary to the evaluations; facilitates the team’s contacts with local stakeholders; sets up meetings, field visits; and provides logistic support during the fieldwork. − Organizes security briefings for the evaluation team and provides any materials as required. c) An internal Evaluation Committee has been formed as part of ensuring the independence and impartiality of the evaluation. This committee will be composed by Ronal Tran Ba Huy, CD; Ilaria Martinatto, Head of Programme; Viviana Sandoval, Head of School Feeding; Claudia Schwarze, M&E Officer; Ivan Touza, Regional Evaluation Officer. The members of the committee will provide inputs to the evaluation process and comment on evaluation products and make key decisions such as internal approval of evaluation deliverables (Annex 6). 95. USDA will be involved in the evaluation at the following stages: Appropriate members of USDA (Programme analyst and M&E lead) will be consulted for comment of the ToR and the inception and final evaluation products Evaluation Reports and will approve these products; participate in key informant interviews with selected evaluators prior to field data collection; and participate in stakeholder meetings and presentation of the evaluation findings, as appropriate. 96. An Evaluation Reference Group will be formed, as appropriate, with representation from the Government (MENFP, PNCS) other school feeding actors (e.g. World bank) and USDA. The ERG members will review and comment on the draft evaluation products and act as key informants to further safeguard against bias and influence (Annex 7). 97. The Regional Bureau: The RB will take responsibility to: − Advise the Evaluation Manager and provide support to the evaluation process where appropriate. − Participate in discussions with the evaluation team on the evaluation design and on the evaluation subject as relevant, as required. − Provide comments on the draft ToR, Inception and Evaluation reports − Support the Management Response to the evaluation and track the implementation of the recommendations. − While the Regional Evaluation Officer Ivan Touza will perform most of the above responsibilities, other RB relevant technical staff may participate in the Evaluation Reference Group and/or comment on evaluation products as appropriate. 98. The Office of Evaluation (OEV). OEV, through the Regional Evaluation Officer, will advise the Evaluation Manager and provide support to the evaluation process when required. It is responsible for providing access to the outsourced quality support service reviewing draft ToR, inception and evaluation reports from an evaluation perspective. It also ensures a help desk function upon request. 7. Communication and dissemination 99. To ensure a smooth and efficient process and enhance the learning from this evaluation, the evaluation team should place emphasis on transparent and open communication with key stakeholders. This will be achieved by ensuring a clear agreement on channels and frequency of communication with and between key stakeholders. All communication between the evaluation team and stakeholders should go through or include the evaluation manager. 100. As part of the international standards for evaluation, WFP requires that all evaluations are made publicly available. Following the final approval of the evaluation report, findings and Final Report 62 | P a g e recommendations shall be shared in various ways, including through discussions with WFP senior management and staff (primarily to enhance strategic and operational aspects) as well as with key partners including McGovern-Dole, concerned Government entities, as well as NGOs as implementing partners, and relevant UN agencies. WFP will publish both the reports and the management response. Final evaluation products of the evaluation will be disseminated or made available to partners in electronic and print form. See an overview of the Communication and Learning plan in Annex 6. 8. Budget 101. For this evaluation, the budget will be based on procurement through Long-term Agreements. Rates may be guided by pre-agreed rates and should consider costs for an external evaluation manager’s support in managing the decentralized evaluation. The evaluation budget is planned under the Mc-Govern-Dole contribution. The total budget for this contract will not exceed USD 210,000. The evaluation budget should include costs associated with international and national travels, daily subsistence and other direct costs. Final Report 63 | P a g e Annex 2: Evaluation Matrix Questions and Sub￾questions Measures/Indicators Main Source of Information Data Collection Method Data Analysis Method Reliability & availability of data 1. To what extent are the programme activities relevant in light of the needs of school children and their families, and education and school meals policies and strategies of national government? Criteria Relevance Sub-questions 1.1 Are programme activities aligned with the objectives of the National School Meals Policy and Strategy (PSNAS)? *Degree of alignment between the programme and National School Meals Policy and Strategy (PSNAS) including: selection criteria; % of local food purchase, % of public schools; age group targeted; constraints due to contextual factors (infrastructure, access, etc.) Secondary data: McGovern￾Dole programme description; PSNAS guidelines WFP M&E data review Primary data: Min. of Educ., PNCS, donors: Canada, WB, AIDB; McGovern￾Dole manager; WFP Programme and M&E staff Secondary data: Document review Primary data: Key informant interviews Comparison between programme objectives and national policy; Comparison between PSNAS targets and actual number of schools in the programme Good Documentation is available and complete 1.2 Was the geographic targeting congruent with the needs of the population, in particular relating to food security, nutrition needs, vulnerability and access to education? *level of food insecurity in selected intervention zones. Secondary data: 2015 IPC report (chronic), VAM Analysis; previous evaluation reports, PNSAS nutrition data; UNICEF data on education; FAO strategy Primary data: Min Educ. Department officials; PNCS; WFP programme staff, incl. Nutrition specialist; USAID Secondary data: Document review; WFP M&E data review Primary data: Key informant interviews Comparative analysis between selected municipalities/scho ols and national studies and data Comparative analysis of documented data and stakeholders’ perceptions/opinio ns Good IPC chronic map 2015 at onset of McGovern-Dole programme 1.3 How has the programme reconciled the issue of having to work with the most vulnerable * Existence of a documented approach/policy to exclude schools from the programme, including * Criteria, procedures, Secondary data: Progress reports to McGovern-Dole; WFP M&E data. Document review; M&E data review Content analysis of reports Fair to Good Final Report 64 | P a g e Questions and Sub￾questions Measures/Indicators Main Source of Information Data Collection Method Data Analysis Method Reliability & availability of data schools while ensuring schools meet the selection criteria and identifying new schools that should be included? communication plan * Perceived fairness of the approach Primary data: McGovern-Dole manager; WFP manager; Departmental Edu Managers; school directors Primary data: Key informant interviews Comparative analysis of documented data and stakeholders’ perceptions/opinio ns Based on stakeholders’ perception Availability of a paper trail detailing decision making? Evaluation Question 2. To what extent has the programme achieved the expected outputs and outcomes at school level and which factors are explaining those results? Criteria Effectiveness Sub-Questions 2.1 Which planned outputs have been achieved and which have not? Degree of achievement of all outputs, including targets, monitored by the programme indicators based on logic model indicators (WFP and McGovern-Dole) *% of open school days when the food is prepared and delivered * Number of meals prepared * Number of schools and individuals involved in activities * WFP’s ability to manage challenges (e.g. Mathew’s aftermath) * type and effectiveness of measures/innovations implemented to improve outputs and outcomes Secondary Data: WFP semi-annual reports to McGovern-Dole; WFP monitoring data sets; WFP SPR/ACR Primary Data: School directors; teachers; pupils; SF management and supervision committees; cooks; WFP programme and M&E staff; WFP and operating partners monitoring officers Secondary Data: Document review; Monitoring data review. Primary Data: Key informant interviews Focus group discussions (with committees and monitoring officers) Comparative quantitative analysis on planned and actual outputs reported Fair to Good Operating partners and WFP monitoring reports complete but noted that some school attendance reports are unreliable. 2.2 Which planned outcomes have been achieved and which have not? *Degree of achievement of all outcomes monitored by the programme indicators based on performance management frameworks (WFP and McGovern￾Dole) and baseline * Improvement in literacy, based on EGRA Indicators (% of students who, by the end of two years of primary Secondary Data: WFP semi-annual reports to McGovern-Dole; WFP M&E data; Min. of Educ. data sets at departmental level Primary Data: School directors, teachers; implementing partners (CSR; WV Secondary Data: Document review Monitoring data review Primary Data: Key informant interviews Focus group discussions (with teachers and with Comparative quantitative data analysis on planned (baseline) and actual outcome for key reported indicators and targets Fair to Good Baseline deemed unreliable and essentially descriptive; good monitoring data from the programme; may be difficult to Final Report 65 | P a g e Questions and Sub￾questions Measures/Indicators Main Source of Information Data Collection Method Data Analysis Method Reliability & availability of data schooling, demonstrate that they can read and understand the meaning of grade level text) * Attendance rate for g/b (%) * Retention rate for g/b (%) by type of school and various sub￾activity combination (e.g. with literacy; with homegrown; with WASH, etc.) * Pass rate for g/b (%) management and monitoring staff); WFP programme and M&E staff; Min. of Educ. National and departmental staff. monitoring staff from CRS, WV) parents Comparative quantitative data analysis of averages of outcome indicators (except literacy) in programme schools with departmental averages for attendance, retention and success. Comparative quantitative data analysis of outcomes for schools with SF only and schools with literacy or WASH components. obtain/compile data from the Ministry of Education. Evaluation Question 3. To what extent does the community involvement in schools (general, parents especially via PTAs, SFCs, directors, etc.) contribute to outcomes and sustainability? Criteria Effectiveness, Sustainability, Efficiency, Impact (long term effects) Sub-Questions 3.1 To what extent do school meals management and supervision * Number of functioning SF management committees (%) Secondary Data: WFP reports and M&E data; WFP Focus Groups 2018 (Tim Swartz Secondary Data: Document review Monitoring data review Quantitative and qualitative analysis of actual Fair to Good Data collected from schools on parental Final Report 66 | P a g e Questions and Sub￾questions Measures/Indicators Main Source of Information Data Collection Method Data Analysis Method Reliability & availability of data committees contribute to the outcomes as defined above and the sustainability of the activities on the long term? − Are the contributions from parents sufficient to achieve planned outcomes? − Are women and men represented equally in the various structures at the school level (SF management committees; supervision committees; parent- teachers committees) / more specifically, what is the level of involvement and participation of women and girls and what are the reasons driving or inhibiting involvement? − How are these gender issues affecting programme implementation and results? − What are the key contributing factors for parent/community participation? * Number of functioning supervision committees (%) * Perceptions of committees’ effectiveness * Average contribution of parents (in-kind and monetary) * % of women/men participating in SFCs SF management and supervision committees * % of women/men leading in SF management and supervision committees * % of SF management and supervision committees where women/men, boys/ girls participate in decision-making *types and magnitude of barriers for women participation to school committees studies); implementing partners reports; Primary Data school directors; teachers; pupils SF management and supervision committees; parents; NGOs; gender specialists Primary Data: Key Informant interviews Focus Group discussions (with SF committees, teachers, pupils and parents) contribution from parents compared to school needs. Quantitative analysis of number of men/ women/ boys /girls in SM management and supervision committees Qualitative analysis of men/ women/ boys /girls’ role in decision-making in committees. contribution uneven Effectiveness based mostly on a small sample and stakeholder perceptions Variable quality of qualitative information sources and accessibility, e.g. some in paper format. 3.2 Does the programme have the potential to create sustainable income * Stakeholders’ perceptions/ideas of income generation activities Secondary Data: 2017 Evaluation of Local Purchasing and School Canteen Secondary Data: Document review Comparative qualitative data analysis of Good Strong documented evidence based on Final Report 67 | P a g e Questions and Sub￾questions Measures/Indicators Main Source of Information Data Collection Method Data Analysis Method Reliability & availability of data generation opportunities for communities? − What types of revenue generation opportunities has the programme helped create to date (e.g. farmers, cooks, others), and what is the quality and sustainability of these activities? − Are there other potential sources of revenue generation that could be fostered through the project? associated with the school meals programming in Haiti * Documented evidence from other programmes in Haiti and in other countries of successful income generation strategies/activities associated with school meals programmes Pilot Project in Petite Rivière et Nippes; WFP case studies of other countries. Primary Data: KII with Min Agric.; FAO; donors (USAID, Canada; Brazil); development NGOs (e.g. Care, AFC); farmers cooperatives Focus group discussions (1 with WFP and implementing partners’ monitoring staff; 1 with cooks; teachers, directors, pupils) Primary Data: Key Informants Interviews Focus group discussions Site visits to various income generation projects linked to school meals programmes documented evidence and stakeholders’ perceptions evaluations/studies . Stakeholders perceptions based on experience with successful strategies and knowledge of local socio-economic factors and culture. Evaluation Question 4. To what extent are the key institutions (national, provincial/district and local stakeholders; international and national implementing partners and NGOs; international donors and multilateral agencies working on school meals in Haiti) and governance structures able to effectively deliver, implement, sustain and/or scale up school meal interventions in the long￾term? Criteria Effectiveness, Sustainability, Efficiency Sub-Questions 4.1 To what extent have specific government capacities been strengthened so far? (expected outputs and outcomes) *Stakeholders’ perceptions of WFP contribution to capacity building of government entities Secondary Data: WFP Monitoring data and semi￾annual reports Primary Data: donors (Canada, USAID, WB); Min. of Educ/PNCS current and former staff, WFP programme staff Secondary Data: Document review Primary Data: Key Informant interviews Comparative qualitative analysis of documented evidence and stakeholders’ perceptions Good Documented by WFP in reports to McGovern-Dole. Stakeholders perceptions based on experience with successful strategies, Final Report 68 | P a g e Questions and Sub￾questions Measures/Indicators Main Source of Information Data Collection Method Data Analysis Method Reliability & availability of data innovation and pilot initiative’s and knowledge of local socio-economic factors and culture. 4.2 At what extent do Haitian national institutions (at central and decentralized levels) currently have the capacity to take over the programme fully or partially? − Is there a full or partial governance structure in place to implement, sustain and scale up school meal interventions in the long-term? − Are key national institutions willing and able to assume the responsibility for school meals within the originally agreed timeline, and what would a more ideal timeline be? − What would need to be in place to allow for a full transition to the Government of Haiti? − What would the cost be for the GoH and what different budget options might there be? *Evidence of government capacity in terms of human and financial resources; management structures and control mechanisms at the central, departmental and school levels. * Evidence of progress made on the SABER exercise completed in 2015 on the five different Strategic Objectives (Policy Framework, Financial Capacity and Coordination, Institutional Capacity, Design and Implementation, Roles of Communities) *Perceptions of national and international stakeholders on change in government capacities to manage a national SF programme *Examples in Haiti of successful school meals initiatives, pilot or otherwise, that could be replicated and contribute to understanding the potential makeup of sustainable national system Secondary data: National Policy and Strategy on School meals; SABER documents; WB/IADB financial studies Primary data: Min. Educ. (national & departmental); PNCS; donors (Canada, USAID, WB, IADB; WFP programme staff. Secondary Data: Document review Primary Data: Key Informant interviews Comparative analysis of recommended with actual government structure and human resources. Comparative quantitative analysis between the total cost of the SF programme at national level (including the food, transportation and administration) and available government budget for SF. Good Strong documented evidence based on the availability of evaluations/studies . Stakeholders perceptions based on experience with successful strategies and knowledge of local socio-economic factors and culture. Final Report 69 | P a g e Questions and Sub￾questions Measures/Indicators Main Source of Information Data Collection Method Data Analysis Method Reliability & availability of data Evaluation Question 5. What are the differences and advantages/disadvantages between the McGovern-Dole model and other approaches, focusing on nutritional and educational outcomes and community engagement, and how could it be adapted to increase effectiveness, sustainability, efficiency and long-term effects? Criteria Effectiveness, Sustainability, Efficiency Sub-Questions 5.1 How does the McGovern-Dole model/Theory of Change compare with other national and international school meals programmes with respect to its contribution to nutritional, educational and community outcomes? * Documented evidence of success factors of the McGovern-Dole Programme in Haiti * Documented evidence of other successful strategies * Stakeholder perceptions on additional strategies or activities that could be implemented in Haiti Secondary data: McGovern-Dole Haiti Programme reports; Evaluations of other school meals programmes (Canada, Nippes, World Bank, IDB, Kore Lavi; McGovern-Dole Program internationally); WFP M&E data Primary data: Donors (USAID, Canada); NGOs; WFP programme staff; McGovern￾Dole programme manager Secondary Data: Document review Primary Data: Key informant interviews Comparative qualitative analysis of outcomes of the McGovern-Dole programme and others in Haiti and internationally in terms of nutrition, education and community involvement. Good Strong documented evidence based on availability of evaluation reports/studies. Stakeholders perceptions based on experience with successful strategies and knowledge of local socio-economic factors and culture. 5.2 What other strategies, based on other experiences in Haiti or elsewhere, that might improve community involvement and sustainability? *Documented evidence of other successful strategies * Stakeholder perceptions on strategies could be implemented in Haiti Secondary Data: 2017 Evaluation of the homegrown project in Nippes; Case studies from other countries (WFP, McGovern-Dole). Primary Data: NGOs; gender specialists; growers’ cooperatives, WFP programme staff; donors (USAID; Canada; Brazil, WB, IADB); NGOs; school directors; parents and SF management and supervision committees, community members. Secondary Data: Document review and analysis Primary Data: Key informant interviews Site visit to homegrown project locations Focus groups discussion with parents and SF management and supervision committees Comparative qualitative data analysis of documented evidence and stakeholders’ perceptions Good Strong documented evidence, based on the availability evaluations/studies . Stakeholders perceptions based on experience with successful strategies and knowledge of local socio-economic factors and culture. Final Report 70 | P a g e Questions and Sub￾questions Measures/Indicators Main Source of Information Data Collection Method Data Analysis Method Reliability & availability of data 5.3 Are there opportunities to add or augment a nutrition sensitive approach in any of those models? *Documented evidence of other successful strategies from Haiti and other countries * Stakeholders’ perceptions of strategies could be implemented in Haiti Secondary Data: Micronutrient/nutrition studies conducted in Haiti; 2017 Evaluation of the homegrown project in Nippes; WFP case studies from other countries Primary Data: Min. Health, UNICEF and WFP nutritionists; school directors; teachers; school management committees, cooks, parents Secondary Data: Document review Primary Data: Key Informant Interviews Focus group discussion (with school teachers, pupils, management and school supervision committees, parents and cooks) Comparative qualitative data analysis documented evidence and stakeholders’ perceptions Good Strong documented evidence. Stakeholders perceptions based on experience with successful strategies and knowledge of local socio-economic factors and culture. 5.4 Has the programme used available resources efficiently? − Are budget ratios and sources for the school meals component in line with sustainability? − What would equitable solutions for the payments of cooks be and are there ways to implement this? * Number of planned versus actual schools supported by the McGovern￾Dole SF programme *Planned versus actual expenditure of the programme? *Cost per meal of this programme compared to other programmes * Planned versus actual contribution from parents * Documented evidence of other successful strategies from Haiti and other countries * Stakeholders’ perception of strategies that could be implemented in Haiti Secondary Data: WFP Standard Project Reports (2015, 2016, 2017); WFP project documents Primary Data School directors, cooks, teachers, SF management and supervision committees, donors; WFP programme staff, Other School Feeding Project calculations. Secondary Data: Document review Primary Data: Key informant Interviews Focus group discussions (with school management and school supervision committees and cooks) Comparative analysis of M&E data Good WFP reporting accurate and stakeholders’ perceptions based on experience and knowledge of socio￾economic factors and culture. Final Report 71 | P a g e Annex 3: Country Context 1. This section explores the relationship between school meals programming in Haiti and the broader socio-economic and political context, in which it is embedded. This section is meant to explain why school meals programming is important in Haiti and to better understand the factors underpinning potential success through the activity and challenges that have to be overcame. The section is also meant to better situate the particular focus of this evaluation, the contribution of McGovern-Dole to improving the delivery of school meals in three parts of the country. As well, this section helps to establish a framework for integrating a longer-term perspective into the evaluation process regarding a potential evolution in WFP supported school meals programming in Haiti. a. Macro-Economic Situation 2. Sharing the Hispaniola Island with the Dominican Republic, Haiti has a current population of approximately 11.24 million people, a considerable increase from the estimated 9.89 million people in 2013.73 The country’s population is the youngest in the Caribbean, with 42 percent 18 years of age or younger.74 It is also the most densely populated and increasingly urban country with greatest concentration of people found in metropolitan Port-au-Prince. Sixty percent of the population resides in urban areas, and this is expected to increase on average by 3.7 percent per year.75 Haiti remains the poorest country in the Americas and is commonly described as one of the most economically unequal. Based on a household survey in 2012, 59 percent of the population was living below the national poverty line of US$2.41 per day with more than 24 percent living below the national extreme poverty line of US$1.23 per day.76 Development assistance to Haiti peaked in 2010 in the aftermath of the earthquake. The years following the earthquake continued to see historic levels of development assistance. By 2015, a new trend began to take hold towards restoring Haiti’s traditional level of development assistance.77 3. The economic growth rate showed a modest increase from 1.2 percent in 2016-2017 to 1.6 percent in 2017-2018, owing primarily to the agricultural sector. However, the gourde has continued to depreciate against the US dollar and inflation remains high.78 The most recent report of the Economic Commission for Latin America and the Caribbean (ECLAC), concluded that Haiti’s macroeconomic performance for 2018 did not meet expectations.79 ECLAC estimated the growth of Haiti's Gross Domestic Product (GDP) to be at 1.4 percent with Inflation at 14.6 percent. The public deficit is 6.5 percent of overall GDP (3.9 percent in 2017).80 In February 2018, the International Monetary Fund (IMF) and the Government of Haiti (GoH) agreed to carry out economic and structural reforms to reduce subsidies for fuel and improve tax collection which was expected to lead to investments in infrastructure, health, education 73 http://worldpopulationreview.com/countries/haiti-population/ 74 2015b. “Population Totale, de 18 Ans et Plus. Ménages et Densités Estimes En 2015.” Port-au-Prince, Haïti: Institut Haïtien de Statistique et d’Informatique. 75 FEWS NET (2018) Haiti Staple Food Market Fundamentals. Famine Early Warning Systems Network. USAID (March 2018) 76 http://www.worldbank.org/en/country/haiti/overview#1 77 http://www.oecd.org/dac/financing-sustainable-development/development-finance-data/America-Development-Aid-at￾a-Glance-2018.pdf 78 http://www.worldbank.org/en/country/haiti/overview#1 79 https://repositorio.cepal.org/bitstream/handle/11362/43965/131/S1800836_en.pdf 80 https://repositorio.cepal.org/bitstream/handle/11362/43965/131/S1800836_en.pdf Final Report 72 | P a g e and social services. The reduction of fuel subsidies ignited broad-based protests that had a destabilizing impact on Haiti that carried over to 2019.81 The IMF reforms are part of a new reality faced by Haiti. The favourable terms of concessional financing once provided by Venezuela has ended. In March 2019, the IMF announced it had an agreement in place to extend a $229 million loan to Haiti with the expectation it would be used for school feeding programmes and for other social objectives.82 At the same time the political instability over the past three decades, has been accompanied by underperformance of government institutions, and poor governance and accountability. The fiscal deficit has resulted in low spending in key areas such as education, and health and sectors.83 4. In 2012, constitutional amendment Article 17.1 was introduced setting a minimum gender quota of 30 percent of women in all levels of public life. Despite this, Haiti remains the lowest ranked country in the Americas in terms of women's political participation.84 Haiti ranks among the worst performers in the world for women's representation in parliament: 187 out of 191 in the IPU ranking.85 There are only four women parliamentarians among over a hundred serving members. 5. Haiti adopted its first 20-year National Gender Equality Policy in 2014 and its first six-year action plan for 2014-2020; This policy builds on Haiti's Strategic Development Plan, which aims to make Haiti an "Emerging Country by 2030" through territorial, economic, institutional and, in particular, social transformation. "Ensuring gender equality" is the central axis and, in principle, guides the actions of all sectors and serve as benchmarks for technical and financial partners through established guidelines. 6. The Haitian economy suffers from a number of inefficiencies. For example, while Haiti has enormous renewable energy potential, there remain significant challenges to having access to clean and efficient energy sources. On average, 85 percent of Haiti’s electricity is derived from imported fossil fuels. At the household level Haitians rely on biomass such as charcoal and wood fuel as their main source of energy.86 7. Unemployment in Haiti is high, estimated at 40 percent. Small-scale agriculture and work found in the urban informal sector are the main sources of employment. Informal enterprises create the majority of new jobs with private formal sector employing only 7.5 percent of the labour force. Wages are low with about 70 percent of people earning less than the minimum wage, which by mid-2017 reached 300 Haitian gourdes.87 8. Women's access to steady and decent employment remains a challenge in Haiti where the majority of jobs are classified as informal, making it almost impossible to find decent working conditions. Overall, women are less likely to be in the labour market - the female participation rate in the labour market is 63.8 percent, compared to 72.6 percent for men.88 Women are mostly in the informal sector and poorly represented in formal jobs. Thus, 86.3 percent of women are self-employed with 75 percent of women working in the informal sector, compared 81 https://www.haitilibre.com/en/news-26501-haiti-economy-poor-performance-of-the-haitian-economy-in-2018.html 82 https://www.yahoo.com/news/imf-reaches-tentative-deal-haiti-229m-loan-150850176.html 83 https://www.haitilibre.com/en/news-26501-haiti-economy-poor-performance-of-the-haitian-economy-in-2018.html 84 UN ECLAC: Gender Equality Observatory for Latin America and the Caribbean. https://oig.cepal.org/en/indicators/legislative-power-percentage-women-national-legislative-body 85 Etat de la situation au 1er janvier 2019 : http://archive.ipu.org/wmn-f/classif.htm 86 https://www.export.gov/article?id=Haiti-Energy 87 FEWS NET (2018) Haiti Staple Food Market Fundamentals. Famine Early Warning Systems Network. USAID (March 2018) 88 PNUD (2018). Human Development Indices and Indicators: 2018 Statistical Update. Briefing note for countries on the 2018 Statistical Update Haiti. Final Report 73 | P a g e to 39 percent of men (EMMUS V). Furthermore, 40 percent of households in Haiti are headed by women.89 Women also account for 57.7 percent of the permanent family labour force and 40 percent of caregivers.90 Their disadvantaged position in the labour market can be partly explained by the unequal distribution of household chores: they report spending twice as much time on domestic work (15.4 hours per week) as men (7 hours per week). Overall, 48.4 percent of women are economically active compared to 66 percent of men.91 b. Human Development 4. In 2017, Haiti’s HDI value stood at 0.498. This placed Haiti at the lower end of the scale at 168 out of 189 countries.92 Roughly 80 percent of the 2.5 million Haitians living in extreme poverty are to be found in rural zones with the Nord-Est and Nord-Ouest Departments recording the highest levels of poverty. About 48 percent of Haitians aged 15 years or more find it necessary to borrow money. The main source of finance is family and friends, followed by private lenders.93 According to EMMUS VI, about three quarters of households (74 percent) have access to an improved source of drinking water, 95 percent in urban areas and 60 percent in rural areas. Furthermore, only 41 percent of households have access electricity at home at national level, with only 17% in rural areas compared with 76% in urban areas. 5. Further complicating this situation, about 40 percent of the population does not have access to essential health services. Preventable conditions such as respiratory infections, diarrheal diseases, tuberculosis, preterm birth complications, and birth￾related complications are among the top causes of death.94 The value of the Gender inequality Index (GII) is high, 0.601, which ranked 144 out of 160 in 2017.95Inequalities between men and women, issues related to reproductive health, empowerment and the labour market partly explain this negative value. 6. In terms of women's reproductive health, the data is alarming. The Pregnancy-Related Mortality Report is 646 deaths per 100,000 live births, and the maternal mortality ratio is 529 deaths per 100,000 live births. With regard to sexual and reproductive health, among15-19-year-olds, 52 percent of girls have already started their reproductive life. The birth rate for teenagers is 37.5 births per 1,000 women aged 15-19. 96 c. Education Sector 6.1.1. Educational Standards 7. In Haiti, while all children have the right to free and quality education, illiteracy and low educational achievement remain major concerns. According to the Ministère de l'Éducation Nationale et de la Formation Professionnelle (MENFP), 74 percent of women 89 World Atlas (2018): https://www.worldatlas.com/articles/female-headed-households-in-the-developing-world-and￾middle-east.html. 90 MARNDR (Ministère de l’Agriculture, des Ressources naturelles et du Développement rural) (2012). Recensement général de l’agriculture (RGA) 2008-2009. Résultats provisoires. Département de l’Ouest, Port-au-Prince, 213 p. 91 MAST (2016). Evaluation des besoins post cyclone Mathieu, secteur emploi et moyens de subsistance. 92 http://hdr.undp.org/sites/all/themes/hdr_theme/country-notes/HTI.pdf. 93 FEWS NET (2018) Haiti Staple Food Market Fundamentals. Famine Early Warning Systems Network. USAID (March 2018) 94 FEWS NET (2018) Haiti Staple Food Market Fundamentals. Famine Early Warning Systems Network. USAID (March 2018) 95 PNUD (2018). Human Development Indices and Indicators: 2018 Statistical Update. Briefing note for countries on the 2018 Statistical Update Haiti. 96 Cayemittes, Michel et al (2016). Enquête Mortalité, Morbidité et Utilisation des services ((EMMUS-VI 2016-2017) Haiti. Calverton, Maryland, USA: MSPP, IHE et ICF International. Final Report 74 | P a g e and 79 percent of men are literate but only 14.7 percent of people in the 15-49 age group are able to read an entire sentence.97 According to EMMUS VI,98 13 percent of women and 9 percent of men have never attended school and a UNICEF report estimates that half of children will not reach the 6th grade.99 Additionally, students often enter school late, sometimes as old as 12 years old, with high rates of repetition, 15 percent on average, and, in some cases, multiple times, further increasing the number of over-aged children.100 This contributes to an already problematic situation of overcrowding further entrenching school system inefficiencies. The situation is generally more problematic in rural areas where attendance starts decreasing sharply after the 4th grade.101 Factors leading to exclusion are diverse but the most important is the cost of studying since 76 percent of schools ask for some kind of payment, even if education is officially free. Parents still need to buy uniforms, books and other material.102 8. At the primary and secondary level, girls' school attendance has recently been higher than boys. While primary school attendance is roughly the same (85 percent girls to 83 percent boys), by high school 51 percent of girls and 41 percent of boys still attend school. The opposite trend is observed for higher education 6.1 percent for women aged 35 to 39 years against 11.8 percent for men. Progress in the first two cycles does not translate into gains for girls either in terms of higher education or the job market as they too often are involved in domestic work. Early pregnancy is another factor, preventing girls from continuing their education. ii Improving Academic Performance 9. The MENFP has tried to improve student performance through such means as instituting a national training policy for teachers and educational personnel. Additionally, the GoH has prioritized primary education and since 2007, has subsidized school fees through the programme “Éducation Pour Tous’ (EPT). As well, starting in 2011, the GoH has been financing the Programme de Scolarisation Universelle Gratuite et Obligatoire (PSUGO). At their peak in the 2011 to 2014 period, these two programmes financed 60 percent of the cost of tuition helping to boost school attendance. But recently, both the EPT and PSUGO suspended school fees subsidies shifting the responsibility onto families.103 In August 2014, the GoH announced 12 policy measures to “stabilize and improve governance and quality in the education system.” Despite these efforts, problems in terms of teacher competence and their motivation persisted leaving a generation of Haitian young people without basic skills and knowledge.104 Only 15 percent of teachers are considered qualified to teach.105 Teacher pay, teaching methods, and validity of teachers’ credentials, are some of the concerns identified. An evaluation conducted in 2009 found that the average third-year student could only read 23 words per minute, which 97 MENFP 2016. 98 Institut Haïtien de l’Enfance (IHE) et ICF. 2018. Enquête Mortalité, Morbidité et Utilisation des Services (EMMUS VI) en Haïti 2016-2017 : Rapport de synthèse. Rockville, Maryland, USA: IHE et ICF. 99 UNICEF (2018), Étude sur les enfants hors de l’école en Haïti - Rapport final 100 http://haiti-progres.com/news/2018/05/09/community-schools-for-haiti-broken-school-system/ 101 UNICEF (2018), Étude sur les enfants hors de l’école en Haïti - Rapport final 102 UNICEF (2018), Étude sur les enfants hors de l’école en Haïti - Rapport final 103 MENFP 2016 104 http://ww.thedialogue.org/PublicationFiles/Education%20in%20Haiti%20-%20The%20Way%20Forward%20- %20FINAL%20-%209-15-08.pdf 105 http://haiti-progres.com/news/2018/05/09/community-schools-for-haiti-broken-school-system/ Final Report 75 | P a g e is well below the estimated speed of 35-60 words per minute needed for understanding basic text. In 2015, a mathematical assessment pilot project conducted by MENFP in public and non￾public schools found student scores were only slightly above what would be achieved through random responses.106 10. The challenges faced by the public education system has resulted in non-state actors filling gaps operating almost all pre-primary schools, more than 80 percent of primary schools, and more than 70 percent of secondary schools.107 There are tuition fees for sending children to privately run schools. Nineteen percent of schools operate in school buildings, the rest make due in private houses, churches, huts or tents. According to the 2010-11 census, only 54 percent of urban primary schools and 33 percent of rural schools had a source of drinking water and less than 20 percent were electrified.108 11. School curriculum in Haiti is seen as not being well-adapted to the needs of students, especially those living in rural areas. Students that leave school early cite a lack of support as one of the primary reasons for dropping out. This is often tied to the fact parents are not in a position to assist them academically.109 Given the largely rural/agricultural background of a good part of the student population, it is believed that opportunities are missed to better connect the agrarian culture of Haiti with the school system to possibly better train children for farming related livelihoods. In general, it is felt that Haitian schools should be striving to become more relevant to the children and communities they are expected to serve.110 d. Food Security and Nutrition 12. Haiti ratified the International Covenant on Economic, Social and Cultural Rights (ICESCR) on October 8, 2013. The ICESCR recognizes the "fundamental right to be free from hunger". Despite this, malnutrition remains a significant concern in Haiti, with more than 3 million people facing severe or moderate chronic food insecurity.111 According to the Institut Haïtien de l’Enfance (IHE), 22 percent, or 264,000 children under 5 years, suffer from chronic malnutrition (stunting or low height-for-age) and 66 percent, or Food and nutrition insecurity disproportionately affects women due to among other things, their education levels, social role and socio-economic status.112 13. The persistent risk of food insecurity combined with other vulnerabilities undermine access to food such as environmental disasters will continue to make Haitian children vulnerable to malnutrition. According to the Technical Centre for Agricultural and Rural Cooperation (CTA), the level of food insecurity is higher in rural areas (49.6 percent of households) than in Port-au-Prince (39.9 percent of households) and other cities (34.6 percent of households). The CTA has concluded that Haitian farmers are unable to 106 MENFP 2016 107 MENFP 2016 108 MENFP 2016 109 http://haiti-progres.com/news/2018/05/09/community-schools-for-haiti-broken-school-system/ 110 http://haiti-progres.com/news/2018/05/09/community-schools-for-haiti-broken-school-system/ 111 Cadre Intégré de Clasification de la Sécurité Alimentaire (IPC) (2015) 112 IPC Analyse de l’Insécurité Alimentaire Aigue Décembre 2018. file:///F:/Consultations/EVALUATION%20PAM%202019/DOC%20travail/ipc_haiti_dec_18__fiche_de_communication_versio n_finale_1_.pdf Final Report 76 | P a g e produce enough food for their own needs.113 According to the Integrated Food Security Phase Classification (IPC), underlying factors of chronic food insecurity are households’ lack of purchasing power, the lack of self-production, inadequate food use habits and low human, natural and physical capital. 114 The National Council for Food Security (CNSA), in its report of the last quarter of 2018, indicated that food shortage is threatening 2.6 million people in Haiti, and an additional 571,000 between March and June 2019 if the situation persists. 14. EMMUS VI found that the nutritional the majority of girls and boys have a normal BMI (87% and 86% respectively). However, it also found nearly 65% of children under 5 years suffer from anemia while 32% of children suffer from vitamin deficiencies. For iodine deficiency, 72% of children aged 6-12 in rural areas and 52% of urban children are affected. Chronic malnutrition is correlated to poverty status and the mother’s level of education. 115 e. Agriculture 15. While remaining the anchor of the Haitian economy, the agricultural sector is nonetheless under duress. In 2017, at the peak of the spring harvest, 3.6 million people were considered to be food insecure and 1.5 million were severely food insecure.116 This occurred in the aftermath of consecutive years of inconsistent agricultural production. A recent WFP and FAO mission to Haiti found three main issues cited by farmers across the country for the lack of production including poor seed distribution, the ineffectiveness of irrigation infrastructure and equipment and finally, a third and surging issue is the decline in the availability of agricultural labour. The low return on agriculture production and unpredictable harvests in recent years has caused people to question farming as a livelihood. This is despite the fact that it is a sector where work is available and product demand is high.117 Since opening the agricultural market in the early 1980’s to imports, Haiti has remained strongly depended on food imports including basic staple foods importing on average are 700 million U.S dollars annually.118 These economic policies favouring food imports has been done so at the expense of small-scale producers. 16.Agricultural production is predominantly rainfed in Haiti and most of it takes place at the small landowner level. Over 95 percent of the production units in the country rely on family labour and productivity levels are low.119 According to the most recent agricultural census, Haiti counted about 1 million smallholders in 2009, of whom about 25 percent were women. Tenure insecurity presents a large problem in Haiti the vast majority of women (71%) do not own any land or housing property and are 113 Duvivier & Fontin (2017) Building the evidence base on the agricultural nutrition nexus: Haiti. Series Agriculture and Nutrition. CTA Working Paper 17/9. Duvivier, Predner and Marie Lesly Fontin. 114 Cadre Intégré de Classification de la Sécurité Alimentaire (IPC) (2015) 115 EMMUS VI (2017) 116 FAO and WFP. 2017. Special Report: FAO/WFP Crop AND Food Security Assessment Mission to Haiti. Available at: https://reliefweb.int/sites/reliefweb.int/files/resources/a-i8279e.pdf 117 FAO and WFP. 2017. Special Report: FAO/WFP Crop AND Food Security Assessment Mission to Haiti. Available at: https://reliefweb.int/sites/reliefweb.int/files/resources/a-i8279e.pdf 118 http://coeh.eu/which-agricultural-policy-for-haiti/. 119 FEWS NET (2018) Haiti Staple Food Market Fundamentals. Famine Early Warning Systems Network. USAID (March 2018) Final Report 77 | P a g e disadvantaged when it comes to inheritance practices.120 In 2011, 39 percent of rural households were female headed.121 17. However, women represent only 15.2 percent of the total 61 million "man-days" in the paid labour force in agriculture. This is usually related to occasional major seasonal operations (sowing, harvesting) that request additional labour and not a stable source of income. On the other hand, women are involved in food processing through women groups and cooperatives and have skills in processing raw products. However, these tasks are carried out with basic tools and technologies, which do not help women reduce their workload. Youth (up to 29 years of age) account for approximately 11 percent of all agricultural producers. This would indicate young people are not committing to agricultural the way previous generations have. Encouragingly, the GoH recently identified agribusiness and the expansion of agricultural investment as a priority for economic development.122 As well, agricultural land in Haiti remains relatively fertile, with the soil less impacted by the use of chemical fertilizers and pesticides.123 f. Environment and Climate Change 18. Haiti ranks 4th on the Long-Term Climate Risk Index (CRI). Haiti is recurrently affected by catastrophes and continuously ranks among the most affected countries both in the long-term index and in the index in recent years.124 Climate change and ongoing environmental degradation in Haiti have completely altered the agricultural-food security equation creating a new and much more challenging and unpredictable context. Climate change has come into full focus as a destructive force having broad economic and social impacts. Climate change in Haiti is occurring in a highly environmentally degraded context. Despite the ongoing dependence on wood as a source of fuel, it is estimated that only one to two percent of the country’s forest remains intact where it was once estimated that 60 percent of the country was covered in forest.125 In addition, across the country water shortages are widespread with as much as half the country not having access to clean drinking water. 19. Large scale environmental disasters, primarily the earthquake in 2010, Hurricane Sandy 2015 and Hurricane Mathew 2016, have caused widespread damage leaving the country constantly rehabilitating the agriculture sector.126 These major events come in addition to the seemingly endless cycle of droughts, storms, and floods that have become a common occurrence in increasingly unpredictable climatic conditions. Drought was experienced in three consecutive years from 2012 to 2015.127 The lack of a broad-based capacity to adapt and build resilience to climate change is affecting agricultural outputs. Four of the last five agricultural seasons have been impacted by 120 Millennium Development Goals (MDG) Report, 2013. 121 FAO (2011) The State of Agriculture, Closing the Gender Gap. 122 https://www.export.gov/article?id=Haiti-Agricultural-Sector 123 https://www.export.gov/article?id=Haiti-Agricultural-Sector 124 Germanwatch, Global Climate Risk Index 2019, Briefing Paper. 125 https://www.oxfam.org/sites/www.oxfam.org/files/rr-climate-change-resilience-haiti-260314-fr.pdf 126 FAO (2017) Haiti and FAO Strengthening Food and Nutrition Security and Enhancing Rural Incomes Aug 2017 FAO Bulletin. Food and Agricultural Organization 127 Haiti Emergency Food Security Assessment, Haitian National Council for Food Security and World Food Programme; December 2015 Final Report 78 | P a g e erratic weather. The increasing climate uncertainty is making it more challenging for farmers to remain committed to farming.128 g. School Meals Programmes in Haiti 20.Haiti has a long history with school feeding initiatives. Historically and more critically in recent times due to disruptions in the agricultural sector, school feeding activities have established themselves as having great importance for not only academic performance but also in addressing issues related to child nutrition. It first began with the assistance of WFP in the 1980’s at about the time that agricultural reforms were introduced leading to increased dependence on imports. In 1997, the GoH created the Programme National des Cantines Scolaires (PNCS), the entity in charge of administering and regulating the national school meals programme. This was a first step in a transition towards greater government involvement in the activity. Efforts to build a sustainable national programme made progress between 2001 to 2006. 21.In subsequent years political instability led the PNCS and other nutrition related activities to be severely reduced. In the period leading up to the earthquake of 2010, some progress was made in re-establishing the PNCS through increased operations and strengthening its role in overseeing school feeding activity in Haiti. In the immediate aftermath of the earthquake, the PNCS was identified as one first entities to respond to the crisis through food distribution. Stakeholders recognized the important role the PNCS played in rebuilding the education sector. School feeding became an integral part of the National Plan for the Reconstruction of Haiti. At about the same time it was identified as playing a role in assuring the sustainability of Haiti’s agriculture sector. 22.After a study initiated in 2011 by the GoH with the support of WFP and the Government of Brazil, it was determined that it was critical that Haiti develop a national school feeding strategy. This has been captured in the Politique et Stratégie Nationale d’Alimentation Scolaire (PSNAS), that was completed in 2016. There is strong consensus regarding the importance of providing children with balanced and healthy meals to ensure their nutritional needs are being met, while attending school. 23.By defining the cornerstones of the school feeding programme; implementation modalities, role of respective actors and potential financing mechanisms, the PSNAS was expected to reinforce the PNCS and broaden its field of action beyond the canteens to embrace all components necessary for the successful implementation of a national school feeding programme.129 At the time of its development, there was a hope that the PSNAS would lead to additional support to ensure the success of the PNCS, notably from MENFP. In fact, the PSNAS was developed, under the impetus of the MENFP, through a participatory and multisectoral process built on consensus reached by all stakeholders within the sector.130 However, despite past and recent efforts, and PNCS leadership, school feeding in Haiti is still almost entirely dependent on donor contributions and on implementing partners for its day-to-day operations. 128 FAO & WFP (2017) 129 MENFP (2016) Politique et Stratégie Nationales D’Alimentation Scolaire (PSNAS). Le Ministère de l'Éducation nationale et de la Formation professionnelle (MENFP) Programme National de Cantine Scolaire (PNCS). 130 MENFP (2016) Politique et Stratégie Nationales D’Alimentation Scolaire (PSNAS). Le Ministère de l'Éducation nationale et de la Formation professionnelle (MENFP) Programme National de Cantine Scolaire (PNCS). Final Report 79 | P a g e 24.For example, in addition to the McGovern-Dole grants, between 2012 and 2017 WFP implemented the “Assistance to the National School Feeding Programme in Haiti”, with an approved budget of USD 124 million, which sought to build the Haitian capacity to manage the national school feeding programme, while distributing mid-day hot meals to 485,000 school aged children in the Haitian public school system. In addition, between 2013 and 2019 WFP, World Vision, Action Against Hunger and CARE administered a multimillion dollar USAID programme, KORE LAVI, with MAST (Ministère des Affaires Sociales et du Travail). Originally scheduled to end in September 2017, USAID extended the programme for two more years after Hurricane Matthew in 2016, which devastated homes and food crops in many regions of the Caribbean country. These programmes were complementary in terms of locations to cover as much of the country’s needs as possible. See Figure 6, in Annex 4. France has also bought food in the Haitian market on an annual basis to help feed 55,000 children, distributed by the PNCS. However, based on interviews with the Agence française de développement, during school year 2018-2019, food was distrusted to schools via other operators. While the PNCS reported not having food to distribute in the school year 2018-2019, the PNSC has recently secured funding from the Fonds National pour l’Éducation to provide a food for more 8,000 children enrolled in the Lower North-West and some communes in southern Haiti during the 2019-2020 school year.131 131 https://www.icihaiti.com/en/news-28227-icihaiti-education-fne-helps-more-than-8-000-children-at-risk-of-food￾insecurity.html. Final Report 80 | P a g e Annex 4: Detailed description of subject of the evaluation 24.1. Budget 1. The McGovern-Dole (McGovern-Dole) funded school meals programme is embedded in the WFP Haiti T-ICSP January 2018-June 2019. The evaluation covered two McGovern-Dole grants that supported a total of 190,000 primary students across almost 600 schools132 About half of the supported schools (49 percent) were located in the Cap-Haïtien, North and North East development corridor, 32 percent were located in the St-Marc, Artibonite corridor and 19 percent were in the Port-au-Prince, West and Centre corridor as Figure 1 shows. − The FY15 grant amounted to USD 10,000,000, and was approved on December 21, 2015 and covered the 2016-17 schoolyear; − The FY16 grant amounted to USD 14,000,000, and was approved on October 26, 2016 and covered the 2017-18 and 2018-19 schoolyears (ending in June 2019); 2. The budget amount for the McGovern-Dole school feeding activities was USD 24,000,000 covering 3 school years in total. No budget revision were reported. Quantities donated were revised to correspond to changing costs of commodities, transportation and financial assistance. Table 5: Budget awarded and total expenditure, by year FY15 – 2016/17 FY16 – 2017/18 FY16 – 2018/19 Budget Awarded USD 10,000,000 USD 6,999,823 USD 6,895,336133 Total Expenditure USD 9,471,551 USD 7,104,664 USD 3,748,427 24.2. Programme Objectives 3. The McGovern-Dole programme funding encompassed two main WFP Strategic Objectives (SO): 24.2.1.1. SO1: Improved Literacy of School-Aged Children; 24.2.1.2. SO2: Increased Use of Health and Dietary Practices 4. These SO are linked to the 3 main objectives of WFP School Meals in Haiti and worldwide: i. Help to ensure that primary school children remain in school to improve their education and nutritional levels; ii. Reduce micronutrient deficiencies and particularly the incidence of anemia; and iii. Enhance the capacities of the Government to enable it to continue to expand its national school meals programme in a sustainable manner. 5. Figure 6 provides an overview of the three McGovern-Dole programme corridors: North and North-East, Gonaives/St-Marc, Centre, and the West regions, including Port-au￾Prince. 132 As of 2019 the number of schools were a little over 400. 133 Calculated based on Year 1 of the FY16 grant (see Second Year Letter of Notification) Final Report 81 | P a g e Figure 5: Map of intervention areas: North, North-East, Gonaives/St-Marc and Port-au-Prince corridors Final Report 82 | P a g e 6. Figure 7 shows the distribution of McGovern-Dole schools by corridor in percentage and Figure 8 the number of schools assisted by year. Figure 6: Distribution of McGovern-Dole supported schools by location, 2018-19 Source: WFP Haiti Country Office Figure 7: Number of McGovern-Dole Schools, school year 2018-19 Source: WFP Haiti, Monitoring data 24.3. Theory of Change of the McGovern-Dole School Meals Programme 7. A key objective of the evaluation was to examine whether the current theory of change was effective in improving the academic achievements of Haitian primary school children, though investigating the causal relationships among the different levels of the results chain, key success factors and challenges encountered that explained the results achieve to date. It also examined whether other elements/activities could lead to stronger results and sustainability. See the methodology section for a list of documents that were reviewed as part of this objective. 8. The theory of change for WFP’s McGovern-Dole funded school meals rested on evidence that providing a nutritious hot meal to children early in the day would help children learn better.134 According to the World Health Organization (WHO), evidence suggests a causal 134 Just, David (2014), https://articles.extension.org/pages/68774/3-ways-nutrition-influences-student-learning-potential-and￾school-performance#_ednref8 Cornell Center for Behavioral Economics in Child Nutrition Programs: https://articles.extension.org/pages/68774/3-ways-nutrition-influences-student-learning-potential-and-school￾performance#_ednref8. Corridor Port-au-Prince WEST CENTRE… Corridor Saint-Marc ARTIBONITE 32% Corridor Cap Haitien NORD NORD-EST 49% 650 382 382 595 497 401 0 100 200 300 400 500 600 700 2016-2017 2017-2018 2018-2019 Planned Actual Final Report 83 | P a g e effect between healthy food and improved nutrition and improved learning ability, leading to better academic performance.135 Hence, the school meals programme in Haiti primary activity was to provide a basic nutritious meal to cycle 1 and 2 children in schools in food insecure locations in the country. It was assumed that the meal would be enhanced by parents’ cash or in-kind contributions, e.g. condiments, fuel, etc. The assumption was that community engagement would contribute to the programme outcomes and increase local management of the programme activities, thus contributing to its sustainability. To support engagement from parents and community, awareness-raising activities were conducted for all the participating schools. 9. Furthermore, numerous studies have found evidence of positive outcomes related to diarrhoeal disease from WASH interventions in schools.136 Therefore, the addition of a WASH component to the school meals were designed to prevent nutrition loss through water-borne diseases and prevent absences from school. In addition to WASH activities such as providing soap, clean water, and teaching good hygiene habits like hand washing, children were given deworming tablets every six months. 10. Finally, there is evidence that literacy lead to both academic and health outcome. There is evidence that low literacy levels in the early grades leads to a range of poor outcomes, both academic and health wise.137 In 42 schools, a literacy component was added to the curriculum to test the assumption that early mastery of Creole and French, in addition to school meals would lead to better academic outcomes for those children. 11. The theory of change also assumed that providing technical assistance at the national and local level in the governance and management of school meals programme would lead to the sustainability of the programme over time. Technical assistance to the Ministry of Education and the Programme National des Cantines Scolaires (PNCS) was designed to develop a national strategy and an implementation manual to lead to a gradual transfer of school meals programming to the government of Haiti. At the local level, technical assistance to schools (directors, school meals management and supervision committees) and to departmental education authorities was designed to 12. The theory of change is captured in a Logic model and Results Framework for McGovern-Dole programming in figures 10, 11 and 12 below. A summary of the key activities that support the theory of change is provided below, followed by some key outputs and outcomes gathered by the Programme. 24.4. Key Activities 13. School Meals: Since the school year 2016/17, the programme has provided school meals made of bulgur, green peas, fortified vegetable oil (+ salt),138 to children in primary, mostly public, schools in five departments (Artibonite, North, North-East, Centre and West). The grain, pulse and oil are provided by McGovern-Dole in-kind donations. The programme 135 World Health Organisation: http://www.euro.who.int/__data/assets/pdf_file/0019/152218/E89501.pdf 136 McMichael, C, (2019). Water, Sanitation and Hygiene (WASH) in Schools in Low-Income Countries: A Review of Evidence of Impact, International Journal of Environmental Research and Public Health: https://www.mdpi.com/1660-4601/16/3/359/html. 137 Gray, Abigail; Goldsworthy, Heather; May, Henry; and Sirinides, Philip. (2017). Evidence for Early Literacy Intervention: The Impacts of Reading Recovery. CPRE Policy Briefs: https://repository.upenn.edu/cgi/viewcontent.cgi?article=1083&context=cpre_policybriefs 138 FY15 had a snack component as well. Final Report 84 | P a g e also provides storage & food preparation equipment, tools and eating utensils. It trains all actors implementing the school feeding programme (Directors, Teachers and Administrative Staff, Cooks and Supervisory Committees) on key skills necessary to ensure a smooth and safe implementation of the school feeding programme (e.g. school feeding management, supply chain, storage and food preparation). In addition, it also builds capacities at community level by training cooks (usually local women) on food safety, preparation and handling to reduce health-related problems in general, by training school staff on child nutrition and health and sensitising parents on the importance of healthy food and regular school feeding. 14. The programme is implemented by WFP and national civil society organizations, contracted as implementing partners monitor all activities at school level, e.g. ensure that food stocks are complete and stored according to WFP standards, and that prepared rations match student attendance. They also monitor parents in-kind and cash donations, hygiene and health activities implemented and activities of school meals management and supervision committees. 15. WASH: From 2018, the international non-governmental organization (INGO) World Vision has provided water purification tablets in 110 schools, every two months, sensitizes communities and train school directors and teachers to improve hygiene practices to prevent diseases. Some of these schools received water chlorine purifiers with replaceable batteries and solar panel battery chargers. World Vision also trained WASH working groups, made up of community members, and coordinated with these groups to ensure that handwashing stations in schools meet national standards. From November 2018 to February 2019, WFP-Haiti distributed water purification tablets in 144 schools with the poorest water quality to improve hygiene practices and prevent diseases. This distribution continued until the end of current school year. In February 2018, WFP distributed soap in 401 schools to improve hygiene practices and prevent diseases. 16. Literacy: Since July 2017 the Catholic Relief Service (CRS) has been carrying Literacy Activities ‘Map Li Net Ale’ in 42 public schools receiving school meals in the West Department. These activities consisted mainly in improving reading and writing skills in French and Haitian Creole, training teachers and administrators on professional teaching modules in bi-monthly coaching sessions, as well as distributing reading materials developed by USAID for both the teachers and students. Progress in oral French and Haitian Creole was measured through the EGRA tests, taken periodically (May and November 2018, while a third one took place in May 2019) for children in the first and second grade. To strengthen the program, CRS also organized so-called ‘reading camps’ for less advanced students which involved a system of tutoring. It also provided library and school supplies to participating schools. 17. Capacity Building activities: Development of activities to address gaps in capacity at the national and local level. WFP supported the roll-out of the National School Feeding Policy and Strategy (PSNAS), with a dedicated senior technical advisor working directly in the Ministry of Education and providing orientations to the PNCS coordination team. In October 2017, a draft version of the National School Cantine’s Operation Manual was submitted to the Minister of Education. WFP supported PNCS with office equipment (computers, office furniture) and 3 vehicles to reinforce the capacities and working conditions in their head and field offices. WFP facilitated the participation of Haiti in the Final Report 85 | P a g e 8 th Regional Latin-American and Caribbean School Feeding Seminar. Figure 9 provides an overview the number of schools by programme activities. Figure 8: Number of schools by activity, school year 2018-19 Source: WFP Haiti 24.5. Partners 18. The key governmental partners involved in the intervention included the Ministère de l’Education Nationale et de la Formation Professionnelle (MENFP) and the Programme National des Cantines Scolaires (PNCS) - the body mandated to coordinate Haiti’s National School Feeding. According to the Nacional School feeding Policy, PNCS role is normative and to ‘coordinate, control, monitoring and evaluation, resource mobilisation, contract management in a strategy of decentralized implementation’. 19. WFP implemented the programme in the field currently (school year 18/19) through five operating partners: Ananda Marga Universal Relief Team (AMURT), La Société d'Agence de Secours et de Bienfaisance aux Enfants Défavorisés (ASEBED), the Bureau diocésain d'éducation Port-au-Prince (BDEPAP), the Bureau de Nutrition et Développement (BND), and World Vision Haïti (WVI). They were in charge of food distributions (only BND and WVI), monitoring and reporting and on reinforcing school committees’ capacities. Two additional partners were in charge of implementing specific activities: Catholic Relief Service (CRS) implemented literacy activities, while World Vision International was in charge of the water and sanitation (WASH) activities to improve health and hygiene. 20. The school community was also an important actor at local level for the management of day-to-day operations. It includes principals, teachers and parents; mainly those who participate in the school meals committees: Comité de supervision and Comité de gestion de la cantine scolaire, the main school/community bodies involved in the delivery of school meals and the supervision of the programme. Parents were supposed to provide cash or in-kind contributions to complement the school meal rations given by the programme. 21. The private sector was mainly represented through transport companies, even though these were contracted via WFP for school year 2017/18. UNICEF and the World Bank were key technical partners in the field of education and school meals, implementing related activities and supporting the programme through coordination and exchange of expertise. 24.6. Gender dimensions of the intervention 401 289 42 110 144 401 204 Total Number of schools attended Number of Public Schools Number of schools attended with literacy activities Number of schools attended with WASH activities # of schools receiving Water purification tablets # of schools reciveiving soap No of schools that received MNP (2017) Final Report 86 | P a g e 25. All children in a community were entitled to receive the same education and same meals, regardless of their gender. McGovern-Dole is not different from other school meals programmes in the sense that it delivers identical food rations to boys and girls. However, UNICEF reports that girls tend to enter the school system at a later age and are more likely to be excluded to support the economy of their homes, what implies that school meals could contribute to reduce the entry age gap.139 26. Also, through its intervention, McGovern-Dole programme aimed at advancing towards gender equality through women participation and leadership in the school committees and better involvement in the management of school meals. It also looked to empower volunteer women cooks by giving them specific training and providing adapted conditions for cooking. 27. The Haiti country office measured all data of interest with gender disaggregation such as the number of beneficiaries, the number of persons trained or education outcome indicators. Some specific indicators were also measured such as the number of women in school committees, including in leading positions. 28. In March 2019, the Country Office (CO) hired a gender specialist to undertake a comprehensive analysis on roles and responsibilities, access and control of resources, decision-making and participation, vulnerabilities and opportunities for women, men, boys and girls in all aspects of the School Meals Programme. g. Activities Implemented 29. The key activity Provide School Meals was done by WFP in collaboration with partners via the provision of a daily hot meal. The partners ensured that quantity and quality of the school meals were in line with WFP and GoH standards. The partners also trained and monitored School Management Committees (SMCs) to ensure that commodities were safely stored and appropriately used for the programme. 30. WFP Haiti developed guidance material for school cooks regarding safe food handling and hygiene. WFP cooperating partners trained up to 3 cooks per school on nutrition and healthy cooking as well as hygiene on a yearly basis. 31. The trainings on food preparation, storage practices and good health and nutrition that complement the provision of school meals were conducted on a yearly basis. In October 2018 WFP organized a training of trainers for 102 partner staff, with training material approved by the Ministry of Health. The partner staff subsequently trained school directors in assisted schools. 32. Storage and Food Preparation Equipment, Tools and Eating Utensils were provided to all schools over the course of the project. 33. WFP supported the roll-out of the National School Feeding Policy and Strategy, with a dedicated senior technical advisor working directly in the Ministry of Education and providing orientations to the new National School Canteens Programme (PNCS) coordination team on the various school meals modalities implemented by operating partners. In October 2017, a draft version of the National School Cantine’s Operation Manual was submitted to the Minister of Education for his orientations on the next steps of the validation process. 34. Whenever possible WFP and its partners conduct monitoring visits jointly with the local representatives of the National School Feeding Agency (PNCS). 139 UNICED, out-of-school Final Report 87 | P a g e 35. For school year 2018-2019, WFP rolled out a strategy to include all local representatives of the Ministry of Education (MENFP) the Departemental Education Directors (DDE) into the management of school meals and support on disciplinary actions against directors. 36. Activities related to literacy were implemented in partnership with Catholic Relief Services (CRS) in 42 public schools in Port Au Prince corridor. During the reporting period, the teachers and school directors were trained on the literacy curriculum according to “Ann Ale” and classroom management. Creole and French reading kits (Ann Ale materials, composed of 3 reading materials for first grade and 4 reading materials for second grade) were distributed to 5,791 students in the first and second grade classes, and 1,103 teaching materials were distributed to 112 teachers and 42 school directors. CRS also conducted studies to evaluate the outcome of the literacy interventions using the EGRA methodology. These studies informed an integral part of the final evaluation. 37. Distribution of water purification tablets and soaps as well as trainings on Health and Dietary Practices were implemented in partnership with World Vision (WVI). The additional distribution of Water Chlorine purifiers to select schools started in February 2018. They were distributed in 110 schools in the 3 corridors. 38. In 2017 and until end of February 2018 Micronutrient Powders (MNPs) were used to enrich school meals in 204 selected schools for 63,000 children, in McGovern-Dole development corridors of Saint Marc and Port-au-Prince. This was part of a pilot project implemented in partnership with Nutrition International (NI) to measure the influence of MNPs on the reduction of micronutrient deficiencies, particularly anemia, in school children in Haiti and to assess their acceptability. The results from this study were made available for this final evaluation and were considered. 39. Additional complementary activities were implemented by various cooperating partners. 128 school gardens were created in Saint Marc corridor. This complementary activity focused on the distribution and cultivation of moringa plants, in alignment with the national policy for school meals. 40. Deworming was undertaken in some schools. WFP pushed to ensure this was scaled up so that all school meals beneficiaries would benefit from deworming. 41. WFP Haiti is also currently conducting a special study to assess the feasibility of increasing diversification of rations by complementing the standard McGovern-Dole ration with locally￾produced food to better meet the nutritious needs of pre-schoolers (aged three to five) and first to sixth graders (aged six to twelve). This study also reviewed how to improve cultural acceptability of school meals. The study was conducted between October and December 2018 and was part of the Performance Monitoring Plan (PMP) agreed with McGovern-Dole. 42. Figures 10 provides a summary of the Programme Logic Model based on the ToC. Fig 11 show the initial Results Framework for the first grant received. Figure 12 presents the revised Results Framework, used for the current programme. Final Report 88 | P a g e Figure 9: Graphic representation of the Theory of Change Incentive for HHs to send/Keep children in school PSNAS Improved Academic Achievements of Primary School Children Parents’ Contributions MGD SO1 - Improved literacy of school-aged children MGD SO 2 - Increased use of health and dietary Practices Improved student health status and learning ability Capacity-building Activities Gender Dimension Implementing Partners Local and National Government Key Issues Activities Project Objectives Expected Impact •In-kind food donations •Storage and monitoring •Safe food handling and hygiene Food Distributions • Distribution of materials • Teachers’ training • Additionnalsupport Literacy • Water purification tablets • Water chlorine purifiers • Sensitization activities WASH •MNP distributions •Deworming •School gardens Other Country Context Low Institutional Capacity Widespread poverty and economic inequalities Low food and nutrition Security Gender Inequalities Frequent Environmental Diasters Education Low enrollment, Absenteism and Dropping-out Illiteracy and low educational achievement Free Public Education but still financial burden Incentive for children to go to/stay in schools Improved teaching and learning quality Reduced absenteeism and drop-out Increased enrollment* Improved pupil health status Improved health knowledge and practices Improved National Capacity to expand national SF in a sustainable manner* * WFP School Meals Objectives in Haiti Reduce Micronutrient Deficiencies* Final Report 89 | P a g e Figure 10: WFP initial McGovern-Dole result framework - School meals programming in Haiti Final Report 90 | P a g e Figure 11: Revised McGovern-Dole results framework Final Report 91 | P a g e Figure 12: Revised McGovern-Dole results framework Final Report 92 | P a g e Annex 5: Evaluation Methodology a. Approach and methodology 1. The evaluation assessed the relevance, effectiveness (achievements of outcomes), efficiency and sustainability of the McGovern-Dole school meals model. It examined the factors that contributed to or impeded the achievement of outputs and outcomes. 2. The original questions in the Terms of Reference were as follow: Question 1: To what extent is the intervention relevant considering the needs of school children and their families, national government’s education and school feeding policies and strategies? Question 2: To what extent has the intervention resulted in the expected outputs and outcomes? How and why were those results achieved? Is there any difference based on gender dimensions, school types (public, private), location (urban, rural) or model (traditional, home-grown)? Question 3: To what extent are the key institutions and governance structures able to deliver, implement, and sustain school meal interventions in the long-term? What would need to be in place to allow for a transition to the Government? Question 4: To what extent is the level of community involvement in schools contributing to the project school feeding results and sustainability? Particularly, what is the level of involvement of women and girls and what are the reasons behind it? Is there any difference of involvement based on school types, location or model? Question 5: What are the differences between the McGovern-Dole model and other approaches? How could the Theory of Change be altered to increase effectiveness and long￾term effect? Question 6: To what degree factors for sustainability (financial, political, cultural, gender, environment…) were considered and which ones are reinforcing the intervention’s sustainability and future transition to the Government? 3. During the inception mission the evaluation team reviewed the evaluation questions with the Evaluation Manager and the Country Office Representative to clarify their meaning and intent. As a result, sub-questions were added to the evaluation questions and they were recognized to reflect the different key stakeholders involved, i.e. school level, community level and the broader institutional level. The evaluation questions were later validated by the McGovern-Dole Manager in Washington and the Regional Bureau Evaluation Officer supporting the evaluation. As a result of these clarifications, the evaluation questions are as follows: Evaluation Questions Criteria 1. To what extent are the programme activities relevant in light of the needs of school children (girls and boys) and their families (mothers and fathers), and education and school feeding policies and strategies of national government? Relevance 1.1 Are the programme activities aligned with the objectives of the National School Feeding Policy and Strategy (PSNAS)? (What is Final Report 93 | P a g e Evaluation Questions Criteria mentioned in terms of gender? Does the PSNAS meet the other government commitments/strategy on gender?) 1.2 Was the geographic targeting congruent with the needs of the population, in particular relating to food security, nutrition needs, vulnerability? 1.3 How has the programme reconciled the issue of having to work with the most vulnerable schools while ensuring schools meet the selection criteria and identifying new schools that should be included? 2. To what extent has the programme achieved the expected outputs and outcomes at school level and which factors are explaining those results? Effectiveness 2.1 Which planned outputs have been achieved and which have not? 2.2 Which planned outcomes have been achieved and which have not? 2.3 Are there notable differences in outcomes based on gender, type of school (public, private), location (geographic), model (traditional, homegrown) or complementary activities (literacy, WASH, school gardens)? 2.4 What other external factors are explaining those results (e.g. other reasons for drop-outs (for boys / girls), lack of academic achievement, low/high enrollment rates)? 3. To what extent does the community involvement in schools (general, parents (mothers and fathers) especially via (PTAs), SFCs (same), directors, etc.) contribute to outcomes and sustainability? Who makes decisions - male, female or both together? Effectiveness, Sustainability, Efficiency, Impact (long term effects) 3.1 To what extent do school feeding management and supervision committees contribute to the outcomes as defined above and the sustainability of the activities on the long term? − Are the contributions from parents sufficient to achieve planned outcomes (which parent decide how much to contribute or both decide)? − Are women and men represented equally in the various structures at the school level (SF management committees; supervision committees; parent- teachers committees) / more specifically, what is the level of involvement and participation of women and girls and what are the reasons driving or inhibiting it? How are these gender issues affecting programme implementation and results? − What are the key contributing factors for parent/community participation (for both genders)? 3.2 Does the programme have the potential to create sustainable income generation opportunities for communities/ women? − What types of revenue generation opportunities has the programme helped generate to date (e.g. farmers, including female producers, cooks, others), and what is the quality and sustainability of these activities? (Are there different challenges for women and men?) − Are there other potential sources of revenue generations that could be fostered through the project (for women and men)? Final Report 94 | P a g e Evaluation Questions Criteria 4. To what extent are the key institutions (national, provincial/district and local stakeholders; international and national implementing partners and NGOs; international donors and multilateral agencies working on school feeding in Haiti) and governance structures able to effectively deliver, implement, sustain and/or scale up school meal interventions in the long-term? Effectiveness, Sustainability Efficiency 4.1 To what extent have specific government capacities been strengthened so far? (expected outputs and outcomes) 4.2 At what extent do Haitian national institutions (at central and decentralized levels) currently have the capacity to take over the programme fully or partially? − Is there a full or partial governance structure in place to implement, sustain and scale up school meal interventions in the long-term? (What is the role of women versus men in these structures?) − Are key national institutions willing and able to assume the responsibility for school feeding within the originally agreed timeline, and what would a more ideal timeline be? − What would need to be in place to allow for a full transition to the Government of Haiti? 5. What are the differences and advantages/disadvantages between the McGovern-Dole model and other approaches, focusing on nutritional and educational outcomes and community engagement, and how could it be adapted to increase effectiveness, sustainability, efficiency and long-term effects? Effectiveness, Sustainability Efficiency 5.1 How does the McGovern-Dole model /ToC compare with other national and international school feeding programmes with respect to its contribution to nutritional, educational and community outcomes? 5.2 What other strategies, based on other experiences in Haiti or elsewhere, might improve community involvement and sustainability? (What would be the role and benefits for both genders/how to increase benefits for women?) 5.3 Are there opportunities to add or augment a nutrition sensitive approach in any of those models? 5.4 Has the programme used available resources efficiently? − Are budget ratios and sources for the school feeding component in line with sustainability? − What would the cost be for the GoH and what different budget options might there be? − What would equitable solutions for the payments of cooks be and are there ways to implement this? 4. The evaluation was theory-based and used a non-experimental design. As noted in the TOR, the evaluation was not meant to asses impact (see ToR, par. 51). However, in line with UNEG standards for evaluation, it examined the relevance, effectiveness, sustainability and efficiency of the programme. While the social, environment and economic contexts in which WFP activities were implemented are highly complex, the actual school feeding programme Final Report 95 | P a g e itself was a relatively straight forward activity with very focused investigative parameters. Furthermore, school feeding have been tested in a variety of contexts. Hence, the use of more experimental designs would not have been recommended. From each school feeding programme, a great deal can be learned. But it was felt that this was best achieved through a mixed method approach depending on secondary data produced by WFP and other stakeholders and collected primary data through semi-structured interviews and focus group discussions with programme stakeholders at the national, regional, community but mostly at the school level where a great deal of information about the operation of each canteen could be learnt as well as their importance for the respective schools, children, families and surrounding communities. Children only participated in focus group discussions (not individual interviews, often in the presence of adults (e.g. management committees). No personal questions or questions that could cause embarrassment or harm were asked. The purpose of the evaluation and discussion was always explained at the beginning of the FGD. The voluntary aspect of their participation was emphasised, i.e. no child was coerced to participate at any time in the course of data collection in accordance with UNNEG Ethical Guidelines for Evaluation. 5. The data collected was triangulated between and within sources of information to the extent possible. As noted, the information generated from evaluation activities at the school level provided the core data for the evaluation. This information was then triangulated through the other methods mentioned above. Given the broader mandate of this evaluation to look at how directional changes could benefit the programme, research activities played a prominent role in the evaluation. Collectively these sources of information provided ample possibilities for triangulation. 6. Given the volume of schools that were visited to develop a representative sample size, and ensure that visited schools were not overwhelmed by the evaluation process, some potentially useful tools such as the Early Grade Reading Assessment (EGRA) standard test were not employed. Lines of questioning with school level stakeholders in relation to child literacy, health and dietary practices were pursued. Additionally, direct observation was employed in schoolyards, school kitchens and when access was provided, school classrooms when for example children were eating school meals. There were 35 schools visited. The CO evaluation manager and WFP programme staff confirmed that it is was a representative sample of the overall programme in each programming zone. The representativity of the schools was validated by the WFP offices in the three zones. 7. Gender was an integral part of the evaluation. Firstly, the evaluation examined outputs and outcomes for both genders. Secondly the data was analysed in light of the needs and strategic interests, i.e. socio-economic empowerment, of both genders. The different data collection methods used ensured that the ‘voice’ of women, men, girls and boys were equally reflected, as well as in the analysis, findings, conclusions and recommendations. The gender specialist devoted substantial time and energy on gender analysis and produced a thirteen￾page document in French. The information of this analysis has been used throughout the report. The analysis gender analysis was shared with the CO evaluation manager. In addition, the ET had access to a gender study of WFP school feeding programming in Haiti, conducted Final Report 96 | P a g e in parallel with the McGovern-Dole programme evaluation.140 This independent gender study examined the gendered cultural norms in schools and the communities they serve. The review of the gender study confirmed the findings of this evaluation. A summary of the key findings of the gender study and the suggestions for action are presented in Annex 10, section d. 8. The evaluation was conducted using an inclusive, participatory approach (it provided for equal participation of key actors and actresses) and respectful of all stakeholders. Some of the measures taken helped in integrating gender dimensions into the different phases of the evaluation. The evaluation describes how the programme affects men and women, girls and boys differently. It examined the extent to which programme interventions affected gender and power relations and provides recommendations to better respond to the needs of women and men, girls and boys and to contribute to positive changes in the relationships between women and men. b. Planning and preparation steps. 9. The assessment team included a Haitian specialist on and the team members were all sensitive to the problem. The review of shared data and documents from the earliest stages of the evaluation and exchanges with the WFP team helped the evaluation team in: preparing the initial report, highlighting gender concerns on which have been given special attention in the subsequent stages. Measures have been identified and implemented for the effective evaluation of gender aspects in preliminary stakeholder analysis, methodology (instruments addressed gender issues) which fed the gender analysis conducted by the gender specialist. 10. Steps to collect and analyze data. Methods and tools for data collection provided information that includes girls and boys, women and men. The constant concerns of the team were, among other things, to ensure that all voices, including those of the most marginalized, are heard and taken into account such as: children (girls / boys), small staffs in schools ( cooks, caretakers); to ensure the collection and analysis of disaggregated data as far as possible by sex and age; cope with contextual constraints in terms of gender equality and at the same time be able to seize contextual opportunities when collecting data. 11. As mentioned earlier, the evaluation served the dual purpose of accountability and learning. The methodology carried out was designed to gather information and data to achieve these two broad but complementary objectives. 12. Objective one: Review of McGovern Dole Food for Education Programme related to the primary responsibility of assessing the impact of the McGovern Dole Food for Education Programme (McGovern-Dole) in contributing to efforts by the GoH in establishing a viable foundation for school feeding activities in the three corridors targeted by the programme where food insecurity is a major concern. 13. The evaluation analysed secondary data that began during the inception mission and carried on with this throughout the next stages of the evaluation. As much as possible, the outcomes in the McGovern-Dole supported schools and other government schools in the 140 Nora Astor, Gender Analysis of WFP Scholl Feeding Program, September 2019. Note that this is not the final version of the report. Final Report 97 | P a g e departments where the programme was implemented was compared. However, the data from government was not always available in a format that could easily be utilised, and this comparison was not always possible. At any rate, the evaluation explored the possibility of comparing WFP related outcome data with national statistics. Despite many attempts to obtain data from the MENFP, this was not possible. The evaluation reviewed outcomes data specifically for the McGovern-Dole supported schools. Key documents that were reviewed included: − 2016 Baseline Report − WFP semi-annual reports to McGovern-Dole − WFP SPR 2015-2017141 14. The team of five consultants (3 international and 2 national) dedicated time to the three programming areas that were representative of the different scenarios under which the food provided by McGovern-Dole was distributed. See Annex 11 for an overview of the team qualifications and roles. This included investigating canteen operations where McGovern￾Dole donated food was used in combination with homegrown food such as rice and vegetables, and/or complemented by development activities, in particular, literacy (run by WFP and run by UNICEF) and Water Sanitation and Hygiene (WASH) that were expressly coordinated with the McGovern-Dole programme. However, as there had been audits conducted of the school meals programme, the evaluation put an emphasis on factors that had contributed to or impeded the achievement of outputs and outcomes, especially key success factors, that highlighted the value added of this programme for Haiti and at the same time explored ways to enhance the existing programme. The objective was to see whether the outputs and outcomes were different under each scenario or if some scenarios lead to better outcomes. 15. To this end, the evaluation team interviewed stakeholders in the Cap Haïtien, Saint￾Marc/Artibonite and Port-au-Prince corridors to directly assess the strengths and weaknesses of the McGovern-Dole Programme. The assessment involved interviewing and conducting focus group discussions at the school level (teachers, school directors, cooks, school committees, children, community leaders) and interviewing government representatives and NGOs at the regional level. The aim was to visit 15 schools under each scenario for a total of up to 45. The evaluation collected qualitative data from several actors: school directors (KII), and FGD with teachers, children, and parents, as part of the various school committees (school meal management committee, supervision committee and school committee) and monitoring officers to obtain their views on the functioning of the programme, as well as on issues such as parent/community participation and revenue generation. In the end the team visited a total of 35 schools, which was deemed sufficient as the ET was told that these schools were representative of the different situations encountered in the programme. This was complemented key stakeholder interviews with WFP staff and with Government staff such as representatives of the MENFP, PNCS, MARND, MSPP and partner organizations such as McGovern-Dole, USAID, and operating partners (NGO) in Haiti. This process began during the inception mission and was carried out through two data collection missions (April 23-May 8 and May 27-31) . 141 The 2018 SPR will be reviewed if available during the evaluation period. Final Report 98 | P a g e 16. A very important element of this analysis was understanding the importance of complementary actions to school feeding initiatives such as school literacy initiatives and WASH activities to the achievement of outcomes. The data collection and analysis involved examining the outcomes achieved under three scenarios: the provision of school meals alone, the provision of school meals with the literacy component and the provision of school meals with the WASH component. 17. Of special importance was to review issues related to linkages between school meals activities and Haiti’s agricultural sector. In this regard it was important to consult with representatives from the Ministère de l ’agriculture, Ressources Naturelles et Developpement Rural (MARNDR) and local farmers associations/cooperatives in the three corridors. A general understanding was established from key institutions and organizations (national, departmental/district and local/school level stakeholders; international and national implementing partners and NGOs; international donors and multilateral agencies working on school meals in Haiti and their perspectives on success factors for school meals initiatives in Haiti reaching the objectives set out in the Politique et Stratégie Nationale de Cantines Scolaires (PSNCS). Interviews with representatives of some of these stakeholders began during the inception mission however, it was necessary to return to some of them with more directed questions regarding issues identified in the evaluation questions and to answer some of the evaluation questions. 18. Interviewees were also conducted with representatives of organizations that could provide insights on other SF models or about income generation activities, particularly for farmers to help answer evaluation question 5. To ensure that this analysis would be useful required additional lines of investigation. It was important to ascertain how well school meals initiatives were being accepted and supported by local communities, particularly the parents of the supported school children, and what could be done differently in this regard. The experience of both WFP (Haiti and other country offices) and other actors in Haiti was researched and analysed. With this objective in mind, this evaluation examined other evaluations of school meals programming in Haiti and in other countries, as well as case studies, which all provided valuable recommendations, lessons and insights.142 19. The review of McGovern-Dole also included Objective Two: Broader Analysis of School Feeding Capacity in Haiti and Broader Context (Evaluation Question 4). The work related to this second objective was linked to the long-term viability of an McGovern-Dole model approach to school feeding in Haiti and specifically what role if any, a model largely build on food importation should have. It was guided in good part by the results of the McGovern-Dole programme. It also involved looking more generally at school meals initiatives in Haiti and current favourable factors and roadblocks to transitioning to a more sustainable independent national school meals system and identifying the types of actions that could be supported by WFP and its partners in the near to mid-term future to better encourage sustainability. It also required to undertake an analysis of the operational context of school meals initiatives, especially in vulnerable areas under study. Much of the information to be used in this regard came from the data and information generated by the McGovern-Dole fieldwork. 142 Other evaluations, studies or cases studies will be sought during the data collection period and relevant lessons and recommendations will be considered in the analysis. Final Report 99 | P a g e 20. A special focus was placed on Haitian governance structures and their current ability to deliver, implement, sustain and/or scale up school meal interventions over the long-term. Establishing an understanding of the GoH’s ability and willingness to manage a national school feeding programme independent of donor assistance and how support from organizations such as WFP evolved to make this happen was an important area of investigation. The news that the GoH requested an IMF loan that in part, was expected to support school meal interventions also had to be understood to the best degree possible. c. Stakeholder Analysis 21. Prior to conducting field work the ET conducted a stakeholder analysis to identify the most relevant stakeholders for this evaluation. The following table provides an overview of the stakeholder analysis conducted during the inception phase and their interest in the McGovern-Dole programme. Table 6: Stakeholder analysis and mapping Stakeholder Interest in the McGovern-Dole School Meals Programme Involvement in Evaluation and likely use Who (specifically for the Evaluation) Internal (WFP) stakeholders Country Office For accountability and learning. In terms of learning, the Country Office is keen to understand the different school feeding models, currently in use in Haiti and learn from the experience of other countries, to ensure future projects are designed around the key beneficiaries, addressing their needs and supporting them in a sustainable manner. Primary stakeholder Involved in all phases of the evaluation. Evaluation Manager: Country Representative; Programme Director and staff; M&E team Regional Bureau Same as above. Secondary Stakeholder Will provide technical assistance and quality assurance in all phases of the evaluation. Regional Evaluation Officer Office of Evaluation (OEV) Being informed about the effectiveness of WFP programmes and that decentralized evaluations deliver quality, credible and useful evaluations. Secondary Stakeholder Will provide quality assurance and approve the final products of the evaluation. OEV Director of Evaluation ad evaluation officers External stakeholders Schools As the primary beneficiaries of the School meals program, they have a direct interest in the evaluation and ensuring that the children are able to learn and succeed in school. Primary Stakeholders school directors, teachers, children, parents. Final Report 100 | P a g e Stakeholder Interest in the McGovern-Dole School Meals Programme Involvement in Evaluation and likely use Who (specifically for the Evaluation) Community members Interested in the children’s ability to learn Secondary Stakeholders Cooks; community members who are part of supervision committee or other school committees; others with an interest in school, e.g. mayors, local NGOs. NGOs and School Feeding implementors WFP Haiti partners with well￾established local Haitian and international NGOs to implement its activities. The results of the evaluation might affect future implementation modalities, strategic orientations and partnerships. Care and other NGOs implement programmes with local producers, including women and are likely interested in finding out how their programming can contribute to school feeding programming in the future. They will participate in interviews and focus group discussions and provide insights into the programme and other school meal models and activities implemented in Haiti and elsewhere. BND and CRS are members of the Evaluation Reference Group Primarily BND, CRS, World Vision, AMURT, ASEBED, BDEPAP. Also, Care and other international and local NGOs. MENFP and PNCS The MENFP and PNCS have a direct interest in knowing whether WFP activities in the country are aligned with its priorities. They are also eager to take over the national school feeding programme. Secondary Stakeholder MENFP and PNCS are members of the Evaluation Reference Group. Will provide input through interviews. Directors and technical staff of MENFP and PNCS. McGovern￾Dole As the main funder of this programme, it has a strong interest in knowing whether their funds have been spent efficiently and if WFP’s work has been effective and contributed to the strategies and programmes of McGovern-Dole. Primary Stakeholder McGovern-Dole is also funding this evaluation. Its role is to review, comment and approve the TORs and the inception and final evaluation reports after the ERG approval; in addition, it participates in a key informant interview with the selected evaluator prior to field data collection. McGovern-Dole Fritzner Cledo, Agricultural Specialist, USDA Haiti Programme Manager Paul Alberghine Senior International Program Specialist | Food Assistance Division | School Feeding and Humanitarian Branch Final Report 101 | P a g e Stakeholder Interest in the McGovern-Dole School Meals Programme Involvement in Evaluation and likely use Who (specifically for the Evaluation) Donors The programme supported by these donors is very similar in terms of provision of a daily meal but has a strong local procurement component. They are interested in the effectiveness of the model and want to learn from other models to improve outcomes for Haitian children. Secondary stakeholders They will participate in interviews. Mainly Canada and France UNICEF and FAO UNICEF is a key actor in the education sector and is interested in better educational outcomes for children. FAO is interested in improving food security, local production and environmental sustainability. Secondary stakeholders UNICEF is represented in the Evaluation Reference Group. They will participate in interviews. UNICEF Head of Education; FAO TBD Ministry of Agriculture, Natural Resources and Rural Development The Ministry is a key stakeholder in the area of agriculture production and will be interested in the evaluation’s findings regarding strategies that could help increase local production through linkages with school meals and local purchases. Secondary stakeholders Its input will be sought during data collection through interviews. Individuals to be identified during the data collection planning phase. Local cooperatives and farmers associations Groups which have participated in homegrown activities will be interested in seeing how homegrown activities can expand and how local farmers can benefit economically from increases in production and the supply of food to schools offering meals to children on a daily basis. Secondary Stakeholders Farmers cooperatives will provide input through interviews. Groups and individuals to be identified during the data collection planning phase. World Bank The World bank was a major implementor of a school feeding programme in Haiti although it has recently scaled down. Its experiences could be valuable for future programmes and the evaluation could inform its future strategy. Secondary stakeholder They will participate in interviews to provide information about its own programme. Head of Education USAID, Brazil, Canada These donors have supported homegrown strategies combined with school meals and would be interested in the findings of the evaluation on the effectiveness of the McGovern-Dole programming and how the activities of the school Secondary stakeholders They will participate in interviews to provide information about the Individuals to be identified during the data collection planning phase. Final Report 102 | P a g e Stakeholder Interest in the McGovern-Dole School Meals Programme Involvement in Evaluation and likely use Who (specifically for the Evaluation) meal programme may be expanded to provide better outcomes for Haitian children. There is also interest from Canada and USAID in environmental sustainability and gender equality. homegrown initiatives they. d. Site mapping 22. The data collection work will began by a preliminary review of documents, which began during the inception phase. It was conducted remotely from April 9-18 and then followed by fieldwork, which begun in Cap Haïtien (April 23 to 27). Information gathering continued in the Artibonite and St- Marc area (April 29 to May 4). After the completion of these first two weeks of field work, the evaluation team continued working remotely (May 6 to 24). As well, this period allowed to better organize and analyse the McGovern-Dole data collected in Cap Haïtien and Gonaives/St-Marc. 23. The next step involved the international members of the evaluation team returning to Haiti but without the data analysis specialist on the team (May 27 to 31). This stage was divided into achieving two distinct tasks: First, completing fieldwork at the level of schools and corresponding communities in the Port au Prince area. Secondly, conducting a series of interviews and holding meetings in relation to Objective Two. Visits to two or three homegrown projects/communities in the vicinity of Port-au-Prince were conducted, including semi structured interviews with farmers associations. It was also thought that if deemed necessary, additional days, could be added during this stage either end of the May 27 to 31 data collection mission. This stage concluded with a full debriefing with WFP including a power point presentation and other stakeholders identified by WFP. 24. The rationale for this phased approach was that because of the tight deadlines between the inception phase and the data collection phase, there was little time to conduct an in-depth review of documents. This phase approach allowed for this. In addition, it allowed the team to analyse the data collected in the first phase and clarify, validate data collected or collect additional data if needed. 25. The key gender consideration in relation to the sites and that were taken into account in the analysis is the urban rural divide. Both men and women have more access to revenue generation activities in urban settings than in rural areas, particularly remote areas and in areas affected by drought (North-North-East and parts of Artibonite where the local economy is based on agriculture. This reduces severely both parents earning capacity, particularly single parent households, which are mostly headed by women. This affects parents’ ability to contribute to the school meals as expected by the programme. Table 7: Summary of site map for field data collection Final Report 103 | P a g e Location Team Members Focus Dates North and North-East corridor All team members (note that each team member will visit one school. WFP local manager (KII), Schools with meals only (KII-FGD); Schools with meals and WASH component (KII-FGD); Community/parents (FGD) Operating partners: local manager KII), monitoring staff (FGD) Departmental Education Officials (KII) Local PNCS (KII) MARNDR: Departmental Agriculture Officials (KII), NGOs (FGD or KII) Producer organizations (FGD or KII) April 23-27 Gonaives and St￾Marc corridors All team members WFP local manager (KII); Schools with meals only (KII-FGD) Schools with meals and WASH component (KII-FGD) Community/parents (FGD) Operating partners monitoring staff (FGD) Departmental Education Officials (KII) Local PNCS (KII) Artibonites Homegrown projects (FGD or KII, NGOs (KII) Producer organizations (FGD or KII) April 29- May 4 Port-au￾Prince corridor (Centre and West) Team Leader, Deputy Team Leader and one team member Schools with meals and WASH component (KII-FGD) Schools with meals and literacy component (KII-FGD) Operating partners monitoring staff (FGD) WFP monitoring staff (FGD) Donors: USAID, France, Brazil (KII) Nippes Homegrown projects (FGD) NGOs (KII) Producer organizations (FGD) Ministry of education directors and technical staff KII) PNCS director and technical staff (KII) WFP School Meals Director (KII) Multilateral Banks: WB, IDB (KII) Operational Partners not met during inception mission (KII) May 4, 6, 7 and May 27-31 e. Sampling of schools 26. The sampling was purposeful rather than random. The main reason for this was to allow for regional representation as well as the inclusion of rural and urban schools, schools that offer meals only, schools with a WASH component and schools with a literacy component. Accessibility was another factor for the choice of schools to include in the sample and needed to be validated by WFP for accessibility by car and distance. It was decided that it Final Report 104 | P a g e should not take more than one to one and a half hour to reach a school from a major centre where the team would be located (e.g. Cap Haïtien, Gonaives, St-Marc or Port-au-Prince). To allow for a degree of randomness, for the schools with a conventional school meal programme, every 18th school was selected. For the WASH component with water purification equipment, every 6th school in the list was selected out of the 110 schools that were selected for this activity. For the literacy component every alternating 3rd and then 2nd school were selected. The lists was cross-checked so that they did not appear twice. For schools being substituted, the next down the list was selected until a suitable candidate was found. The list of schools visited is shown in the table below. Given the similarity across schools it was decided by the evaluation manager that visiting 35 schools would suffice instead of the 45 planned. This is reflected in the number of interviews and focus groups conducted at school level. Table 8: List of Schools visited by region and type No Region Commune School Name WFP school Number Type 1 Nord Cap Haïtien École Nationale de Marie Rose PAM-2017-09-01165 WASH 2 Nord Dondon École Nationale de Saint Maxime PAM-2017-09-01193 WASH 3 Nord Est Fort Liberté École Nationale De Haut Madeleine PAM-2017-09-01286 WASH 4 Nord Est Trou du Nord École Saint Dominique Savio PAM-2017-09-01400 WASH 5 Nord Est Terrier Rouge École Nationale De Colonie /Grand-Bassin PAM-2017-09-01373 Conventional 6 Nord Cap Haïtien École Nationale de l’Union PAM-2017-09-01170 Conventional 7 Nord Cap Haïtien École Nationale Marie Auxiliatrice PAM-2017-09-01183 Conventional 8 Nord Grande Riviere Du Nord École Nationale de Galifète PAM-2017-09-01199 Conventional 9 Nord Plaisance École Nationale Colline Gaubert PAM-2017-09-01230 Conventional 10 Nord Est Capotille École Nationale de Cana PAM-2017-09-01270 Conventional 11 Nord Est Ouanaminthe Centre Éducatif National St-Esprit de Grand Chemin PAM-2017-09-01338 Conventional 12 Nord Est Sainte Suzanne École Nationale de Ste￾Suzanne PAM-2017-09-01348 Conventional 13 Artibonite Gonaives École Congréganiste Saint Pierre Claver PAM-2017-09-01042 Conventional Final Report 105 | P a g e No Region Commune School Name WFP school Number Type 14 Artibonite Gonaives École Nationale Jules Zéphirin PAM-2017-09-01068 Conventional 15 Artibonite Gros Morne École Nationale de Pendus PAM-2017-09-01095 Conventional 16 Artibonite Gonaives École Congréganiste Saint Pierre Claver PAM-2017-09-01042 Conventional 17 Artibonite Anse Rouge Ecole Presbytérale Notre-Dame de l'Assomption PAM-2017-09-01010 Conventional 18 Artibonite Anse Rouge Institution Mixte Senèque Quintilien PAM-2017-09-01013 Conventional 19 Artibonite Saint-Michel de l'Attal Institution Mixte Kelly Augustin PAM-2017-09-01142 Conventional 20 Artibonite St-Marc École Nationale de Chatelas PAM-2017-09-01122 Conventional 21 Artibonite St-Marc Institution Mixte Desir Croicy Conventional 22 Artibonite St-Marc École Nationale Béthel de Canneau PAM-2017-09-01121 WASH 23 Artibonite St-Marc Ecole Presbytérale de Louère PAM-2017-09-01125 WASH 24 Ouest Croix-Des￾Bouquets École Nationale Tamarin Santo PAM-2017-09-00646 Literacy 25 Ouest Ganthier École Nationale De Jonc PAM-2017-09-00650 Literacy 26 Ouest Croix-Des￾Bouquets École Nationale De Sibert PAM-2017-09-00640 Literacy 27 Ouest Arcahaie École Nationale De Délugé PAM-2017-09-00599 Literacy 28 Ouest Arcahaie École Nationale De Saintard PAM-2017-09-00603 Literacy 29 Ouest Arcahaie École Nationale Du Drapeau PAM-2017-09-00605 Literacy 30 Ouest Cabaret École Nationale D Cazale PAM-2017-09-00610 Literacy 31 Ouest Cabaret École Nationale De Foveau PAM-2017-09-00613 Literacy 32 Ouest Arcahaie École Nationale De Délugé PAM-2017-09-00599 Literacy 33 Ouest Arcahaie École Nationale De Saintard PAM-2017-09-00603 Literacy Final Report 106 | P a g e No Region Commune School Name WFP school Number Type 34 Ouest Ganthier École Nationale de Saint Pierre PAM-2017-09-00649 WASH 35 Ouest Ganthier École Nationale de fonds parisien PAM-2017-09-00635 Literacy f. Data Collection Methods and Tools 27. As mentioned previously, the evaluation used a combination of primary and secondary data to generate findings and recommendations and primary qualitative data was needed to complement and explain the quantitative results that WFP has collected during the programme’s implementation. These data collection techniques were complementary and allowed for the triangulation of data across different sources. The evaluation matrix in Annex 2 provides an overview of the evaluation questions and sub-questions with the key sources of evidence and associated data collection methods. 28. In carrying out the evaluation the following evaluation tools were used for primary data collection: a) Key stakeholder interviews: This technique was used in variety of contexts. We tailored the interview guides for each category of respondents, e.g. WFP; government partners, schools, parents, etc. b) Focus Group Discussions (FGD): Focus groups were only be conducted at the school/community level and with monitoring officers. For each school visited, the approach was to interview the school director, and conduct a focus group discussion with the canteen supervision and management school meal committees and representatives of parent associations, canteen cooks, and as many students as possible given that the visits would be conducted during school hours. It must be underlined that the volume of people that were consulted depended on a number of factors such as availability. Where possible the team observed school activities as the serving of meals, the space where the cooks make the food, latrines and use of WASH equipment. Table 9: Research techniques by category of stakeholders, and totals Method Stakeholder Planned Actual Key Informant Interviews Total: Up to 76 School directors Up to 45 35 Donors: USAID, Canada, France Brazil Up to 4 3 Ministry of Education - National level 2 0 - Departmental level 3 3 PNCS (national and departmental level) 4 4 MANRD (national and departmental) 3 2 WFP programme director; Gender Analyst 2 2 Implementing partners 5 4 Focus group discussion with CRS trainers 1 1 NGOs up to 2 per region Up to 6 4 Producer organizations up to 2 per region Up to 6 4 Final Report 107 | P a g e Method Stakeholder Planned Actual Multilateral Banks (IDB) 2 2 Focus Groups Total: Up to 141 Monitoring Staff Operating Partners - 1 per region 3 0 Monitoring staff WFP 1 1 School Meal Management and Supervision committees; parents and community members Up to 45 34 FGD with teachers* Up to 45 0 FGD with children Up to 45 30 * During field data collection, the evaluation team found that it would be impossible to pull out five or six teachers during classes. However, teachers who were members of school committees and canteen management committees did participate in focus group discussions. g. Data Analysis 29. The primary qualitative data collected and compiled from data collection activities (fieldwork and desk review) was entered in a data collection template by source and indicator. This allowed to categorize and triangulate data across the different sources, method and researcher using a coding strategy based on key words. Themes and patterns then emerged from this exercise. The qualitative data analysis was complemented by data analysis of secondary quantitative data that began during the inception mission and was carried on throughout the next stages of the evaluation. The second stage was to generate findings for each of the evaluation questions. These findings were validated with stakeholders before drafting the report. 30. Overall, there was sufficient valid and reliable quantitative data from WFP to answer the evaluation questions. Please see the evaluation matrix in Annex 8 for a more detailed description of the different sources of data and their quality. A baseline study was conducted for FY15 in late 2016 and an end line study was planned for the end of the 3 years. The baseline focused on indicators related to schools’ organizational structures, students’ enrollment and performance, infrastructure, compliance with good health and hygiene practices, among others. This evaluation’s TORs mentioned that the baseline ‘did not dedicate sufficient resources to qualitative data collection and did not go beyond the PMP.’ As a result, it was expected that the final evaluation would explain ‘why the indicators were the way they are and how the program could address this, if it was identified as an issue. The evaluation however used key quantitative indicators of the baseline, which WFP to develop its extensive monitoring and control system including partner’s assessments, school monthly monitoring report, monitoring visits and dashboards. 31. This monitoring data was made available to the evaluation team, with some of the data previously cleaned and organized. These data provided a good overview of the main issues related to the management of the program by school authorities and committees. Other reports, which were used for the evaluation came from the WFP’s SPR and the McGovern￾Dole semi-annual reports, with a good reliability of data. The collection of primary quantitative data was limited due to the size of the sample of schools. Nevertheless, the ET quantified generally the qualitive data. For example, when 100% of respondents had a similar response, it was reported as “all”; most meant that at least 75 percent of respondents Final Report 108 | P a g e had a similar response; “the majority” signified more than half and ‘some’ less than half. It must be kept in mind that semi-structured interviews and focus group discussions do not produce data as quantifiable as surveys. As mentioned above the analysis of quantitative data was based on the data collected by WFP CO, including the baseline, which key results are presented below. h. Some Key Findings and Indicators from the Baseline Study (from the Baseline Executive Summary) 32. The results of this study are the findings of an in-depth exploration of the three main sources of data collection, namely, schools, students and communities, in order to provide precise answers to a set of questions formulated beforehand. These results, as articulated, aim to provide program managers with the necessary level of understanding to better articulate program actions in the implementation of the feeding program in the schools. 33. The study showed that in terms of school organization, there is a SMC in 93.3% of the schools’ facilities. As to whether these structures play their role effectively, it was immediately revealed that the members of this structure are only getting bogged down in daily activities associated only with the management of the school canteen’s routine operations. However, their statutory responsibilities, as defined in the TORs of their constitution, are more extensive. It should be noted that the function of president, for example, is occupied by the school's principal in at least 50% of the cases. However, the statutory regulations indicate that the SMC members must be democratically elected by the general assembly. Thus, it seems that there is a lack of a more formal normative framework that would have allowed the members of this structure to more adequately fulfil the assigned mission. 34. For example, in 2015-2016, the students who had to repeat their class are estimated at 21.08%. If it is recognized that the school meal has a positive impact on the students' academic performance, how can the SMCs, the teachers, and the principals be involved in a joint effort to maximize the benefits of this program in order to reduce the repetition rate in the schools. This is a relevant issue that could help refocus the role of the SMC with its statutory rules. 35. In addition, the issue of school infrastructure management requires an attention from the community actors in the schools, particularly the SMC. For example, the study found that only 66.7% of the schools have latrines. When it comes to the upstream functioning of the canteen, there is the management of the storage room where the food is stored before being sent to the kitchen for cooking. The service personnel seem to know the basic rules associated with secure storage of goods. In fact, 90.7% of the schools have a storage room whereas pallets are available in 88.7% of the schools. For the kitchens’ compliance with minimum operating standards, only 48% of the schools meet these requirements. 36. There is a lack of equipment and cooking utensils in the kitchens. Indeed, the study revealed that there are no plates in 32% of the schools. We must admit that these survey data do not come from a systematic inventory of equipment and utensils available at the schools. It was not possible to confirm all the information provided from the school property inventory records properly maintained by the SMC. 37. When it comes to the management of the school canteen, the staff seem to have some knowledge of the rules associated to cleanliness, hygiene and food safety preparation. In fact, by reviewing good practices in this field, 94% of the schools have their SMC members Final Report 109 | P a g e or their service staff capable of identifying and applying at least three good practices. However, in terms of training, needs are identified in food product stock management, in school health and hygiene, in nutrition and food safety preparation. 38. In its report of cause-effect beneficial with academic performance, unanimously shared by the various groups of stakeholders, the school canteen plays an undeniable role. All the actors (principals, teachers, parents) intervening in the school environment recognize it besides. Even the direct beneficiary who is the student is aware of the benefits he derives from the school meal program, the most important of which is his academic success. 39. The study highlighted the relationship between community actors and the management of the program. Their involvement and participation, despite the difficulties encountered with pay issues, continue to demonstrate their commitment to the implementation of the program in their schools. The question remains on how to create greater synergy in the actions of its stakeholder groups to maximize the benefits and long-term outcomes of the program. 40. Indeed, the new paradigm of development implies the development of local capacities in the effective management of their problems in the long term. This program, in itself, represents a driving force around which could be grafted several dynamics that could influence the quality of education. The statements included in the ToR defining the organizational and structural profile of the SMC, for instance, contain the seeds to give this new drive to these schools. 41. Although most community actors have certain grievances about the program that they generally formulate in the form of demands for adjustment in the supply of food in the schools, it remains nevertheless, that it is a program that local communities rely on. With its capacity building component, this program could leverage to help improve the quality of education while providing a balanced diet to the students. It is with this understanding that this study attempts to make the following recommendations to the program managers. Monitoring Indicators Baseline 2016-2017 1. Percentage of schools with improved storage equipment, food preparation tools and eating utensils as a results of McGovern-Dole assistance 1.1. Percentage of schools with improved storage equipment 77% 1.2. Percentage of schools with food preparation tools 79% 1.3. Percentage of schools with eating utensils 51.30% 2. Percentage of targeted schools equipped with fuel efficient stoves as a result of McGovern￾Dole Assistance 4.70% 3. Proportion of household involved in briquettes' purchase for improved stoves 4.1. Proportion of household involved in briquettes' purchase 4.52% 4.2. Proportion of household involved in propane gas' purchase 3.36% 4. Percentage of schools’ management committees who can identify at least three improved health and hygiene practices 94% 5. Number of students regularly (80%) attending McGovern-Dole 5.1. Number of female students regularly attending McGovern-Dole supported classrooms 18762 5.2. Number of male students regularly attending McGovern-Dole supported classrooms 19526 Final Report 110 | P a g e Monitoring Indicators Baseline 2016-2017 supported classrooms / schools (female/male) 5.3. Total number of students regularly (80%) attending McGovern-Dole supported classrooms / schools (female + male) 38288 6. Percent of student missing less than 20 days a year due to illness 3.2% 7. Number of schools using an improved water source 7.1. Number of schools using water coming from a Tap 63 7.2. Number of schools using a protected drilling￾well water source 25 8. Pass rate (Female/Male) in the assisted schools 8.1. Pass rate female in the assisted schools 74.20% 8.2. Pass rate male in the assisted schools 73.90% 8.3. Pass rate in the assisted schools 74.04% 9. Retention rate (Female/Male) in the assisted schools 9.1. Retention rate female in the assisted schools 94% 9.2. Retention rate male in the assisted schools 96% 9.3. Retention rate in the assisted schools 95% Source: Baseline survey 2017, AGV/WFP. i. Limitations 42. One limitation in terms of the quantitative data is that at the time WFP had not yet collected end line data, which would have provided a fuller set of data to analyse. However, as mentioned above, WFP collected sufficient data (baseline and two school years) to draw conclusions of the achievement of outputs and outcomes. j. Risks and Mitigation Strategy 43. Semi-structured interview can yield a lot of information from which it is difficult to generalize. To mitigate this, we limited the number of questions in interviews so that we would have a more focused set of responses. Furthermore, the qualitative research was based on a set of assumptions that came from the secondary data, e.g. school meals play a role in attracting and keeping kids in school; they are valued by parents, insufficient contribution from parents prevents varying the menu; etc. Interviews were also exploratory and not meant as generalizations. The semi-structured interviews and focus group discussions were used to examine these in the local context to generate hypothesis’ that WFP could test in its programme in the future. A key mitigation measure to mitigate limitations in the research was to triangulate the data gathered through the qualitative data collection. 44. During the initial piloting in Cap Haïtien the evaluation team established a shared approach for conducting focus groups. The team used a common protocol for conducting FGD. Topics of discussion focused on parents and community engagement. The focus group discussions were developed at a later stage based on the initial findings from key informant interviews. See draft interview guides in Annex 12. 45. The inception mission allowed the evaluation team to develop a better understanding of what data collection activities were possible to undertake but at the same time the team was very aware of the possibility that further political unrest could disrupt the evaluation. Final Report 111 | P a g e The evaluation team remained in contact with WFP regarding the security situation in Haiti. In addition, the national members regularly monitored social media platforms to remain informed regarding any potential situations. Every measure was taken from a planning standpoint to ensure the success of the evaluation and at the same time, the safety of the evaluation team. The fieldwork was planned early so that the team could adjust the dates of the fieldwork if necessary and allow the conducting of the fieldwork before the end of the school year, which the CO anticipated to end around June 5, as the official calendar is often not fully respected, or activities such as exams would be disrupted otherwise. If needed and sufficiently safe the national team members would carry out more of the fieldwork. k. Ensuring Quality 46. WFP's decentralised evaluations conformed to WFP and UNEG ethical standards and norms. Accordingly Goss Gilroy Inc was responsible for safeguarding and ensuring ethics at all stages of the evaluation cycle. This included, but was not limited to, ensuring informed consent, protecting privacy, confidentiality and anonymity of participants, ensuring cultural sensitivity, respecting the autonomy of participants, ensuring fair recruitment of participants (including women and socially excluded groups) and ensuring that the evaluation resulted in no harm to participants or their communities. 47. The evaluation respected the norms and standards of the United Nations Evaluation Group (UNEG) that were first established in 2005 through the document “Norms and Standards for Evaluations in the United Nations system.”143 The evaluation team respected the United Nations evaluation functions that should ensure credible and useful evidence to inform and strengthen the work of the WFP in Haiti. In 2005, UNEG adopted the foundational document, Norms and Standards for Evaluation in the United Nations System. The evaluation was conducted by an independent team of consultants who have no conflict of interest, i.e. have no stake in the programme being evaluated. This being said, the evaluation team ensured that it provided a balanced account of the relevance and effectiveness of the programme activities. The evaluation findings reflected the views of all partners and participants, including donors, implementers, and beneficiaries. These issues were monitored and managed during the implementation of the evaluation. If any additional ethical issues arose during the implementation of the evaluation, they were recorded and managed in consultation with the Evaluation Manager. 48. At the level of data collection, the evaluation team ensured to test data collection tools, e.g. the first interviews and focus groups allowed to test the instrument for clarity. We ensured that all team members had a common understanding of the questions. We held a meeting at the beginning of data collection to compare our first interviews to ensure that they were understood by stakeholders as intended. The data collected was stored in a common document accessible by all team members. This allowed for the peer review of data collected by different team members. Furthermore, as mentioned earlier, the team held frequent meetings to discuss their findings. This allowed to find out if there were unanticipated issues or findings that needed further exploring as part of the study. The 143 UNEG (2005) Norms for Evaluation in the UN System. Towards a UN system better serving the peoples of the world; overcoming weaknesses and building on strengths from a strong evidence base. United Nations Evaluation Group Final Report 112 | P a g e team made use of the guidelines and technical notes developed by WFP at each stage of the evaluation and for deliverables ensured that they met WFP’s expectations and standards. l. Details on Evaluation Stages and Timelines Stage 1: Document Review 49. During this stage the evaluation team started their in-depth review of documents. This included reports from WFP to McGovern-Dole, including output and outcome data; other reports and evaluations conducted on school meals in Haiti; case studies and lessons learned from Haiti and other countries. Stage 2: Preparation for Field Work Phase (April 1 to 19) 50. Logistical matters were ongoing. Prior to arriving in Haiti, the evaluation continued to undertake data analysis related to McGovern-Dole and other matters related to WFP programming in Haiti. Discussions were held with WFP to identify additional stakeholders that needed to be interviewed especially as it related to Objective Two. Stage 3 Field Work Two Cap Haïtien (April 22 to 27) 51. Upon arrival in Haiti the international team members travelled directly by air to Cap Haïtien joining up with the Haitian members of the team. It was suggested that WFP’s Monitoring and Evaluation Manager, join the evaluation team for the first few days in Cap Haïtien where the precise methodology for gathering data at the school/community level was tested and finalized collectively by the team. After testing the approach, the evaluation team was divided into two smaller teams to allow visits to more schools. With two Haitian nationals on the team and the hiring of translators for the international members of the team, there was considerable flexibility to use all team members in different configurations. 52. During the fieldwork planning stage (April 1-19), the team endeavoured to identify production projects with farmers associations and cooperatives to gage their interest and capacity to produce for school meals initiatives. The field work was based on a six-day work week. Data collection were undertaken from Monday to Friday. Generally, data collection in schools was conducted in the morning, between 8 am -1 pm. Afternoons were devoted to typing and entering notes into data collection templates or other data collection activities, including meeting with parents if it could be arranged. Saturdays were devoted to visits and focus groups to groups of producers or moving from one location to another and team analysis work. Sunday was a rest day. Stage 4 Field Work Saint Marc/Artibonite (April 29 to May 4) 53. The evaluation team spent the week in the Saint Marc area further employing the techniques developed in Cap Haïtien. Before beginning the work in Saint-Marc the team met to determine if any modifications should be made in the approach. As well, at this time the best ways in terms of combinations of team members were made clear. Upon completing the field work in Saint-Marc, the team will travel to Port au Prince to provide an informal debriefing to WFP. Stage 5 Remote Activities Largely in Support of Objective Two (May 6 to 17) 54. After the completion of these first two stages the evaluation team worked remotely (May 6 to 17) beginning to work more proactively on Objective Two of the evaluation through a desk review of documents. Some Skype interviewing were also undertaken. As well this period of time was used to organize the data collected in Cap Haïtien and Saint Marc. Final Report 113 | P a g e Stage 6 Finalize Data Collection in Port-Au-Prince May 27 to 31 Port au Prince 55. For this stage, evaluation team members were separated again into two small teams. However, for this stage the tasks were divided. One team was dedicated to completing fieldwork in schools in the Port au Prince area. A second team conducted interviews and meetings related to Objective Two. It was felt that five working days would be sufficient for completing the McGovern-Dole related field work and completing the interviews for the broader situational analysis. However as noted above, it was deemed that if it was necessary, an additional day or two might be added. At the end of this work period, a full debriefing with power point presentation was made to WFP. Stage 7 Analysis and Draft Report (June1 to 14) 56. Preparation and submission of draft 0 of final report. Stage 8 Final report (June 14 to August 9) 57. Draft 0-1-2-3 of the final report. The following table provides an overview of the key stages, activities and timelines of the evaluation. Table 10: Summary of Evaluation Activities and Timelines Stages Dates Key Activities Stage 1 April 1 to 19 Remote work literature and document review Stage 2 April 2 to 18 Preparation for fieldwork (Easter Holiday April 19-21) Stage 3 April 23 to 27 Fieldwork Cap Haïtien area Stage 4 April 29 to May 3 (May 1 is a holiday in Haiti) Fieldwork in Saint-Marc/Artibonite area May 6-7 Fieldwork in Port-au-Prince area Stage 5 May 10 to 2h Remote work McGovern-Dole data analysis and Information gathering for contextual analysis (other models and governance) of SM in Haiti Stage 6 May 27th to 31st Fieldwork Port au Prince area and interviews and meetings for contextual analysis (other models and governance), debriefing for WFP staff. Presentation of preliminary findings to WFP at the end of the mission Stage 7 June 1 to 26 June 26- July 7 (WFP) Continue analysis and prepare draft final report (draft zero) and submit to WFP Haiti and regional evaluation managers for initial review (June 17-21). Stage 8 July 8-12 QA) Update report based on comments from WFP Haiti and regional evaluation managers and submit draft 1 for quality assurance (QA), which needs 7 days. Respond to comments from QA and submit draft 2 to July 15 to 19 (WFP) July 22-25 July 26- August 8 (RG) August 9 (final version) for comments from WFP Haiti office, which needs 1 week. Respond to comments from WFP Haiti office and submit to Reference Group Review by reference group, which needs two weeks Prepare final version of the evaluation report. Present report remotely. Final Report 114 | P a g e Table 11: Overall evaluation activities and timelines Timelines in weeks Activities 1 Mar 2 3 4 Apr 5 6 7 8 May 9 1 0 1 1 12 Jun 13 1 4 1 5 16 Jul 17 1 8 1 9 20 Aug 21 Inception mission to Haiti (March 6-12) Draft inception report (March 26) WFP review of inception report (March 26-29) Revised Inception Report (April 8) Review by WFP and final version (April 9-15) Start document review (April 15) Logistic for fieldwork (April 1-18) Data collection mission 1 / Cap Haïtien, Artibonite, PAP (April 23-May 4) Finalize document review Data collection mission 2 / PAP (May 27-31) Present preliminary findings (May 3 - May 31) Data analysis and draft zero (June 1-26) Draft 0 review by WFP (June 27-July 7) Prepare Draft 1 submit to QA (July 8) Draft 1 review by QA (July 8-July 15) Prepare Draft 2 and submit to WFP (July 16-20) Draft 2 review by WFP (July 21-26) Prepare Draft 3 and submit to RG (July 27-August 03) Draft 3 review by RG (August 4-8) Submit final evaluation report (August 9) Present report to stakeholders Final Report 115 | P a g e Annex 6: Documents Reviewed Title Link to Main Evaluation Matrix Questions 1. Policy and Guidance La Politique et Stratégie Nationale d’Alimentation Scolaire PSNAS 1.1 Manuel d’opération du PNCS, version 1.0 – draft 0, dans le cadre de la mise en oeuvre de la politique et stratégie nationales d’alimentation scolaire (PSNAS), September 2017 1.1 School Feeding Programme, McGovern – Dole, FY 15: 2016 / 17, FY 16: 2017 / 18 and 2018 / 19, March 2019 All Rethinking School Feeding: Social Safety Nets, Child Development, and the Education Sector, 2009 All State of School Feeding Worldwide, 2013 All Theory of Change Model, School Feeding 5.1 Corporate Evaluation Strategy (2016-2021), May 2016 All School Feeding Strategy Handbook, June 2014 All School Meals, Monitoring Framework and Guidance, November 2017 All • SABER: SABER - Approche Systémique pour Améliorer les Résultats de l’Éducation 4 Rubriques du cadre SABER pour l’alimentation scolaire 4 SABER‐Alimentation Scolaire, Outil de collecte de données Pour Alimentation scolaire 4 SABER–School Feeding, Manual for SABER-SF Exercise, May 2016 4 MENFP Plan décennal d’éducation et de formation, Octobre 2018-Septembre 2028 4 2. Programme Planning Food for Progress and McGovern-Dole Indicators and Definitions, August 2016 2 Education en chiffres - MENFP 2015-16 1; 2 Aperçu de la situation d’insécurité alimentaire chronique en Haïti, Octobre 2015 1.1 WFP 2015-021-1_Insécurité Alimentaire Chronique Haiti_Octobre2015_A4 1.1 IPC Analyse de l’insécurité Alimentaire Aigue, Décembre 2018 1.1 La situation de la sécurité Alimentaire du pays, Journée Mondiale de l’Alimentation, Octobre 2017 1.1 Local and Regional Food Aid Procurement Program, Fact Sheet, March 2018 1.1 Performance Monitoring Plan (PMP), 2015 2 E-mail sending McGovern-Dole Letter not requiring MAD Indicator, 2016 1.1 • McGovern-Dole 2016 Performance Monitoring Plan (PMP), 2016 2 WFP-Haiti FY 2016 McGovern-Dole Results Framework, Results Framework #1 (4 files) 2 WFP-Haiti FY 2016 McGovern-Dole, Results Framework #2 (3 files) 2 WFP Haiti FY16 McGovern-Dole Project Level Results Framework, 2016 (unable to open file) 2 McGovern-Dole Second Year Letter of Notification for Agreement # FFE 521-2016/016- 00 - Haiti 2 • Other Logframes CSP Detailed Logframe – Main, May 2017 (in PDF and Word) 2 Full Logframe 2.0 for the project 200150 (Development Project): Assistance to the National School Feeding Programme in Haiti 2 Final Report 116 | P a g e Title Link to Main Evaluation Matrix Questions Plan de suivi, d'examen et d'évaluation du PSP/PSPP, June 2018 2 Cadre de mesure de rendement (CMR), Novembre 2017 2 • WFP Project Documents HAITI Development Project 200150, B/R No.: 3, “Support for the National School Meals Programme” All • Monitoring Data BDD compilée sept @ juin 2017 3.1 Compilation Rapports Techniques 2014-2016 3.1 Données de suivi FAM 2015-2016 3.1 Ecole - Outcome 2015 3.1 Fiche Ecole Data Feb-Jun17 3.1 Fiche Elève Data Feb-19Oct17 3.1 • School feeding data McGovern-Dole - 2016 Compilation Rapports Technique Cantine Scolaire 3.1 ▪ Monitoring Report Données de suivi FAM 2016 3.1 ▪ Outcome Measurement Database Taux Ins-Ret 2016 3.1 School feeding data McGovern-Dole – NEW – 2017 ▪ Monitoring Report Fiche Elève Data 2017 3.1 Report Sep-Dec 2017 3.1 ▪ Monthly Report BDD compilée sept 2016 @ juin 2017 3.1 ▪ Outcome Measurement Données Outcome School feeding (Retention Enrollment Passrate) V2 3.1 ▪ Output Data Consolidation des outputs-SPR 12.12.2017 avec correction M&E V. 30.01.2018. 3.1 • School feeding data McGovern-Dole – NEW - 2018 ▪ Enquête Structure Copy of ME18_Enquete_Structure_CS_v13_BND Résulat final complet 3.1 ▪ Monitoring Report Report Jan-Jun 2018 3.1 ▪ Outcome Measurement Outcome data 2018 2.2 ▪ Output Data ACR 2018-Other Output SF 2.1 Tableau des irregularités et des sanctions appliqués aux écoles dans le cas de non￾respect du contrat 2.1 • School feeding data McGovern-Dole – NEW - Plan 2019 Programme Cantine Scolaire - Liste des écoles pour réchauds à gaz All • School Lists Liste des écoles 2016-2017 actualisée 09092016 All Liste écoles MNP 2017-2018 All Final Report 117 | P a g e Title Link to Main Evaluation Matrix Questions School List 17-18 All • Data_Baseline_AGV_WFP Base_Ecole_Bénéficiaire.sav All Base_Ecole_Témoin.sav All Base_Eleves_Bénéficiaires.sav All Base_Eleves_Non_Bénéficiaires.sav All 5. Reporting • McGovern-Dole Semi Annual Reports Performance Indicators (Results): HAITI FY15 Award, December 2015 to March 2016 2 Performance Indicators (Results): HAITI FY15 Award, April to September 2017 2 Semi-Annual Report (April 1 – September 30) - Narrative 2 Performance Indicators (Activities): Haiti FY 2016 Award, April to September 2017 2 Haiti Semi-annual Report Narrative, 1st April 2018 – 30th September 2018 2 Haiti Semi-annual Report Narrative, 1st April 2018 – 30th September 2018 2 Commodity Direct Distribution: FY15 Award, October 2016 to March 2017 2 Performance Indicators (Results): HAITI FY15 Award, April to September 2016 2 Haiti (McGovern-DoleFY16) Semi-Annual Report April 2018-31 September 2018 18.10.2018 2 Haiti’s McGovern-Dole School Feeding Programme, Narrative Progress Report, (01/04/2016 - 30/09/2016) 2 Semi annual Report Narrative 1st October 2017 – 31st March 2018, Haiti, Narrative 2 Haiti’s McGovern-Dole School Feeding Programme, Narrative Progress Report, (01/10/2015-31/03/2016) 2 Semi annual Report Narrative October 1, 2016 – March 31, 2017 2 • Standard Project Report (SPR) Assistance to the National School Feeding Programme in Haiti, Standard Project Report 2015 2 Assistance to the National School Feeding Programme in Haiti, Standard Project Report 2016 2 Assistance to the National School Feeding Programme in Haiti, Standard Project Report 2017 2 6. Special Studies • EGRA Projet d’achats locaux pour les cantines scolaires de Petite Rivière de Nippes, Juin 2016 5 CRS, Ligne de base – HAITI GAGNE - Mai 2018, LANGUE – CREOLE & FRANÇAIS, (Cahier de Stimuli) 2.2 Fey done _ formation peer tutoring 24 Oct 2018 2.2 PROJET, Mc Govern Dole Food for Education and Child Nutrition Program (FFE) 2.2 • Gender Enquête Mortalité, Morbidité et Utilisation des Services en Haïti 2016-2017 (EMMUS￾VI) All Nora Pistor, Gender Analysis of WFP Haiti – School Feeding Program, September 2019 All • MNP Final Report 118 | P a g e Title Link to Main Evaluation Matrix Questions Étude formative sur l’utilisation des poudres de micronutriments dans les Cantines scolaires en Haiti, Rapport final, Septembre 2016 5.1 Évaluation du processus de mise en oeuvre du projet «Micronutrient for Every Meal Initiative (MEMI) » en Haïti, Rapport final, Janvier 2018 5.1 Process Indicators (to Monitor Activities and their Outputs) 5.1 Final Report to the World Food Programme in Haiti, Micronutrients for Every Meal Initiative, Sept 2017 - March 2018 (School year 2016 – 2017), April 2018 5.1 Étude nutritionnelle en milieu scolaire en Haïti, mesures anthropométriques, Rapport de l’Étude, Décembre 2017 5.1 • WASH Évaluation d’utilisation du purificateur d’eau distribué par la World Vision, April 2018 5.1 7. Other • Case Studies: Smart School Meals, Nutrition-Sensitive National Programmes in Latin America and the Caribbean, A Review of 16 Countries, July 2017 5 Case Study 1: Botswana — National School Feeding Programme 5 Case Study 2: Brazil — National School Feeding Programme 5 Kenya: Home-Grown School Meals Programme 5 Global School Feeding Source Book, Lessons from 14 Countries, 2016 5 Research Report - Part II, Food Distribution, Production, and Consumption in Haiti (with special emphasis on the emerging role of snack foods and prospects for marketing high quality peanut-based snack foods), April 2015 5 Research Report - Part I, Food Distribution, Production, and Consumption in Haiti (with special emphasis on the emerging role of snack foods and prospects for marketing high quality peanut-based snack foods), April 2015 5 Learning from Evaluations of School Feeding: a Synthesis of Impact Evaluations This synthesis forms Volume I of Annex I to the report ‘School Feeding Policy: a Policy Evaluation’, OE/2012/002, June 2012 5 Kore Lavi, Cantine Scolaire Pilote-Plateau Central, Décembre 2015 5 • Canada Documents Étude d’évaluation: programme de cantines scolaires PAM – Canada 2015-2017, Novembre 2017 5 Cadre de mesure de rendement (CMR), Novembre 2017 5 WFP Haiti – Annual Report, WFP School Feeding Programme, October 2015 – June 2016 5 Thème Principal: Capitalisation des études et expériences sur les réchauds et énergies utilisées pour la préparation des repas en milieu scolaire, Rencontre du groupe de travail dans le cadre de la mise en œuvre de la PSNAS, juillet 2016 5 Home-Grown School Feeding, A Framework to Link School Feeding with Local Agricultural Production 5 How School Meals Contribute to the Sustainable Development Goals, A Collection of Evidence Fighting, February 2017 5 ▪ WV Avr-Jun 18 Rapport narratif trimestriel du partenaire coopérant World Vision, Pour la période d’Avril 2018 à Juin 2018, Juin 2018 5 ▪ WV Jan-Mar 18 Final Report 119 | P a g e Title Link to Main Evaluation Matrix Questions Rapport Trimestriel de La Gonâve, Programme Cantine Scolaire, Décembre 2017 5 Rapport Trimestriel du département du centre, Programme Cantine Scolaire, Avril 2018 5 WV-Matrice pour rapportage des outputs-Octobre 17- Mai 18.xls 5 WV-Matrice pour rapportage des outputs-Janvier- Mars 18_.xls 5 Les constats de la mission au centre/ les actions à prendre par la WV, E-mail 5 ▪ WV Sep-Dec 17 EDU - Rapport trimestriel partenaire coopérant, Octobre 2016 à Janvier 2017 5 • Etude Nutrition Vincent 20180825 nutrition status data x4 5 20180825 telephone interview nutrition 5 20180825 telephone interview nutrition.xml 5 • Homegrown School Feeding Projet d’achats locaux pour les cantines scolaires de Petite Rivière de Nippes, Juin 2016 5 Termes de Référence (TDR) révisés Étude sur le « Projet Pilote d’Achats Locaux pour les Cantines Scolaires à Petite Rivière de Nippes » 5 Evaluation of Local Purchasing and School Canteen Pilot Project in Petite Riviere de Nippes, Report 1 of 3, January 2017 5 Evaluation of Local Purchasing and School Canteen Pilot Project in Petite Riviere de Nippes, Report 2 of 3, June 2017 5 Evaluation of Local Purchasing and School Canteen Pilot Project in Petite Riviere de Nippes, Report 3 of 3, February 2018 5 • Maps School locations Map School Feeding 180523.jpg All WFP 2016-034_School_Responsible-Partners_A1.jpg All WFP 2018-117_School_Location_A1.jpg All WFP 2018-117_School_Location-Roads_A1.jpg All Tableau des irrégularités et des sanctions appliquées aux écoles dans le cas de non￾respect du contrat, 2018-2019 1.3 • Studies by Tim Schwartz Focus Groups, Evaluation of Local Purchasing and School Canteen Pilot Project in Petite Riviere de Nippes, September 2018 5 Research Report, part I, of 2 parts, Food Distribution, Production, and Consumption in Haiti, (with special emphasis on the emerging role of snack foods and prospects for marketing high quality peanut-based snack foods), April 2015 5 Research Report, part 2, of 2 parts, Food Distribution, Production, and Consumption in Haiti, (with special emphasis on the emerging role of snack foods and prospects for marketing high quality peanut-based snack foods), April 2015 5 Evaluation of Local Purchasing and School Canteen Pilot Project in Petite Riviere de Nippes, Report 1 of 3, January 2017 5 Evaluation of Local Purchasing and School Canteen Pilot Project in Petite Riviere de Nippes, Report 2 of 3, January 2017 5 • UNICEF Documents Final Report 120 | P a g e Title Link to Main Evaluation Matrix Questions Rapport de l’évaluation de base EGRA, Projet Haïti Gagne, Lire, Écrire et Réussir, Octobre 2017 2.2 The Essential Package: Twelve Interventions to Improve Health and Nutrition of School-age Children 5.1 Essential Package Implementation Guide, July 2010 5.1 Liste actualisée Écoles Haïti Gagne 5.1 Projet Haïti Gagne: Lire, Écrire et Réussir, Plan International, Enquête de la ligne de base, Rapport final, Septembre 2017 5.1 Étude sur les enfants hors de l’école en Haïti, Rapport final, Novembre 2018 1 World Bank World Bank Group (2018), Child and Adolescent Health and Development Optimizing Education Outcomes: High-Return Investments in School Health for Increased Participation and Learning, Volume 8. Disease Control Priorities Third Edition. 1-2-3 Climate-Agriculture Cadre de Programme Pays (CPP Haiti 2013-2016), Juillet 2013 1 Climate Change Information Fact Sheet, HAITI, September 2015 1 Haiti Ag Climate, Executive Summary 1 Une étude exhaustive et stratégique du secteur agricole/rural haïtien et des investissements publics requis pour son développement, Chapitre 8. Renforcement du système d’innovation, Version finale, Juin 2016 1 CTA Working Paper 17/09, Building the Evidence Base on the Agricultural Nutrition Nexus: Haiti, June 2017 1 Haiti and FAO, Strengthening food and nutrition security and enhancing rural incomes, 2017 1 HAITI Staple Food Market Fundamentals, FEWS Net, March 2018 1 Republic of Haiti Agricultural and Agroforestry Technological Innovation Programme (PITAG), Final project design report, Main report and appendices, IFAD, March 2018 1 Climate Risk and Adaptation Country Profile, April 2011 1 Community Government Private Sector When Academics Become Humanitarians: A Post-Disaster Programme by the State University of Haiti, Summary Report, November 2017 5 McGovern-Dole International Food for Education and Child Nutrition Program, Fact Sheet, November 2018 5 School feeding data McGovern-Dole • 2016 ▪ Monitoring Report Données de Suivi FAM 2016 2.1 ▪ Monthly Report Rapports Technique Cantine Scolaire 2.1 ▪ Outcome Measurement Database Taux Ins-Ret 2016 2.1 Outcome report 2016 2.2 • 2017 ▪ Monitoring Report Fiche Eleve Data 2017 2.1 Final Report 121 | P a g e Title Link to Main Evaluation Matrix Questions Report Sep-Dec 2017 2.1 ▪ Monthly Report BDD compilée sept 2016 @ juin 2017 2.1 ▪ Outcome Measurement Données Outcome School feeding V2 2.2 ▪ Output Data Consolidation des outputs-SPR 2017. 2.1 • 2018 ▪ Data study McGovern-Dole (Data baseline AGV) Base_Ecole_Bénéficiaire.sav (unable to open file) 2.1 Base_Ecole_Témoin.sav (unable to open file) 2.1 Base_Eleves_Bénéficiaires.sav (unable to open file) 2.1 Base_Eleves_Non_Bénéficiaires.sav (unable to open file) 2.1 ▪ Enquete Structure ME18_Enquete_Structure_CS_v13_BND 2.1 ▪ Monitoring Report Report Jan-Jun 2018 2.1 ▪ Monthly Report School Report Sep18- Jan19 2.1 ▪ Outcome Measurement Échantillonage outcome 2018 2.1 Outcome data 2018 2.2 ▪ Output Data ACR 2018-Other Output SF 2.1 • 2019 ▪ Monitoring Report Données Suivi 2019 2.1 Données suivi 2019_Group 2.1 Final Report 122 | P a g e Annex 7: Stakeholders Interviewed Organization Name Title Gender Ambassade de France en Haïti Thomas Poirier Attaché de Coopération Male Ananda Marga Universal Relief Team (AMURT) Richard Beauséjour Directeur Opérationnel Male Association de producteurs RACPABA Dieula Bien Aimé Directeur Opérationnel Male Association des producteurs Agricoles de Verettes (APAV) Directeur Opérationnel Male Bureau de Nutrition et Développement (BND) Michelle Routhier Coordonnatrice de Programmes Female BND – Bureau de Nutrition et Développement Emmline Toussaint Grants Manager Female Bureau de Nutrition et Développement (BND) Jonas Abraham Agronome Male CARE Jelson Select Superviseur education pour Care Et cantines Male CARE Haiti Mathieu Lucius Chef de Projet Male CARE Haiti Jean Jude Valcourt Manager Cantine Male Centre d'étude et de coopération internationale Edriss Gonel Coordonnateur Régional Nord Male Catholic Relief Services (CRS) Lucka Jouthe Beauvil Literacy Programme Manager Female Direction Départementale du l’Éducation (DDE) du Nord-Est Fritz Jacques Directeur Départemental Male InterAmerican Development Bank (IADB) Mathieu Vladimir Operations Associate Male InterAmerican Development Bank (IADB) Sabine Rieble-Aubourg Social Sector Department, Education Division4 Female Ministère de l’agriculture des ressources naturelles et du développement rural Gary Augustin Consultant Male Programme d'innovation technologique pour l’agriculture et l’agroforesterie du Nord￾Est (PITAG) Frantz Estimable Regional Coordinator Male Ministère de l’agriculture des ressources naturelles et du développement rural Nord Pierre Mary Brutus Directeur Départemental Adjoint Male Final Report 123 | P a g e Organization Name Title Gender Ministère de l’agriculture des ressources naturelles et du développement rural Nord Frantz Dorvil Directeur Départemental Male Ministère de la santé publique et de la population (MSPP) Dr. Marhonne Directeur Male Organisation des femmes de l’Artibonite OFDAD Roseline Executive Director Female PALMIS ENEJI Jean Farreau Guerrier Directeur Général Male PALMIS ENEJI Dafney Jean Pierre Business Development Manager Female Programme d'innovation technologique pour l’agriculture et l’agroforesterie du Nord￾Est (PITAG) Jimps Donatien Manager Male Programme national de cantine scolaire (PNCS) Phanèse Laguerre Coordonatrice Générale Female Programme national de cantine scolaire (PNCS) Nord-Est Hamlil Bernardin Directeur Départemental Male Programme national de cantine scolaire (PNCS) Nord Ary DASSAS Coordonnateur régional Male Société Holding d'investissement S.A. Dafney Jean Pierre Business Development Manager Female SWITCH S.A. F. Maxwell Marcelin General Manager Male Réseau de coopérative caféière Nord/Nord-Est Luckner Bonheur Excecutive Director Male UNICEF Mirko Forni Head of Education Male USDA / FAS Fritzner Cledo Agricultural Specialist Male USAID Edwidge Vilbrun Programme Éducation Female USAID Parola Joseph Programme Éducation Female USAID Jonathan Greenham Contractor - Technical Advisor Male World Food Program Ronald Tran Ba Huy Country Director Male World Food Program Raphaël Chuinard Deputy Country Director Male World Food Program Ilaria Martinatto Program Director Female World Food Program Antoine Morel-Vulliez Head of Social Protection Male World Food Program Petorking Charles M&E Data Focal Point Male World Food Program Ilaria Martinatto Program Director Female World Food Program Viviana Sandoval Head of School Feeding Female World Food Program Claudia Schwarze Head of Monitoring and Evaluation Female World Food Program Damieta Mendes Head of Education Female Final Report 124 | P a g e Organization Name Title Gender World Food Program Leana Chery Gender Specialist Female World Food Program James Randolph Romain M&E Data Focal Point Male World Food Program Thomas Gabriel VAM Officer Male World Food Program Emilie Swalens VAM Officer Female World Food Program Rossella Bottone Regional Monitoring Advisor Female World Food Program Myrlande Norelia Officier Nutrition Female World Food Program Sylviory Dubuisson Chargé PAMSI Male World Vision International Lesly Michaud Head of Programme and Operations Male World Bank Yves Jantzem Head of Education Male World Bank Michel-Ange Augustin Consultant Male Final Report 125 | P a g e Annex 8: Data Collection Instruments Guide 1: Schools (directors, committees), Parents and Cooks Introduction: The World Food Programme (WFP) is undertaking an evaluation of the School Meal Programme. Consultants from the firm Goss Gilroy Inc. (GGI) are conducting the evaluation on behalf of the WFP. As part of the evaluation process, interviews and focus groups are being conducted to assess the effectiveness and sustainability of the programme. Your participation is voluntary and all interviews are confidential. Information gathered through this interview will be aggregated and no individual will be identifiable in the final report. If you have questions about the evaluation you can contact Louise Mailloux at lmailloux@ggi.ca. To start, please describe your role or involvement in the school meals programme. 1. How well is the meal programme working in your school in terms of logistics, quantity and quality of food provided, storage, etc.? 2. Which factors have contributed to the success of the school meals programme in your school? What factors have impeded the success of the programme? What could be done differently? Consider: logistics; complaint mechanism; community participation; school infrastructure; ongoing financing. What could be done differently? 3. How are women/men, boy/girls represented in the management and supervision committees? What is discussed in these meetings? How are decisions made? What could be done to make them work better? 4. Is the targeted age group of children 5-12 years of age as beneficiaries for the programme correctly defined or should it be expanded to include children of the 3rd cycle? Why? How could it be done with the current level of food provided? 5. What can be done locally to increase the variety and nutrition value of food in the meals, in addition to what is provided by WFP? Who would be involved? What would need to be in place to make it work? 6. Is the food distributed by WFP locally available? What would be the likely impact on markets of purchasing locally? 7. In addition to having food provided, what could be done to help children have better learning outcomes? 8. Are parents (fathers and mothers) contributing to school meals at the school? How well is it working? What could be done to improve participation? Are there other potential sources of revenue generation that could be fostered? 9. How are cooks compensated? Is this compensation perceived as fair? What would equitable solutions for the compensation of cooks and are there ways to implement this? 10. What are strategies developed by to operate the canteen taking into account constraints and unforeseen: insecurity, inflation? 11. Have women gained knowledge and skills through their participation in the school meals programme? To what extent can they be used for their own needs or to sustain actions in their communities? Final Report 126 | P a g e 12. Are women and men represented equally in the various structures at the school level (SF management committees; supervision committees; parent-teachers committees and decision-making? What is the level of involvement and participation of women and girls and what are the reasons driving or inhibiting it? 13. What could be done to encourage the wider community to become more involved in the school meal programme and to support the school more generally? What other training could be of value to parents and teachers as a means to improve the programme's sustainability? Energy Efficiency 14. What are the main energy sources used in schools? Have there been efforts to improve energy efficiency? What have been the result of the experience? What have been Questions for schools with literacy activities 15. How well are the activities to teach children to read and write Créole and French working? What could be improved? How? 16. Is the school meals programme helping to reinforce efforts to improve literacy? Please explain What other factors if any contribute to the success of the literacy activities? Are there any challenges? Questions for schools with WASH activities 17. Who are managing the WASH activities? And boys and girls involved in managing the equipment? Are the WASH activities introduced in the school(s) making a difference for children? How? What is working well? What could be improved? How? 18. Do you have any additional comments? Thank you for your input Final Report 127 | P a g e Guide 2: WFP and Operating Partners Introduction: The World Food Programme (WFP) is undertaking an evaluation of the School Meal Programme. Consultants from the firm Goss Gilroy Inc. (GGI) are conducting the evaluation on behalf of the WFP. As part of the evaluation process, interviews and focus groups are being conducted to assess the effectiveness and sustainability of the programme. Your participation is voluntary and all interviews are confidential. Information gathered through this interview will be aggregated and no individual will be identifiable in the final report. If you have questions about the evaluation you can contact Louise Mailloux at lmailloux@ggi.ca. To start, please tell me about your role in the school meals programme. 1. How well is the school meal programme working in your school(s) in terms of logistics, quantity and quality of food provided, storage, etc.? What could be done differently? Which factors have facilitated the programme in the school? What factors have impeded the success of the programme? 2. Are women/men, boy/girls equally represented in the school meals management and supervision committees? How well are they working? What could be done to make them work better? 3. What can be done locally to increase the variety of food and nutritional value in the meals, in addition to what is provided by WFP? Who would be involved? What would need to be in place to make it work? 4. Which activities linked with environmental sustainability in the school meal programme have worked/not worked? Why? What have been challenges and opportunities in this regard? How can women be empowered to take a leadership role in environmental sustainability issues with regards to the school meal programme? 5. In addition to food being provided, what could be done for children to improve their learning outcomes? 6. What could be done to increase the number of parents (fathers and mothers) who contribute to the school meal at the school? What could be done differently? 7. What could be done to encourage the wider community to become more involved in the school meals programme and to support the school more generally? What have been other experiences in Haiti or elsewhere in terms of community involvement and what have been the results and lessons learned? 8. How are the cooks at the school compensated? What could be done locally to ensure that cooks get adequate compensation for their time and effort? Are there other potential sources of revenue generation that could be fostered? 9. Based on the experience of other communities in Haiti or in other countries, what can be done differently/locally to increase the quantity and variety food as well as nutritional value of meals? 10. How well are the activities to teach children to read and write Créole and French working? (question will be adapted for children) What could be improved? How? 11. Are the WASH activities introduced in the school(s) making a difference for children? How? (questions will be adapted for children FG) What is working well? What could be improved? How? Final Report 128 | P a g e 12. What is the role of the departmental education authorities in relation to the school meal programme? Could/should this role be increased? How? 13. Could the government manage the school meals programme without external support at this stage? Why? 14. What would need to be in place to allow for a full transition to the Government of Haiti taking on full responsibility for school feeding in the country? 15. Are there any factors impeding this at the moment? How can they be overcome? 16. What can be done to better facilitate the involvement and support of the agricultural sector and the Ministry of Agriculture in school meals programme? 17. Do you have any other comments to make about the school meals programme? Thank you for your input Final Report 129 | P a g e Guide 3: Ministry of Education (national and departmental) and PNCS Introduction: The World Food Programme (WFP) is undertaking an evaluation of the School Meal Programme. Consultants from the firm Goss Gilroy Inc. (GGI) are conducting the evaluation on behalf of the WFP. As part of the evaluation process, interviews and focus groups are being conducted to assess the effectiveness and sustainability of the programme. Your participation is voluntary and all interviews are confidential. Information gathered through this interview will be aggregated and no individual will be identifiable in the final report. If you have questions about the evaluation you can contact Louise Mailloux at lmailloux@ggi.ca. To start, please describe your role in the school meals programme. How familiar are you with the School meal program supported by McGovern Dole? 1. Is the programme reaching the most vulnerable children and communities? 2. What are the main difficulties to working with most vulnerable schools? What are their characteristics? How can they be addressed if at all? 3. Who are the local actors involved in the school selection and exclusion processes? Are communities and departmental educational authorities involved in the school selection? 4. Is the food served to school children in adequate quantity to cover children dietary requirements, taking into account the age groups and gender? How could nutrition be improved if at all? 5. Is the targeted age group of children 5-12 years of age as beneficiaries for the programme correctly defined? or should it be expanded to include children of the 3rd cycle? IS there a way to reach more vulnerable children with school meals with the funding available? 6. How does the school meal programme benefit boys and girls? Are there differences in outcomes for boys and girls overall? Literacy activities 7. How well are the activities to teach children to read and write Créole and French working? What could be improved? What have been the results for children? Are they different for boys and girls? Why? 8. Does the school feeding programme play a role in efforts to improve academic achievement for boy and girls? WASH activities 9. To your knowledge, are the WASH activities introduced in school(s) making a difference for children? Why? What is working well? What could be improved? How? Environmental Sustainability Activities 10. Which activities linked with environmental sustainability in the school meal programme have worked/not worked? Why? What have been challenges and opportunities in the Haitian context in this regard? How can women be empowered to take a leadership role in environmental sustainability issues with regards to the school meal programme? Capacity Building Final Report 130 | P a g e 11. What technical assistance was received for the government to be able to manage the school meals programme nationally? What have been the main results of capacity-building activities of national and local institutions? 12. What would need to be in place to allow for a full transition to the Government of Haiti taking on full responsibility for school feeding in the country? - At the legal and policy levels; - At the technical level; - At financial level; - At organizational level? 13. Are there any factors impeding this at the moment? How can they be overcome? What would be the cost of a nationally managed school meal programme? 14. Other activities 15. What can be done to better facilitate the involvement and support of the agricultural sector and the Ministry of Agriculture in school meals programming? Thank you for your input Final Report 131 | P a g e Guide 4: Donors, Multilateral Agencies Introduction: The World Food Programme (WFP) is undertaking an evaluation of the School Meal Programme. Consultants from the firm Goss Gilroy Inc. (GGI) are conducting the evaluation on behalf of the WFP. As part of the evaluation process, interviews and focus groups are being conducted to assess the effectiveness and sustainability of the programme. Your participation is voluntary and all interviews are confidential. Information gathered through this interview will be aggregated and no individual will be identifiable in the final report. If you have questions about the evaluation you can contact Louise Mailloux at: lmailloux@ggi.ca. To start, please describe your role in the school meals programme. How familiar are you with the WFP School Meal Programme supported by McGovern Dole? 1. How should schools be selected and who should be involved in the selection process? Should local actors be more involved? If so, how? 2. Is it better to give food to fewer schools and focus on the most vulnerable or provide less for a larger number of children? What could be fair and have a greater impact? 3. Based on the experience of other communities in Haiti or in other countries, what can be done in the context of Haiti to increase the quantity, variety and nutritional value school meals? 4. How important are secondary activities such as literacy and WASH, to the success of school meals programmes and vice versa? Do they make a significant difference in academic achievement for girls and boys (e.g. ability to learn, success)? How? 5. Which activities linked with environmental sustainability in the school meal programme have worked/not worked? Why? What have been challenges and opportunities in the Haitian context in this regard? How can women be empowered to take a leadership role in environmental sustainability issues with regards to the school meal programme? 6. What technical assistance was received for the government to be able to manage the school meals programme nationally? What was the outputs and outcomes of the technical assistance received? 7. What would need to be in place to allow for a full transition to the Government of Haiti taking on full responsibility for school feeding in the country? - At the legal and policy levels; - At the technical level; - At financial level; - At organizational level? 8. Are there any factors impeding this at the moment? How can they be overcome? What would be the cost of a nationally managed school meal programme? 9. What can be done to better facilitate the involvement and support of the agricultural sector and the Ministry of Agriculture in school meals programme? How can initiatives to increase local production help improve the socio-economic status of women or empower in the process? 10. What else could possibly be done to attract more attention and support for school meals programmes and generally make them more dynamic? 11. Do you have any additional comments to make about school meals programming? Final Report 132 | P a g e Guide 5: NGOs and Producer Organizations Introduction: The World Food Programme (WFP) is undertaking an evaluation of the School Meal Programme. Consultants from the firm Goss Gilroy Inc. (GGI) are conducting the evaluation on behalf of the WFP. As part of the evaluation process, interviews and focus groups are being conducted to assess the effectiveness and sustainability of the programme. Your participation is voluntary and all interviews are confidential. Information gathered through this interview will be aggregated and no individual will be identifiable in the final report. If you have questions about the evaluation you can contact Louise Mailloux at: lmailloux@ggi.ca. To start, please describe your role in school meals programming. How familiar are you with the School meal program supported by McGovern Dole? 1. Based on the experience of other communities in Haiti or in other countries, what can be done in the context of Haiti to increase the quantity and variety of food being bought locally? 2. For local producers: Based on your experience what has helped or hindered your participation in school meal programmes? 3. What is the role of women and men in food production for school meals in your area? Are women benefiting equally from their involvement in food production for school meals? How? 4. Are you aware of experiences in Haiti in terms of community involvement in school meal programming and what have been the results and lessons learned? 5. What needs to be in place for the Haiti to be able to take over the responsibility of the national school meal programme? Are there any factors impeding this at the moment? How can they be overcome? 6. What can be done to better facilitate the involvement and support of the agricultural sector and the Ministry of Agriculture in school meals programme? 7. What else could possibly be done to attract more attention and support for school meals programmes and generally make them more dynamic? 8. Do you have any other comments regarding school meals programming? Thank you for your input Final Report 133 | P a g e Annex 9: Additional Evidence for EQ 1 1. Table 12 compares the PNSAS and McGovern-Dole objectives in order to assess the alignment between the two policies. This analysis is based on documents and not on the actual implementation of the program, which can slightly differ. Table 12: Comparison between PNSAS and McGovern-Dole programme objectives PNSAS Objectives McGovern-Dole Objectives and analysis GO1. Provision of quality food services through the universal provision of quality food in schools, student attendance is continuous at the basic school level, cognitive ability is high, student nutrition is adequate, and in shocks, schoolchildren and their families benefited from food support strengthening their resilience, while promoting the use of environmentally friendly energy sources. SO 1.1: The design and implementation of school feeding is effective and efficient, and has fully capitalized on educational benefits for targeted populations while promoting community participation. McGovern-Dole program documents mention the support to the GoH in analyzing options for ‘the optimization of the costs of its school feeding program by undertaking a cost-analysis of all the school feeding models in place’ Improving education through literacy activities is at the core of the program and one of the program SOs. It includes the distribution of reading kits and school supplies. Level of Alignment: Good SO 1.2: Complementary nutritional health interventions The composition of the food basket and the implementation of complementary nutritional health interventions make it possible to ensure adequate nutrition for schoolchildren (fortification, nutritional SO2 of the McGovern-Dole program in Haiti aims at the ‘Increased Use of Health and Dietary Practices’, what confirms the central role of health interventions in the McGovern-Dole Theory of Change. Six activities are related to this objective including health and nutrition practices, distribution of water purification tablets and soap, among others.144 WASH interventions are at the core of the program and have been implemented in every school, to the exception of water purifiers that have been distributed to no more than 110 schools. Deworming and school gardens have also been implemented in some areas. Nevertheless, some of the activities mentioned in the PSNAS are not part of the design of the McGovern-Dole program. Food baskets, composed of three products, do not allow for the 144 McGovern-Dole 2.1. Improved Knowledge of Health and Hygiene Practice; McGovern-Dole 2.2. Improved knowledge of Safe Food Preparation and Storage Practices; McGovern-Dole 2.3. Increased Knowledge on Nutrition; McGovern-Dole 2.4. Increased Access to Clean Water and Sanitation Services; McGovern-Dole 2.5. Increased Access to Preventive Health Interventions; McGovern-Dole 2.6. Increased Access to Requisite Food Prep and Storage Tools and Equipment. Final Report 134 | P a g e PNSAS Objectives McGovern-Dole Objectives and analysis education, deworming, water and sanitation). improvement of food diversity. Some complementary nutritional health activities are not considered such as the purchase of fresh food products or food fortification, excepted fortified oil. Also, while the PSNAS call for a snack and a hot meal amounting to 1040 Kcal per day (65 percent of the daily food intake), McGovern￾Dole daily food ration has a total energy content limited to 601 kcal per day. Level of Alignment: Fair to Good SO 1.3: Response to shocks: Interventions facilitating the reopening of classes following a shock No mention is made of the use of the program as a safety net in case of an adverse shock. Schools were used to facilitate the targeting of families impacted by the earthquake. Level of Alignment: Low SO 1.4: Environment. Use of environmentally friendly energies helps to limit the degradation of forest cover and encourages the use of such energies by communities near schools. No mention is made of the use of renewable energy sources or energy-saving stoves in program documents. The indictor on ‘improved fuel efficient stoves’ was removed for FY16. Level of alignment: Low GO2: Support Local Economy and Local Food Production Thanks to the purchase of local food products and the involvement of the private and voluntary sectors, local economies are revitalized and producers increase their food production in a sustainable manner and benefit from agricultural and agro-business services specifically geared to those supplying food. SO 2.1: Local Purchase: exclusive use of local products, acquired through a gradually established purchasing mechanism, taking into account the cost, the capacity of the operators, the quality of the food and the stability of the food supply. McGovern-Dole does not use the local and Regional Food Aid Procurement (LRP) mechanisms in Haiti and there is no plan to do it on the short term (see below). Program documents mention the conduction of a feasibility study on the incorporation of locally grown peanuts and the support to the improvement of locally produced peanuts. Level of alignment: Low Final Report 135 | P a g e PNSAS Objectives McGovern-Dole Objectives and analysis SO 2.2: Agricultural and Agribusiness Services The agricultural and agribusiness services of the MARNDR, the MCI and the technical partners are coordinated and oriented towards the producers and actors participating in the school feeding supply efforts. Mention is made to the support to activities focusing on improving the quality and reducing post-harvest losses of locally produced peanuts. Level of alignment: Low SO 2.3: Private and associative sector The private and voluntary sector is engaged throughout the supply chain from school feeding, from agricultural production to food distribution, through processing and the preparation and distribution of dishes in schools. Developing the private sector is not among the program’s objectives. Level of alignment: Low GO 3: National Capacity Development. National institutions are able to finance, coordinate and manage the implementation of a universal, decentralized school feeding program with structures capable of ensuring the accountability of operators at the level of Departments and Communes. SO 3.1: Strengthening National Institutions The PNCS and the Local Authorities strengthened and supported by Municipal Commissions of Education are able to implement a universal SO1 Foundational Results include 4 objectives (McGovern-Dole 1.4.1 to 1.4.4) aiming at increasing the capacity of government, improve the policy and regulatory framework, increase government support and engagement of local organization and community groups. The McGovern-Dole mentions among its activities to ‘develop and implement a comprehensive capacity strengthening plan that will systematically address gaps in capacity, boost national and community ownership, and ensure a substantial move towards the establishment of a national school feeding Final Report 136 | P a g e PNSAS Objectives McGovern-Dole Objectives and analysis and decentralized school feeding program program’. It includes: training to the GoH at national and decentralized levels, monitor the progress of the National School Feeding Policy (NSFP), lead the development of an overall School Feeding Operational Guideline, including strategies for topics surrounding HGSF145. However, most activities could not carried out as expected (see Question 4). Local authorities have been also involved through joint monitoring visits and have been informed of any decision. Level of Alignment: Good (in terms of policies) SO 3.2: Financing National mechanisms ensure the financing of universal school feeding and funds are disbursed on time at all levels of implementation. SO1 Fundamental Results mention as a priority to ‘Assist MENFP/GoH to develop annual funding strategies for national school feeding program’ and work with the School Feeding Task Force to advocate for the inclusion of a budget line for school feeding. So far no budget line has been included in the national budget. Level of Alignment: Good SO 3.3: Coordination. The multi-sectoral coordination of the implementation of the national school feeding program is effective at the central level as well as the Departments. A large number of potential allies and partners are mentioned in the Result Framework: government (DINEPA, FAES, PNCS, MAST, MARNDR, MDE, MENFP), donors, UN agencies. These collaborations are mentioned in broad terms in project documents and few concrete examples of such collaboration are given. Level of Alignment: Fair SO 3.4. Accountability The governance, control and monitoring and evaluation structure guarantees the accountability of actors at all levels of implementation and is supported, at the municipal level, by No specific mention is made of accountability system in project documents but accountability systems implemented by WFP in the context of the McGovern-Dole program are very comprehensive. Level of Alignment: Good 145 Agreement WFP- McGovern-Dole FY16. Final Report 137 | P a g e PNSAS Objectives McGovern-Dole Objectives and analysis municipal education committees and parents' committees. . SO 3.5. Communities participate in the design, implementation, local governance and evaluation (social audit) of the school feeding program and contribute significantly to its funding. McGovern-Dole program aims at boosting national and community ownership, as mentioned in the objective 2.7.4. Increased the engagement of Local Organization and Community Groups through the development of community strategy to strengthen participation and ownership of the school meals program. Level of Alignment: Good Final Report 138 | P a g e Figure 13: Map of Chronic Food Insecurity by Livelihood Area, IPC 2015 Final Report 139 | P a g e 2. The IPC considered the following factors: Food consumption (SCA, HDDS, /HHS, number of meals, frequency of meals, Vita A, minimal dietary diversity among kids and women, ratio of spending on starch, percent of children aged 6 to 23 months receiving and acceptable minimum diet (RMA), Nutritional indicators (anaemia prevalence, stunting), Vulnerability (poverty threshold, ratio of food expenses, iodized salt, possessions related to likelihood), Food availability, access, use , Access to water, food production, price, food basket, employment level . The Chronic IPC published in 2015 was based on data up to 2013 – some being used in the third section of this chapter. It important to mention here that the IPC use livelihood areas as the main unit of analysis, and not departments. It explains why one department can have more than one classification. Table 13: Vulnerability ranking in McGovern-Dole-assisted districts146 Ranking (1=best; 43=worse) Departmen t Districts Communes (In bold the Commune s part of the McGovern￾Dole Program) Complete/ Partial Coverage Househol d Hunger Scale (Moderat e + Severe) Food Consumptio n Group (Bordeline + Poor) Food Diversit y Index (less than 3 groups) Synthetic Score of Food Security (Severe + Moderate ) West Port-au￾Prince Port-au￾Prince, Delmas, Cité Soleil, Tabarre, Carrefour, Pétionville, Kenscoff, Gressier Partial 24 32 21 32 Croix-des￾Bouquets Croix-des￾Bouquets, Thomazea u, Ganthier, Cordillon, Fonds-des Partial 10 22 8 20 146 In order to better interpret the Food Security Index, it is important to know that districts were ranked from 1 to 43 according to four food security indicators, where 1 is the least food insecure and 43 is the most food insecure. Only districts where McGovern-Dole program is active are considered. The table shows that almost of the districts in which the programme operates are vulnerable. The main indicators reported are: The Household Hunger Scale (HHS) - a proxy of food access, built around 3 questions on perceptions of a household on varying degrees of hunger, by the number of times the household has experienced hunger within past 30 days; the Food Consumption Score is based on dietary diversity, frequency of food consumption and nutritional intake of Food Diversity Index: households were asked about the 23 foods consumed during the last 24 hours preceding; The Food Security Synthetic Score was determined by triangulation of the three previous indicators. Final Report 140 | P a g e Ranking (1=best; 43=worse) Bois, Verrettes Arcahaie Arcahaie, Cabaret Complete 9 16 4 13 North Cap￾Haïtien Cap￾Haïtien, Quartier Morin, Limonade Complete 28 40 35 38 L’Acul du Nord Acul du Nord, Plaine du Nord, Milot Complete 29 38 33 30 Saint Raphael St￾Raphael, Dondon, Pignon, Ranquitte, La Victoire Complete 39 25 31 26 Grand Rivière du Nord Grand Rivière du Nord, Bahon Complete 12 37 39 33 Borgne Borgne, Port￾Margot Partial 43 41 43 43 Plaisance Plaisance, Pilate Partial 11 33 24 31 North-East Fort￾Libert é Fort￾Liberté, Ferrier, Perche Partial 27 20 19 21 Ouanamin the Ouanamin the, Capotille, Mont￾Organisé Complete 31 24 20 23 Trou du Nord Trou du nord, Sainte￾Suzanne, Terrier Rouge, Caracol Partial 38 31 13 28 Final Report 141 | P a g e Ranking (1=best; 43=worse) Vallières Vallières, Carice, Monbain Crochu Complete 21 2 18 2 Artibonite Gonaïves Gonaïves, Ennery, L’Estère Partial 1 21 11 15 Gros Morne Gros Morne, Terre Neuve, Anse Rouge Partial 2 4 1 1 Saint￾Marc Saint￾Marc, Verrettes, La Chapelle Partial 35 43 40 42 Dessalines Dessalines, Petite Rivière de l’Artibonit e, Grande Saline, Desdunes Partial 36 42 42 41 Marmelade Marmelad e, Saint￾Michel de l’Attalaye Complete 19 6 17 7 Centre Mirebalais Mirebalais, Saut d’eau, Boucan Carré Partial 17 14 28 17 3. Temporary and permanent suspensions have been common over the programme’s duration, as the number of beneficiary schools declined. At the end of the school year 2017- 18, WFP was asked to reduce the number of attended schools from 497 to 401 due to budgetary restrictions. Therefore, in August 2018, all beneficiary schools were assessed by WFP and its partners through the basis of an objective criteria covering aspects related to poor management, school enrollment, quality of reporting, food management, and regular operation of the canteen, among others. No criteria related to the vulnerability of the school was used. However, in vulnerable areas, an excluded school is, when funding is available, and Final Report 142 | P a g e is replaced by a new school in the same commune, making sure that vulnerable communities are not losing this important support.147 4. The WFP country office (CO)has set up an elaborate system of exclusion with clear criteria and protocols. A monitoring document (Table des irrégularités) lists 12 potential irregularities such as problems of hygiene or inadequate stock management. Each infraction is accompanied by a list of actions that are to be implemented at the first, second and third occurrence. In most cases, when WFP monitors or partners witness an infraction, he/she informs the WFP hotline. The first action taken is to send a letter to inform the focal point at the DDE, BDS and PNCS and to suspend temporarily the delivery of food items. If corrective measures are not taken, WFP can decide on the termination of the programme or request to replace the school director involved in the infraction. Table 14: List of Suspension Criteria WFP Assessment Criteria148 Examples of irregularities reported149 Evaluation of the infrastructure, availability of kitchen utensils; Supervision Committee of the school; Case of irregular management of the school meals (delays in submission or reports, irregular use of food items, exclusion of schools for non-payment of the contribution) Large difference between reported and actual attendance figures; Less than five days of canteen per week without valid justification; Difference between physical and actual stock; inadequate use of monitoring tools; food diversion to pay the cooks, for transportation, for personal use; feeding cycle 3 children. Source: WFP Monitoring Documents, Tableau des irréguarités 147 Interview with WFP School Feeding Program. 148 Implemented at the end of the 2017/18 school year to reduce the number of schools participating to the program. 149 Errors that can lead to a temporary or permanent exclusion of the program if not respected. Final Report 143 | P a g e Annex 10: Additional Evidence for Question 2 a. Key Outputs, Planned versus actual schools and beneficiaries b. Assessment of all output indicators of the PMP 1. Table 14 provides a list of all the output indicators that have been used during the program (FY15 and 16), and how have they have been treated. The main indicators have been treated in the core of the evaluation report. Table 14: Summary of output achievement assessment against programme indicators Indicators Assessment McGovern-Dole SO1 – Improved Literacy of School-aged children Number of individuals benefiting indirectly from McGovern-Dole-funded interventions Target achieved but decreasing. Number of individuals benefiting indirectly from McGovern-Dole-funded interventions Indicator later cancelled This indicator applies to programs where take￾home rations benefits the whole household and has not been calculated. Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text See the section on outcome indicators in the main text Number of textbooks and other teaching and learning materials provided as a result of McGovern-Dole assistance Target Achieved See main text Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of McGovern-Dole assistance Target Almost Achieved Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of McGovern-Dole assistance Target Achieved School Administrators were trained on cantine management and on the use of monitoring tools. Number of school administrators and officials trained or certified as a result of McGovern-Dole assistance Target Achieved Number of students regularly (80 percent) attending McGovern-Dole supported classrooms/schools Target Achieved See outcome section: attendance rate. Final Report 144 | P a g e Indicators Assessment Number of students enrolled in school receiving McGovern-Dole assistance Target achieved but decreasing See above: Number of individuals benefiting indirectly from McGovern-Dole-funded interventions: Number of educational policies, regulations, and/or administrative procedures in each of the following stages of development as a result of McGovern-Dole assistance (Stage 2): Target Not Achieved. During the three-year duration of the program, WFP contributed to the writing and dissemination of two major documents: - The National School Feeding Policy (PNSAS) was approved in January 2018 by the Minister of Education and other ministries; - WFP also supported the writing of the Policy’s Strategic Implementation Plan - The School Feeding Operational Guideline (SFOG), which was developed and submitted to the Ministry of Education and he still pending for approval. Value of new public and private sector investments leveraged as a result of McGovern-Dole assistance (Host Government) Indicator not calculated then cancelled for FY16 Number of public-private partnerships formed as a result of McGovern-Dole assistance Indicator not calculated then cancelled for FY16 McGovern-Dole SO 2: Increased Use of Health and Dietary Practices Percent of school-age children receiving a minimum acceptable diet Not calculated and then cancelled for FY16 Activity 1: Capacity Building: Local, regional, national level Number of School Feeding Operational Guidelines/Handbooks developed as a result of McGovern-Dole assistance Not calculated and then cancelled for FY16 Number of school feeding policy dissemination events organized as a result of McGovern-Dole assistance Target Achieved for FY 15 then Indicator Cancelled for FY16 Dissemination events included the launch of the Table Sectorielle en Alimentation Scolaire and the National School Feeding Policy document (July 16) and the Regional Rice Fortification Workshop (Aug 16) Number of sensitization campaigns implemented at the community level on the importance of school feeding and education as a result of McGovern-Dole assistance: Target achieved then cancelled for F16 Cooperating partners have led the organization of presentation meetings in each school at the beginning of the school year, including the entire school communities. Number of south-south cooperation/exchange activities with GoH participation supported as a result of McGovern-Dole assistance Target not achieved for FY 15 then cancelled Organization of the Regional Rice Fortification Workshop in August 2016. Activity 2: Distribution: Storage Equipment, Food Preparation Tools and Eating Utensils Final Report 145 | P a g e Indicators Assessment Number of schools with improved storage equipment, food preparation tools and eating utensils as a result of McGovern-Dole assistance Activity and indicator cancelled Number of schools with improved fuel efficient stoves as a result of McGovern￾Dole assistance Activity and indicator cancelled Number of schools equipped with cooking and eating utensils, banners/information and visibility materials as a result of McGovern-Dole assistance: Indicator cancelled Even though these indicators do not appear in the PMP for FY16, activities have been implemented and McGovern-Dole semi-annual narratives report eating and cooking utensils were provided to 89,085 schools. Activity 3: Provide School Meals All Indicators repeated from above Activity 4: Training: Commodity and Warehouse Management Number of supply chain, warehouse and commodity management training conducted as a result of McGovern-Dole assistance Indicator cancelled Number of local sub recipient partner members trained as a result of McGovern-Dole assistance: Indicator cancelled Activity 5: Training Food Preparation and Storage Practices Number of trainings on food preparation and storage practices conducted as a result of McGovern-Dole assistance: Indicator cancelled Number of individuals trained in food preparation and storage practices conducted as a result of McGovern-Dole assistance: Indicator cancelled Number of schools provided with training materials on storage, management, and tracking donated commodities as a result of McGovern-Dole assistance: Target almost achieved then cancelled for FY16 WFP Haiti developed guidance material for school cooks regarding safe food handling and hygiene, approved by the Ministry of Health. World Vision disseminated training materials on food preparation, storage practices and good health and nutrition practices at the school level, that have been approved by the Ministry of Health. Activity 6: Training: Good Health and Nutrition Practices Number of individuals trained in child health and nutrition as a result of McGovern-Dole assistance: Target not achieved Final Report 146 | P a g e Indicators Assessment Number of training on good health and nutrition practices conducted as a result of McGovern-Dole assistance Indicator cancelled Number of trainings on good health and nutrition practices conducted as a result of McGovern-Dole assistance: Indicator cancelled In all assisted schools, cooks , monitoring staffs of Cooperating Partners and government staffs were trained in hygiene food preparation and nutrition by World Vision. However the total number of people trained (2,152) is inferior to the annual target of 3438 individuals trained. Number of schools using an improved water source Target achieved World Vision distributed 110 water purification (water chlorine purifiers with replaceable batteries) in assisted schools in the three USG development corridors. Other WASH activities (not referred in the PMP) include the distribution of 15,600 water purification tablets. c. Key Outcomes: planned versus actual; Enrollment; Pass and Retention Rates 2. At the Outcome level, the key outcome indicators are the enrolment rate, the pass rate and the retention rate. Figures are currently only available at the national level for all the Schools that WFP supports through different grants. See figures 16, 17 and 18 below. Figure 14: Percentage change in enrollment rate, year-per-year (WFP national level) Source: WFP Monitoring Data 6 0.3 4.8 5.62 -3.2 6 -1.1 5.4 4.29 -2.4 6 -0.7 5.1 4.95 -3.4 Project End Target SPR 2015 SPR 2016 SPR 2017 ACR 2018 Male Female Overall Final Report 147 | P a g e Figure 15: Pass rate in WFP assisted schools (WFP national level) Source: WFP Monitoring Data Figure 16: Retention rate WFP assisted primary schools (WFP national level) Source: WFP Monitoring Data 60 60 60 77.77 78.85 77.85 79.2 81.1 80.2 72.4 74 73.1 72 74 73 Male Female Overall Pass rate in WFP-assisted primary schools Project End Target SPR 2015 SPR 2016 SPR 2017 ACR 2018 96 95 95.5 93 92 92.4 92.5 92 92.2 92 91.6 91.8 91 91 91 Male Female Overall Retention Rate in WFP-assisted Primary Schools Project End Target SPR 2015 SPR 2016 SPR 2017 ACR 2018 Final Report 148 | P a g e Figure 17: WFP monitoring data of the McGovern-Dole Programme Result: MGD SO1 Improved Literacy of School-Age Children F 0 88383 0 0 92595 NA M 0 88382 0 0 94249 NA Number of individuals benefiting indirectly from USDA-funded interventions 0 176765 0 0 0 313423 313423 NA Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text NA NA NA NA NA NA NA NA NA 6 12 Number of textbooks and other teaching and learning materials provided as a result of USDA assistance NA NA NA NA NA 6440 6280 20531 21383 NA 19511 Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance NA NA NA NA NA 120 120 NA 112 NA 116 Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance NA NA NA NA NA 30 30 NA 29 34 34 Number of school administrators and officials trained or certified as a result of USDA assistance NA NA NA NA NA 40 40 40 42 42 42 Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance NA NA NA NA NA 160 160 108 112 115 116 Number of daily school meals (breakfast, snack, lunch) provided as a result of USDA assistance 0 57978783 0 0 20731569 16963996 16963996 NA 11679000 5007491 8249810 F 0 0 92595 M 94294 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance 176765 0 0 186889 103439 103439 NA 158967 144294 116735 116735 Number of individuals benefiting directly from USDA-funded interventions. The program does not distribute take home rations. 107107 107227 Performance Monitoring Plan - Haiti FY2015 and FY 2016 McGovern-Dole MGD Reporting Period Baseline 1 Oct 17 - 31 March 1 April - 30 Sept 1 Oct 18- 31 March Performance Indicator and Activity output indicator Gender PMP FY15 PMP FY16 1 Oct 15 - 31 March 1 April - 30 Sept Target FY 2016 1 April - 30 Sept 1 Oct 16 - 31 March Target FY 2017 Target FY 2018 Result: MGD 1.2.1/1.2.1.1 Reduced Short-Term Hunger/Increased Access to Food Result: MGD 1.2.1.1/1.3.1.1 Increased Access to Food (School Feeding) Result: MGD 1.2.1.1/1.3.1.1/2.5 Increased Access to Food (School Feeding)/ Number of school-aged children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance 176765 144294 116735 158967 144294 158967 103439 103439 NA Final Report 149 | P a g e Result: MGD 1.3 Improved Student Attendance F 53030 0 0 89817 M 53030 0 0 91465 Result: MGD 1.3.4 Increased Student Enrolment F NA 0 94929 92595 M 95633 94294 Result: MGD 1.4.2 Improved Policy and Regulatory Framework Number of educational policies, regulations, and/or administrative procedures in each of the following stages of development as a result of USDA assistance (Stage 2) 2 0 1 1 1 2 5 NA 2 0 1 Value of new public and private sector investments leveraged as a result of USDA assistance (Host Government) TBD 0 0 0 0 0 NA NA NA NA Number of public-private partnerships formed as a result of USDA assistance TBD 0 0 0 NA NA NA NA Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance 350 0 0 595 382 382 NA 497 115 401 94057 Performance Monitoring Plan - Haiti FY2015 and FY 2016 McGovern-Dole MGD Reporting Period Baseline 1 Oct 17 - 31 March 1 April - 30 Sept 1 Oct 18- 31 March Performance Indicator and Activity output indicator Gender PMP FY15 PMP FY16 1 Oct 15 - 31 March 1 April - 30 Sept Target FY 2016 1 April - 30 Sept 1 Oct 16 - 31 March Target FY 2017 Target FY 2018 Number of students enrolled in school receiving USDA assistance 176765 76245 Number of students regularly (80%) attending USDA supported classrooms/schools Result: MGD 1.4.4 Increased Engagement of Local Organizations and Community Result: MGD 1.4.3/1.4.4 Increased Government Support/ Increased Engagement of local Organizations and Community Groups 62063 72407 103439 103439 120815 109663 170181 153825 128482 NA NA Final Report 150 | P a g e Percent of school-age children receiving a minimum acceptable diet 20 0 0 NA NA NA NA NA Activity 1: Capacity Building: Local, regional, national level Number of school feeding policy dissemination events organized as a result of USDA assistance 3 1 1 1 NA NA NA NA Number of sensitization campaigns implemented at the community level on the importance of school feeding and education as a result of USDA assistance NA 1 1 NA NA NA NA Number of south-south cooperation/exchange activities with GoH participation supported as a result of USDA assistance 3 0 1 0 NA NA NA NA Number of schools with improved storage equipment, food preparation tools and eating utensils as a result of USDA assistance 325 0 0 0 NA NA NA NA Number of schools with improved fuel efficient stoves as a result of USDA assistance 325 0 0 0 NA NA NA NA Number of schools equipped with cooking and eating utensils, banners/information and visibility materials as a result of USDA assistance 325 0 0 0 NA NA NA NA All Repeated From above Performance Monitoring Plan - Haiti FY2015 and FY 2016 McGovern-Dole MGD Reporting Period Baseline 1 Oct 17 - 31 March 1 April - 30 Sept 1 Oct 18- 31 March Performance Indicator and Activity output indicator Gender PMP FY15 PMP FY16 1 Oct 15 - 31 March 1 April - 30 Sept Target FY 2016 1 April - 30 Sept 1 Oct 16 - 31 March Target FY 2017 Target FY 2018 Result: MGD SO 2: Increased Use of Health and Dietary Practices Activity 2: Distribution: Storage Equipment, Food Preparation Tools and Eating Activity 3: Provide School Meals Final Report 151 | P a g e Number of supply chain, warehouse and commodity management training conducted as a result of USDA assistance 4 0 0 0 NA NA NA NA Number of local sub recipient partner members trained as a result of USDA assistance 15 0 0 0 NA NA NA NA Number of trainings on food preparation and storage practices conducted as a result of USDA assistance 130 0 0 1 NA NA NA NA Number of individuals trained in food preparation and storage practices conducted as a result of USDA assistance 650 0 0 0 NA NA NA NA Number of schools provided with training materials on storage, management, and tracking donated commodities as a result of USDA assistance 650 0 0 595 NA NA NA NA Activity 6: Training: Good Health and Nutrition Practices F 650 0 0 140577 M 650 0 0 86334 Number of training on good health and nutrition practices conducted as a result of USDA assistance 130 0 0 1 NA NA NA NA Number of trainings on good health and nutrition practices conducted as a result of USDA assistance 1300 0 0 226911 NA NA NA NA Number of schools using an improved water source NA NA 25 25 NA NA 40 70 Commodities Vegetable Oil 134,01 150,0 20,0 91,6 Bulgur 2301,27 Bulgur wheat 1516,0 518,0 1054,7 Green Peas 326,48 Nuts 314,8 Pulses 323,8 108,8 283,0 Activity 5: Training Food Preparation and Storage Practices Performance Monitoring Plan - Haiti FY2015 and FY 2016 McGovern-Dole MGD Reporting Period Baseline 1 Oct 17 - 31 March 1 April - 30 Sept 1 Oct 18- 31 March Performance Indicator and Activity output indicator Gender PMP FY15 PMP FY16 1 Oct 15 - 31 March 1 April - 30 Sept Target FY 2016 1 April - 30 Sept 1 Oct 16 - 31 March Target FY 2017 Target FY 2018 Scheduled to arrive btw April and August 2016 Distribution s to start at Number of individuals trained in child health and nutrition as a result of USDA assistance Activity 4: Training: Commodity and Warehouse Management Two weeks of distribution reported in following report 3438 3438 NA 941 418 793 Final Report 152 | P a g e d. Additional Evidence on Gender (cross-cutting issue) 3. Concurrently to the evaluation, WFP conducted an independent gender analysis of WFP school feeding programing in Haiti.150 It examined the gendered cultural norms in schools and the communities they serve. The key findings relevant to the McGovern-Dole evaluation and the recommendations of the gender analysis report are presented below. Key findings: Gender roles in the schools and households 4. Typically, female teachers are working with younger students and male teachers with elder students or more advanced classes. Most committees are headed by male chairpersons or presidents who are responsible for all major and ultimate decisions. Female committee members appear to be less involved in decision-making, less vocal than their male counterparts and usually only sharing their ideas after being explicitly encouraged to. Women as well as men reported constraints for women to enter leadership positions for three reasons: women’s lack of political connections which were needed to get elected or nominated for certain positions, women’s lacking affiliations to existing support networks, and women’s lack of self-confidence due to their subordinate position in the society. Female school directors participating in the research reported constant barriers that endanger their positions, including threats and intimidations by male members of the communities. 5. Education materials, which are part of the national curriculum, contain socio-cultural gender bias and stereotypes, including division of labour between women and men. Furthermore, girls are represented only 33% as active participants (e.g. sports), while boys are consistently representing the absolute majority of all descriptions and drawings of exercises. Parents, teachers and students, only have a basic knowledge of human rights and gender equality laws. Boys and girls do not generally interact much at school. 6. At household level, men were reported to be the major decision makers. Most parents and community entities participating in the research displayed a deeply rooted socialization of male domination with strong constructions of images and narratives of masculinity and femininity. 7. Roles and responsibilities around the preparation of food at home are highly gendered with women being mainly responsible. Boys participation in household chores was usually seen as an exception in the case the female responsible, i.e. mother or sister, was not there to accomplish them. Regarding the distribution of prepared food at home, women and girls are typically the last in the hierarchy to be served (or serving themselves last). In precarious situations of food scarcity or food insecurity, this can mean that women may receive less of important nutrients. Access to Resources 8. Women were mostly aware of their lack of access to and control over resources, i.e. economic resources (finance), land titles, water and education, compared to men who more often were land title owners, better educated, and earning an income. Many women shared feeling disempowered and economically disadvantaged through societal and familial structures. 150 Nora Astor, Gender analysis of WFP School Feeding Program, September 2019. Final Report 153 | P a g e These barriers were perceived as unsurmountable and keeping them from community-based and individual development. 9. Gendered concepts, such as the male leadership in households and society, women’s subordination and responsibility for reproductive tasks, were often linked to the Christian belief system as taught through the local churches. Sexual Education and Gender-based Violence 10. Sexual education is not part of the school curriculum and sexuality remains a taboo not communicated about. Sexual harassment and gender-based violence against girls was identified as a problem but no specific mechanisms were in place to address issues or improve behaviour. School Canteens and WAH Facilities 11. Cooking, cleaning and serving food is seen a female responsibility, justified by their ability to undertake these tasks compared to men and boys. Boys and men carry items needed in the canteen, such as fire wood and charcoal and heavy bags to the kitchen but these tasks are also accomplished by the female cooks in the schools. 12. Kitchen facilities were deemed as inadequate in rural schools where cooking is done on an open fire with firewood. Cooks work fulltime as volunteers without payment in difficult situations. 13. In several schools, latrines were not sufficient to meet the needs and lacking sanitary disposals for girls during menstruation. Suggestions for Action 1) Suggestions for action in schools • Develop a strategy for behavior change in schools to achieve equal sharing of nutrition related responsibilities in schools. o Provide trainings for male students in food preparation, distribution, cleaning in schools and including household related tasks in the homes. o Arrange students’ competitions around topics of food preparation as well as nutrition related information. o Include parents, mostly fathers as well as mothers, in these trainings and in competitions. • Develop partnership with Sommet Education (or similar local NGOs in other departments where available). o Students can be supported to learn to play musical instruments, singing, dancing and theater o Theater groups could be established and used in order to disseminate important messages around gender and nutrition as part of the SBCC. • Improvement of school meals. o Offer remuneration for cooks. o Improve cooking infrastructure in schools, e.g. by providing improved cooking stoves, protection from animals, tables etc. o Diversify meal composition for students including local products for canteen. Final Report 154 | P a g e o Include a ratio for cooks to eat. o Improve storage of food to avoid deterioration of ingredients. o Revise hygiene situation in schools, consider auto-sufficient models e.g. with rain water harvesting, local soap production or alternative means. o Involve women in food delivery chain and all remunerated positions in the SFP, not only as cooks. • Organize events in schools to raise awareness on nutrition and gender equality. o Develop handicraft items to raise money for school improvement. o Organize competition, awareness raising events to gain community sensitization and contribution to be used for school improvement. • Introduce sex education, GE and hygiene in school curricula or extra-curricular learning. o Develop training contents with teachers and school staff. o Develop learning sessions for students to be sensitized, involving parents where possible. o Develop extra-curricular activities, e.g. gender sensitive games to strengthen inter-gender socialization of students. • Include sports and environmental protection into curricular and extra-curricular activities. o Develop action learning tools, e.g. in combination with sports classes. o Design girls’ sports program, e.g. in cooperation with UN Women, promoting football or other sports particularly for girls in schools. o Ensure girls’ equal participation in related activities, e.g. tree planning not only for boys, football lessons etc. • Develop strategy for Zero Tolerance of sexual harassment in schools. o Organize sensitization sessions for teachers, school staff on sexual harassment, e.g. in cooperation with local women’s groups or organizations. o Develop strategy to offer professional counseling possibility for female and male students, with particular focus on sexual harassment, non-violent communication, through specifically trained female Monitoring Agents. • Establish appropriate WASH facilities in the schools. o Revise latrine buildings and ensure double number of latrines are made available for girls. o Ensure gender appropriateness of latrines and washing areas, e.g. considering hygiene needs of menstruating girls/ women, garbage disposal etc. 2) For communities/ local level: • Improve awareness in communities on gender equality related to education and nutrition. o Develop communication and awareness campaigns on sharing of household responsibilities including food preparation. o Sensitize communities/ families on harmful effects of girls being raised as child caretakers, develop strategies to address the issues, e.g. through (participatory) Final Report 155 | P a g e cost-benefit analysis, awareness raising on family planning and appropriate strategy development in communities. o Cooperate with local religious, important cultural and political entities in the communities, as well as with parents groups in the schools to develop appropriate messages. o Offer training and sensitization particularly for local religious and political stakeholders in the communities on GE in education and nutrition. • For the effective implementation of the SBCC, identify and use female role models from the local area as messengers for women’s empowerment. o Identify local singers, actresses/ actors to create attractive messages, e.g. songs, radio snippets, banners, posters, community events etc. o Set up protection measures for these women before they are being exposed to prevent from physical attacks from the communities. o Offer training, awareness raising on women’s empowerment, gender equality, women’s rights etc. o Offer self-defense courses for women. • Strengthen existing women’s organizations in the communities. o Involve local women’s organizations staff in training and campaign development on GEWE. o Create sounding board including female trainers and promoters in communities with support/ monitoring of local women’s organizations. • Provide training for local partners, including AMURT staff and local government officers. o Topics: GE, sexuality and sex education, hygiene, nutrition. o Offer refreshment/ in-depth seminars on a regular basis, e.g. biannually. • Improve awareness on sexuality and education. o Raise awareness on physical integrity and boundaries, non-violent communication and negotiation possibilities between all genders for students, parents, professors at schools. o Design sustainable, visible and audible material, e.g. in cooperation with UNICEF for teaching material and methods, based on already existing communication and sensitization material. 3) For government and partners: • Develop joint Gender Action Plan. o Involve in participatory dialogue with all partners including school directories o Involve religious leaders and cultural entities from local areas. o Develop TOC/ detailed logframe for GEWE in program. Determine the details for implementing Action Plan and Strategy (responsibilities, resources, scope, outcomes, outputs, indicators, timelines). • Offer GE training and capacity building plan for partners. o Include partners from central and departmental levels (BDE, BND). o Involve MCFDF/DCFDFs for capacity building and knowledge sharing (consider knowledge sharing rather than cooperation). Final Report 156 | P a g e • Revise directly discriminating practices supported in the programme. o Change strategy of volunteer female cooks to end direct discrimination of female program participants, e.g. by providing fixed salary or adequate guaranteed compensation for cooks. o Improve cooking situation in the school canteens, e.g. by including improved cooking stoves. • Support the establishment of appropriate WASH facilities in the schools. o Revise latrine buildings and ensure double number of latrines are made available for girls. o Ensure gender appropriateness of latrines and washing areas, e.g. considering hygiene needs of menstruating girls/ women, garbage disposal etc. • Reduce gender stereotypical/ directly discriminating contents in school education. o Analyze gender stereotypes in school books in coop. with BDE. o Reform discriminatory school book contents and include gender equality perspective to advance women’s/ girls’ equal participation in all fields of learning and life. o Advocate for gender sensitivity in education contents with political decision makers accordingly. • Strengthen opportunities for the empowerment of women agricultural producers. o Promote women’s production in potential cash crops, i.e. veritable, cacao, mandarins, carrots. o Increase women’s participation in local organizations in agricultural production through provision of targeted trainings for women. o Provide further sensitization sessions for both female and male agricultural producers in the communities on gender equality, non-violent communication and negotiation. • Conduct further research in other Departments, e.g. in Artibonite (and other areas as necessary): on food taboos, nutrition and gender related customs, existence and capacities of local women’s organizations. Final Report 157 | P a g e Annex 11: Findings, Conclusions and Recommendations Findings Conclusions Recommendations EQ1 Relevance The programme activities are generally aligned with the National Policy and Strategy on School Feeding (PSNAS) in terms of educational benefit, nutrition and health but less so in some areas, such as support to the local economy, environmental safeguards, and meeting recommended nutritional standards for school feeding. The programme is also not sufficiently shock responsive. The evaluation concludes that the McGovern-Dole programme is highly relevant for the children it serves who receive a basic nutritious meal daily. School feeding acts as a major safety net for vulnerable households, given the high levels of food insecurity in the areas where programme is implemented. Despite its lack of variety and local produce, this daily meal is valued by parents, schools and children alike. For some children, this is the major meal of the day, sometimes the only meal. The school canteen was found to be very important in rural areas, where poverty rates are poverty rates are almost twice those of urban areas, and vital for regions that have been hit by drought and failing crops in recent years. While the programme targets food insecure areas, and the public-school system, where the most economically and socially disadvantaged children go to learn, the targeting system could be more systematic Final Report 158 | P a g e to ensure that the most vulnerable children and schools benefit from the programme. While geographic targeting was not undertaken systematically, the programme has a stronger presence in food insecure regions, rural areas and public schools, where needs are greater. However, some of the most vulnerable children risk being excluded from the programme due to inadequate school feeding management and the exclusion of 3rd cycle students from school feeding. The evaluation also found that the monitoring system WFP has established to ensure proper management of the food supply chain and management of food stocks at school level can undermine programme goals by risking excluding some of the most vulnerable children, due to mismanagement at school level. The programme could provide more support to the most vulnerable schools to prevent their exclusion from the programme when they cannot comply with school feeding management requirements. Another issue highlighted in the evaluation is the exclusion of students of the higher grades 3 rd cycle from school feeding which raises moral and ethical issue in the Haitian context, where many girls and boys in primary school are overaged. WFP needs to explore sources of funding to include children of the 3rd cycle in school feeding Recommendation 6: Include vulnerability as a criterion (as defined in the PSNAS: education, nutrition, safety nets) in the school selection and suspension processes. Provide additional support (e.g. more training) to schools struggling to comply with programme rules. WFP has to establish a strategy to integrate the three school cycles. The practice of sanctioning schools for feeding older students needs to be reviewed. In the event that McGovern-Dole will not fund feeding activities for students in the 3rd cycle, WFP Haiti should seek alternative funding. Final Report 159 | P a g e where they are attending alongside with cycle 1 and 2. While the programme is generally aligned with the national policy and strategies for school feeding (PNSAS), the limited use of local purchases and support for local agriculture, present in WFP school feeding programming funded by other donors, should be seen as a notable gap in the MDG funded programme. The evidence shows that local purchases are highly valued by the communities and schools where they are implemented, providing benefits for children and the local economy that merit pursuing in the future. Local purchases not only help make the basic meals more interesting for children but also help to address micronutrient and vitamin deficiencies. While it was demonstrated micronutrient powders were effective in reducing these deficiencies, the consumption of MNP went largely counter to the general objectives of the PNSAS (consuming locally grown food) and local culture. Local purchases also benefit local producers who are members of the communities Final Report 160 | P a g e where the schools are located, which is another value added. Other shortcomings in terms of alignment with the PNSAS, include the failure to address the extensive use of firewood and charcoal to prepare meals, in light of the extensive level of deforestation in Haiti, notwithstanding the health hazards borne mainly by the women who more often than not toil in inadequately aired kitchen. The evaluators consider that given the recurrent extreme and devastating climatic events in Haiti and its rank as the 4 th most affected country in the world, there is an opportunity to make future school feeding programming more shock responsive. The absence of emergency mechanism in a context of high vulnerability warrants a review of the programme, ensuring that the strategy that is adopted is also gender responsive. See recommendation 5. Recommendation 7: Due to the frequency of emergencies in Haiti, an analysis should be carried out together with donor and Haitian authorities on the use of the McGovern￾Dole school feeding platform as a response mechanism to slow and rapid-onset emergencies, such as droughts, earthquakes and hurricanes and political and economic instability. WFP should take the opportunity to ensure that it is gender-responsive and promote gender equality. EQ2. Effectiveness Overall, the programme has been effective, achieving most The programme has been effective in outputs and outcomes. The evidence Recommendation 4: WFP should capitalize on CRS’s experience and consider expanding the literacy activities Final Report 161 | P a g e planned outputs, such as the number of children attended, meals served, committees trained in canteen management, nutrition and hygiene, as well as number of teachers trained and certified. It also has been effective in achieving outcomes, particularly in schools where a literacy component was implemented. strongly suggests that providing a daily meal plays and other complementary activities such as WASH bolsters attendance, allowing a greater number of boys and girls to acquire certain level of education by maintaining them in school longer despite dire economic circumstances. However, the data did not show conclusively that school feeding has a strong effect on students’ academic performance. As discussed, the baseline data did not show a significant difference in pass rates between beneficiaries and non-beneficiary children. This suggests that other factors such as competency of teachers (as noted in the Annex 3, only 17% of teacher are qualified to teach), lack of school materials, over crowding, teacher absenteeism and lack of training play a greater role academic performance. piloted as part of the current programme. Materials challenging gender stereotypes should be developed as part of a next phase. This could be done in partnership with the MENFP, UNCEF, CRS and other NGOs with expertise in promoting gender equality and education. Overall, the programme achieves planned outcomes, with targets nearly achieved for enrollment, attendance and retention and surpassed for Indeed, the literacy component piloted in some 42 schools has helped improve student performance in reading Creole, even though results were much lower than expected. The short implementation period was given as the explanation for Final Report 162 | P a g e pass rate, with similar results for girls and boys. those modest results. However, the evaluation of the pilot showed that beneficiaries had EGRA results twice as high as those of children from control groups. This indicates that, in addition to school meals, which help bolster attendance, there is a need to continue investing in teacher training, and adequate teaching materials. While structural issues still need to be addressed at national level, the programme can provide tangible benefits to the children it serves. The evaluation did not find significant differences based on gender, type of school (public, private), location (geographic), model (traditional, homegrown) or complementary activities WASH, school gardens), except for schools which benefited from the literacy component. EQ3. Community Participation Final Report 163 | P a g e The involvement of parents (women and men), principals and committees have contributed to the achievements of outputs and outcomes but, in many cases, the contribution of parents is insufficient to meet the canteen needs on a daily basis or for school feeding to become sustainable over time. While both men and women contribute to the functioning of school canteens. Women contribute disproportionally more, particularly cooks. Participation decision-making through committees, but decision-making position, such as chairperson is occupied almost exclusively by men. The current level of involvement of parents and community is insufficient to ensure long term sustainability The day-to-day operations and management of school feeding and its sustainability depends on the participation and leadership of a continuum of actors at all levels, from the schools and local communities to the national level, e.g. MENFP, PNCS and other ministries and organizations with a stake in education, food security, nutrition, economic development, climate change etc. School principals, parents (women and men) and teachers have played an important role in programme implementation, thanks to the solid norms supporting parent committee roles and responsibilities at local level. However, these measures and the committees’ activities have not generated sufficient cash or in-kind contribution for a fully functioning school feeding programme, let alone ensuring sustainability at the community level. Women’s leadership in committees also needs to be strengthened, Lack of resources from parents, especially in light of rapid price increases recently and in vulnerable areas affected by drought, are factors that have to be Recommendation 2: Develop a gender-transformative strategy for community engagement, using proven approaches that employ social mobilisation within a community development approach to be implemented by NGOs. The suitability of financing tools such as Village Savings and Loans should be investigated in terms of their potential application in support of school feeding and income generation activities. This could also include private sector actors such as SogeBank or Digicel or one of their affiliates such as Mon Cash. Final Report 164 | P a g e of school feeding at community level. The lack of a strategic approach to social mobilisation may be the explanation. Overall, the contribution of the McGovern-Dole programme to income generation and the local economy has been minor. However, WFP programming to support local purchase and other ‘homegrown’ models provide substantial benefits to local communities and producers, including for women. considered in the lack of contributions from parents. There is clearly a need for a better community mobilization strategy, as the efforts to date through monitoring officers alone have not yielded expected results. Support for revenue generation at the local level, which other programmes have been found useful to achieve better outputs and outcomes. There is also a need to devise strategies to ensure that leadership is exercised more equally at school level, while taking into consideration that a large proportion of households are headed by women, both in rural and urban areas, with all the economic and social barriers that this entails. Approaches such as Village Savings and Loans that allow small groups of people to come together to finance social and economic related ambitions could be valuable in the context of school feeding, particularly to support income generation activities and should be actively explored. Furthermore, based of the findings of the evaluation, there is reason to believe that support to local purchases and other homegrown models and support the local Final Report 165 | P a g e economy, including women, may help foster greater community engagement and support for the school canteens. Whatever the strategy developed, it will need to be gender-transformative in essence and seek to change perceptions and behaviours on appropriate gender roles among women and men, boys and girls. EQ4. Government Capacity While some important elements policy framework for school feeding are in place or underway at the national level, the Haitian Government lacks the institutional or financial capacity to manage the programme independently, even partially, in the short term. Stakeholders agree that building capacity of decentralized government structures, as well as promoting local production can Ultimately sustainability rests on the ability of the MENFP to mobilize resources to take charge of the school feeding programme. The findings of the evaluation show that despite the progress that has been made over time (PNSA), legislation, governance structures, operations manual, etc., these are insufficient for even partial transfer of the programme to the GoH at this juncture. The SABER exercise, provided ample evidence of this and there is consensus among international partners that the GoH needs to budget for SF, resolve crucial governance issues and Recommendation 1: To enhance sustainability of the national school feeding programme, the Haiti CO should continue to support the validation of normative documents and governance structures and assist with their adoption and application. At the same time, capacity building efforts at decentralized government structures should continue, including awareness raising and training on gender equality. WFP could partner with the Ministère à la Condition Feminine et aux Droits des Femmes151 to develop material to that effect. 151 Ministry for the Status of Women and Women's Rights. Final Report 166 | P a g e bolster capacity until crucial governance issues are resolved at the national level. develop the capacity and strong financial systems to adequately manage the resources. The findings point to the need for WFP to continue supporting the capacity building process at national level, while focusing efforts on decentralized structures and community level. This will help develop capacity at all levels of government for an eventual transfer. Q5. Programme Advantages and Disadvantages and Possible Improvements The key advantage of the MDG model is that the cost of the basic meal provided to schools is much lower than models relying on local purchases. The key disadvantage is that it contributes almost nothing to the local economy. Other models that include local purchases demonstrate that more dynamic and collaborative actions can be nurtured through school feeding with higher nutritional and economic benefits With regards to strategies to promote the generation of revenue, evidence points to the experience in Nippes as a suitable model and also contribute to better nutritional outcomes. At any rate, intersectoral collaboration could support these objectives through innovative approaches. As discussed, there is a strong case for increasing local purchases, with numerous advantages which, in the longer term outweigh the higher initial costs. In recent years several donors have demonstrated their interest in promoting local purchases, e.g. Canada, France, the USA, Brazil. It is also clear from the findings Recommendation 3: Consider increasing local purchases and support local producer organizations, especially women using gender transformative approaches. In addition, WFP should seek to promote complementary activities related to nutrition and food growing activities to provide an opportunity for children and their families to learn agricultural practices, and in particular protected agriculture such as greenhouses, better suited to current climatic challenges. WFP’s section in charge of climate change should reach out to the Innovation Department of MARNDR and IADB both who expressed an openness to collaborate with WFP in this area. WFP should explore the possibility of turning schools into community hubs through partnerships with community-level service providers. Final Report 167 | P a g e achieved for local communities. Other countries, particularly in the Caribbean region, which face similar conditions as Haiti have developed initiatives that integrate innovative climate adaptation measures to enhance community involvement, as well as create local economic opportunities. Financial and credit mechanisms to support income generation and households’ expenditures, such as Village Savings of Loans, Savings and Internal Lending Community and schemes as ‘Mon Cash’ and ‘Sogesol’ and other more formal financial vehicles, are increasingly accessible in Haiti. WFP has also used mechanism such as cash transfers and food for work to build community assets. that climate change and gender will need to be part of strategies. Regarding gender, as shown, women face numerous barriers in the agricultural sector, related to land tenure, which limits their access to productive resources, including as credit. Strategies should be designed to help overcome these barriers. Regarding climate change, the experience from other countries, including from WFP, has shown that schools can provide learning for children and economic opportunities for youth and women, though climate adapted food growing, using greenhouses and hydroponics. The experience of other WFP CO promoting greenhouses and hydroponic gardening (see Innovation Accelerator) should be studied, but perhaps as importantly, the experiences of other Caribbean countries including Jamaica and Barbados. The evaluation found that there was interest to pilot these innovative strategies from the Innovation Unit of the MARNDR and IADB. Final Report 168 | P a g e Developing partnerships between the education and health sector and related organizations at the community level has the potential to achieve better educational outcomes for vulnerable children. Gender As discussed above, two particular gender issues need to be addressed as a priority. One is the issue of the compensation for cooks, which is not commensurate with their daily input; essentially, they are working fulltime in hard conditions as volunteers, which virtually all stakeholders at school lamented. The second issue concerns both the health and working conditions of the cooks. WFP needs to give serious consideration to helping provide less environmentally damaging and more healthy ways of cooking the food provided to the children on a daily basis, particularly through a cookstove strategy. There are opportunities for WFP to provide improved cookstoves adapted to the local circumstances by partnering with foundations (e.g. World Kitchen), Recommendation 5: A comprehensive cook stove strategy should be established. This strategy should aim to work with available circumstances to use the most appropriate and practically feasible cooking stove set ups. This should include the elimination of all occupational health and safety risk for women cooks such as those related to cooking on open fires in closed spaces, burning of wood in urban areas where for example, propane stoves could be used and the overall reduction of unwanted environmental impacts. To this end, a partnership with foundations, NGOs and the private sector should be actively explored. Final Report 169 | P a g e implementing partners (e.g. BND and others) and the private sector. Final Report 170 | P a g e Final Report 171 | P a g e List of Acronyms ACR Annual Country Report AMURT Ananda Marga Universal Relief Team APP Accountability to Affected Populations ASEBED La Société d'Agence de Secours et de Bienfaisance aux Enfants Défavorisés AtB Accountability to Beneficiaries BDEPAP Bureau diocésain d'éducation Port-au-Prince BND Bureau de Nutrition et Développement CAD Canadian Dollar CD Country Director CECI Centre d'étude et de coopération internationale CNSA Conseil national pour la sécurité alimentaire/ National Council for Food Security CO Country Office COMET WFP Programme Management System CP Country Programme CSP Country Strategic Plan CRS Catholic Relief Services CTA Technical Centre for Agricultural and Rural Cooperation CVQ Caribbean Vocational Quafifications DAEP Direction d'appui à l'enseignement privé et du partenariat DCD Deputy Country Director DDE Direction Départementale de l'Éducation / Departmental Education Directorate DEQAS Decentralized Evaluation Quality Assurance Support DINEPA Direction Nationale de l’Eau Potable et de l'Assainissement / National Directorate for Water and Sanitation EGRA Early Grade Reading Assessment EMMUS Enquête Mortalité, Morbidité et Utilisation des. Services EPT Éducation Pour Tous / Education for all ECLAC Economic Commission for Latin America and the Caribbean EM Evaluation Manager ET Evaluation Team FAES Fonds d'Assistance Économique et Sociale FAO Food and Agriculture Organization FGD Focus Group Discussion FY Fiscal Year GDI Gender Development Index GDP Gross Domestic Product GEEW Gender Equality and Empowerment of Women GGI Goss Gilroy Inc. GII Gender Inequality Index GO Government Objective Final Report 172 | P a g e GoH Government of Haiti HDDS Household Dietary Diversity Score HDI Human Development Index HGSF Home Grown School Feeding HHS Household Hunger Scale HQ Head Quarters ICESCR International Covenant on Economic, Social and Cultural Rights ICSP ICSP – Interim Transitional Country Strategic Plan IADB InterAmerican Development Bank IFAD International Fund for Agricultural Development IMF International Monetary Fund IHE Institut Haïtien de l’Enfance IHSI Institut Haïtien de Statistique et d’Informatique / Haitian Statistics Institute ILO International Labour Organization IPC Integrated Food Security Phase Classification Kcal Kilo Calorie KII Key Informant Interviews LRP Local and Regional Food Aid Procurement Programme MDE Ministère de l'Environnement McGovern-Dole McGovern Dole MO Monitoring Officers MT Metric Tonne NGO Non Governmental Organization OEV Office of Evaluation MARNDR Ministère de l’agriculture, ressources naturelles et développement rural / Ministry of Agriculture, Natural Resources and Rural Development MAST Ministere des Affaires Sociales et du Travail / Ministry of Social Affairs and Labour MENFP Ministère de l'Éducation nationale et de la Formation professionnelle / Ministry of Education and Vocational Training MSPP Ministère de la santé publique et population / Ministry of Public Health and Population PAP Port-au-Prince PEQH Bank Providing an Education of Quality in Haiti PITAG Programme d’Innovation Technologique pour l’Agriculture et l’Agroforesterie / Technological Innovation for Agriculture and Agroforestry Program PMP Performance Monitoring Plan PNCS Programme National de Cantine Scolaire / National School Feeding Program PTA Parent-Teacher Association PSNAS Politique et Stratégie Nationales d’Alimentation Scolaire PSUGO Programme de Scolarisation Universelle Gratuite et Obligatoire / Universal free and obligatory schooling programme QA Quality Assurance REDI Rural Enterprise Development Initiative RG Reference Group Final Report 173 | P a g e REO Regional Evaluation Manager RMA Resting Metabolic Assessment SABER Systems Approach for Better Education Results SCA Score de Consommation Alimentaire / Food Consomption Score SF School Feeding SFC School Feeding Committees SPR Standard Programme Report SO Strategic Outcome SOGESOL Société Générale de Solidarité S.A. SOP Standard Operating Procedure T-ICSP Transitional - Interim Transitional Country Strategic Plan ToR Terms of Reference ToC Theory of Change UN United Nations UNEG United Nations Evaluation Group USAID United States Agency for International Development USD United States Dollar USDA United States Department of Agriculture USG United States Government UNICEF United Nations International Children's Emergency Fund. VAM Vulnerability Analysis and Mapping VSLA Village Savings and Loans Associates SFVP School Food Vendors Project SILC Savings and Internal Lending Community WASH Water, Sanitation and Health WB World Bank WCK World Central Kitchen WFP World Food Programme WHO World Health Organization WVI World Vision International Haiti Country Office https://fr1.wfp.org/countries/haiti