Foreign Agricultural Service, United States Department of Agriculture Côte d’Ivoire McGovern-Dole Project Midterm Evaluation January 2019 This publication was produced at the request of the United States Department of Agriculture. It was prepared independently by IMPAQ International, LLC. DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Mid-Term Evaluation of “Support for the Integrated School Feeding Program” in Côte d’Ivoire The United States Department of Agriculture (USDA) McGovern-Dole (MGD) project in Cote D’Ivoire was implemented by World Food Programme (WFP) from 2015 to 2020. The project’s primary objective is improving nutrition and health outcomes and increasing the literacy of 125,000 school aged children in 613 rural public primary school in seven high-priority regions of Côte d’Ivoire. Agreement Number: No. FFE-681-2015/006-00 Project Duration: 2015-2020 Implemented by: WFP Evaluation Authored by: IMPAQ International, LLC Individual Authors Deichsel Sakari, Research Analyst Maria DiFuccia, Qualitative Lead Michaela Gulemetova, Team Leader Edoxi Kindané, Field Manager Marc Masson, Research Analyst Elnaz Safarha, Quantitative Lead Prepared Decentralized Evaluation Mid-Term Evaluation of “Support for the Integrated School Feeding Program” in Côte d’Ivoire September 2016 to June 2018 Evaluation Report January 2019 World Food Programme Country Office in Côte d'Ivoire Evaluation Manager Moyabi Sylla Prepared by IMPAQ International, LLC Sakari Deichsel, Research Analyst Maria DiFuccia, Qualitative Lead Michaela Gulemetova, Team Leader Edoxi Kindané, Field Manager Marc Masson, Research Analyst Elnaz Safarha, Quantitative Lead DECENTRALIZED EVALUATION FOR EVIDENCE-BASED DECISION MAKING Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) i | Page Acknowledgements The authors would like to thank the World Food Programme (WFP) Country Office in Côte d'Ivoire and the U.S. Department of Agriculture (USDA) for their financial support. The authors would also like to thank the Steering Committee and the Technical Committee of the project, as well as AVSI, the Ministry of National Education, Technical Education and Vocational Training (MENETFP), the Directorate of School Canteens (DCS), the Ministry of Agriculture and Rural Development (MINADER), and the National Agency for Support to Rural Development (ANADER) for supporting and facilitating the rollout of data collection. The authors have very much enjoyed their collaboration with WFP and would especially like to thank Moyabi Sylla, Alti Bema, and Adeyinka Badejo-Sanogo. The authors also highly appreciated the welcome provided by Elly Bahati, Lorenzo Manzoni, and the whole team at AVSI. The authors thank Rébéka Kakou for her support and expertise doing qualitative interviews, particularly with the women’s production groups. The authors thank Mont Horeb for their professionalism and flexibility with data collection. The authors also thank all the enumerators for their excellent work in the field. The authors also acknowledge Guy Kacou, the head of research at Mont Horeb, for providing technical insights on cognitive testing of the instruments and managing the fieldwork team. At IMPAQ, the authors thank their colleagues Drs. Sara Borelli and Kajal Gulati for their technical expertise during the evaluation design and analysis, and Jonathan Simonetta for providing guidance and quality control. Disclaimer The opinions expressed in this report are those of the authors, and do not necessarily reflect those of WFP or USDA. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP or USDA of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) ii | Page Table of Contents Table of Figures...................................................................................iii Executive Summary ............................................................................. v Context..................................................................................................................................................... v Methodology.......................................................................................................................................... vi Key Findings ......................................................................................................................................... vii Recommendations................................................................................................................................ x 1. Introduction.................................................................................... 1 1.1. Overview of the Evaluation Subject.............................................................................. 1 1.2. Context..................................................................................................................................... 4 1.3. Evaluation Methodology and Limitations..................................................................... 5 2. Evaluation Findings ........................................................................ 9 2.1. Evaluation Criteria 1 – Relevance.................................................................................. 9 2.2. Evaluation Criteria 2 – Effectiveness ..........................................................................11 2.3. Evaluation Criteria 3 – Efficiency .................................................................................15 2.4. Evaluation Criteria 4 – Impact......................................................................................17 2.5. Evaluation Criteria 5 – Sustainability .........................................................................32 3. Conclusions and Recommendations ........................................... 37 3.1. Overall Assessment/Conclusions..................................................................................37 3.2. Lessons Learned and Good Practices..........................................................................37 3.3. Recommendations..............................................................................................................38 Annex 1: Evaluation Matrix............................................................... 43 Annex 2: Evaluation Methodology....................................................48 Annex 3: Documents Reviewed / Bibliography .............................. 55 Annex 4: Stakeholders Interviewed................................................. 57 Annex 5: Data Collection Tools......................................................... 58 Annex 6: Additional Figures.............................................................. 59 Annex 7: Project Results Framework............................................... 76 Annex 8: Terms of Reference............................................................ 79 Annex 9: MGD Indicators ..................................................................80 Annex 10: Map of MGD School Feeding Program............................ 87 Annex 11: List of Abbreviations .......................................................88 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) iii | Page Table of Figures Figure 1: Project Effects Using DID Method ................................................................................. 19 Figure 2: Reading Assessment Scores by Grade (Midline Only) ...................................................... 20 Figure 3: Reading Proficiency by Grade and Gender (MGD Only) .................................................... 20 Figure 4: Teacher Absences by Region (Midline Only) ................................................................... 21 Figure 5: Student-Reported Teacher Absences (Midline Only) ........................................................ 21 Figure 6: Access to School Materials by Region............................................................................ 22 Figure 7: Access to Reading Materials (Mobile Library, Reading Board, etc.) .................................... 22 Figure 8: Teachers Having Participated in AVSI Trainings by Region (Midline MGD) .......................... 22 Figure 9: Schools Having Received Training on Teaching of Reading............................................... 23 Figure 10: Number of Students out of 10 Described as Attentive by Teachers on a Typical Day (Midline MGD Only) .............................................................................................................................. 23 Figure 11: Student Attentiveness by Region (Midline MGD Only).................................................... 23 Figure 12: Students with Perfect Attendance (Midline Only) .......................................................... 24 Figure 13: Average Student Enrolment per Grade (All Schools)...................................................... 24 Figure 14: Ratio of Girls to Boys Enrolled in CM1 & CM2 (MGD Schools).......................................... 25 Figure 15: Frequency that Students Ate at the School Canteen...................................................... 26 Figure 16: Portion of Days that Canteen Provided Minimum Dietary Diversity .................................. 26 Figure 17: Teacher Training on Health Practices........................................................................... 27 Figure 18: By Region, Canteen Managers Know Three Food Storage and Food Preparation Best Practices ............................................................................................................................................. 27 Figure 19: Type of Access to Water in Schools............................................................................. 28 Figure 20: Distance to School’s Water Source, Where Available (Midline MGD Only) ......................... 29 Figure 21: Most Common Problems with Schools’ Water Source..................................................... 29 Figure 22: Access to and Quality of Sanitary Facilities .................................................................. 29 Figure 23: Sufficient Food Preparation Equipment........................................................................ 30 Figure 24: Sufficient Food Storage Equipment ............................................................................. 30 Figure 25: Stove Type in the Canteens ....................................................................................... 31 Figure 26: Stove Type in the Canteens, by Region ....................................................................... 31 Figure 27: Student Sample Distribution by Gender, Grade, and Type of Respondent ........................ 59 Figure 28: Reading Proficiency by Grade..................................................................................... 59 Figure 29: Teacher Attendance According to School Records and Self-Reported............................... 59 Figure 30: Self-Reported Teacher Absences by Region (Midline Only) ............................................. 60 Figure 31: Access to Reading Materials (Mobile Library, Reading Board, etc.) by Region (Midline MGD Only)...................................................................................................................................... 60 Figure 32: Usefulness of AVSI Trainings (Midline MGD Only) ......................................................... 60 Figure 33: Number of Students out of 10 that were Attentive, According to Teachers (Midline MGD Only)...................................................................................................................................... 61 Figure 34: By Region, Number of Students out of 10 that were Attentive, According to Teachers (Midline MGD Only) .............................................................................................................................. 61 Figure 35: By Region, Average Student Recorded Attendance (Midline Only) ................................... 61 Figure 36: Average Student Enrolment per Class (MGD Schools with Take Home Ration Schools Separated).............................................................................................................................. 62 Figure 37: Average Student Enrolment per Class (MGD Schools with Take Home Ration Schools Separated).............................................................................................................................. 62 Figure 38: Ratio of Girls to Boys Enrolled by Grade (MGD Schools with Take Home Ration Schools Separated).............................................................................................................................. 62 Figure 39: Ratio of Girls Enrolled to Boys Enrolled by Grade (MGD Schools with Take Home Ration Schools Separated) .................................................................................................................. 63 Figure 40: Number of Days Students Missed Due to Illness (Midline Only) ...................................... 63 Figure 41: Students Missing More than 10 Days Due to Illness (Midline Only).................................. 63 Figure 42: Foods Eaten in Household during the Previous Day ....................................................... 64 Figure 43: By Region, Foods Eaten in Household During the Previous Day (MGD Midline).................. 64 Figure 44: By Region, Foods Eaten in Household during the Previous Day (MGD Midline) .................. 65 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) iv | Page Figure 45: Coping Strategies Used by Household ......................................................................... 65 Figure 46: By Region, Days* Coping Strategies Used (Midline MGD)............................................... 66 Figure 47: By Region, Percent* Using Coping Strategies (Midline MGD) .......................................... 66 Figure 48: By Region, Percent* Using Coping Strategies (Midline MGD) .......................................... 67 Figure 49: Percent of School Days during which Canteen Operated by Region ................................. 67 Figure 50: By Region, Percent of School Days during which Canteen Operated ................................ 68 Figure 51: Parents’ Awareness of the School Canteen Project........................................................ 68 Figure 52: Students Eating through the School Canteen Program................................................... 68 Figure 53: Frequency that Students Eat at the School Canteen...................................................... 68 Figure 54: Foods Used to Prepare Canteen Meals (Midline Only) .................................................... 69 Figure 55: By Region, Canteen Managers Can Cite Three Health & Hygiene Practices ....................... 69 Figure 56: By Region, Canteen Managers Can Cite Three Health & Hygiene Practices ....................... 69 Figure 57: Teacher Training on Health Practices........................................................................... 70 Figure 58: Nutrition-Related Trainings at School .......................................................................... 70 Figure 59: Type of Access to Water of the School......................................................................... 70 Figure 60: Distance to School’s Water Source, Where Available (Midline MGD Only) ......................... 70 Figure 61: Most Common Problems with Water Source ................................................................. 71 Figure 62: Access to and Quality of Sanitary Facilities .................................................................. 71 Figure 63: Deworming Pills and Micronutrient Pills Distributed to Students ...................................... 71 Figure 64: Deworming Pills and Micronutrient Pills Distributed to Students ...................................... 71 Figure 65: Access to Food Storage and Equipment....................................................................... 72 Figure 66: Access to Food Storage and Equipment....................................................................... 72 Figure 67: Sufficient Food Preparation Equipment........................................................................ 72 Figure 68: Sufficient Food Storage Equipment ............................................................................. 73 Figure 69: Stove Type .............................................................................................................. 73 Figure 70: Stove Type by Region ............................................................................................... 73 Figure 71: Reading Assessment Scores by Treatment Group, Phase, and Grade............................... 74 Figure 72: Food Consumption Score Group by Gender of Head of Household ................................... 75 Figure 73: Household Dietary Diversity Score by Gender of Head of Household ............................... 75 Figure 74: Reduced Coping Strategies Index by Gender of Head of Household*............................... 75 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) v | Page Executive Summary 1. This evaluation report is for the mid-term evaluation of the “Support for the Integrated School Feeding Program” in Côte d’Ivoire, a school feeding and early grade reading project implemented by World Food Programme (WFP) and funded by the McGovern￾Dole (MGD) International Food for Education and Child Nutrition Program of the United States Department of Agriculture (USDA). Under MGD, the USDA awarded USD 35,678,500 to WFP over five years, which includes an in-kind donation of 24,600 metric tons (MT) of food. The project’s primary objective is improving nutrition and health outcomes and increasing the literacy of 125,000 school aged children in 613 rural public primary school in seven high-priority regions of Côte d’Ivoire. 2. This mixed-methods evaluation is commissioned by the WFP Country Office in Côte d'Ivoire and covers the period from September 2016 to June 2018. The two main objectives of this mid-term evaluation are to: • Assess and report on the progress made in achievement of the results, compared with the baseline, for the school meals project as it was implemented during the first two years. • Assess and identify key achievements and challenges, draw lessons, and identify best practices for learning. It provides evidence-based findings to enlighten operational and strategic decision-making, improve in partnership coordination, and inform sustainability. Findings from this evaluation will be actively disseminated, and lessons will be incorporated into relevant lesson sharing systems. Context 3. Since 1989, to encourage enrolment in primary school and address the challenge of lunchtime hunger at school, the Government of Côte d’Ivoire (GoCI) has administered a national school feeding program with the support of WFP. In 2000, the GoCI integrated nutritional dimensions in addition to educational objectives, aiming for sustainability by encouraging the production of local communities in the form of women’s production groups (WPGs). This national program, entitled the “Integrated Program for the Sustainability of School Canteens” (PIPCS), aims to address the problems of chronic child malnutrition, which is high at 23.2 percent for boys and 19.9 percent for girls under 5 years old, and poor performance in primary education, with 63 percent of the population being illiterate1 and only 75.1 percent of children finishing primary school in 2016.2 It encourages girls' education and ensures their retention in school. This national program resulted in the operation of 5,708 school canteens across the country in 2014- 2015, providing hot and balanced meals to 1,104,138 elementary school children. 4. However, as indicated in the GoCI’s national school feeding policy (2018-2025) and strategy (2018-2022), the provision of food to the canteens remains a significant challenge. In 2016-2017, the government’s program was only able to deliver 18 days out of 100 school days of food to the canteens.3 WFP complements the GoCI’s efforts through its MGD funding, which covers a total of 613 rural public primary schools with 125,000 students benefiting from hot and balanced meals across seven priority regions 1 MICS 2016: Enquête a indicateurs multiples, Côte d’Ivoire. Abidjan : Institut National de la Statistique 2 MENETFP/DSPS, 2017: Statistiques scolaires de poche 2016-2017, MENETFP. 3 MENETFP/DCS. Stratégie Nationale d’Alimentation Scolaire en Côte d’Ivoire (2018-2022), Côte d’Ivoire. Abidjan. P. 13 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) vi | Page (Poro, Bagoue, Tchologo, Bounkani, Gontougo, Bafing, and Cavally). In addition, 10,000 girls in upper grades (CM classes) in three regions (Bagoué, Poro, and Tchologo) benefit from take-home rations each year (a total of 50,000 girls throughout the project). Through this effort, MGD-supported schools were able to deliver on average 72 out of 100 school days of food during the 2016-2017 school year.4 Concurrently, WFP provides technical assistance to the GoCI to enhance the sustainability of the national school meals program. 5. The main components of the MGD project are: 1) providing school meals to primary school students; 2) delivering take-home rations for girls in upper grades (CM1/CM2); 3) distributing deworming tablets and micronutrients, 4) improving student literacy education; 5) training canteen management staff in the use of good health and food practices; 6) providing cutlery and equipment for the preparation and storage of food to the canteens; 7) constructing improved stoves and training canteen staff in their use; and 8) building capacity (both technical and financial) of women’s production groups (WPGs) to contribute production to the school canteens of their villages or communities. 6. The two main planned outcomes of the intervention are improvement in the literacy of school-aged children and increased use of health and dietary practices. These two outcomes are expected to result from a combination of more consistent teacher attendance, better access to school supplies and materials, improved access to learning materials for reading, and increased skills and knowledge of teachers. The other outcomes that fall under the increase use of health and dietary practices include improved knowledge of health and hygiene practices, safe food preparation and storage practices, nutrition, and the use of improved stoves, as well as access to preventive health interventions. Methodology 7. The evaluation was designed to evaluate the “Support for the Integrated School Feeding Program”, financed by the USDA’s MGD Program, against the following evaluation criteria: relevance, effectiveness, efficiency, impact, and sustainability (the comprehensive set of evaluation questions are found in the conceptual framework in Annex 1). In order to respond to these questions, the evaluation team employed quantitative surveys (ultimately reaching 104 schools, among which 68 treatment [i.e. MGD] and 36 control [i.e. non-MGD]) and qualitative focus group discussions (FGDs) with beneficiaries, along with key informant interviews (KIIs) with top-level project stakeholders. As part of its FGDs with beneficiaries, the evaluation team met with 10 WPGs supported by the MGD project since 2017 (designated as "established") and 10 WPGs supported since 2018 (designated as "new"). For the impact evaluation, the evaluation team designed a difference-in-differences (DID) method to compare the changes in reading proficiency between the population of beneficiaries (the treatment group) and the population that is not benefiting from the project (the comparison group). 8. Limitations and challenges of the evaluation included inaccurate or missing data from baseline, a delay in data collection until the beginning of the school year 2018-2019, and difficulties following the same subsample over time. However, measures were taken to mitigate these issues as much as possible, including revising the quantitative tools 4 Ibid. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) vii | Page developed by the baseline evaluator to more accurately capture the required MGD indicators; developing and collecting data with a teacher survey to improve the key performance indicator measurements; and measuring the project effect on literacy outcomes for at least one year, instead of two, at midline to maintain statistical power. Key Findings 9. The key findings of the evaluation team are summarised below, structured according to the evaluation criteria. Results related to relevance, effectiveness, efficiency, and sustainability are primarily informed by the qualitative analysis, while the results related to impact are primarily informed by the quantitative analysis. Relevance 10. The project is highly integrated into and complements the government’s efforts. Beneficiaries agree that the food delivery, school improvements, and literacy activities aspects of the intervention are relevant to their needs. 11. While the project aligns with Government priorities and was developed in collaboration with the MENETFP, which concurred with the project proposal prior to submission, some representatives from the DCS said they were not consulted during the final stages of the project design phase. Because of this, these representatives believed that several activities were not realistically planned given the timeframe and resources. 12. By encouraging or allowing children to attend school with the promise of a lunchtime meal, the MGD project is also directly relevant to the needs of parents of the students by reducing the time they needed to devote to childcare during the school week– increasing the amount of time they can devote to their livelihoods. 13. All the WPGs reported that receiving materials and training was relevant and highly appreciated. Women in the more established groups appreciated that prior to distributing materials, representatives met with them to determine their needs. This ensured that the equipment received was directly targeted and relevant to the individual groups. 14. The literacy component of the project was welcomed by parents; they said that education is the key to future success for both their sons and daughters. Many talked about their own challenges not knowing how to read and write and hoped for something better for their children. Effectiveness 15. The absence of a regularly provisioned school canteen, seen as a major challenge which previously discouraged or prevented children from attending school or staying for the full day, was addressed under WFP’s MGD project according to the school management committees (COGES) and parents. 16. Parents appreciated not having to prepare meals for their children, thus freeing their time for other livelihood activities. 17. Members in the established5 WPGs stated that their production has grown substantially due in part to the technical support received through the project. All groups were able to increase their contributions to the school canteens from last year to this year. 5 The sample of WPGs contains 10 groups supported by the MGD project since 2017 (designated as "established") and 10 groups supported since 2018 (designated as "new"). Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) viii | Page 18. Overall, parents were happy with the quality of the teachers in MGD schools, the education their children receive, and the progress they are making in school. 19. Some planned activities are behind schedule, such as the support to WPGs and the training of COGES. These delays were explained by ambitious expectations in mobilizing additional funding. In addition, threats to implementation include the low financial means of beneficiaries, who report not being able to afford school-related costs, including school uniforms, textbooks, and the daily 25 FCFA6 canteen fee. Other factors influencing project effectiveness include poor school infrastructure (lack of running water or latrines and insufficient school table-benches and classrooms) and teacher absenteeism. 20. The WPGs still rely on men, as there are several agricultural activities that are labour intensive and typically done by males. For some of the established WPGs, integrating youths to do this work has been successful, especially with the support of the village chief. In other groups, however, the members said their efforts to incentivize younger community members has been unsuccessful so far. Efficiency 21. Overall, the project is efficient as it provides beneficiaries with school meals, take home rations (THR), and literacy instruction. 22. There are some activities where the complementary funds have not been mobilized as anticipated, specifically the capacity building activities for WPGs and COGES. As a result, WFP made strategic choices that benefited the primary components of the project at the expense of secondary components. 23. Some government stakeholders and local community members suggested that by providing meals every day, the project discourages the community from taking ownership of the canteens. 24. It is difficult to monitor the project using school records. Student and teacher attendance reports appear to be inflated, as the official reports show almost no absenteeism, conflicting with the qualitative results. Canteen records did not appear inflated, but some were missing or incomplete. 25. External factors influencing the efficiency of the WPGs include climate variation and lack of irrigation systems. Other factors include the low incomes, high-illiteracy rates, and lack of management skills of the community members, which prevent them from fully executing certain activities. Impact 26. The impact analysis shows that the MGD intervention significantly improves the reading proficiency of students. Exposure to the MGD project increases the reading assessment literacy scores of students in treatment schools at midline by 1.2 points (p<0.01) relative to comparison group, which is approximately one-level increase in the reading assessment. When looking only at the students who have been exposed to the project for two years (CP1 students from baseline who are now in CE2 in 2018-2019), the estimated impacts are somewhat larger (1.7 points). 6 The 25 FCFA canteen fee per child/day is mandated by the Government of Cote d’Ivoire to cover the salary of canteen workers, firewood, and condiments, with a portion also going to the IEP and COGES for activity monitoring. It is equivalent to roughly 0.05 USD. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) ix | Page 27. The project effect is twice as great for male students, who show almost two levels of improvement in reading proficiency, compared to one level for girls. Girls and boys have similar levels of reading proficiency in CP1 and CP2, but starting in CE1, the gains for boys are much greater. 28. Enrolment increased at MGD schools for girls in CM grades where THR are offered. In the three regions with THR, the average enrolment increased by almost 10 girls in the upper grades (from 33.9 CM 1 and CM2 girls at baseline to 42.5 students at midline). In contrast, in regions where THRs are not offered, girls’ CM1 and CM2 enrolment did not increase as much (from 17.7 to 22.2). The ratio of girls to boys enrolled in these two grades increased from 0.88 to a near-parity ratio of 0.96 in THR schools, while the ratio increased from 0.7 to 0.81 with no THR. 29. The percentage of school days during which food was provided was nearly 100 percent in each region, but varied slightly, ranging from to Poro (88 percent) to Bafing (100 percent). There were no substantive differences between the days of canteen services available to boys and girls. This was a large increase from baseline, where schools on average only provided meals on 57 percent of the days when school was in session. 30. Almost all parents (94 percent) of children enrolled at MGD schools were aware of the school canteen project, and almost all (92 percent) reported that their children ate at the canteen in the previous school year. Over 80 percent of students reported eating at the canteen regularly (three days a week or more). 31. Most MGD canteens served food containing the minimum dietary diversity on more than 80 percent of school days at midline, compared to 40 percent of days at baseline. 32. While most canteen managers were able to identify three or more health and hygiene practices at midline (88 percent), the number had decreased slightly from baseline (93 percent). 33. Among canteen managers who reported that there was a canteen management committee at their school, 33 percent indicated that they received nutrition training, compared to 41 percent at baseline. Teachers in more than half of MGD schools (54.4 percent) received trainings on nutrition. 34. About half of both MGD (53 percent) and non-MGD (50 percent) schools did not have any adequate access to water, similar to the baseline numbers. In MGD schools, about one-third (32 percent) of those with water access had a source on the school grounds, while 43 percent had a water source less than a 15-minute walk. The most common problem preventing access to water in both MGD (59 percent) and non-MGD (50 percent) schools was broken water pumps. 35. Slightly more MGD schools (59 percent) than non-MGD schools (50 percent) have sanitary facilities on-site. Less than one-third (31 percent) of MGD schools and only one-quarter (25 percent) of non-MGD schools have gender-separated sanitary facilities. 36. The number of MGD schools receiving deworming treatments decreased from baseline to midline (from 72 to 64 percent), as did the number of schools receiving micronutrient supplements (53 to 26 percent). 37. The percentage of canteen managers who reported having sufficient food preparation equipment increased from baseline to midline (from 19 to 32 percent), as did sufficient food storage equipment (from 10 to 38 percent). Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) x | Page 38. Across all regions, 46 percent of canteens had an improved stove with a chimney, instead of just a traditional stove or a furnace without a chimney. This includes 100 percent of canteens in Cavally and Tchologo where there is a stove. This is large increase from baseline, where only 7 percent of school canteens had an improved stove. Sustainability 39. While it is still early to fully assess the impact of the WPGs and whether this component of WFP’s sustainability plan is viable, the members of the most established WPGs which began receiving project support in 2017 have indicated that their work has started to have an effect. In the ten established groups, the members have substantially increased their canteen contributions from 2017 to 2018. Most of these groups reported that their generated income was now sufficient to purchase their own seeds, fertilizer, and equipment maintenance without needing external financial support from the project. 40. In the established groups, women reported the material and trainings received through the project is making their groups stronger and therefore more likely to sustain contributions after the project is over. For example, the tricycles, grain mills, and scales save time and money, and the financial training and market access has allowed them to sell produce for a higher price at the markets.7 41. For all WPGs, social networks are created, and members rely on each other as a coping strategy mechanism in the case of adverse social shocks, such as an illness or death in the family. In the more established groups, this social support goes even further, thanks to the money gathered through income generating activities and to the financial management training received as part of the project. These groups are able to give small loans to their members. These auxiliary activities develop WPGs as community institutions, with functions beyond supporting their school canteen, increasing their sustainability along with the benefits they provide to their communities. 42. The factors preventing full effectiveness of the project’s WPGs in supporting their school canteen are the same as those that hinder sustainability. Lack of access to physical and natural capital prevent WPGs from becoming stronger and autonomous, while climate variability and the associated shocks threaten their production, both for their members and the school canteen they support. 43. Turnover is a threat to sustainability at all levels–at both the local level (cooks, COGES, school canteen management committees, WPGs, and teachers), and at the regional and national level (regional advisors, elected government officials, and WFP/partner staff). Strengthening the capacities of local communities (COGES and school canteen committees, in addition to the WPGs) and government partners will increase the likelihood of the project achieving sustainable, long-term impact. Recommendations 44. Based on the findings and conclusions of this evaluation, the recommendations of the evaluation team are outlined below. The target group for each recommendation is clearly identified. The opinions expressed in these recommendations are those of the authors, and do not necessarily reflect those of WFP or USDA. 45. Recommendation 1: Strategically Reduce the Proportion of Canteen Days Covered in MGD Schools. Currently, WFP provides meals throughout the entire 120- 7 The MGD project’s support to WPGs takes into account the specific needs of the groups and as such, the material and training WPGs receive may differ. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) xi | Page day school term. As heard from government stakeholders and local beneficiaries, this discourages parents and WPGs from contributing money and/or food to the canteens. At the same time, few WPGs reported being close to being able to adequately support a school canteen without the MGD project’s support and it is unclear when they will be able to.8 Considering the critical importance of WPGs in the project’s sustainability plan, WFP should immediately perform an assessment, informed by other projects supporting WPGs in similar contexts, to estimate the time needed for WFP-supported WPGs to fully support a school canteen and determine when WPGs could realistically be expected to graduate from the project and under what conditions. After evaluating what local communities can support, WFP should integrate these findings into the transition plan it is developing with the GoCI and indicate how, when, and in which schools and regions school feeding will be transitioned to the GoCI’s national program. WFP should start reducing the dependence of canteens on MGD commodities by the start of the 2019- 2020 school year, replacing these commodities with GoCI and WPG contributions, and find other beneficial uses for the surplus commodities, such as increasing the number of girls receiving THRs by expanding this offer to all 7 project regions. 46. Recommendation 2: Reinforce DCS and DPFC M&E Capacity at the National and Local Level. The MGD project’s monitoring network relies heavily on the GoCI’s own network of local and regional government school feeding and teaching advisors to collect indicator data. However, the data collected through this system does not include key MGD indicators, particularly on student attendance and their absences due to sickness, teacher absences, and food diversity and security, which cannot be effectively calculated through evaluation. In addition, while the MGD project delivers annual M&E training to DCS and DPFC, several national-level DCS personnel requested that WFP provide them with additional M&E training and capacity building to help them better support the MGD project and other school feeding related projects. By the end of the 2018-2019 school year, WFP personnel should host a workshop with M&E personnel from DPFC and DCS to jointly reassess their training needs, develop a training plan that will respond to them, and consider strategies to improve MGD monitoring efficiency and effectiveness. Through this workshop, the MGD implementing partners should address the issue of accurately monitoring teacher and student attendance data, which remains a challenge, and the possibility of consolidating government monitoring activities where relevant. By the start of the 2019-2020 school year, WFP should support DCS and DPFC in modifying existing data collection tools and methods for indicators relevant to project implementation and reporting, while also consolidating existing school-level data collection visits across the school feeding and literacy components to more efficiently and frequently visit schools. 47. Recommendation 3: Improve Field Presence and Beneficiary Interaction. There are currently only eight WFP field monitors who are responsible for over 600 schools. They are working with government school feeding advisors who, as best they can, mobilize communities, supervise day-to-day canteen management, and collect indicator data, but have limited capacity in actuality. As WFP is trying to build both local capacity and long-term sustainability, they should consider working with the appropriate government stakeholders and local partners to increase the capacity of the local and regional school feeding advisors, as described in Recommendation 2. However, this 8 Through WFP technical assistance, WPGs are expected to incrementally increase their production while continuing to contribute 1/3 to their associated school canteen, with the end goal of fully supporting the canteen in conjunction with the Government’s national school canteens program. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) xii | Page does not replace regular, direct interaction between WFP and its project beneficiaries, which is essential for responding to local needs in a timely manner, more accurately capturing M&E data, and effectively disseminating innovations in the field. To efficiently improve its field presence and increase its touch-points with beneficiaries, the MGD project should consider using the data collection networks, agents, and activities already being used for the literacy component, such as community focal points and school monitoring visits, to more frequently visit schools and communities for other components of the project. 48. Recommendation 4: Increase Coordination, Communication, and Planning with the DCS. While the MGD project is deeply integrated with the GoCI, WFP could improve its advance planning of activities involving the DCS, with a clear and transparent view of the available funds and the project timeline. While DCS representatives participated in meetings to develop WFP’s proposal for the MGD project, multiple DCS key informants indicated that they were not informed of some late stage meetings, where they could have shared threats to some of the proposal’s assumptions. For example, these informants thought that the WPGs, a key component of this project, are not yet supported at sufficient number to provide project sustainability partly due to insufficient funds stemming from underestimating the cost of supporting WPGs. Having supported WPGs for nearly two decades, the DCS and ANADER has an intimate knowledge of these groups. The DCS informants thought their feasibility, given the resources and timeline, could have been more accurately described if they were fully included in discussions during the entire planning phase. Several DCS representatives also said that they were forced to start activities later than anticipated due to delays or uncertainty in receiving funds and were sometimes unaware of what MGD funding is allotted for their work. WFP should more frequently include DCS, DPFC, and other GoCI partners in conversations on the design, budgeting, and implementation of MGD project activities. In turn, the GoCI should ensure representatives from all relevant departments within it participate in the MGD project meetings organized by WFP to review and approve the tasks and budgets pertaining to their work. 49. Recommendation 5: Reinforce Girls’ Education. The results of the reading assessment showed that male student improved twice as much as female students. AVSI should work with DPFC and its field staff to adjust its programming to ensure girls benefit from the project as much as boys. While no beneficiaries in the FGDs said that girls were discriminated against at school, the evaluation team recommends that, by the start of the 2019-2020 school year, AVSI should conduct a study to explain these results and subsequently adapt its approach to ensure girls participate, read, and learn at the same rate as boys. Considering that teachers reported that girls in their class were roughly as attentive as boys, the evaluation team suggests that AVSI and DPFC consider integrating or reinforcing a gender focus in their teacher training curriculum. In addition, considering that THRs have shown to be effective in the three regions where it was offered in increasing girls’ attendance and keeping them in school, WFP should consider using a surplus MGD commodities to expand the THR offering to all seven regions by the start of the 2019-2020 school year. 50. Recommendation 6: Increase Female Representation in COGES. WFP targeted COGES to include 50 percent women as members, however, most in the evaluation team’s sample did not include this many women, and in fact, some COGES were exclusively male. By the start of the 2019-2020 school year, WFP should encourage women to participate in its anticipated basic literacy training for WPGs and canteen Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) xiii | Page management committees to also join their school’s COGES, as well as make additional efforts to sensitize COGES of the need to integrate women in their leadership. 51. Recommendation 7: Strengthen COGES Capacity to Support MGD Activities. The COGES help their school and its canteen in many ways which could be enhanced by expanded MGD training and support. All COGES agreed they could use additional training in managing COGES activities, including accounting, commercial farming, and fundraising. While the current MGD project is designed to offer only limited support to COGES, by the end of the project, to the extent feasible WFP should explore offering training and financial support to COGES in developing commercial activities or building infrastructure for the school, either directly or through partnerships with relevant Government structures or UN agencies and other organizations. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 1 | Page 1. Introduction 52. This evaluation report is for the mid-term evaluation of the “Support for the Integrated School Feeding Program” in Côte d’Ivoire. This evaluation is commissioned by the World Food Programme (WFP) Country Office in Côte d'Ivoire and will cover the period from September 2016 to June 2018. 53. The two main objectives of this mid-term evaluation are for accountability and for learning purposes. In particular, this evaluation will: • Assess and report on the progress made in achievement of the results, compared with the baseline, for the school meals project as it was implemented during the first two years. • Assess and identify key achievements and challenges, draw lessons and identify best practices for learning. It will provide evidence-based findings to enlighten operational and strategic decision-making, improvement in partnership coordination, and sustainability. Findings from this evaluation will be actively disseminated and lessons will be incorporated into relevant lesson sharing systems. 54. The expected users for this Evaluation Report are the WFP Country Office and its decision-making partners, Regional Bureau, WFP Headquarters in Rome, Evaluation Bureau (OEV), WFP Executive Board (EB), Government (Ministry of Education, Technical Education and Vocational Training [MENETFP] and Ministry of Agriculture and Rural Development [MINADER]), United Nations country team, Steering Committee, Technical Committee, AVSI, and the United States Department of Agriculture (USDA) Foreign Agriculture Service (FAS). 1.1. Overview of the Evaluation Subject 55. In support of the national integrated school feeding program in Côte d'Ivoire, WFP was awarded funding from the McGovern-Dole International Food for Education and Child Nutrition Program of the United States Department of Agriculture (USDA). This funding supports the school feeding program over the period from September 2015 to July 2020 in seven regions (Poro, Bagoué, Tchologo, Bounkani, Gontougo, Bafing, and Cavally), with the goal of improving nutrition and health outcomes and increasing the literacy of school aged children in 613 rural public primary schools these high-priority areas. 56. In addition to the school feeding component, the project provides support to literacy learning. The international NGO AVSI was selected to improve the reading skills of assisted students through the literacy component. In collaboration with the Ministry of National Education (MENETFP), AVSI designed reading improvement tools and trained teachers in new reading techniques. 57. The current project activities are to: 1) Provide daily hot lunches composed of rice (150g / per day / student), split peas (30g / day / pupil), and vegetable oil (10g / day / student) to 125,000 primary school students at 613 schools in seven targeted departments; 2) Provide take home rations (THR) of 50kg of rice to 10,000 regularly attending CM1/CM2 girls, three times per year; 3) Train canteen management staff once a year to enhance management and administration capacities, in the areas of food preparation, nutrition, reporting, filling out data collection tools, using the mobile data collection tool (ODK), and building improved fuel-efficient stoves; 4) Train 613 school management committees (COGES) every two years in community mobilization, school canteen management, and relevant crosscutting issues affecting the community; 5) Provide food preparation and storage tools and equipment (cooking kits, energy efficient Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 2 | Page cook stoves, wooden pallets for commodity storage); 6) Strengthen women’s production groups (WPGs) established around school canteens through enhanced agricultural knowledge, access to structured markets, and adult literacy (50/year); 7) Distribute deworming tablets twice a year to all students; 8) Provide technical assistance, limited financial support, and trainings to the Government of Cote d’Ivoire in the development of a national school feeding policy and enhancement of monitoring, logistics and commodity management capacity; 9) Developing reading improvement toolkits for teachers, community members, school administrators and Ministry officials as a guide to quality reading instruction that combines best practices in literacy; 10) Distributing suitable French language books and mobile libraries in targeted primary schools; 11) Improve distribution and use of Government’s existing literacy materials; 12) Conduct four professional literacy instruction workshops; 13) Build capacity of Government and communities to improve literacy instruction through Reading Promotion Circles and an Early Grade Reading Symposium, in addition to implementing other reading-related interventions. Timing 58. The duration of the current project is from September 2015 to July 2020. The evaluation team’s mid-term evaluation is timed to allow it to find early signals of the effects of the project and allows time for modifications or mid-course corrections. Objectives and Activities 59. The two main objectives are to improve nutrition and health outcomes and to increase literacy of school aged children in high-priority rural areas. The project aims to reach— directly or indirectly—more than half a million beneficiaries. The key project activities are described above. Outcomes 60. The two main planned outcomes of the intervention are improvement in the literacy of school-aged children and increased use of health and dietary practices. These two outcomes are expected to result from a combination of more consistent teacher attendance, better access to school supplies and materials, improved access to learning materials for reading and increased skills and knowledge of teachers. The other outcomes that fall under the increase use of health and dietary practices include improved knowledge of health and hygiene practices, increased knowledge of safe food preparation and storage practices, increased knowledge of nutrition, and also access to preventive health interventions. 61. The WFP proposes to go beyond these two results to put into perspective the other equally important results such as ensuring children receive nutritious lunches that feed their bodies and minds, improving enrolment, retention, and full day attendance of boys and girls in schools, support to local production, protecting the environment through the construction of improved fuel efficient stoves, and the reinforcement of the capacity of the government for the management and implementation of the school feeding program. 62. The mid-term evaluation of the WFP “Support to a Sustainable School Feeding Programme (DEV 200465)” project, which operated from 2013 to 2016, highlighted that the planned transfer of responsibility for WFP-supported school canteens to the government did not occur, the food distributed was insufficient to generate the expected impact, and the limited support to WPGs reduced the potential sustainability of the Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 3 | Page project.9 This evaluation will pay particular attention to whether these stated shortfalls have been corrected in the MGD project. Results Framework 63. The results framework developed for this intervention is based on the two main planned objectives and linking project activities in a chain of logical connections. For each projected outcome, there are foundational results that include increased capacity of government institutions and improved policy and regulatory framework as well as increased government support and engagement of local organizations and the community in general. The mechanisms of change are clearly presented through project activities, planned outputs, and planned outcomes. For example, providing school meals reduces short-term hunger, increases school attendance, and improves attentiveness, all which are relevant to the improved literacy of school aged children. The project results framework is presented in Annex 8. Partners 64. The key partners involved in the intervention are WFP, AVSI, and the GoCI, through 1) the MENETFP, including the School Canteens Directorate (DCS) and 2) the Ministry of Agriculture and Rural Development (MINADER) et l’Agence Nationale d’Appui au Développment Rural (ANADER). 65. These partners collaborate within two structures, the MGD Steering Committee, which oversees the implementation of the project and is chaired by the MENETFP, and the MGD Technical Committee, which includes technicians from all project stakeholders who provide technical support for the implementation of the project. Resource requirements and funding situation 66. The total award of the USDA MGD Food for Education Program to WFP for the implementation of the school feeding program and the literacy component, as well as the activities listed above, is USD 35,678,500 over five years, which includes an in-kind donation of 24 600 metric tons (MT) of food to be distributed by WFP over the life of the project. Gender dimensions of the intervention 67. In order to address the gender disparity in girls’ school enrolment, the intervention provides THR to 10,000 girls from the CM classes (the last two grades of primary education). Several other WFP Country Offices in West and Central Africa have provided take-home rations to girls, which has improved girls’ enrolment and retention at the primary school level. Take-home rations act as an incentive for parents to keep girls in school. 68. Another gender dimension is providing support to 50 WPGs per year who will allow their schools to graduate from MGD and transition to the GoCI’s national program. Anchored in the government’s vision for “One school, One canteen, One women’s group,” WPGs are community-based groups that receive technical support from the GoCI or its partners, and jointly cultivate food commodities from communal plots to contribute to the school canteens. These allow women to develop their own economic power through 9 Côte d’Ivoire, Projet de Développement 200465 – «Appui au Programme Intégré de Pérennisation des Cantines Scolaires » Evaluation d’Opération du PAM Mi-Parcours (2013-2016). PAM: Décembre, 2015. https://www.wfp.org/content/c%C3%B4te-d%E2%80%99ivoire-dev-200465-support-sustainable-school-feeding￾programme-operation-evaluation-t Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 4 | Page which they can influence decisions in their communities. The groups are committed to share 30 percent of their production in order to supply schools. By the end of the project, WFP expects to support a total of 200 WPGs, including 10,000 women. WFP is mobilizing additional funding to provide technical support to these groups through ANADER and other local partners. 69. Finally, WFP will build the capacity of the COGES to oversee the school canteens, expecting to train 12,500 COGES members, and target women as 50 percent of the members of these committees (6,250 women). By providing training in community mobilization, school canteen management, and relevant crosscutting issues affecting the community, WFP will strengthen COGES’ ability to provide leadership and oversight to the school and canteen with effective female representation. 1.2. Context 70. Since the end of the post-election crisis, Côte d'Ivoire's economic recovery has been notable, with the country experiencing one of the highest growth rates in sub-Saharan Africa. According to the World Bank's latest Living Standards Measurement Survey from 2015, the poverty rate declined from around 51 percent in 2011 to 46 percent in 2015. This improvement triggered by recent economic recovery has affected both rural and urban areas. Nevertheless, poverty remains a predominant rural phenomenon, marked by inequalities in access to essential services and gender disparities. Approximately 30 percent of children under 5 years old suffer chronic malnutrition and 13 percent of the population is food insecure. 71. Women face particularly acute challenges in schooling. In 2012, 63 percent of women in Côte d’Ivoire were illiterate, compared to 49 percent of men. Girls were also more often educationally disadvantaged than boys. Only 14 percent of girls reached secondary school, compared to 30 percent of boys. In primary school, 9 girls were enrolled for every 10 boys, and 34 percent of girls abandoned their education prematurely, compared to 28 percent of boys.10 72. As soon as it became independent, Côte d'Ivoire pledged to reach a schooling rate of 100 percent. As a result, it has placed education as a national priority by allocating more than 40 percent of the budget in the sector. In September 2015, the GoCI passed law n°2015-635 to make school mandatory and free for children between 6 and 16 years old. However, a number of factors slowed down the aspiration, including the thorny problem of noon hunger, which was soon faced by many children whose schools were located several kilometres from the family home. The adequate and comprehensive response to this important problem required the conduct of a social policy based in particular on school meals. 73. Since 1989, the GoCI has administered a national school feeding program with the support of WFP. In 2000, the GoCI integrated nutritional dimensions in addition to educational objectives, aiming for sustainability by encouraging the production of local communities in the form of WPGs. This national program, entitled the “Integrated Program for the Sustainability of School Canteens” (PIPCS), aims to address the problems of chronic child malnutrition, which is high at 23.2 percent for boys and 19.9 percent for girls under 5 years old (MICS 2016), and poor performance in primary education, with 63 percent of the population being illiterate and only 75.1 percent of 10 Bailly-Traore, Anna and Charles Kapie. December 2017. “Etude sur l’Impact du Programme d’Aide Alimentaire du PAM sur les Roles de Genre : Rapport Preliminaire.“ WFP Country Office in Côte d’Ivoire. Page 18. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 5 | Page children finishing primary school in 2016. This national program resulted in the operation of 5,708 school canteens across the country in 2014-2015, providing hot and balanced meals to 1,104,138 elementary school children. 74. In April 2012, the Government of Cote d’Ivoire, through the MENETFP’s Directorate of School Canteens (DCS), with the technical assistance of the WFP and the United Nations Development Program (UNDP), developed the strategy for the National School Feeding Program for the period 2012-2017. The priority areas of the intervention were defined through an analysis based on a composite indicator of the level of food insecurity, gross enrolment rate, gross completion rate, and prevalence of chronic malnutrition. Thus, the following regions have been identified as priority areas for school feeding interventions: priority 1 (Cavally, Guémon, Poro, Bagoué, Tchologo, Bafing), priority 2 (Worodougou, Béré) and priority 3 (Gontougo and Bounkani). 75. Aligned with this national strategy, from September 2013 to December 2016, WFP implemented the previous iteration of the subject of this evaluation, entitled: "Support for the Integrated Programme for Sustainable School Feeding". With an expected number of 571,000 beneficiaries, this project targeted 29 percent of all school canteens and 15 percent of all public primary schools nationally. This project covered 1,634 school canteens in the 10 priority regions. Building on the progress from this project, WFP conducted a vulnerability analysis and mapping (VAM) exercise to select the most food insecure communities within 7 of the 10 priority regions for the 2015-2020 MGD project, in coordination with the DCS, although WFP continues to provide school meals with funding from other donors in 6 other regions in Côte d’Ivoire (which are not included as part of this evaluation). 1.3. Evaluation Methodology and Limitations Evaluation Design 76. To address the research questions at midline, the evaluation team built on the quantitative methodology used at baseline to ensure consistency in the approach and to minimize the introduction of new biases. In addition, the evaluation team added a qualitative component to complement the quantitative analysis and shed light into the processes and mechanisms of change and the factors that may have affected implementation. 77. To assess the causal effect of the MGD project after two years of implementation on student’s reading proficiency11, the evaluation team designed a difference-in￾differences (DID) method at baseline. This method compares the changes in outcomes between the population of beneficiaries (the treatment group) and the population that is not benefiting from the project (the comparison group) by following the same students over time. Evaluation Questions 78. The evaluation team organized the evaluation questions under the following criteria: relevance, effectiveness, efficiency, impact, and sustainability. Given the breadth of the questions for this evaluation, the evaluation team has compiled them into a comprehensive conceptual framework, in Annex 1. For each evaluation question, the evaluation team lists data collection strategy, data analysis method, and the responsible party for collecting and measuring them. In Annex 1, the evaluation team also maps 11 Reading proficiency is defined as ability to achieve the grade-appropriate reading level by USDA. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 6 | Page each evaluation question to their corresponding indicators or assessment criteria, required by USDA, based on the MGD results framework. For example, the evaluation team listed the output indicators under efficacy (e.g., number of girls receiving take￾home rations as a result of USDA), and outcomes indicators under impact (e.g., proportion of students at the end of grade 2 who demonstrate reading proficiency at their grade level). Sampling Frame 79. As requested in the revised TOR and in compliance with the requirement for consistent DID methodology, at midline, the evaluation team followed the same 99 school impact evaluation sample used for the baseline evaluation and added 9 extra treatment schools which the evaluation team randomly selected, maintaining the regional distribution, from the 169 MGD schools sampled at baseline. 80. The evaluation team targeted a sample of 1,753 students and their corresponding households, out of the 125,000 student beneficiaries, to ensure proportional representation of respondents within 104 schools12 across the 7 regions, among which 68 treatment schools and 36 control schools. The evaluation team also ensured a balanced sample of girls and boys to disaggregate data by gender where applicable. In each school, the evaluation team also surveyed teachers in Grades CP1-CM2, the principal, and the canteen manager. Exhibit A2.1 in Annex 2 shows the distribution of sample respondents by region. 81. In collaboration with WFP, the evaluation team selected 20 project sites to visit. At each site selected, the evaluation team conducted focus group discussions (FGDs) with CM students, WPGs,13 parents, and COGES members. Where possible, the evaluation team separated the parent focus groups by gender to allow respondents to speak more freely and to compare mothers’ and fathers’ perceptions. Finally, the evaluation team conducted key informant interviews (KIIs) with internal and external stakeholders, as described in Annex 4. In all mixed-gender FGDs and KIIs, the evaluation team actively encouraged women’s participation to ensure the evaluation team’s evaluation incorporated their experiences and recommendations. Data Collection and Analysis Methods 82. Surveys. To answer evaluation questions, the evaluation team reviewed instruments that were used during baseline and revised the tools to more accurately capture the MGD indicators and fill in the gaps from baseline (e.g., add or revise questions related to student attendance, teacher attendance, minimum acceptable diet, etc., particularly to distinguish between girls and boys). The evaluation team also developed a teacher survey that it calibrated into Ivorian context to capture teachers’ training needs and use of the new techniques. 83. Reading Assessment. To measure the impact on reading skills of school children and analyse any change from baseline, the evaluation team administered the same reading assessment tool, ASER. To avoid any possible bias in reading outcomes, the team updated the test content. The evaluation team ensured that the updated test has the 12 Among the 108 selected schools, the evaluation team learned that 4 control schools closed since baseline, which the evaluation team excluded from the evaluation. 13 The sample of WPGs contains 10 groups supported by the MGD project since 2017 (designated as "established") and 10 groups supported since 2018 (designated as "new"). Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 7 | Page same level of complexity as the one used at baseline to be able to compare students’ reading skills between baseline and midline. 84. Qualitative Tools. In addition, the evaluation team administered KIIs and FGDs protocols to conduct an in-depth stakeholder analysis, including at the beneficiary level. The tools assisted in interpreting the quantitative findings by contextualizing and filling in gaps from the baseline, as several of the MGD indicators are not directly comparable. 85. Data Collection. In collaboration with its data collection partner, Mont Horeb (MH), and its fieldwork managers, the evaluation team recruited and trained 40 experienced enumerators, with local language skills in addition to French, to collect the midline evaluation data in September–October 2018 (at the beginning of the school academic year, 2018-2019). MH organized these 40 enumerators into 10 teams of three individuals and one supervisor, including both men and women, to collect data in the seven target regions. The evaluation team and MH’s fieldwork managers closely followed the ten teams daily to oversee the quality of the data that enumerators collected and provide them with technical support. The evaluation team conducted a stakeholder analysis with internal and external stakeholders, and conducted qualitative interviews with local stakeholders and beneficiaries. 86. Evaluation Analysis. After the field activities were completed, the evaluation team conducted a final review of the collected data. The evaluation team then compiled the survey responses into a master file for the performance-evaluation analysis. Where the accuracy/completeness of the baseline quantitative data allow, the evaluation team conducted a cohort comparison to measure the project’s progress in reaching its target goals and provided descriptive analysis (percentages and averages) where the comparison was not feasible in order to establish base values for comparison later at endline. When possible, the evaluation team also conducted subgroup analyses by grade, student gender, highlighting emerging patterns. 87. For the impact evaluation, the evaluation team compared the changes in reading outcomes between students who attended MGD schools (treatment group) with students who were enrolled in non-MGD project (comparison group) to estimate the two-year project’s effect on literacy growth. The evaluation team surveyed students from the same schools in baseline and midline so that it can construct the change in reading outcomes for each cohort of student. Since the evaluation team collected the midline data at the beginning of the 2018-2019 school year, to be able assess their reading outcomes at the end of their grade, it had to measure their reading proficiency retrospectively which excluded CM2 cohort from the analysis. Validity and Reliability 88. To measure project impact between baseline and midline, the evaluation team collected surveys and reading assessments at the same schools selected by the baseline evaluator (INS). The evaluation team and WFP were unable to meet with INS to assess the validity of the baseline sampling and data collection. However, having designed and validated the reading assessment for the baseline evaluation, the evaluation team is confident in the validity of the related indicators at baseline and the evaluation team’s ability to infer causal relationships between the project and progress in literacy. 89. Before midline data collection, the evaluation team conducted an adaptation workshop with a group of local reading, curriculum, and assessment experts from MENETFP with support from AVSI, as the evaluation team successfully did at baseline, to ensure that the updated ASER test is still culturally appropriate and consistent with Côte D’Ivoire’s Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 8 | Page learning standards for Grades 1–6. The long-form methodology in Annex 2 presents the structure of the ASER reading test in Exhibit A2.2, including the test’s levels and corresponding grades and reading skills. 90. In addition, the evaluation team held a validation workshop at midline with DCS and WFP to review all of the survey instruments. Based on the discussion, adjustments to the instruments were made prior to data collection. The evaluation team also shared its inception report, including the surveys and qualitative protocols, with WFP, AVSI, and Government stakeholders to address their comments prior to any field work. 91. Quantitative and qualitative data were compared to cross-validate findings across data collection methods and verify the reliability of beneficiary responses. To ensure inter￾rater reliability, an IMPAQ research analyst performed daily quality checks on key variables, including reading assessment and food consumption scores, and followed-up quickly as necessary to address any significant differences in survey responses by enumerator. In addition, qualitative data was independently collected, with the findings and transcripts reviewed by IMPAQ home office staff. 92. Modifications and Limitations. There were some limitations and challenges in the MGD midline evaluation. These include: inaccurate or missing data from baseline; delay in data collection until the beginning of the school year 2018-2019; change in the project implementation plan for training teachers; and following the same subsample over time. Exhibit A2.3 in Appendix 2 describes the limitations and challenges faced in the evaluation team’s midline evaluation in further detail, along with the mitigation strategies it implemented and their possible impact on the findings. 93. Ethical Considerations. During the evaluation the following ethical issues were considered for the design, data collection, data analysis, reporting and dissemination: providing sufficient information about the evaluation to participants; obtaining informed consent from participants; collecting confidential data from participants, assuring them that their identity will not be revealed; and analysing and presenting de-identified and aggregate findings. The following safeguards and measures to manage these issues were in place: submitting the research protocol and data collection instruments to Advarra Institutional Review Board (IRB); training enumerators on respondents safety and confidentiality, with particular consideration given to safeguarding child respondents; obtaining teachers’ and principals’ consent to survey their students along with each child’s individual assent; keeping the data in a secured place to protect respondents’ personal information; and active monitoring of data collection for compliance with IMPAQ quality and ethical guidelines. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 9 | Page 2. Evaluation Findings 94. The evaluation findings and the evidence to substantiate them are presented below. They are structured as a response to each evaluation criteria and subsequent evaluation questions in turn. 2.1. Evaluation Criteria 1 – Relevance 95. The first set of evaluation questions aim to assess the relevance and design of the intervention. The findings in this section are informed from the evaluation team’s document review and stakeholder analysis, and address the following questions: • To what extent was the design of the interventions in line with the needs of the target population—women, girls, boys, and men? • To what extent is the design of the intervention aligned with national policies, strategies, and programs? • Do the project design and implementation arrangements complement other donor￾funded and government initiatives? • Is the project designed to reach the right people with the right type of assistance? Internal Stakeholders (WFP) and Partners/Sub-Recipients (Government and AVSI) 96. The MGD project was designed in collaboration with the GoCI to reach the most vulnerable population in Côte d’Ivoire and was guided by a national school feeding strategy that the GoCI developed in 2012 with the technical assistance of WFP and UNDP. This national strategy identified ten priority regions for school feeding interventions, based on their level the level of food insecurity, gross enrolment rate, gross completion rate, and prevalence of chronic malnutrition. Ensuring that its activities are relevant to national priorities, the WFP implements the current MGD project in seven of those regions and distributes THR for girls in three of the seven regions (Bagoué, Poro, and Tchologo) to reduce their high gender enrolment disparity. 97. Key stakeholders from the GoCI confirmed that the MGD project is highly integrated into and complements the government’s efforts to provide nutritious school meals to all primary school children in the country. Representatives from the DCS described how the activities fit into the National School Feeding Program, which was developed in 1989 in partnership with WFP, and appreciated WFP’s technical support in developing the country’s first national school feeding policy, enacted in March 2018 and covering 2018 to 2025. 98. The project also complements other donor- and Government-funded initiatives. Specifically, WFP has partnered with UNICEF and the Global Partnership for Education to target infrastructure improvements at the same schools receiving MGD funding. However, as confirmed by the evaluation team’s quantitative and qualitative data collection at midline, these infrastructure improvements have not yet begun in the schools in the evaluation team’s sample. 99. While the project is highly aligned with Government priorities, some representatives from DCS said they were not consulted during the final stages of the design phase. Because of this, stakeholders believed that several activities were not realistically planned. For example, the original proposal anticipated providing meals to 125,000 students in 1,000 rural schools, but the average school enrolment in these regions is much higher. While the number of student beneficiaries remained as proposed, WFP had to reduce the number of schools they worked in, which government officials said could have been decided earlier had they been more involved in the planning phase. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 10 | Page Some government officials also thought that many proposed activities were not realistic, for example, the allocated resources and timeline for the number of improved stoves, the training of COGES, and the capacity building of WPGs. The training of 200 WPGs in particular was mentioned by multiple stakeholders as being ambitious given the timeframe and resources. 100. Several GoCI key informants mentioned wanting to receive additional training on the monitoring and evaluation (M&E) practices and tools that WFP personnel use on the MGD project. This would not only increase their current technical capacity to implement MGD, but also empower them to improve management of the national school feeding program and other projects. The MGD project delivers annual M&E training to DCS and DPFC, suggesting that either these trainings may not be attended by all relevant GoCI personnel or do not pertain to M&E topics most relevant to the Government’s work. External Stakeholders (Beneficiaries) 101. At the beneficiary level, parents and members of the COGES agreed that aspects of the intervention are relevant to their needs, specifically food delivered to the canteen, school improvements, and literacy activities. According to beneficiaries, a critical challenge for their communities before the MGD project began was attracting students to school and consistently serving meals at the school canteen. 102. By encouraging or allowing children to attend school with the promise of a lunchtime meal, the MGD project is also directly relevant to the needs of parents of the students by reducing the time they needed to devote to childcare during the school week and so increasing the amount of time they can devote to their livelihoods. Some parents also spoke of the psychological relief in knowing their children will not go hungry. 103. The WPGs reported that receiving high-quality seeds, herbicides, and fertilizers through the project is relevant and highly appreciated. Prior to MGD support, financial access to these inputs was challenging, and they had to adjust their practice due to shortages (for example, reduce the amount of recommended fertilizer as they did not have enough). In the ten established WPGs, members received even more materials, including watering cans, gardening tools, scales, tricycles, corn mills, and oxen. Women in these groups appreciated that prior to distributing materials, Development Training and Consulting Office (BFCD) staff met with them to determine their needs.14 This ensured that the equipment received was directly targeted and relevant to the individual groups. 104. In addition to addressing food insecurity, which the WPGs in the north said was particularly severe, members said that the project addresses school level outcomes such as the improvement of children’s attention, attendance, and concentration in class. The WPGs and students’ mothers said that because food is provided at the canteen, they save time and money as they no longer have to make food for school lunches and their children stay in school for their afternoon classes. 105. The literacy component of the project was welcomed by parents, as they said that education is the key to future success for both their sons and daughters. Many talked about their own challenges not knowing how to read and write and hoped for something better for their children. However, some parents were disappointed that some children 14 For the 10 established WPGs, WFP is working with BFCD, an Ivoirian NGO, instead of ANADER to assess WPG needs and deliver technical assistance. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 11 | Page in their communities who earned diplomas are currently jobless. Some parents reported heavily investing in their children’s education, who are unemployed or took a long time to find work. This demotivates parents, with some reporting that there is no value in sending children far in school. Key findings and conclusions – Relevance • The project is highly integrated into and complements the government’s efforts. Beneficiaries agree that the food delivery, school improvements, and literacy activities aspects of the intervention are relevant to their needs. • While the project is highly aligned with Government priorities, some representatives from DCS said they were not consulted during the final stages of the design phase. Because of this, stakeholders believed that several activities were not appropriately realistically planned given the timeframe and resources. • The MGD project is also directly relevant to the needs of parents of the students by reducing the time they needed to devote to childcare during the school week– increasing the amount of time they can devote to their livelihoods. • The WPGs reported that receiving materials and training was relevant and highly appreciated. • The literacy component of the project was welcomed by parents; they said that education is the key to future success for both their sons and daughters. Many talked about their own challenges not knowing how to read and write and hoped for something better for their children. 2.2. Evaluation Criteria 2 – Effectiveness 106. The second set of evaluation questions seeks to evaluate the project’s effectiveness. The results in this section are based on the qualitative data (for quantitative results, please see Evaluation Criteria 3 – Impact in section 2.4). The results below aim to address the following: • What is the progress of project implementation—is the project on track to carry out all activities as planned? • To what extent have the interventions so far responded to the needs of the beneficiaries—women, girls, boys, and men? • What are the main factors influencing the achievement, whether or not of the results/ objectives, of the intervention? • Are any changes required to increase the project effectiveness? Progress of Project Implementation and Response to Beneficiary Needs 107. At the beneficiary level, parents and COGES members agreed that aspects of the intervention addressed community needs, particularly the food delivered to the school canteens. The absence of a regularly provisioned school canteen, seen as a major challenge which previously discouraged or prevented children from attending school or staying for the full day, was addressed under WFP’s MGD project according to the COGES and parents. One group of parents also shared that the MGD canteens even attracted students enrolled in other schools nearby. This information was confirmed by key informants at the national level, who also mentioned that children from nearby schools were enrolling in MGD project schools to benefit from the canteen. 108. According to parents, by encouraging or allowing children to attend school with the promise of a nutritious meal, the MGD project is effective by reducing the economic burden on parents and making their children’s lunch during the school week, which increases the amount of time they can devote to their livelihoods. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 12 | Page 109. While all COGES agreed that the project helped their communities and the MGD food relieves their limited budgets, most COGES members and parents raised concern about the diversity of the meals served to students in the canteen. While respondents agreed that enough rice was provided, along with split peas and vegetable oil, the COGES and newer WPGs said they struggled to grow or buy enough vegetables and animal protein to complement the MGD staples and create varied and nutritious meals. Some asked that different food to be provided in addition to split peas, rice, and vegetable oil, as they did not consider these foods diverse or plentiful enough to make a healthy meal. They brought up examples of previous projects which brought canned sardines or beef to the canteen. Most students reported that they enjoyed the school meals, but a few children said their meals were not nutritious because the sauce did not contain enough vegetables. 110. However, in the ten established WPGs, dietary diversity was not raised as an issue. Women in these groups reported that since receiving project support they were able to provide vegetables to the canteens to create nutritious and balanced meals. They also reported receiving training on nutritionally diversifying the types of meals they could make with their crops. Women said that they learned how to identify symptoms of malnutrition (which they had previously attributed to witchcraft) and were given guidelines on nutrition. By following these guidelines, they stated that in their communities there are now fewer cases of malnutrition. 111. From the discussions with the ten established WPGs, members stated that their production has grown substantially due in part to the technical support received through the project. All groups were able to increase their contributions to the school canteens from last year to this year, with one group increasing their contribution from one ton of food last year to four tons this year. Members credited their higher production yields to learning how to efficiently plot seeds and distribute fertilizer, as well as on preventing crops from being destroyed by insects and disease. Prior to MGD support, many members said they threw seeds in their field and mixed the crops. Learning how to plant seeds in rows and appropriately apply fertilizer and pesticides was cited by the women as an essential part of increasing their yields. 112. According to parents, the project helped their children in lower grades decipher letters, particularly for their children at the CP levels. Overall, parents were happy with the quality of the teachers and the progress of their children, although some parents said the teacher training for the older grades (CE and beyond) was less effective. Even so, they said their children still benefited from attending class with a trained teacher. 113. Multiple stakeholders believe that WFP is behind on several of their planned activities because they did not accurately estimate the required involvement and cost. The activities related to the WPGs were most frequently cited, but others were mentioned as well, including those related to school infrastructure improvement and capacity building of the canteen management committees and COGES. Both internal and external stakeholders spoke of being behind on activities, and when asked, said that underestimations of the WPG support budget were made at the design phase, including underestimating the amount of external funds that could be leveraged to help support some activities. 114. Although the MGD project intends to train COGES members every other year in community mobilization and school canteen management, most COGES members interviewed for this evaluation reported not receiving any training. One COGES mentioned attending a school management training from WFP in 2017. Other COGES reported receiving training from government advisors on creating and managing a Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 13 | Page sustainable fund for their COGES. However, it was unclear if this training was related to the MGD project or not. 115. In addition, the COGES were targeted to include 50 percent women as members, however, in the evaluation team’s focus groups, most did not include this many women, and in fact, some COGES were exclusively male. The COGES who did include women stated that the female committee members had the same rights as men, and even if they performed different work, women and men were equally involved. Some COGES had women in leadership positions. In other COGES, women were excluded from leadership, explained by members as being because women were naturally different from men, were culturally quieter, and/or were illiterate. 116. COGES also differed in their operational involvement in the MGD project, with those in some communities checking the quality of food delivered to the canteen while others not participating in the supervision of these deliveries. Factors Influencing Achievement 117. As stated by parents and children, the main factor influencing project effectiveness is financial constraints. Some parents said that they are not able to afford the indirect costs of sending their children to school, and struggle to afford the daily 25 FCFA canteen fee15 and the annual costs of books, school uniforms, and COGES contributions. Many COGES indicated that they lost or received less government funding in recent years, although the official number of subsidized COGES in Cote d’Ivoire has not decreased. Instead, COGES have solicited parents for contributions to fund their activities for the school, creating tension in the community and draining parents’ resources. For example, last year, one COGES estimated that only 58 out of 178 students paid the annual COGES contribution. Some parents threatened to pull their children out of school if they were forced to pay the COGES contribution. However, these contributions are critical to giving the COGES the means to repair school infrastructure and build additional classrooms, sanitary installations, and teacher housing. 118. In 2015, an independent operation evaluation of WFP’s support to the GoCI’s national school canteens program recommended that WFP advocate and support the DCS in implementing a special project to reduce the cost of students who were unable to pay to the canteen fee. In its management response, WFP agreed to perform a study of community participation in the school canteens and specifically mechanisms for free/subsidized school meals, and help the government implement the government￾accepted recommendations from the study.16 This action was to be completed by October 2016. 119. While the daily canteen fee is heavily subsidized, according to parents, COGES, and CM students in the FGDs, it was cited as a significant financial barrier to some children accessing MGD meals. One group of parents said the 25 FCFA canteen fee was especially difficult to manage during the off season for the cashew harvest (generally September to January). After the harvest starts in February, parents are once again able to pay 15 The 25 FCFA canteen fee per child/day is mandated by the Government of Cote d’Ivoire to cover the salary of canteen workers, firewood, and condiments, with a portion also going to the IEP and COGES for activity monitoring. It is equivalent to roughly 0.05 USD. 16 Côte d’Ivoire DEV 200465 Support to a Sustainable School Feeding Programme: A mid-term Operation Evaluation. https://www.wfp.org/content/c%C3%B4te-d%E2%80%99ivoire-dev-200465-support-sustainable-school-feeding￾programme-operation-evaluation-t Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 14 | Page the fee, and their children are able to eat at the canteen. In the FGDs, students in this community also mentioned sometimes not eating at the canteen because they could not afford the fee. Students in other communities gave similar reasons as to why they did not always eat at the canteen, although the majority of students in the focus groups indicated that they regularly ate at their canteens. In the communities with the established WPGs, women said that the fee was not a barrier for them or their children, but did acknowledge that as members of the WPG, they were more financially stable than other women in their community. 120. According to parents and students, in some schools the canteen was only open 3-4 days a week. This is validated by the canteen managers surveyed, who indicated that on average across all regions the canteen was open only 93.1 percent of actual school days, with strong regional variation. According to WFP, potential reasons for why the canteen is not open every day include some schools not operating on a five-day schedule and frequent teacher strikes and demonstrations. At the beginning of the 2018-2019 school year, when the evaluation team’s focus groups occurred, children at one school said that their canteen had not yet opened for the year, and therefore they had to eat at home. Doing so often prevents them from returning to school on time in the afternoon. Some students indicated that when the canteen was closed they do not attend school at all, but others said they still came to school to be with their friends or because they valued their education. 121. According to parents, COGES, WPGs, and students, school infrastructure is another factor influencing effectiveness. Some parents reported that the classrooms are overcrowded, with four or more students sitting on benches designed for two students. Some canteens had no tables or seats, so children had to eat sitting on the floor. In many schools, COGES reported that teacher housing was unavailable or in reportedly unacceptable conditions. COGES members and CM students mentioned that access to water was also a key challenge that forced students or canteen workers to walk long distances to collect water for school meals and other school utilities, delaying meals or forcing students to miss class. 122. Another factor influencing project effectiveness as mentioned by parents and students is teacher absenteeism. In one focus group, the students said that last year they had to miss class on four separate occasions because their teacher left to conduct personal business in town. 123. Some students said they did not take books from the mobile library because they were worried that they or a family member would damage them. In at least one school, it is the policy (or at least the perception of the students) that they must pay to replace any damaged book. This policy dissuaded children from taking full advantage of MGD￾provided learning materials, and potentially decreases project effectiveness. 124. Some COGES’ members shared instances where the rice has been of poor quality, which they believe could be linked to improper or lengthy storage before the school received its delivery. This information was collaborated by a government stakeholder who mentioned an instance where rice was delivered in powdered form and also where oil was delivered within a month of the expiration date. To prevent food waste, in these situations the rice was treated so it could still be used, and the oil was distributed to multiple schools who prioritized its use. WFP also mentioned that the slow food consumption pace in 2017 combined with annual importing of food from the United States led to longer storage times. However, WFP addressed this by re-scheduling food Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 15 | Page import to be done in tranches throughout the year and implemented capacity building for the DCS on proper food storage and distribution. The government stakeholder who raised this problem agreed that it has since been addressed. 125. The WPGs still rely on men, as there are several agricultural activities that are labour intensive, and typically done by males. For example, ploughing with oxen, soil preparation, and using farming equipment such as rice threshing machines and tillers can be particularly physically demanding or gender specific. This is especially true for WPGs that are composed of older women as their age does not allow them to physically support field work. As a result, these groups depend on male volunteers or paid labour services. One group reported that the cost of day labourers was not affordable, particularly in areas with more lucrative crops like sugar cane. As these members said, a plot of rice requires physical effort that their own members cannot provide, including clearing grass in the field, transplanting rice, cutting it, beating it, and loading it in the trucks. As a result, this group is required to hire wage labourers at a great expense to their overall profit. WFP has noted this and is currently advocating integrating youth as members of the WPG to support their activities. For some of the established WPGs, integrating youths has been successful, especially with the support of the village chief. In two groups, though, the members said that their efforts to incentivize younger community members have been unsuccessful so far. Key findings and conclusions – Effectiveness • The absence of a regularly provisioned school canteen, seen as a major challenge which previously discouraged or prevented children from attending school or staying for the full day, was addressed under WFP’s MGD project according to the COGES and parents. • Parents appreciated not having to prepare meals for their children, thus freeing their time for other livelihoods activities. • Members in the established WPGs stated that their production has grown substantially due in part to the technical support received through the project. • Some planned activities are behind schedule, such as the support to WPGs and the training of COGES. These delays were explained by ambitious expectations in mobilizing outside funds. • Threats to implementation include the low financial means of beneficiaries, who report not being able to afford school-related costs, including school uniforms, textbooks, and the daily 25 FCFA canteen fee. • Other factors influencing project effectiveness include poor school infrastructure (lack of running water, latrines, and school benches) and teacher absenteeism. • The WPGs still rely on men, as there are several agricultural activities that are labour intensive and typically done by males. 2.3. Evaluation Criteria 3 – Efficiency 126. This section explores whether objectives were achieved on time and what factors led to efficiency. Although a cost-benefit analysis is outside the scope of this evaluation, through using qualitative data the team endeavoured to address the following evaluation questions: • How efficient is the targeting? • Does the assistance reach the right beneficiaries (girls, boys, men, and women) in the right quantity and quality and at the right time? • Is the project efficient in terms of costs and costs/beneficiary? • What are the external and internal factors influencing efficiency of the project? Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 16 | Page 127. Overall, the project is efficient in that it generally provides beneficiaries with the outputs as described in the proposal, specifically providing school meals, THR, and literacy instruction. 128. There are some activities that have yet to fully materialize as proposed, as the complementary funds have not been mobilized as anticipated. For these activities, strategic choices were made that benefited the primary components of the project at the expense of secondary components. For example, the literacy and numeracy training for WPGs/COGES were not prioritized, as indicated by national and local stakeholders. 129. During the stakeholder analysis, some government stakeholders suggested that WFP should reduce the number of school days for which it provides MGD commodities and reorient the extra food to serve more schools. According to these officials, and corroborated through FGDs with COGES, WFP’s current strategy discourages parents and community members from contributing money and/or food to the canteens, as basic commodities are provided by MGD for every day school is in session 130. It is difficult to accurately and regularly monitor student and teacher attendance (including absences due to sickness) and canteen outputs. When reviewing school records, the evaluation team found that attendance rates of teachers and students conflicted with the qualitative data. The school records showed almost perfect attendance for teachers and students, while in the evaluation team’s FGDs respondents reported frequent teacher absenteeism, and instances where children missed school. Teachers self-reported missing school at a substantially higher rate than what the school records indicated. As for the canteen records, in the evaluation team’s survey of MGD canteen managers, referring to March 2018, 39 percent did not know the expected number of meal recipients for the month; 10 percent could not disaggregate by gender the number of meal recipients; and 28 percent could not indicate the number of days the canteen served four food groups. 131. External factors influencing efficiency include climate variation and lack of irrigation systems that affect the crop yield of the WPGs. Members from all groups spoke of their reliance on rain to water their crops. When rain is plentiful, groups can grow vegetables and rice to support the canteen, but during the dry season from April through July, they are unable to grow these crops. 132. Other external factors influencing efficiency include the low incomes, high-illiteracy rates, and lack of management skills of the community members, and therefore of the COGES, which prevent them from fully executing several activities to improve the infrastructure of the schools or to start taking ownership of the canteens. The remoteness of the schools also affected the efficiency of several activities, due to the cost of exporting materials and expertise to these schools. For example, the construction of improved, fuel-efficient stoves was more difficult and expensive than originally planned, because the materials and expertise are limited in isolated rural areas. Key findings and conclusions – Efficiency • Overall, the project is efficient as it provides beneficiaries with school meals, THR, and literacy instruction. • There are some activities where the complementary funds have not been mobilized as anticipated. For these activities, strategic choices were made that benefited the primary components of the project at the expense of secondary components. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 17 | Page • Some government stakeholders and local community members suggested that by providing meals every day, the project discourages the community from taking ownership of the canteens. • It is difficult to monitor the project using school records. Student and teacher attendance reports appear to be inflated, as the official reports show almost no absenteeism, conflicting with the qualitative results. Canteen records did not appear inflated, but some were missing or incomplete. • External factors influencing the efficiency of the WPGs include climate variation and lack of irrigation systems. Other factors include the low incomes, high-illiteracy rates, and lack of management skills of the community members, which prevent them from fully executing certain activities. 2.4. Evaluation Criteria 4 – Impact 133. This section forms the largest part of the evaluation team’s evaluation findings, and is divided into two main sections: • Impact Analysis, where the evaluation team uses a difference in differences approach to measure the effects of the MGD intervention on students’ reading outcomes; and • Performance Analysis, where the evaluation teams use a descriptive approach using the survey findings to measure changes in outcomes, including triangulating where relevant examples of project impact informed by the stakeholder analysis. 134. The evaluation team address the following evaluation questions to assess changes as a result of the intervention: • What are the midterm effects of the operation on the beneficiaries in terms of: (a) improvement of school indicators; (b) improvement of pupil reading skills; (c) capacity building of groups? • What are the reasons for the observed effects? • Are there unintended effects on the beneficiaries? What have been the gender￾specific impacts, particularly with regards to girls’ schooling? • To what degree have the project outcomes made progress toward positive long￾term effects on targeted beneficiaries (girls, boys, men, and women), households, communities, and institutions? 2.4.1 Impact Analysis 135. As described in the methodology section in Annex 2, the evaluation team used a difference in differences (DID) method to measure the changes in reading outcomes over time between students who attend schools that receive the MGD interventions17 (treatment group) and students in schools that do not receive the project (comparison group/Non-MGD schools). 136. Given the change in implementation of teacher training in 2017-2018, across different grades at midline, students have not been exposed to the teacher training consistently. The only cohort of students who have been fully exposed to the project interventions for two years, including two-years of trained teachers, for which the evaluation team also has their baseline values are CE2 students (who were in CP1 at baseline). Therefore, to maintain the power of the study, the evaluation team restricted its sample to compare the change in students’ reading skills from baseline (when they were in grades CP1, CP2 and CE1) to midline (when they are in grades CE2, CM1, and CM2), with the change that occur among similar students from comparison schools. The 17 School canteen and literacy activities. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 18 | Page evaluation team’s key outcome of interest is the improvement in children’s reading proficiency at grade level due to at least one-year project effects. 137. It is important to note that since the midline data collection was done at the beginning of the 2018/2019 school year, the midline reading skills of CE2, CM1, and CM2 students reflect their reading ability at the end of the previous school year (i.e., reading ability at CE1, CE2, and CM1 level, respectively). To estimate this effect, the evaluation team ran the following model: ΔMGD= [(𝑌𝑌1MGD − 𝑌𝑌0MGD) − (𝑌𝑌1non-MGD − 𝑌𝑌0non-MGD)] Where: • 𝑌𝑌0 and 𝑌𝑌1 denote reading outcomes at baseline and midline respectively; and • MGD and non-MGD denote schools that are receiving MGD interventions and comparison group that are not receiving any activities. 138. To obtain more precise estimates, the evaluation team included other covariates, such as the gender and grade of the student, as well as availability of story books at their homes. The evaluation team also controlled for the head of household’s educational attainment and gender as significant differences between their values in treatment and comparison groups at baseline were observed. In addition, because students are nested in a particular grade from a given school, literacy outcomes are correlated to each other. To account for this correlation, the evaluation team clustered the standard errors at the school level. 139. At midline, the evaluation team tried to assess the same students from baseline. When they were not available to be surveyed for reasons such as dropping out of school or moving away, the evaluation team sampled new students randomly to replace them. However, some of these replacements might not have been exposed to the project if they have just joined the school for the 2018-2019 school year. As a robustness check, the evaluation team estimated its model by restricting the sample to include only the students who have been in the school for at least a year18 by academic year of 2018- 2019. 140. Figure 1 summarizes the impact results. The second column, highlighted in grey, represents the holistic project’s impact on children’s literacy outcomes (with at least one-year exposure to the trained teachers). As shown in Figure 1, exposure to the MGD project increases the reading assessment literacy scores of students in treatment schools at midline by 1.2 points (p<0.01) relative to comparison group, which is approximately one-level increase in the reading assessment. The evaluation team also estimated the model by restricting the sample to include only the students who have been exposed to the project for two years (CP1 students from baseline who are now in CE2 in 2018-2019). The estimated impacts are somewhat larger (1.7 points) when the evaluation team limits the sample (fifth column). However, the project effect remains significant at all levels. 141. In addition, the data show that gender plays an important role in students’ reading scores. As shown in the exhibit, the project teacher training is more likely effective for boys as compared to girls. The data show that the project has significant positive effects 18 To identify these students, the evaluation team asked all students when they started in their school and excluded those who mentioned 2018-2019 as an answer (i.e., students who just joined the school and have not been exposed to the program yet). Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 19 | Page on both boys and girls, but the effect was doubled for male students (almost 2 levels improvement for boys compared to almost 1 level for girls). Also, the effect on girls is only statistically significant at the 10 percent level. Figure 1: Project Effects Using DID Method Difference At least one-year exposure Two-year exposure Overall Sample Female Male Overall Sample 𝑌𝑌1 𝑀𝑀𝑀𝑀𝑀𝑀 − 𝑌𝑌0 𝑀𝑀𝑀𝑀𝑀𝑀 4.3*** (0.239) 4.1*** (0.271) 4.7*** (0.256) 4.45*** (0.357) 𝑌𝑌1 𝑛𝑛𝑛𝑛𝑛𝑛−𝑀𝑀𝑀𝑀𝑀𝑀 − 𝑌𝑌0 𝑛𝑛𝑛𝑛𝑛𝑛−𝑀𝑀𝑀𝑀𝑀𝑀 3.12** (0.336) 3.2*** (0.433) 2.9*** (0.360) 2.72*** (0.358) ∆ 𝑀𝑀𝑀𝑀𝑀𝑀 (Project effect) 1.2*** (0.417) 0.91* (0.511) 1.8*** (0.439) 1.7*** (0.503) N 1,740 1,036 704 446 Source: Student Survey; Coefficient (Standard Error); *p-value < 0.1 ** p-value < 0.05 *** p-value < 0.01; Standard errors shown in parentheses are clustered at the school level. 142. While both boys and girls are positively impacted by the MGD intervention, this indicates that girls are not benefiting as much. As shown in greater detail in the performance analysis section below, girls and boys have similar levels of reading proficiency at CP1 and CP2, but starting in CE1, more boys are reading proficient. This trend was similar at baseline, and also in the non-MGD schools, and this achievement gap between girls and boys appears to be exacerbated by the intervention. The qualitative data does not explain why, and in fact, parents spoke often of the importance of educating their daughters. Multiple groups of parents spoke of being inspired by national female role models and believed that by sending their girls at school, they can become ministers, members of parliament, and managers: “big ladies” who are holding “big positions.” Therefore, more investigation needs to be done to determine what is driving this gap (teachers’ attitudes, lower attendance by girls, parents’ selection of who to send to school, etc. are all ideas to be explored). 2.4.2 Performance Analysis 143. The performance evaluation was designed to measure changes in outcomes over the life of the project with regards to USDA’s two strategic objectives, improved literacy of school aged children and increased use of health and dietary practices. 144. The findings below are from the student reading assessment, and the household, student, school, canteen, and teacher surveys. The MGD indicators can be found in annex 9 and project results framework in annex 7. 19 MGD SO1: Improved Literacy of School Aged Children 145. Grade-level reading proficiency increased at each grade level between baseline and midline. The gains observed in MGD schools were larger than the gains observed in non-MGD Schools at each grade level. As seen in Figure 2, at every grade level (CP1 through CM1), a larger share of students in MGD schools were at or above grade-level reading proficiency than in non-MGD schools. 19 At baseline, to measure MGD 1.2.1.1 (Increased Access to School Feeding) the evaluators used composite scores for household food consumption, dietary diversity, and coping strategies as three of the indicators. These indicators were captured at midline to maintain consistency across the two evaluations (see Figure 72 through Figure 74 in Annex 6). However, being household scores capturing data unrelated to the scope of activities in the MGD program, the evaluation team does not believe that these indicators are an appropriate measure of this program, and are therefore not reported in the main body of the report. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 20 | Page 146. While the rate at which girls were able to reach reading proficiency more than doubled at nearly every grade level, their gains were outpaced by boys, who more than tripled the level of reading proficiency of their counterparts at baseline in every grade except CE1, as show in Figure 3. This is similar to the findings of the evaluation team’s impact evaluation, which showed that while the MGD intervention significantly increased the reading proficiencies of all students, boys benefited to a greater extent. When looking at this trend by grade, it appears that this achievement gap between boys and girls starts particularly growing starting at the CE2 level. More detailed scores by grade level can be found in Figure 71 in Annex 6. Figure 2: Reading Assessment Scores by Grade (Midline Only) Source: Student Survey. The vertical black line indicates the threshold for grade-level reading proficiency. Students to the left of the threshold are below proficiency. Students to the right of the threshold are at or above proficiency. The sample size was 964 at baseline and 1753 at midline. CM1 girls in MGD schools were oversampled at midline to gather sufficient observations to be able to analyse the effects of the take-home-rations project in those schools. Figure 3: Reading Proficiency by Grade and Gender (MGD Only) Mid MGD Base MGD Grade Boys Girls All Boys Girls All CP1 43.8% 46.8% 45.2% 3.6% 5.7% 4.6% CP2 39.7% 35.8% 37.7% 12.9% 15.0% 13.9% CE1 51.1% 45.7% 48.5% 24.6% 23.2% 23.9% CE2 45.6% 27.3% 36.5% 6.9% 13.5% 10.0% CM1 30.2% 17.4% 20.5% 10.0% 5.0% 7.5% Total 41.0% 26.6% 32.2% 11.6% 12.5% 12.0% Source: Student Survey. The sample size was 604 at baseline and 1149 at midline. CM1 girls in MGD schools were oversampled at midline to gather sufficient observations to be able to analyse the effects of the take-home-rations project in those schools. More Consistent Teacher Attendance 147. Based on records kept by principals, 95 percent of teachers attended school at least 90 percent of the time. However, based on teachers’ self-reported absences, only 80 percent of teachers were in attendance for 90 percent or more days of school. This ranged from a low of 61 percent of teachers in Cavally to a high of 97 percent of teachers in Bounkani (see Figure 4). No schools in Bounkani region were able to provide data Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 21 | Page collectors with records kept by school principals, so these self-reported figures cannot be corroborated. Figure 29 and Figure 30 in Annex 6 also illustrate teachers’ attendance. 148. While unable to say for certain, one hypothesis to explain the difference in official and self-reported absences is perhaps due to inflation of attendance at the school level. The evaluation team’s field-level data collection found that while most schools were able to show records, they rarely reported absences. This was also true for student attendance, with school records showing very low levels of student absenteeism. In the case of teacher attendance, the evaluation team tends to put more trust in the teachers’ reports, which not only appears more realistic, but also is corroborated by reports from parents and students in the FGDs about the prevalence of teacher absenteeism. Figure 4: Teacher Absences by Region (Midline Only) Source: Teacher Survey. The sample size was 268. Teaches were asked how many days they missed as a result of each of several causes. 149. When students were asked if they were prevented from attending class because their teacher was absent, 64 percent of all students said that this happened “sometimes.” This data matches the teachers’ self-reported attendance, as compared to the official school records. A small number (1.3 percent) of students in both MGD and non-MGD schools said that this happened “almost all the time,” though students reporting this were concentrated in a few schools. Overall, student-reported teacher absenteeism was about the same in MGD and non-MGD schools. Figure 5: Student-Reported Teacher Absences (Midline Only) How often could you not have school because the teacher was absent when you arrived? MGD Non-MGD Total Never 37.3% 30.5% 35% Sometimes 62.2% 66.7% 63.8% Almost all the time 0.4% 2.8% 1.3% Observations 1,149 604 1,753 Source: Student Survey. Better Access to School Supplies and Teaching Materials and Improved Literacy Instructional Materials 150. Nearly all MGD schools (91 percent) were provided with school materials (books, school supplies, etc.), while a majority (72 percent) of non-MGD schools were also provided with school materials. These figures are about the same as from baseline. Large changes that can be observed in Bounkani MGD schools and Tchologo non-MGD schools (see Figure 6) may be attributable to the small number of schools within each region. 0% 20% 40% 60% 80% 100% Records: Attending 90% Self-reported: Attending 90% Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 22 | Page Figure 6: Access to School Materials by Region Region Bafing Bag. Bounk. Cavally Gont. Poro Tcho. Total Obs. MGD Midline 100% 100% 85.7% 100% 82.4% 94.7% 83.3% 91.2% 68 Baseline 83.3% 100% 28.6% 100% 100% 100% 83.3% 89.7% 68 Non￾MGD Midline 80% 100% 50% 83.3% 66.7% 60% 75% 72.2% 36 Baseline 80% 100% 100% 83.3% 83.3% 70% 25% 75% 36 Source: School Survey. Sample size was 104 at baseline and midline. 151. The majority of students enrolled in MGD schools had access to mobile libraries, and many students also had access to other materials, including reading boards and illustrated boards. In non-MGD schools, only 10 percent of students had access to such learning materials. Regionally, students in Bagoué (23 percent) and Tchologo (25 percent) were most likely to say that they had no access to reading materials (see Figure 31 in Annex 6). Figure 7: Access to Reading Materials (Mobile Library, Reading Board, etc.) Reading Material MGD Non- MGD Sculpted plastic letters 14.7% 0.2% Junior dictionaries 17.8% 0.7% Illustrated board 43.3% 3.5% Reading boards 47.4% 4.8% None 13.8% 89.7% Mobile library 67% 5.5% Responses 2343 630 Observations 1149 604 Source: Student Survey. Respondents could select multiple choices. Increased Skills and Knowledge of Teachers 152. The vast majority of teachers at every grade level said that they had participated in trainings provided by AVSI. Overall, this included 90 percent of all teachers, with a low of 84 percent at CE1 and a high of 96 percent at CE2. Of the 10 percent of teachers who did not participate in an AVSI training, 70 percent selected the “other” option. Of the 13 teachers who specified their “other” answer, 7 indicated they were either not assigned to a MGD school at the time of training, while the remainder were either not invited to participate in the training because of the training targeted CP teachers only or they were unaware of the training in their school. This reinforces the finding that nearly all targeted teachers received the MGD project’s training intervention. Although nearly all trained teachers (98 percent) found AVSI’s training to be useful or very useful, only a minority (22 percent) were utilizing four or more of the tools and materials provided and were also using the General Grid for Student Performance. Still, 96 percent of those that had been trained said that they were putting into practice their AVSI training. (See Figure 32 in Annex 6.) Figure 8: Teachers Having Participated in AVSI Trainings by Region (Midline MGD) Region Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total Percent 81.3% 93.3% 86.5% 95% 90.3% 90.9% 91.7% 90% Obs. 32 45 37 20 72 99 24 329 Source: Teacher Survey. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 23 | Page Figure 9: Schools Having Received Training on Teaching of Reading Region Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total MGD Percent 100% 100% 100% 100% 100% 100% 83.3% 98.5% Obs. 6 8 7 5 17 19 6 68 Non-MGD Percent 40% 66.7% 50% 16.7% 66.7% 70% 25% 50% Obs. 5 3 2 6 6 10 4 36 Source: School Survey. Improved Attentiveness 153. On average, every grade level of teachers said that more than 6 in 10 of their students were attentive on a typical day. Teachers of CM2 students said that their students were most likely to be attentive (7.5 out of every 10 students). There were only small differences in the reported attentiveness of girls and boys in class. Teachers in Bafing rated their students as the most attentive (7.8), while teachers in Poro rated their students as the least attentive (6.3). Figure 10: Number of Students out of 10 Described as Attentive by Teachers on a Typical Day (Midline MGD Only) Grade CP1 CP2 CE1 CE2 CM1 CM2 Total Girls 6.9 6.9 6.5 6.4 6.6 7.3 6.8 Boys 6.8 6.6 6.3 6.3 6.7 7.7 6.7 All 6.8 6.8 6.4 6.4 6.6 7.5 6.7 Observations 52 60 57 55 52 54 330 Source: Teacher Survey. Figure 11: Student Attentiveness by Region (Midline MGD Only) Region Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total Girls 7.7 6.6 6.7 6.4 6.8 6.5 6.9 6.8 Boys 7.8 6.7 7.1 7.1 7.2 6 5.8 6.7 All 7.8 6.7 6.9 6.8 7 6.3 6.4 6.7 Obs. 32 45 37 20 72 100 24 330 Source: Teacher Survey. Improved Student Attendance 154. On average, students were recorded as having attended school on 99% of school days. No students were recorded as having attended fewer than 80 percent of school days. According to school records, 52.7 percent of students in MGD schools did not miss a single day of school, compared to perfect attendance in 40.1 percent of students in non￾MGD schools. (See Figure 35 in Annex 6 for regional disaggregation.) 155. Like the teacher attendance records, the student attendance records appear to be artificially inflated, based on what the evaluation team heard during its FGDs with parents and students, who stated that they did sometimes miss school. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 24 | Page Figure 12: Students with Perfect Attendance (Midline Only) MGD Non-MGD Student did not miss a single day of school 52.7% 40.1% Source: School Survey. The sample size was 916. Increased Student Enrolment 156. In contrast to what the evaluation learned from the stakeholder analysis, overall enrolment decreased in MGD schools. However, when looking at the data more closely, this appears to be due to a large decrease in the enrolment of boys and girls in CP1 and CP2.20 All other grades saw increases in average enrolment of both girls and boys. In MGD schools, girls’ enrolment in CM1 increased from 16.4 girls per school to 17.7 and enrolment in CM2 increased from 12.8 to 15.8 per class from baseline to midline. Figure 13: Average Student Enrolment per Grade (All Schools) CP1 CP2 CE1 CE2 CM1 CM2 All Girls Mid MGD 23.1 21.8 21.6 19.9 17.7 15.8 114 Base MGD 29 25.2 21.3 18.9 16.4 12.8 113 Mid non-MGD 18.9 18 18.3 15.8 15.1 12.3 83.7 Base non-MGD 21.2 17.4 16.9 12.5 10.6 10.8 69.4 Boys Mid MGD 20.8 21.2 20.8 20.7 18.7 18.3 114 Base MGD 29 26.9 23.1 20.5 18.8 17.3 123 Mid non-MGD 18.7 18.4 17.5 17.4 15.7 13.4 86.3 Base non-MGD 21.7 18.6 18.4 17.3 14.5 14.5 80.2 All Mid MGD 43.9 43 42.4 40.6 36.4 34.1 228 Base MGD 58 52.1 44.5 39.4 35.2 30.1 236 Mid non-MGD 37.6 36.4 35.8 33.2 30.3 25.7 170 Base non-MGD 42.8 35.9 35.3 29.8 24.5 25.3 150 Source: School Survey. Sample size was 207 schools. 157. Increases in girls’ CM1 and CM2 enrolment (from 33.9 to 42.5) in regions where THR are offered contributed to the overall increases in girls’ enrolment. In contrast, in regions where THR are not offered, girls’ CM1 and CM2 enrolment did not increase as much (from 17.7 to 22.2). The ratio of girls to boys enrolled in these two grades increased from 0.88 to a near-parity ratio of 0.96 in take-home-rations schools, while the ratio increased from 0.7 to 0.81 in non-take-home-rations schools. (See Figure 36, Figure 37, Figure 38, and Figure 39 in Annex 6 for more details on enrolment in schools with THR.) 20 The evaluation team is unable to offer an evidence-based reason for this discrepancy, however, one hypothesis is that data was collected at the beginning of the school year at midline (compared to the end of the school year at baseline), and therefore enrolment records may not be complete. The team anecdotally heard from some program partners that children in rural areas may still be returning to their village after spending a summer working away from home. However, this topic was not explored in-depth at midline. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 25 | Page Figure 14: Ratio of Girls to Boys Enrolled in CM1 & CM2 (MGD Schools) Source: School Survey. The red line indicates gender parity in enrolment. Sample size was 133 schools. Reduced Health Related Absences 158. According to their parents, last year, 4.1 percent of students enrolled in MGD schools missed more than 10 days due to illness. Again, the number of self-reported absences are greater than what is reported through the official student attendance records at the school. On average, the number of days missed was slightly lower for girls (2.8) than boys (3.3), but more girls were also absent for more than 10 days at a higher rate (4.2 percent for girls and 4 percent for boys). Students at Non-MGD schools missed more days of school on average (3.3) and were more likely to have missed more than 10 days of school due to illness (6.4 percent). Eight percent of boys enrolled in non-MGD schools missed more than 10 days of school due to illness. (See Figure 40 and Figure 41 in Annex 6.) MGD SO2: Increased Used of Health and Dietary Practices 159. The percentage of school days during which food was provided was nearly 100 percent in each region, but varied slightly, ranging from to Poro (88 percent) to Bafing (100 percent). There were no substantive differences between the days of canteen services available to boys and girls. This was a large increase from baseline, where schools on average only provided meals on 57 percent of the days when school was in session. (See Figure 43 and Figure 50 in Annex 6 for total statistics and regional disaggregation.) 160. Almost all parents (94 percent) of children enrolled at MGD schools were aware of the school canteen project, and almost all (92 percent) reported that their children ate at the canteen in the previous school year. While the majority of children ate at the canteen regularly (three days a week or more), only about half (53 percent) of children ate at the canteen every school day. (See Figure 51 and Figure 52 in Annex 6 for parents’ awareness and children’s participation in the feeding project.) Qualitative data from parents and students suggest this may be because students did not have the 25 CFCA daily fee, or the canteen was not operational five days a week. WFP staff noted that while technically there are 120 school days in a year, it is rare for schools to actually have this many days in reality. 0.0 1.0 Mid THR Base THR Mid non-THR Base non-THR CM1 CM2 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 26 | Page Figure 15: Frequency that Students Ate at the School Canteen Source: Household Survey. The sample size was 1,143. 161. The percentage of school days during which canteens served food groups constituting minimum dietary diversity (MDD) varied by region. However, the low number of schools in some regions with sufficient canteen records on the number of food groups and ingredients served partly explains this variation. Most regions served MDD more than 80 percent of the school days, but schools in Bounkani served a meal meeting MDD requirements only 50 percent of the school days and in Cavally, records regarding dietary diversity were not available. Schools in Tchologo served an MDD meal 100 percent of canteen days. There was no difference between the MDD meals available to boys and girls. Overall, the percentage of school days where MDD was provided increased greatly from baseline, where only 41 percent of the school feeding days met this requirement. For greater detail on the foods served by region, see Figure 54. 162. The focus groups with the WPGs indicate that in some schools, the lunch provided is more nutritionally diverse. Specifically, the ten established WPGs said they were taught how to cook a balanced diet for children and how to derive nutrients from their local crops. The workers in the school canteens where they contribute their food (including vegetables) have been trained on how to cook a balanced diet for school children. Figure 16: Portion of Days that Canteen Provided Minimum Dietary Diversity Mid MGD Base MGD Region Percent Obs. Percent Obs. Bafing 83% 2 33% 5 Bagoué 83% 6 45% 4 Bounkani 50% 2 53% 6 Cavally N/A 0 30% 3 Gontougo 83% 9 39% 15 Poro 83% 10 43% 17 Tchologo 100% 3 36% 1 Total 83% 32 41% 51 Source: Canteen Survey. Minimum dietary diversity is defined as a day’s meals consisting of at least four of the following food groupings: 1) cereals, 2) tubers and root vegetables, 3) legumes and nuts, 4) dairy products, 5) meat, fish, and offal, 6) eggs, 7) high vitamin A fruit and vegetables, and 8) other fruit and vegetables. 0% 20% 40% 60% 80% 100% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total Every day Four Three Two One Never Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 27 | Page Increased Knowledge of Health and Hygiene Practices 163. While most canteen managers were able to identify three or more health and hygiene practices at midline (88 percent), the number had decreased slightly from baseline (93 percent). In Cavally, less than half of canteen managers (40 percent) could cite three or more health and hygiene practices. (See Figure 55 and Figure 56 in Annex 6.) Since canteen managers also serve as teachers, these results could potentially be explained by teacher turnover in these schools. 164. Teacher training on health practices at midline is somewhat similar to what it was at baseline. While skills-based health education and deworming trainings were almost unchanged, the number of MGD schools receiving HIV/AIDS prevention trainings nearly halved from 22 percent to 11 percent. These trends were generally present in non-MGD schools as well, except for skills-based health education where the proportion of teachers receiving the training decreased from 28 percent to 17 percent. (See Figure 57 in Annex 6.) Figure 17: Teacher Training on Health Practices MGD Non-MGD Type of Training Midline Baseline Midline Baseline Skills-based Health Education 38.8% 35.3% 17.1% 27.8% Deworming 62.7% 67.6% 33.3% 36.1% HIV/AIDS Prevention 11.8% 22.1% 8.3% 22.2% Observations 68 68 36 36 Source: School Survey. Increased Knowledge of Safe Food Preparation and Storage 165. At the time of the evaluation, WFP indicated the proportion of MGD canteen managers trained in safe food preparation and storage was 100 percent in only two regions (Bafing and Cavally) and about 50 percent in the five other regions, with the remainder to be trained in December 2018 and March 2019. Suggesting that the project effectively increased canteen manager knowledge, MGD canteen managers’ knowledge of safe food preparation and storage practices increased modestly in all regions between baseline and midline (from 70 percent to 82 percent for food prep and from 76 percent to 83 percent for food storage). However, this was not a strong trend; while some regions’ canteen managers’ knowledge of safe food preparation and storage practices increased, others decreased. Figure 18: By Region, Canteen Managers Know Three Food Storage and Food Preparation Best Practices Region Bafing Bagoué Boun. Cavally Gont. Poro Tcho. Total Midline Food Storage 67% 75% 100% 60% 88% 84% 100% 83% Food Prep 83% 75% 100% 0% 94% 90% 80% 82% Baseline Food Storage 83% 50% 86% 60% 94% 68% 83% 76% Food Prep 83% 75% 86% 20% 81% 68% 50% 70% Obs. 6 8 7 5 16 19 6* 67 Source: Canteen Survey. Only five schools were available in Tchologo at midline. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 28 | Page Increased Knowledge of Nutrition 166. Only 12 percent of canteen managers in MGD schools at midline reported that members of the canteen management committee received training in nutrition. Among canteen managers who reported that there was a canteen management committee at their school (79 percent at midline, 64 percent at baseline), only 33 percent indicated that they received nutrition training, compared to 41 percent at baseline. 167. While the MGD project focuses on training canteen managers, these managers also serve as teachers and rotate out of the canteen manager position, meaning teachers’ knowledge of topics relevant to canteen management can help ensure the acting canteen manager has the required knowledge. Teachers in more than half of MGD schools (54.4 percent) received trainings on nutrition. Only a small number (8.3 percent) of non-MGD schools received the same trainings. Non-MGD schools also did not receive any student nutrition training or vegetable garden training, while a modest number of MGD schools did receive these trainings. Increased Access to Safe Water 168. Access to improved water sources is about the same between MGD (44 percent) and non-MGD schools (47 percent). But about half of both MGD (53 percent) and non-MGD (50 percent) schools did not have adequate access to water. In MGD schools, about one-third (32 percent) of those with access had water access on the school grounds, while 43 percent were less than a 15-minute walk to a water source, and one-quarter were 15 minutes or further from a water source. In the evaluation team’s qualitative data collection, beneficiaries mentioned lack of water on school grounds as being problematic, as either students (generally girls) or canteen staff would have to fetch water for the school canteen. For students, they left their lessons during the day for this task. This additional task for girls should be explored as a potential reason which may partially explain the gender disparity in reading proficiency, as it may result in girls having less classroom time than boys. 169. The most common problem preventing access to water in both MGD (59 percent) and non-MGD (50 percent) schools was broken water pumps. (See Figure 59, Figure 60, and Figure 61 in Annex 6) Figure 19: Type of Access to Water in Schools Source: School Survey. Sample size at baseline and midline was 104. 0% 20% 40% 60% 80% 100% Mid MGD Base MGD Mid non-MGD Base non-MGD No water Unprotected well Protected well Village pump Tap Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 29 | Page Figure 20: Distance to School’s Water Source, Where Available (Midline MGD Only) Source: School Survey. Sample size was 68. Figure 21: Most Common Problems with Schools’ Water Source Source: School Survey. The sample size was 56 at baseline and 37 at midline. 170. Slightly more MGD schools (59 percent) than non-MGD schools (50 percent) have sanitary facilities on-site. Less than one-third (31 percent) of MGD schools and only one-quarter (25 percent) of non-MGD schools have gender-separated sanitary facilities. Less than one-third of either MGD schools (26 percent) or non-MGD schools (31 percent) have separate sanitary facilities for teachers and students. (See Figure 62.) Figure 22: Access to and Quality of Sanitary Facilities Source: School Survey. The sample size was 104 at baseline and midline. 0% 20% 40% 60% 80% 100% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total In the school grounds Less than 15 min walk Over 15 min walk 0% 20% 40% 60% 80% 100% Mid MGD Base MGD Mid non-MGD Base non-MGD Broken pump Animals occupying water source Dried up water source Water is used for agriculture Other Distance 0% 20% 40% 60% 80% 100% Mid MGD Base MGD Mid non-MGD Base non-MGD Has sanitary facilities Separated by gender Separate for teachers Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 30 | Page Increased Access to Preventative Health Interventions 171. The number of schools that reported receiving deworming treatments decreased from baseline to midline in both MGD (from 72 percent to 64 percent) and non-MGD schools (from 64 percent to 56 percent).21 Distribution of micronutrient supplements fell by more than half in MGD schools (from 53 percent to 26 percent) from baseline to midline. (See Figure 63 and Figure 64.) Increased Access to Requisite Food Preparation and Storage Tools and Equipment 172. In general, canteen managers said they were not provided with enough food preparation equipment (68 percent). However, this is an improvement from baseline, where 81 percent of canteen managers said they did not have enough food preparation equipment. Figure 23: Sufficient Food Preparation Equipment Preparation Equipment Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total Midline 17% 38% 14% 20% 31% 53% 0% 32% Baseline 17% 25% 14% 20% 19% 26% 0% 19% Obs. 6 8 7 5 16 19 6* 67 Source: Canteen Survey. Only 5 schools were available in Tchologo at midline. 173. About the same percentage of MGD schools have access to improved food storage equipment at midline as had access at baseline. Access to stock cards was the only food-related equipment that changed in accessibility from baseline, increasing from 65 percent to 100 percent of MGD schools. The percentage of canteen managers with enough food storage equipment grew modestly from 10 percent at baseline to 38 percent at midline. No canteen managers in Bounkani, Cavally, or Tchologo said that they had enough food storage equipment. (See Figure 65 and Figure 66.) These numbers could be impacted by WFP’s delivery of more than 1000 storage pallets to DCS, along with pallet jacks, to equip schools for the 2018-2019 school year. Figure 24: Sufficient Food Storage Equipment Storage Equipment Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total Midline 50% 63% 0% 0% 44% 53% 0% 38% Baseline 0% 13% 0% 0% 31% 5% 0% 10% Obs. 6 8 7 5 16 19 6* 67 Source: Canteen Survey. Only 5 schools were available in Tchologo at midline. 174. Across all regions, 46 percent of canteens had an improved stove with a chimney, instead of just a traditional stove or a furnace without a chimney. This includes 100 percent of canteens in Cavally and Tchologo where there is a stove. This is large increase from baseline, where only 7 percent of school canteens had an improved stove. (See Figure 69 and Figure 70.) 21 WFP reports that the Ministry of Health conducted a national deworming exercise in 2018, distributing over 1 million deworming tablets nation-wide, including to all MGD supported schools. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 31 | Page Figure 25: Stove Type in the Canteens Source: Canteen Survey. The sample size was 68 at baseline and midline. Figure 26: Stove Type in the Canteens, by Region Source: Canteen Survey. The sample size was 68 at baseline and midline. Key findings and conclusions – Impact • The impact analysis shows that the MGD intervention significantly improves the reading proficiency of students. Exposure to the MGD project increases the reading assessment literacy scores of students in treatment schools by approximately one level in reading compared to the non-MGD schools. Students who have been exposed to the project for two years have larger estimated impacts (1.7 levels). • The project effect is twice as great for male students, who show almost two levels of improvement in reading proficiency, compared to one level for girls. Girls and boys have similar levels of reading proficiency in CP1 and CP2, but starting in CE1, the gains for boys are much greater. • Enrolment increased at MGD schools for girls in CM grades where THR are offered. In the three regions with THR, the average enrolment increased by almost 10 girls in the upper grades • The percentage of school days during which food was provided was nearly 100 percent in each region. This was a large increase from baseline, where schools on average only provided meals on 57 percent of the days when school was in session. • Most MGD canteens served food containing the minimum dietary diversity on more than 80 percent of school days at midline, compared to 40 percent of days at baseline. • Among canteen managers who reported that there was a canteen management committee at their school, 33 percent indicated that they received nutrition training, compared to 41 percent at baseline. Midline None/Unknown Traditional stove Furnace without chimney Stove with chimney Baseline None/Unknown Traditional stove Furnace without chimney Stove with chimney 0% 20% 40% 60% 80% 100% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total None/Unknown Traditional stove Furnace without chimney Stove with chimney Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 32 | Page • About half of both MGD (53 percent) and non-MGD (50 percent) schools did not have any adequate access to water, similar to the baseline numbers. • Slightly more MGD schools (59 percent) than non-MGD schools (50 percent) have sanitary facilities on-site. Less than one-third (31 percent) of MGD schools and only one-quarter (25 percent) of non-MGD schools have gender-separated sanitary facilities. • The number of MGD schools receiving deworming treatments decreased from baseline to midline (from 72 to 64 percent), as did the number of schools receiving micronutrient supplements (53 to 26 percent). • The percentage of canteen managers who reported having sufficient food preparation equipment increased from baseline to midline (from 19 to 32 percent), as did sufficient food storage equipment (from 10 to 38 percent). • Across all regions, 46 percent of canteens had an improved stove with a chimney, instead of just a traditional stove or a furnace without a chimney. This is large increase from baseline, where only 7 percent of school canteens had an improved stove. 2.5. Evaluation Criteria 5 – Sustainability 175. In assessing sustainability, the evaluation team specifically focused on WPGs, as these were proposed as the foundation of the project’s handover strategy. The first section presents a case study of these groups, addressing the following evaluation question: • To what extent are WPGs contributing to school canteen supply, and is there evidence that their contributions will continue or scale up after the project ends? 176. The second section includes an overall assessment of the project’s progress towards sustainability considering the following questions: • To what extent are the results of the activities likely to be sustainable? • What are the key factors likely to affect sustainability of the project? 2.5.1 Women’s Production Groups Description 177. Twenty WPGs were supported by the project as of the data collection period for this midterm evaluation. The evaluation team’s case study involved in-depth discussions with members of all 20 WPGs, 10 of which were newly supported (since June 2018), and 10 of which were more established (supported by the project from 2017). These groups are mainly composed of women who account for 80 percent of the total number of members in the newer groups and over 90 percent of members in the established groups. The few men that are part of the group generally have a higher education level and more agricultural experience, and as such, they support women with administrative management and physically challenging work. For example, prior to the start of the agricultural season, men decide in relation with the technical support structures when to start land preparation and apply specific soil management techniques. 178. These groups have been being supported by the local ANADER staff prior to WFP support through the MGD project. This support stems from a government support that usually provides such groups with a minimum package of capacity building activities under the umbrella of the GoCI’s national program of agricultural development and other projects with technical and financial partners. The groups the evaluation team interviewed have been established for a minimum of three years, with some established over 10 years ago. The number of members ranged from 30 to around 100 or more, while most groups said their membership numbers fluctuated. The average age of the women for the Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 33 | Page majority of the groups was 35, although there was one group were the women were much older (the youngest member was 42 years old). A common characteristic of all the women is their illiteracy. 179. The main crops grown by these groups depends on their geographic location. While maize and rice are typically grown in the northern part of the country, cassava and yam are cultivated in the eastern part. In the ten established groups, women spoke positively about learning how to grow and use vegetables including eggplant, okra, peppers, tomatoes, onions, and greens. At the household level, they have increased the variety of foods they serve to their families by involving vegetables and leaves in their cooking. The schools these women support have also diversified their lunch offering, due to the nutritional training and the donation of vegetables by the WPGs. For the ten newer WPGs in the evaluation team’s sample, many have initiatives to grow vegetables, but this is still in an early stage of development. As reported by four newer WPGs, the main barrier to growing vegetables is the lack of water or irrigation systems. 180. Each group has enrolment and internal contribution fees, ranging from 2,000 FCFA to 10,000 FCFA. One group requires that members be married and follow specific rules pertaining to respect, punctuality, and participation. Some groups are required to give a portion of their production to the village chief as a reward for providing land to the group. 181. All groups started contributing to the canteens before the MGD project started their support. However, the importance placed on food donation and the production capacity varies from one group to the next. The ten established groups reported much higher production rates and reported being able to donate more food to the canteens thanks to the support received through the project. Early Indicators of Success 182. Even though it is too early to fully assess the impact of these groups, the members have indicated that their work has started to have an effect. For example, the indirect impact their support to the canteen is the increase in school attendance, attentiveness, and nutritional outcomes of school age children. Members also reported that it is likely that the project indirectly helps parents secure a balanced diet for their kids and reduces their food expenses. Depending on the leadership of the group, the activities went beyond providing food to the canteen. For example, in one group, thanks to the revenue yielded from their agricultural activities, they were able to buy hand washing devices for their school. 183. In the ten more established WPGs, the gains are more apparent. The members cited specific examples of how they benefited from materials received. For example, by having their own scales, women avoid being cheated as they know the true weight of their harvest. Women also mentioned that prior to being given watering cans, they used cups to water their crops. In addition to using the materials for their own purposes, these groups also rent out their equipment to generate extra income for their groups. Not only do they save time from using grain mills to grind maize (rather than using a mortar and pestle), they charge community members a daily fee (usually around 500 FCFA) to use the mill. 184. According to members in the ten established groups, there are community level effects greater than the growth of their individual WPGs. Although they charge community members to use the grain mills, women said that this ultimately saved money for community members, as they no longer have to travel long distances to process their Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 34 | Page crops. The mills also transform the grains into different types of eatable foods (attieke, couscous, etc.) for donation to the school canteens. 185. Women in the ten established groups also appreciated receiving marketing and grouped sales training. Prior to the MGD project, they used to sell their harvest individually. After their training, they are now grouping their sales and bargain collectively for a more favourable price at the market. One group said that they have been put in contact with the Ivorian board for the purchase of their agricultural products. Members in all groups appreciated the local support of ANADER (for the newer groups) and BFCD (for the established groups) on the ground. The continuous technical assistance provided is a strength and asset for the project to achieve sustainability. 186. Transparency is another key to the success of WPGs. As acknowledged by participants, when all members are given the groups revenue figures by the leadership, it encourages the members to continue supporting the group activities. Beyond transparency, a clear commitment of an engaged president is also key to the WPGs’ success. For the newer groups, community sensitization on the role of the members is necessary according to participants. As these WPGs are still in their infancy, there have not been substantial income distributions for their members. There are some members who complained about the lack of returns to participating in the group, so it is important for strong leaders and other group members to motivate all participants to avoid the collapse of their WPG. 187. Women mentioned that their participation in group activities in a primarily female environment strengthens the social capital among group members. Within these groups, social networks are created, and members rely on each other as a coping strategy mechanism in the case of adverse social shocks, such as an illness or death in the family. One group gave an example of paying for a life-saving surgery for a member of their WPG. A common refrain the evaluation team heard from the women was: “If someone is in trouble, the group helps her.” 188. In the more established groups, this social support goes even further, thanks to the money gathered through income generating activities and to the financial management training received as part of the project. These groups are able to give small loans to their members. In addition, most of these groups reported that their generated income was now sufficient to purchase their own seeds and fertilizer 189. The groups are also places where members learn soft skills that benefit them within their families and communities. As members of some groups stated, the older women participants provide social advice that members appreciate. Threats to Sustainability 190. The factors preventing groups’ effectiveness are the same as those that hinder sustainability. Lack of access to physical and natural capital are factors that could prevent groups from becoming stronger and autonomous. For example, there is an increasing scarcity of agricultural land in Côte d'Ivoire, and as groups grow, finding land for vegetable crop production might become challenging in a near future. While some groups have property rights or own the land, most of the groups interviewed were given access to land through a customary land rights whereby the village chief shares part of his relatives’ land to the group. Women members said that there is real risk that the owners will withdraw these lands to take advantage of a rise in the price of cash crops such as coffee and cocoa. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 35 | Page 191. Some of the newer WPGs supported by the MGD project since June 2018 raised concerns about being able to sell the produce they plan to grow, and requested assistance in bringing their produce to market. In response to these concerns, WFP plans to deliver this kind of technical assistance in the coming years as part of the package of services it works with ANADER to provide to the WPGs it supports. The ten established WPGs in the evaluation team’s sample appreciated this aspect of their training from BFCD, specifically mentioning the economic benefits of grouping their produce in the markets. 192. Factors related to climate variability also threaten sustainability. Rain variability from one year to another represents a serious threat to all groups’ agricultural activities. This variability leads several groups to state that despite the provision of agricultural inputs, insufficient or too much rain limits their production. Group members said this could be partially alleviated if they gained access to water retention and irrigated facility systems, saying lack of infrastructure prevents their growth. 2.5.2 Overall Assessment of Sustainability 193. In March 2018, with technical support from the WFP, the GoCI published the country’s first national school feeding policy, covering the period of 2018 to 2025. It sets an ambitious goal: “By 2025, all students in Côte d’Ivoire enrolled in the basic education structures take a warm and balanced meal every school day at the canteen.” It lists specific educational, health/nutritional, local production, and environmental objectives in line with UN Sustainable Development Goals (SDGs) 3, 4, 5, 8, and 15. Importantly, it emphasizes the multi-sector approach required to implement a national school feeding program, places specific responsibilities on the MENETFP and partnering government entities, and establishes the national structures for steering and coordinating the GoCI’s efforts towards reaching full school feeding coverage by 2025. This is a strong foundation for future technical assistance to support the GoCI in effectively taking over school feeding in all MGD and other WFP-supported school canteens, eventually ensuring the project’s national ownership and sustainability. 194. However, at the local level, turnover remains a threat to sustainability at all levels – cooks, COGES, school canteen management committees, WPGs, and teachers. For example, if a new cook is not trained to use an improved stove correctly, it will crack. New cooks might not know how to properly cook split peas and will therefore leave them in the warehouse instead of using them as part of the daily school meals. Turnover also affects the support groups affiliated with each school, specifically the school canteen management committees, the COGES, and the WPGs. The evaluation team’s qualitative research found that these groups do not receive regular training or continual support frequently enough to counter these groups’ turnover rates (for example, from the survey data, only 33 percent of canteen managers reported receiving training in nutrition) 195. Teacher turnover is also a threat to sustainability, as teachers in the MGD schools are taught pedagogical and literacy techniques but some are then assigned to a different school. Local stakeholders mentioned that they struggled to retain teachers in their community because they lack adequate teacher housing, sanitary installations, and water sources. In these situations, in addition to losing the benefits of the MGD teacher training, the school loses the services of a government-funded, certified teacher and may have to rely on untrained volunteer teachers. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 36 | Page 196. In addition to the WPGs, additional groups such as the COGES, who are well acquainted with and already assist their schools’ needs, are key to the community taking ownership of the project activities. Strengthening their capacities and implicating them as partners in the project will increase the likelihood of the project achieving sustainable, long term impact. 197. The primary sustainability plan of this project is to handover the schools to become fully part of the National School Canteens Program, where schools will benefit from both government and community support. To that extent, government stakeholders indicate that sustainability efforts should include additional training for government staff in monitoring and evaluation and commodity management and logistics. While WFP provides M&E and logistics training to DCS every year, several government stakeholders mentioned wanting to learn more about the monitoring and evaluation component, as this would not only increase their current technical capacity with the MGD project, but also empower them to improve other projects. Key findings and conclusions – Sustainability • In the ten established WPGs, the members have substantially increased their contributions from last year to this. Most of these groups reported that their generated income was now sufficient to purchase their own materials without needing external financial support from the project. These women reported the material and trainings received through the project is making their groups stronger and therefore more likely to sustain contributions after the project is over. • Within the WPGs, social networks are created, and members rely on each other as a coping strategy mechanism in the case of adverse social shocks, such as an illness or death in the family. • The factors preventing full effectiveness of the project’s WPGs in supporting their school canteen are the same as those that hinder sustainability. Lack of access to physical and natural capital prevent groups from becoming stronger and autonomous, along with factors related to climate variability. • Turnover is a threat to sustainability at all levels–at both the local level (cooks, COGES, school canteen management committees, WPGs, and teachers), and at the regional and national level (regional advisors, elected government officials, and WFP/partner staff). Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 37 | Page 3. Conclusions and Recommendations 198. Based on the findings presented in the previous section, an overall assessment that responds to the evaluation questions is provided below. This is followed by 8 recommendations of how WFP and its partners can take action to build on the lessons learned. 3.1. Overall Assessment/Conclusions 199. Committed to providing 24 600 MT of daily hot meals and THR to a total of 125,000 primary school students at 613 rural schools over five years, WFP is largely successful in reaching its operational objectives at midline while delivering significant, positive impact on beneficiaries in seven of the most vulnerable regions of Côte d’Ivoire. 200. For MGD strategic objective 1 (SO1), Improved Literacy of School Aged Children, students in the MGD schools have significantly increased their reading levels at midline. Relative to the comparison schools, student in schools with school feeding and trained teachers increased their reading levels by approximately one grade-level since the project started. The data show that the effect of being exposed to the project for two years is even stronger. However, while there are significant positive effects for both boys and girls, the effect was doubled for male students (almost 2 levels of improvement for boys compared to almost 1 level for girls). 201. In addition, the evaluation team also found a disproportional number of men in the membership of the COGES, and in fact, some groups were exclusively male. The leadership in the COGES also skewed toward male. This risks side-lining female voices in the management of their children’s school and perpetuating patriarchal gender roles. 202. At the beneficiary level, parents and members of the COGES agreed that the MGD project was highly relevant in addressing the critical challenges of attracting students to school and consistently serving meals at the school canteen. The impressive reading gains measured at every grade level in MGD-supported schools confirm the transformative impact the approach designed and implemented by WFP and AVSI had on its student beneficiaries. 3.2. Lessons Learned and Good Practices 203. Good Practice 1: Local Recipes and Cookbooks for Split Peas. Distributing an alternative source of protein to school feeding project beneficiaries may result in significant cost savings which allow implementers to reach more beneficiaries with nutritious meals. However, beneficiaries frequently disliked the split peas or found it difficult to prepare it for optimal taste and nutritional value. Multiple key informants from the Government and WFP said they faced that difficulty with one of the selected MGD commodities, split peas, in Côte d’Ivoire. However, DCS with WFP support were able to address it by developing locally-inspired and validated recipes incorporating the commodity into dishes commonly served in each region in Cote d’Ivoire. These recipes, presented in a large-print cookbook with pictures, reportedly addressed the problem of children rejecting split peas, canteen workers failing to properly prepare them, and split peas piling up unwanted in storehouses. This was confirmed in FGDs with CM student beneficiaries, where many enjoyed the split peas and few disliked them. This allowed for an efficient allocation of USDA project funding which can be replicated in similar contexts. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 38 | Page 204. Good Practice 2: Strong and Efficient Presence in Beneficiary Communities. The MGD project relies heavily on the GoCI’s own monitoring network of DPFC education advisors and DCS extra-curricular advisors in charge of school canteens. This allows for frequent data collection, reduces cost to the project, and facilitates the project’s future take-over by the GoCI. However, this layer of separation from end-beneficiaries may hamper communication between WFP and local beneficiaries on the latter’s needs. Supplementing the GoCI’s efforts, AVSI efficiently maintained a local presence by employing community members as focal points for supporting project field logistics and monitoring efforts. With this network of focal points, AVSI visited each MGD school at least three times per year to collect information on student and teacher progress, as well as monitor the use of project-provided teaching and learning equipment. While there are opportunities to consolidate and improve MGD monitoring efforts, the model being implemented in Cote d’Ivoire is a good example of both government integration and efficient field monitoring. 205. Good Practice 3: Operational Integration with Government. The MGD project is greatly integrated into the national school feeding structures. WFP’s alignment with Government strategies fostered strong partnership at the national and regional level which allowed the project to be implemented through a train-the-trainers model with the Government’s existing network of regional advisors and technicians. This helps practically build the capacity of DCS and DPFC civil servants to administer activities both independently and effectively at the end of the project. 206. Good Practice 4: National School Feeding Policy. In March 2018, with technical support from the WFP, the GoCI published the country’s first national school feeding policy, covering the period of 2018 to 2025. It lists specific educational, health/nutritional, local production, and environmental objectives in line with UN Sustainable Development Goals. Importantly, it emphasizes the multi-sector approach required to implement a national school feeding program, places specific responsibilities on the MENETFP and partnering government entities, and establishes the national structures for steering and coordinating the GoCI’s efforts towards reaching full school feeding coverage by 2025. 3.3. Recommendations 207. Based on the findings and conclusions of this evaluation, the recommendations of the evaluation team are outlined below. The target group for each recommendation is clearly identified. The opinions expressed in these recommendations are those of the authors, and do not necessarily reflect those of WFP or USDA. 208. Recommendation 1: Strategically Reduce the Proportion of Canteen Days Covered in MGD Schools. Currently, WFP provides meals throughout the entire 120- day school term. As heard from government stakeholders and local beneficiaries, this discourages parents and WPGs from contributing money and/or food to the canteens. At the same time, few WPGs reported being close to being able to adequately support a school canteen without the MGD project’s support and it is unclear when they will be able to.22 Considering the critical importance of WPGs in the project’s sustainability plan, WFP should immediately perform an assessment, informed by other projects supporting WPGs in similar contexts, to estimate the time needed for WFP-supported 22 Through WFP technical assistance, WPGs are expected to incrementally increase their production while continuing to contribute 1/3 to their associated school canteen, with the end goal of fully supporting the canteen in conjunction with the Government’s national school canteens program. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 39 | Page WPGs to fully support a school canteen and determine when WPGs could realistically be expected to graduate from the project and under what conditions. After evaluating what local communities can support, WFP should integrate these findings into the transition plan it is developing with the GoCI and indicate how, when, and in which schools and regions school feeding will be transitioned to the GoCI’s national program. WFP should start reducing the dependence of canteens on MGD commodities by the start of the 2019-2020 school year, replacing these commodities with GoCI and WPG contributions, and find other beneficial uses for the surplus commodities, such as increasing the number of girls receiving THRs by expanding this offer to all 7 project regions. 209. Recommendation 2: Reinforce DCS and DPFC M&E Capacity at the National and Local Level. The MGD project’s monitoring network relies heavily on the GoCI’s own network of local and regional government school feeding and teaching advisors to collect indicator data. However, the data collected through this system does not include key MGD indicators, particularly on student attendance and their absences due to sickness, teacher absences, and food diversity and security, which cannot be effectively calculated through evaluation. In addition, while the MGD project delivers annual M&E training to DCS and DPFC, several national-level DCS personnel requested that WFP provide them with additional M&E training and capacity building to help them better support the MGD project and other school feeding related projects. By the end of the 2018-2019 school year, WFP personnel should host a workshop with M&E personnel from DPFC and DCS to jointly reassess their training needs, develop a training plan that will respond to them, and consider strategies to improve MGD monitoring efficiency and effectiveness. Through this workshop, the MGD implementing partners should address the issue of accurately monitoring teacher and student attendance data, which remains a challenge, and the possibility of consolidating government monitoring activities where relevant. By the start of the 2019-2020 school year, WFP should support DCS and DPFC in modifying existing data collection tools and methods for indicators relevant to project implementation and reporting, while also consolidating existing school-level data collection visits across the school feeding and literacy components to more efficiently and frequently visit schools. 210. Recommendation 3: Improve Field Presence and Beneficiary Interaction. There are currently only eight WFP field monitors who are responsible for over 600 schools. They are working with government school feeding advisors who, as best they can, mobilize communities, supervise day-to-day canteen management, and collect indicator data, but have limited capacity in actuality. As WFP is trying to build both local capacity and long-term sustainability, they should consider working with the appropriate government stakeholders and local partners to increase the capacity of the local and regional school feeding advisors, as described in Recommendation 2. However, this does not replace regular, direct interaction between WFP and its project beneficiaries, which is essential for responding to local needs in a timely manner, more accurately capturing M&E data, and effectively disseminating innovations in the field. To efficiently improve its field presence and increase its touch-points with beneficiaries, the MGD project should consider using the data collection networks, agents, and activities already being used for the literacy component, such as community focal points and school monitoring visits, to more frequently visit schools and communities for other components of the project. 211. Recommendation 4: Increase Coordination, Communication, and Planning with the DCS. While the MGD project is deeply integrated with the GoCI, WFP could Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 40 | Page improve its advance planning of activities involving the DCS, with a clear and transparent view of the available funds and the project timeline. While DCS representatives participated in meetings to develop WFP’s proposal for the MGD project, multiple DCS key informants indicated that they were not informed of some late stage meetings, where they could have shared threats to some of the proposal’s assumptions. For example, these informants thought that the WPGs, a key component of this project, are not yet supported at sufficient number to provide project sustainability partly due to insufficient funds stemming from underestimating the cost of supporting WPGs. Having supported WPGs for nearly two decades, the DCS and ANADER has an intimate knowledge of these groups. The DCS informants thought their feasibility, given the resources and timeline, could have been more accurately described if they were fully included in discussions during the entire planning phase. Several DCS representatives also said that they were forced to start activities later than anticipated due to delays or uncertainty in receiving funds and were sometimes unaware of what MGD funding is allotted for their work. WFP should more frequently include DCS, DPFC, and other GoCI partners in conversations on the design, budgeting, and implementation of MGD project activities. In turn, the GoCI should ensure representatives from all relevant departments within it participate in the MGD project meetings organized by WFP to review and approve the tasks and budgets pertaining to their work. 212. Recommendation 5: Reinforce Girls’ Education. The results of the reading assessment showed that male student improved twice as much as female students. AVSI should work with DPFC and its field staff to adjust its programming to ensure girls benefit from the project as much as boys. While no beneficiaries in the FGDs said that girls were discriminated against at school, the evaluation team recommends that, by the start of the 2019-2020 school year, AVSI should conduct a study to explain these results and subsequently adapt its approach to ensure girls participate, read, and learn at the same rate as boys. Considering that teachers reported that girls in their class were roughly as attentive as boys, the evaluation team suggests that AVSI and DPFC consider integrating or reinforcing a gender focus in their teacher training curriculum. In addition, considering that THRs have shown to be effective in the three regions where it was offered in increasing girls’ attendance and keeping them in school, WFP should consider using a surplus MGD commodities to expand the THR offering to all seven regions by the start of the 2019-2020 school year. 213. Recommendation 6: Increase Female Representation in COGES. WFP targeted COGES to include 50 percent women as members, however, most in the evaluation team’s sample did not include this many women, and in fact, some COGES were exclusively male. By the start of the 2019-2020 school year, WFP should encourage women to participate in its anticipated basic literacy training for WPGs and canteen management committees to also join their school’s COGES, as well as make additional efforts to sensitize COGES of the need to integrate women in their leadership. 214. Recommendation 7: Strengthen COGES Capacity to Support MGD Activities. The COGES help their school and its canteen in many ways which could be enhanced by expanded MGD training and support. All COGES agreed they could use additional training in managing COGES activities, including accounting, commercial farming, and fundraising. While the current MGD project is designed to offer only limited support to COGES, by the end of the project, to the extent feasible WFP should explore offering training and financial support to COGES in developing commercial activities or building Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 41 | Page infrastructure for the school, either directly or through partnerships with relevant Government structures or UN agencies and other organizations. 215. In addition to the above recommendations formed from the evaluation findings, below the evaluation team present suggestions provided directly from beneficiaries in the field. They represent the unprompted suggestions of project beneficiaries and should be addressed as and where possible. Nonetheless, these suggestions may not be feasible for WFP or its partners to implement considering the scope of the current MGD project and the resources available, but should be considered for future projects. 216. Suggestion 1: Fund and train COGES to be able to implement school infrastructure improvement and financing activities. Multiple COGES requested funding to help them manage schools. Many COGES members said that subsidies they receive from the government or funds they collect from parents are insufficient, preventing them from purchasing teaching materials and textbooks or making repairs and improvements to school latrines, benches, classrooms, water pumps, and teacher housing. Some also requested materials and training to be able to launch their own agricultural activities and sell their crops to fund the school. Others requested electrification for school buildings so that children can study at night, with the help of the study groups the COGES or parents organize after school. Finally, COGES members suggested that activity and record keeping training multiple times a year would be helpful, especially considering that many reported not having received any training. 217. Suggestion 2: Offer incentives for teachers to stay in the community. Several COGES addressed the issue of teacher turnover as the main barrier for the benefits of pedagogical training staying in the targeted community. Some mentioned plans to build better housing for teachers to encourage them to stay in the communities. Some parents suggested that the MGD project help build lodging rooms for teachers to encourage them to stay. 218. Suggestion 3: Provide additional materials. Parent groups mentioned that they need additional materials and tools to be able to adequately produce food for the school canteen. One parent group mentioned that their WPG received only 30 work boots for 135 members. Another parent group mentioned that insufficient amounts of plates made it impossible to serve all students at once and so the canteen served lunch in waves. Finally, both COGES and parents would like the MGD project to provide more teaching and learning materials for the schools. 219. Suggestion 4: Provide additional capacity building for WPGs. All WPGs, both established and newer, requested literacy training, indicating that it would help them with business interactions and reduce their reliance on non-members. Some of the newer groups mentioned wanting training on animal husbandry, so they could provide meat to their canteens and sell the surplus for additional income. 220. Suggestion 5: Support community study groups. Many parents and COGES in FGDs brought up the after-school study groups formed and run by community volunteers to help their children memorize the lessons of the day. In some villages, everyone participated in these study groups, but in others not all parents made their children join them (although the COGES were trying to sensitize them). In other communities, families sought out tutors to make their children repeat their lessons. These groups and tutors are important since many parents are illiterate and can’t supervise or check their child’s work. Some parents and COGES suggested that Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 42 | Page electrification of their schools would help children in their community participate in these after-school study groups. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 43 | Page Annex 1: Evaluation Matrix Evaluation Questions Illustrative Indicators or Assessment Criteria Data Source/ Collection Methods Data Analysis Method I. Relevance  To what extent was the design of the interventions in line with the needs of the target population—women, girls, boys, and men?  To what extent is the design of the intervention aligned with national policies, strategies, and programs?  Do the project design and implementation arrangements complement other donor-funded and government initiatives?  Is the project designed to reach the right people with the right type of assistance?  Beneficiaries’ perception of usefulness (by gender)  Perception of the project activities in alignment with other national activities (e.g., ministry policies, other WFP initiatives)  Beneficiaries’ perception in applicability (by gender)  FGD (Parents, SMC members, women’s agricultural groups)  KII (USDA staff, government officials, project staff, partners)  Qualitative Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 44 | Page Evaluation Questions Illustrative Indicators or Assessment Criteria Data Source/ Collection Methods Data Analysis Method II. Effectiveness  What is the progress of project implementation—is the project on track to carry out all activities as planned?  To what extent have the interventions so far responded to the needs of the beneficiaries—women, girls, boys, and men?  What are the main factors influencing the achievement, whether or not the results/ objectives, of the intervention?  Are any changes required to increase the project effectiveness?  Number of textbooks and other teaching and learning materials provided with the help of USDA  Number of target schools where students have additional reading material with the help of USDA  Proportion of students in targeted schools who regularly consumed a meal before or during the school day (by gender)  Number of girls receiving take-home rations as a result of USDA assistance (by new/continuing)  Number of teachers / educators / teaching assistants trained or certified as a result of using the USDA assistance (by type, by sex)  Number of targeted schools with access to better food preparation and storage equipment.  Beneficiaries’ perception on effectiveness of the program  Teacher survey  School survey  KII (project staff, partners)  FGD (Parents, SMC members, women’s agricultural groups)  Qualitative  Quantitative  Number of textbooks and other teaching and learning materials provided with the help of USDA  Number of students enrolled in schools receiving assistance from the USDA (by gender)  Gender ratio, primary  Number of people who received health education and child nutrition as a result of using the USDA (by gender)  Number of students who received deworming (by gender)  Number of school-age children receiving school meals daily (lunch) as a result of assistance from the USDA (by gender; new and continuing)  Number of daily school meals (lunch) provided to school children as a result of assistance from USDA  Number of rations provided following USDA assistance  Number of daily school meals (lunch) offered to school children as a result of using the USDA  WFP and AVSI monitoring records Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 45 | Page Evaluation Questions Illustrative Indicators or Assessment Criteria Data Source/ Collection Methods Data Analysis Method III. Efficiency  How efficient is the targeting?  Does the assistance reach the right beneficiaries (girls, boys, men, and women) in the right quantity and quality and at the right time?  Is the project efficient in terms of costs and costs/beneficiary?  What are the external and internal factors influencing efficiency of the project?  Stakeholders’ perceptions of cost-effectiveness  Percentage of stakeholders reporting that beneficiary targets are met (by gender)  Type of other beneficiaries that can be reached; type of assistance that can be provided to them (be gender)  List of factors influencing efficiency of the project  FGD (parents, SMC members, women’s agricultural groups)  KII (government officials, project staff, partners)  Qualitative IV. Impact  What are the midterm effects of the operation on the beneficiaries in terms of: (a) improvement of school indicators; (b) improvement of pupil reading skills; (c) capacity building of groups?  What are the reasons for the observed effects?  Are there unintended effects on the beneficiaries? What have been the gender￾specific impacts, particularly with regards to girls’ schooling?  Proportion of students at the end of two years of elementary school who demonstrate that they can read and understand the meaning of grade level text (by gender)  Number of teachers in the target schools who demonstrate the use of new techniques and educational tools for quality after using the USDA  Proportion of teachers in the target schools and teach attending school regularly (at least 90% of school days) per school year  Number of teachers/educators/teaching assistants in targeted schools who demonstrate the use of new and good technical and pedagogical tools (by type, by sex)  Proportion of pupils identified as attentive in classrooms by their teachers (by gender, class)  Proportion of households with acceptable food consumption (by gender of household head)  Student survey and ASER  Teacher survey  School survey  Household survey  School management committee survey  Women’s agricultural group survey  Quantitative Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 46 | Page Evaluation Questions Illustrative Indicators or Assessment Criteria Data Source/ Collection Methods Data Analysis Method  To what degree have the project outcomes made progress toward positive long-term effects on targeted beneficiaries (girls, boys, men, and women), households, communities, and institutions?  Index of survival strategies (average) (by gender of household head)  Dietary Diversity Score (by gender of household head)  Proportion of pupils regularly (80%) frequent in classes/schools supported by USDA (by gender)  Proportion of students who miss more than 10 school days a year due to illness (by gender)  Number of members of the management committees and members of women’s production groups who are aware of the importance of education  Proportion of school-age children receiving a minimum acceptable diet (by gender)  Proportion of members of the school management committee and canteen management staff who can identify at least three health and hygiene practices (by gender)  Proportion of members of the school management committee and the canteen management staff who can identify at least three practices for safe preparation and storage of food  Proportion of schools using an improved water source  Proportion of schools with improved sanitation facilities  Number of people benefiting directly from interventions financed by USDA (by gender and date)  Number of people indirectly benefiting from interventions financed by USDA  Number of schools targeted leaders that demonstrate new and good technical and pedagogical tools (by type, by sex)  Number of school officials trained or certified as a result of using the USDA (by sex)  Number of beneficiaries of social safety nets who participate in productive safety nets following assistance from the USDA (by gender; new and continuing)  WFP and AVSI monitoring records V. Sustainability Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 47 | Page Evaluation Questions Illustrative Indicators or Assessment Criteria Data Source/ Collection Methods Data Analysis Method  To what extent are the results of the activities likely to be sustainable?  What are the key factors likely to affect sustainability of the program?  To what extent are women’s agricultural groups contributing to school canteen supply, and is there evidence that their contributions will continue or scale up after the project ends?  Value of public and private investment leveraged as a result of using the USDA (host government)  Annual growth rate of the budget allocated by the government to the National Directorate of School Canteens  Number of policies in the areas of health and nutrition of children, regulations and administrative procedures, according to their level of development, due to the help of the USDA (for stage)  Number of policies in the education sector, regulations, and administrative procedures, according to their level of development, due to the help of the USDA (for stage)  Number of parent–teacher associations or similar governance structures “school” sustained as a result of using the USDA  Number of public–private partnerships set up as a result of using the USDA (nutrition, education, health, multi-sectoral, and other)  Number of government staff who are trained in management of food and monitoring and evaluation (by gender)  Number of policies in the areas of health and nutrition of children, regulations, and administrative procedures, according to their level of development, due to the help of the USDA (from steps 1, 2, and 5)  Number of public–private partnerships set up as a result of using the USDA (female producing groupings)  WFP monitoring data  KII (USDA staff, government officials, project staff, partners)  FGD (parents, SMC members, women’s agricultural groups)  Qualitative Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 48 | Page Annex 2: Evaluation Methodology Evaluation Design 221. To address the research questions at midline, the evaluation team used a mixed￾methods evaluation design for collecting, analysing, and reporting both quantitative and qualitative data integrated into a coherent midline evaluation of the MGD project. The evaluation team built on the quantitative methodology used at baseline to ensure consistency in the approach and to minimize the introduction of new biases. In addition, the evaluation team added a qualitative component to complement the quantitative analysis and shed light into the processes and mechanisms of change and the factors that may have affected implementation. 222. To assess the causal effect of the MGD teacher training project after two years of implementation on student’s reading proficiency23 that is the confirmatory outcome (i.e., primary outcome of interest) for the impact evaluation, the evaluation team designed a difference-in-differences (DID) method at baseline. In this method, the evaluation team intended to compare the changes in outcomes between the population of beneficiaries (the treatment group) and the population that is not benefiting from the project (the comparison group) following the same students over time. This method enabled the evaluation team to take into account any differences between the treatment and comparison groups that are constant over time. The evaluation team designed the DID method to collect the same outcomes from both the treatment and comparison groups at midline in a similar fashion as was done at baseline. Evaluation Questions 223. The evaluation team organized the evaluation questions under the following criteria: relevance, effectiveness, efficiency, impact and sustainability. Given the breadth of the questions for this evaluation, the evaluation team has compiled them into a comprehensive conceptual framework, in Annex 1. For each evaluation question, the evaluation team lists data collection strategy, data analysis method, and the responsible party for collecting and measuring them. In Annex 1, the evaluation team also maps each evaluation question to their corresponding indicators or assessment criteria, required by USDA, based on the MGD results framework. For example, the evaluation team listed the output indicators under efficacy (e.g., number of girls receiving take￾home rations as a result of USDA), and outcomes indicators under impact (e.g., proportion of students at the end of grade 2 who demonstrate reading proficiency at their grade level). Sampling Frame 224. At baseline for the overall project evaluation, the National Statistics Institute (INS) applied a proportional sampling to select 169 schools out of the 613 beneficiary schools. From a pre-established group of 200 comparison primary schools with similar socio￾economic and geographic characteristics, INS drew a sample of 56 comparison schools to be included in the evaluation. For the impact evaluation of reading outcomes at baseline, after adjustments to the selection of sites to accommodate for the field work, the final sample consisted of 94 primary schools out 225 (58 treatment and 36 23 Reading proficiency is defined as ability to achieve the grade-appropriate reading level by USDA. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 49 | Page comparison schools) across the 7 regions: Bafing, Bagoué, Boukani, Cavally, Gontougo, Poro, and Tchologo. 225. As requested in the revised terms of reference (TOR) and in compliance with the requirement for consistent DID methodology, at midline, the evaluation team followed the same 99 impact evaluation sample in baseline and added 9 extra treatment schools which it randomly selected them, maintaining the regional distribution, from 169 visited MGD schools at baseline. For the quantitative component, the sample size of students was determined using the following formula 𝑛𝑛 = 𝑍𝑍2 𝑝𝑝(1 − 𝑝𝑝) 𝑑𝑑2 𝑘𝑘 Where: • 𝑛𝑛 is the sample size • 𝑍𝑍 = 1.96 is the probability statistic associated with a confidence level of 95 percent • 𝑝𝑝 = 0.13 is the baseline estimator of prevalence of food security provided in the TOR • 𝑘𝑘 = 1.5 is the cluster parameter allowing adjustment based on the fact that households of students from the same schools share similar characteristics • 𝑑𝑑 = 0.05 is the tolerated margin of error. 226. The evaluation team multiplied 𝑛𝑛, the sample size obtained by the above formula, by 6 to ensure sufficient precision by gender (boys and girls) and school grade (CP, CE, and CM). Additionally, the evaluation team adjusted the sample size to allow for 95 percent response rate. The evaluation team further increased the final sample size to 1,753 students and their corresponding households to ensure proportional representation of respondents within 104 schools24 across the 7 regions, including 68 treatment (i.e. MGD) schools and 36 control (i.e. non-MGD) schools. 227. AVSI provided the evaluation team with the list of the enrolled students for the academic year of 2018-2019. In each primary school, the team surveyed 12 students (2 from each grade CP2, CE1, CE2, and CM1 and 4 from CM2)25 and their corresponding households from each school, and an additional 12 students from CM2 from the selected schools in Poro, Bagoué, and Tchologo regions26. The evaluation team randomly selected new students in Grades CP2 and CE1 and, and to the extent possible, tracked students in Grades CE2, CM1, and CM2 from baseline to survey. For any students in Grades CE2, CM1, and CM2 who could not be found at midline, the evaluation team mitigated this attrition by randomly replacing them with a new sample of students to meet the required sample size from the same grade/school (i.e., referred to throughout as replacements). In the remaining 9 schools, where reading assessment was not administered at baseline, the evaluation team also randomly sampled new students—1 boy and 1 girl from CP2, CE1, CE2, CM1, and 2 boys and 2 girls from CM2. In each school, the evaluation team also surveyed teachers in Grades CP1-CM2, the principal, and the canteen manager. Exhibit A2.1 shows the distribution of sample respondents by region. 24Among the 108 selected schools, the evaluation team learned that 4 control schools got closed since baseline which the evaluation team excluded them from the evaluation. 25 The evaluation team oversampled CM2 students from 2018-2019 school year because the students who were CM2 in the previous 2017-2018 school year are no longer in school and cannot be surveyed. 26 The additional 12 students in CM2 were all female and were added to account for the take-home ration indicator. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 50 | Page Exhibit A2.1: Student Sample Distribution by Region and Type of Respondent* MGD Region CP1 CP2 CE1 CE2 CM1 Total Bafing 12 11 13 12 24 72 Bagoué 16 17 18 18 127 196 Bounkani 14 14 16 16 26 86 Cavally 12 14 16 8 10 60 Gontougo 26 51 39 41 55 212 Poro 33 40 43 58 186 360 Tchologo 13 12 24 25 89 163 Total 126 159 169 178 517 1,149 Non-MGD Region CP1 CP2 CE1 CE2 CM1 Total Bafing 13 12 14 10 16 65 Bagoué 10 5 5 8 18 46 Bounkani 14 4 2 4 24 Cavally 14 17 10 12 19 72 Gontougo 12 12 25 8 20 77 Poro 25 18 21 31 123 218 Tchologo 26 45 8 12 11 102 Total 114 113 83 83 211 604 Source: Student Survey. The sample size was 1753. *Gender disaggregated frequencies are included in Figure 27. 228. In collaboration with WFP, the evaluation team selected 20 project sites to visit. At each site selected, it conducted FGDs with CM students, parents, women’s production group (WPG) members, and school management committee (COGES) members. Where possible, the evaluation team separated the parent focus groups by gender to allow respondents to speak more freely and to compare mothers’ and fathers’ perceptions. Finally, the evaluation team conducted interviews with internal and external stakeholders, as described in Annex 4. In all mixed-gender FGDs and KIIs, the evaluation team actively encouraged women’s participation to ensure its evaluation incorporated their experiences and recommendations. Data Collection Methods 229. Surveys. To answer evaluation questions, the evaluation team reviewed and closely examined instruments that were used during baseline and revised the tools to more accurately capture the MGD indicators and fill in the gaps from baseline (e.g., add or revise questions related to student attendance, teacher attendance, minimum acceptable diet, etc., particularly to distinguish between girls and boys). The evaluation team built on and improved baseline tools, including school, household, student, and school canteen management surveys. The evaluation team also developed a teacher survey and calibrated it into the Ivorian context to capture their training needs and their use of the new gained techniques. In addition, the evaluation team administered KIIs and FGDs protocols to interpret the quantitative findings by contextualizing and filling in gaps from the baseline, as several of the MGD indicators are not directly comparable. 230. Reading Assessment. Importantly, to measure the impact on reading skills of school children and analyse the change in skills over time, the evaluation team administered the same reading assessment tool, ASER. Because of the possibility that either students have access to the test from their older cohorts or teachers have become aware of the assessment and started preparing students for the test, the evaluation team revised the version of the test. To avoid any possible bias in reading outcomes, the evaluation Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 51 | Page team updated the test content of the existing versions. The evaluation team ensured that the updated test has the same level of complexity as the one used at baseline to be able to compare students’ reading skills between baseline and midline. Before midline data collection, the evaluation team conducted an adaptation workshop with a group of local reading, curriculum, and assessment experts from MENETFP with support from AVSI, as the evaluation team successfully did at baseline, to ensure that the updated ASER test is still culturally appropriate and consistent with Côte D’Ivoire’s learning standards for Grades 1–6. The final version of the test included 11 levels (A – K), which roughly correspond to the reading standards for each grade level (see Annex 5 for the ASER tool). Exhibit A2.2 presents the structure of the ASER reading test, including the test’s levels and corresponding grades and reading skills. Exhibit A2.2. ASER-Reading Test Structure Level Corresponding Grade Reading Skills Level 0 None None Level A Grade 1 (CP1) – Lower level Identify letters Level B Grade 1 (CP1) – Upper level Read simple sounds Level C Grade 2 (CP2) – Lower level Read complex sounds Level D Grade 2 (CP2) – Upper level Decode simple words (1-2 syllables) Level E Grade 3 (CE1) – Lower level Decode complex words (2-3 syllables) Level F Grade 3 (CE1) – Upper level Read simple sentences Level G Grade 4 (CE2) – Lower level Read complex sentences Level H Grade 4 (CE2) – Upper level Read simple stories Level I Grade 5 (CM1) – Lower level Answer reading comprehension questions on simple stories Level J Grade 5 (CM1) – Upper level Read complex stories Level K Grade 6 (CM2) Answer reading comprehension questions on complex stories Source: IMPAQ. 231. Qualitative Tools. In addition, the evaluation team administered KIIs and FGDs protocols to interpret the quantitative findings by contextualizing and filling in gaps from the baseline, as several of the MGD indicators are not directly comparable. Evaluation Analysis 232. The evaluation team compiled the survey responses into a master file for the performance-evaluation analysis. Where the accuracy/completeness of the baseline quantitative data allow, the evaluation team conducted a cohort comparison to measure the project’s progress in reaching its target goals and provided descriptive analysis (percentages and averages) where the comparison was not feasible in order to establish base values for comparison later at endline. When possible, the evaluation team also conducted subgroup analyses by grade, student gender, highlighting emerging patterns. 233. For the impact evaluation, the evaluation team compared the changes in reading outcomes between students who attended MGD schools (treatment group) with students who were enrolled in non-MGD project (comparison group) to estimate the two-year project’s effect on literacy growth. The evaluation team surveyed students Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 52 | Page from the same schools in baseline and midline so that it could construct the change in reading outcomes for each cohort of student. Since the evaluation team collected the midline data at the beginning of the 2018-2019 school year, to be able assess their reading outcomes at the end of their grade, it had to measure their reading proficiency retrospectively which excluded CM2 cohort from the analysis. Validity and Reliability 234. To measure project impact between baseline and midline, the evaluation team surveyed the same sample selected by the baseline evaluator (INS). The evaluation team and WFP were unable to meet with INS to assess the validity of the baseline sampling and data collection. However, having designed and validated the reading assessment for the baseline evaluation, the evaluation team is confident in the validity of the related indicators at baseline and its ability to infer causal relationships between the project and progress in literacy. 235. In addition to validating the reading assessment tool at baseline and midline, the evaluation team validated its other survey instruments at midline in a validation workshop with DCS and WFP, adjusting its instruments as necessary prior to data collection. The evaluation team also shared its inception report, including its surveys and qualitative protocols, with WFP, AVSI, and Government stakeholders to address their comments prior to field work. 236. Quantitative and qualitative data were compared to cross-validate findings across data collection methods and verify the reliability of beneficiary responses. To ensure inter￾rater reliability, an IMPAQ research analyst performed daily quality checks on key variables, including reading assessment and food consumption scores, and followed-up quickly as necessary to address any significant differences in survey responses by enumerator. In addition, qualitative data was independently collected by IMPAQ’s two fieldwork managers, with findings and transcripts reviewed by IMPAQ home office staff. Modifications and Limitations 237. There are some limitations in the MGD midline evaluation. The evaluation team discussed some in the inception report, but it learned about the others during the fieldwork. Exhibit A2.3 describes the limitation of the evaluation team’s midline evaluation, the mitigation strategies it implemented, and their possible impact on the findings. Exhibit A2.3: Challenges and Solutions Challenges Solutions Possible Impact on the Findings Inaccurate or missing data from baseline Revised the quantitative tools developed by INS to more accurately capture the required MGD indicators The changes helped fix the caveats of the baseline surveys, but at the same time made the baseline and midline outcomes incomparable for some of the indicators. However, the evaluation team does not consider this as an issue as this improved the tools to capture the indicators more rigorously at endline with establishing base values at midline. Lack of teacher survey at baseline Developed a teacher survey and collected data from teachers in all the treatment schools instead of asking the school principal to improve the key performance indicator measurements. Collecting data directly from teachers was a different methodology that was not taken at baseline. This change improved the precision of the measurement, but made the key indicators related to teachers incomparable to baseline. The evaluation team reported findings descriptively at midline without comparing changes over time and will re-administer survey at endline to see differences from midline. Delay in data collection until the beginning of the school year 2017-2018 The evaluation team used the reading skill information in the For the impact evaluation, the evaluation team intended to follow the same students in Grades CP1, CP2, CE1 and CE2 at baseline and survey them again at midline when Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 53 | Page Challenges Solutions Possible Impact on the Findings beginning of the school year as a proxy for what the reading skills were during the end of the previous school year. The evaluation team also oversampled CM2 students at midline to meet the target. they were at CE1, CE2, CM1 and CM2, respectively, excluding CM1 and CM2 student cohorts at baseline who were gone from the school at midline. The delay in midline data collection left the evaluation team with one less cohort as CE2 students at baseline graduated in 2018-2019 from school. However, oversampling CM2 students at midline helped the evaluation team reach its target without losing any power even with one less cohort. Change in the implementation plan for training teachers in all grade Measured the project effect on literacy outcomes for at least one year, instead of two, at midline to maintain the power. Based on the original implementation plan at baseline, only teachers in CP1 and CP2 were supposed to receive the training in the project at baseline. In the 2016-2017 school year, teachers in CP1 (a new cohort) and CP2 received the training. However, a year after that, teachers in all Grades (CP1-CM2) received the MGD training. Therefore, across different grades at midline, students have not been exposed to the teacher training consistently. Therefore, the evaluation team was not able to show the impact results exclusively for the two￾year project effects (i.e., the effect of canteen and literacy activities both together for two years) given their small sample. The evaluation team assessed the effect of the project on students’ reading scores with at least one year of exposure to the project to maintain the power. However, the evaluation team estimated its regression model also with restricting the sample to include only CE2 students at midline (who were in CP1 at baseline) and the findings stayed statistically significant, regardless. Follow the same subsample over time and include additional schools for the reading assessment The evaluation team used a DID and followed the same cohort of students rather same individuals in the same schools. At midline, the evaluation team was able to track 244 out of the 1106 students from the baseline sample, i.e., 22 percent. Therefore, the results will not be coming from the same students over time but same cohorts. The evaluation team estimated its model by excluding students who joined the school in academic year of 2018-2019 and have not been exposed to the project. The regression robustness checks show no difference in the project effects. Ethical Considerations 238. WFP's decentralised evaluations must conform to WFP and UNEG ethical standards and norms. The contractors undertaking the evaluations are responsible for safeguarding and ensuring ethics at all stages of the evaluation cycle. This includes, but is not limited to, ensuring informed consent, protecting privacy, confidentiality and anonymity of participants, ensuring cultural sensitivity, respecting the autonomy of participants, ensuring fair recruitment of participants (including women and socially excluded groups) and ensuring that the evaluation results in no harm to participants or their communities. 239. During the evaluation the following ethical issues were considered for the design, data collection, data analysis, reporting and dissemination: providing sufficient information about the evaluation to participants, obtaining informed consent from participants, collecting confidential data from participants, assuring them that their identity will not be revealed, analysing and presenting de-identified and aggregate findings. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 54 | Page 240. The following safeguards and measures to manage these issues were in place: submitting the research protocol and data collection instruments to Advarra Institutional Review Board (IRB); training enumerators on respondents safely and confidentiality, with particular consideration given to safeguarding child respondents; obtaining teachers’ and principals’ consent to survey their students along with each child’s individual assent; keeping the data in a secured place to protect respondents’ personal information; and active monitoring of data collection for compliance with IMPAQ quality and ethical guidelines. 241. Prior to collecting data, the evaluation team submitted protocol documents to Advarra IRB on September 11, 2018 (Pro00029763). IRB approval is necessary to ensure that proposed evaluations comply with local and international rules and procedures. The evaluation team received approval from Advarra IRB on September 18, 2018. The approved documentation for this evaluation includes: • School, canteen manager, teacher, household, and student, including ASER, surveys • Research protocol • Informed consent and assent forms • Qualitative protocols for key informant interviews and focus group discussions 242. The evaluation team used these IRB-approved instruments to collect data. Prior to administering the survey, enumerators were trained on procedures for interviewing respondents, protecting respondent privacy and confidentiality, and securing the data, with a particular focus on safeguarding children at school. During the midline data collection, the survey team first obtained teachers’ and principals’ consent to survey their students. Then they asked for students’ assent, assuring children that their participation was voluntary and that they could terminate the survey at any point. After data collection, the evaluation team protected the privacy and confidentiality of respondents by storing the data on secure servers and separating personally identifiable information from the survey data. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 55 | Page Annex 3: Documents Reviewed / Bibliography Document Type Titles of documents received Project related documents Appraisal mission report SNAS 2018 – 2022 Project document (including Logical Framework in Annex) TOR Project proposal FY2015 McGovern-Dole Proposal Monitoring & Reporting M&E Plan A3. Evaluation Plan Food Distribution and Post-distribution Monitoring Reports Monthly hot meals reports; Monthly THR report Donor specific reports Performance Monitoring Plan (USDA) Partners Annual reports from cooperating partners (i.e. AVSI) Analysis of the Performance Evaluation of CP Students in Reading for the year 2016/2017; Analysis of the CP Teachers' Evaluation t on the Use of New Teaching-Learning Strategies; State of the Schools 2016-2017 List of partners (Government, NGOs, UN agencies) by location/ activity/ role/ tonnage handled TOR Evaluations/ Reviews Evaluations/ reviews of past or on-going operations/ interventions Baseline Evaluation Report, Midline Evaluation Report of “Support to a Sustainable School Feeding Programme” (2013-2016), Preliminary Report on Impact Study of the WFP Food Assistance Program on Gender Roles Maps Map of the intervention TOR Food/Cash/voucher Distribution Location Map TOR Food Security Map TOR Other documents collected by the team (including external ones) Decentralized Evaluation Quality Assurance System (DEQAS) DEQAS Document Country Poverty Profile Survey on Household Living Standards in Côte d’Ivoire Baseline Survey Questionnaires Survey Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 56 | Page USDA Indicators USDA Food for Progress and McGovern-Dole Indicators and Definitions Baseline Databases Baseline McGovern Dole Questionnaires Baseline School Sample List of schools and school samples Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 57 | Page Annex 4: Stakeholders Interviewed Dates / Tools Locations/sites Stakeholders July 18 KII USDA Headquarters Washington, DC, United States Representatives from USDA September 20 KIIs DCS (MENETFP) Office Abidjan, Côte d’Ivoire Director of DCS and 14 Representatives from DCS, including those responsible for M&E, nutrition, stove amelioration, accounting, logistics/provisions, and communication September 21 KIIs WFP Office Abidjan, Côte d’Ivoire 4 Representatives from WFP, including project coordination/business support; nutrition, market access/ resilience/ livelihoods ; national projects September 24 KIIs AVSI Office Abidjan, Côte d’Ivoire Representatives from AVSI including project management, M&E September 24-25 KIIs WFP Office Abidjan, Côte d’Ivoire Representatives from WFP, including M&E; education; accounting; and supply chain October 5-24 FGDs; Questionnaires Poro, Bagoué, Tchologo, Bounkani, Gontougo, Bafing, and Cavally, Côte d’Ivoire School principals; Parents; Teachers; Students; School management committees; WPGs October 29 KIIs ANADER (MINADER) Office Abidjan, Côte d’Ivoire Representatives from ANADER October 29 KII DPFC (MENETFP) Office Abidjan, Côte d’Ivoire Representative from DPFC January 3-7 FGDs Korhogo, Côte d’Ivoire WPGs Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 58 | Page Annex 5: Data Collection Tools Quantitative Tools 1. Student Questionnaire 2. Teacher Questionnaire 3. Canteen Manager Questionnaire 4. Household Questionnaire 5. School Questionnaire Qualitative Tools 6. KII Protocol: USDA Staff 7. KII Protocol: Project Staff / Partners 8. KII Protocol: Government Stakeholders 9. FGD Protocol: Parents 10. FGD Protocol: School Management Committees (COGES) 11. FGD Protocol: WPGs 12. FGD Protocol: CM Students Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 59 | Page Annex 6: Additional Figures Figure 27: Student Sample Distribution by Gender, Grade, and Type of Respondent MGD CP1 CP2 CE1 CE2 CM1 Total Boys 64 78 88 90 126 446 Girls 62 81 81 88 391 703 Total 126 159 169 178 517 1,149 Non-MGD CP1 CP2 CE1 CE2 CM1 Total Boys 57 60 38 41 55 251 Girls 57 53 45 42 156 353 Total 114 113 83 83 211 604 Source: Student Survey. The sample size was 1753 at midline. Figure 28: Reading Proficiency by Grade Source: Student Survey. The sample size was 964 at baseline and 1753 at midline. Figure 29: Teacher Attendance According to School Records and Self-Reported Bafing Bag. Boun. Cavally Gont. Poro Tcho. Total Official Records Average Attendance Rate 96.2% 94.5% NA 93.5% 96.5% 98.7% 96.7% 96.6% Attending 90% or More 100% 95% NA 77.3% 93.5% 96.5% 96% 94.4% Obs. 31 40 NA 22 46 86 25 250 Self￾reported Average Attendance 94.8% 90.1% 97.8% 92% 92.8% 89.9% 87.5% 92% Attending 90% or More 95.8% 65.7% 97% 61.1% 82.8% 75.6% 81.3% 79.9% Obs. 24 35 33 18 64 78 16 268 Source: School Survey and Teacher Survey. Teacher attendance records for schools in Bounkani region are not available. 0% 20% 40% 60% 80% 100% Mid MGD Base MGD Mid non-MGD Base non-MGD CP1 CP2 CE1 CE2 CM1 Total Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 60 | Page Figure 30: Self-Reported Teacher Absences by Region (Midline Only) Source: Teacher Survey. The sample size was 268. Teachers were asked how many days they missed as a result of each of several causes. Figure 31: Access to Reading Materials (Mobile Library, Reading Board, etc.) by Region (Midline MGD Only) Reading Material Bafing Bagoué Boun. Cavally Gontougo Poro Tchologo Total None 11.1% 23.0% 1.2% 0% 11.8% 10.6% 25.2% 13.8% Sculpted plastic letters 13.9% 13.8% 11.6% 26.7% 7.6% 20.8% 9.2% 14.7% Junior dictionaries 18.1% 6.6% 10.5% 30.0% 11.8% 30.8% 9.8% 17.8% Illustrated boards 29.2% 18.9% 51.2% 53.3% 55.2% 49.4% 41.7% 43.3% Reading board 26.4% 33.7% 51.2% 63.3% 55.2% 49.2% 50.9% 47.4% Mobile library 75.0% 56.1% 68.6% 98.3% 71.7% 71.1% 49.1% 67% Responses 125 298 167 163 452 835 303 2343 Observations 72 196 86 60 212 360 163 1149 Source: Student Survey. Respondents could select multiple choices. Figure 32: Usefulness of AVSI Trainings (Midline MGD Only) Percent Found AVSI training useful 97.8% Put AVSI training to work 96.0% Using AVSI training materials 21.6% Observations 296 Source: Teacher Survey. 0% 20% 40% 60% 80% 100% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total Average Attendance Rate Teachers Attending 90% or More Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 61 | Page Figure 33: Number of Students out of 10 that were Attentive, According to Teachers (Midline MGD Only) Source: Teacher Survey. Sample size was 330. Figure 34: By Region, Number of Students out of 10 that were Attentive, According to Teachers (Midline MGD Only) Source: Teacher Survey. Figure 35: By Region, Average Student Recorded Attendance (Midline Only) Region Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total Percentage 99% 99% 99% 97% 98% 98% 100% 99% Obs. 88 166 63 78 230 149 142 916 Source: School Survey. 0 1 2 3 4 5 6 7 8 9 CP1 CP2 CE1 CE2 CM1 CM2 Total Girls Boys All 0 1 2 3 4 5 6 7 8 9 Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total Girls Boys All Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 62 | Page Figure 36: Average Student Enrolment per Class (MGD Schools with Take Home Ration Schools Separated) CP1 CP2 CE1 CE2 CM1 CM2 All Girls Mid THR 26.5 26.6 26.4 24.2 22.3 20.2 42.5 Base THR 32.8 28.2 24.6 22.1 19.4 15.6 33.9 Mid non-THR 19.1 17.1 16.5 15.7 12.6 11.2 22.2 Base non-THR 24.7 22.2 17.7 15.3 12.8 9.21 17.7 Boys Mid THR 23.5 24.5 25.3 24.1 23.2 21.2 44.3 Base THR 31.6 27.9 25.9 23 20.7 19.2 38.7 Mid non-THR 17.5 18 16 17.4 13.9 15.3 27.5 Base non-THR 26 25.9 20.1 17.7 16.6 14.8 25.1 Source: School Survey. Sample size was 133 schools. Figure 37: Average Student Enrolment per Class (MGD Schools with Take Home Ration Schools Separated) Source: School Survey. Sample size was 133 schools. Figure 38: Ratio of Girls to Boys Enrolled by Grade (MGD Schools with Take Home Ration Schools Separated) CP1 CP2 CE1 CE2 CM1 CM2 CM1 & CM2 Mid THR 1.13 1.09 1.04 1.00 0.96 0.96 0.96 Base THR 1.04 1.01 0.95 0.96 0.94 0.81 0.88 Mid non-THR 1.09 0.95 1.03 0.91 0.90 0.73 0.81 Base non-THR 0.95 0.85 0.88 0.87 0.77 0.62 0.70 Source: School Survey. Sample size was 133 schools. 0 5 10 15 20 25 30 35 Mid THR Base THR Mid non￾THR Base non￾THR Mid THR Base THR Mid non￾THR Base non￾THR Girls Boys CP1 CP2 CE1 CE2 CM1 CM2 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 63 | Page Figure 39: Ratio of Girls Enrolled to Boys Enrolled by Grade (MGD Schools with Take Home Ration Schools Separated) Source: School Survey. Sample size was 133 schools. Figure 40: Number of Days Students Missed Due to Illness (Midline Only) Days missed due to illness? MGD Non-MGD Total Boys 3.3 4.5 3.8 Girls 2.8 3.5 3 Total 3 3.9 3.3 Observations 801 421 1222 Source: Household Survey. Figure 41: Students Missing More than 10 Days Due to Illness (Midline Only) Missed more than 10 days due to illness MGD Non-MGD Total Boys 4% 8% 5.4% Girls 4.2% 5.3% 4.6% Total 4.1% 6.4% 4.9% Obs. 801 421 1222 Source: Household Survey. 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Mid THR Base THR Mid non-THR Base non-THR CP1 CP2 CE1 CE2 CM1 CM2 CM1 & CM2 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 64 | Page Figure 42: Foods Eaten in Household during the Previous Day MGD Non-MGD Foods Midline Baseline Midline Baseline Cereals 89.9% 95.9% 93.9% 96.9% Roots, tubers, etc. 34.7% 46% 25.9% 37.2% Vegetables 79.7% 93% 81.3% 90.8% Fruit 8.1% 73.7% 9% 81.9% Meat 22.6% 48.9% 26.5% 47.8% Eggs 3.4% 15.7% 4.2% 17.7% Seafood 54.8% 63.6% 45.9% 56.5% Legumes, nuts, seeds 45.1% 37.6% 41.1% 43.8% Milk 19.8% 40.2% 19.6% 47.8% Oils and fats 52.4% 84.3% 51.4% 83.5% Sweets 34.8% 67.8% 35.3% 67.8% Condiments 62.2% 85.8% 63.2% 88% Responses 5833 5142 2998 3228 Observations 1149 681 603 425 Source: Household Survey. Figure 43: By Region, Foods Eaten in Household During the Previous Day (MGD Midline) Food Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total Cereals 98.6% 99.5% 73.3% 85.0% 65.1% 97.8% 100.0% 89.9% Roots, tubers, etc. 30.6% 14.8% 79.1% 46.7% 88.7% 15.0% 6.1% 34.7% Vegetables 90.3% 83.7% 98.8% 83.3% 90.6% 72.5% 60.7% 79.7% Fruit 13.9% 2.0% 2.3% 15.0% 19.3% 5.3% 4.9% 8.1% Meat 25.0% 24.5% 20.9% 18.3% 29.3% 16.1% 27.6% 22.6% Eggs 0.0% 0.5% 0.0% 1.7% 3.3% 6.4% 4.3% 3.4% Seafood 45.8% 66.3% 50.0% 65.0% 69.8% 43.6% 49.1% 54.8% Legumes, nuts, seeds 29.2% 53.6% 11.6% 58.3% 17.9% 66.7% 42.3% 45.1% Milk 5.6% 21.4% 5.8% 3.3% 10.9% 26.9% 33.7% 19.8% Oils and fats 55.6% 69.9% 66.3% 33.3% 57.1% 39.7% 51.5% 52.4% Sweets 20.8% 38.8% 24.4% 15.0% 21.7% 45.0% 43.6% 34.8% Condiments 65.3% 61.7% 69.8% 93.3% 40.6% 73.1% 50.3% 62.2% Responses 346 1052 432 311 1090 1829 773 5833 Observations 72 196 86 60 212 360 163 1149 Source: Household Survey. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 65 | Page Figure 44: By Region, Foods Eaten in Household during the Previous Day (MGD Midline) Source: Household Survey. The sample size was 1149. Figure 45: Coping Strategies Used by Household Mid MGD Base MGD Mid non-MGD Base non-MGD Coping Strategy Percent* Days Percent Days Percent Days Percent Days Consuming less preferred foods for cheaper 28.5% 1.0 33.5% 0.8 31.2% 1.0 31.5% 0.7 Depend on aid from relatives or friends 11.8% 0.2 16.5% 0.3 15.3% 0.3 14.8% 0.3 Decrease the amount of food during meals 19.7% 0.6 24.7% 0.6 22.4% 0.6 19.1% 0.4 Reduced quantities consumed by adult / parent for the benefit of young children 22.4% 0.7 21.6% 0.5 23.2% 0.7 12.5% 0.4 Reduce the number of meals per day (skipping one or two meals in the day) 14.4% 0.4 13.1% 0.3 14.8% 0.3 6.4% 0.1 Buying food on credit 23.2% 0.6 18.9% 0.4 23.6% 0.6 21.7% 0.4 Eating wild foods / Culturally not allowed 2.4% 0.1 5.1% 0.1 2.5% 0.1 3.3% 0.1 Consuming seeds 12.1% 0.5 14.8% 0.4 15.6% 0.6 7.8% 0.2 Begging for food or money to buy food 1.9% 0.0 3.7% 0.1 4.8% 0.1 2.8% 0.1 Send household members to eat elsewhere or to live with relatives or friends 3.1% 0.1 4.9% 0.1 4.8% 0.1 2.6% 0.1 Spend 1 or more days without eating 3.9% 0.1 5.3% 0.1 4.8% 0.1 2.8% 0.0 Have the children work 10.4% 0.2 4.9% 0.1 10% 0.3 7.5% 0.1 Total 1766 1136 1042 564 Cases 1149 681 603 425 Source: Household Survey. Households could report using more than one strategy. *Percentage of households using the strategy at least one day per week; average number of days per week that households use the strategy (this includes households using the strategy 0 days per week). Bolded strategies are the strategies used to create the reduced index. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 66 | Page Figure 46: By Region, Days* Coping Strategies Used (Midline MGD) Coping Strategy Bafing Bag. Boun. Cavally Gont. Poro Tcho. Total Consuming less preferred foods for cheaper 1.5 0.7 1.1 1.5 0.2 1.3 1.4 1.0 Depend on aid from relatives or friends 0.7 0.1 0.3 0.4 0.1 0.3 0.1 0.2 Decrease the amount of food during meals 1.2 0.6 0.4 1.3 0.1 0.7 0.5 0.6 Reduced quantities consumed by adult / parent for the benefit of young children 1.4 0.3 0.5 1.2 0.1 0.9 0.9 0.7 Reduce the number of meals per day (skipping one or two meals in the day) 0.8 0.1 0.3 1.1 0.1 0.6 0.3 0.4 Buying food on credit 0.9 0.6 0.2 1.0 0.1 0.8 0.5 0.6 Eating wild foods / Culturally not allowed 0.2 0.0 0.1 0.1 0.1 0.1 Consuming seeds 1.6 0.3 1.0 0.0 0.3 1.3 0.5 Begging for food or money to buy food 0.2 0.0 0.1 0.0 0.0 Send household members to eat elsewhere or to live with relatives or friends 0.3 0.0 0.1 0.0 0.1 0.0 0.1 Spend 1 or more days without eating 0.3 0.0 0.1 0.0 0.1 0.1 0.1 Have the children work 0.9 0.0 0.3 0.3 0.2 0.2 Responses 270 153 112 102 92 791 246 1766 Observations 72 196 86 60 212 360 163 1149 Source: Household Survey. Households could report using more than one strategy. *Average number of days per week that households use the strategy (this includes households using the strategy 0 days per week). Where table shows 0.0 days, the strategy is used less than 0.05 days on average, but more than 0 days. Bolded strategies are the strategies used to create the reduced index. Figure 47: By Region, Percent* Using Coping Strategies (Midline MGD) Coping Strategy Bafing Bag. Boun. Cavally Gont. Poro Tcho. Total Consuming less preferred foods for cheaper 54.2% 15.3% 29.1% 28.3% 8.5% 40.3% 32.5% 28.5% Depend on aid from relatives or friends 29.2% 6.6% 14% 16.7% 5.2% 17.2% 4.3% 11.8% Decrease the amount of food during meals 45.8% 14.8% 16.3% 23.3% 3.8% 27.8% 17.2% 19.7% Reduced quantities consumed by adult / parent for the benefit of young children 52.8% 6.6% 20.9% 28.3% 4.7% 33.3% 25.2% 22.4% Reduce the number of meals per day (skipping one or two meals in the day) 33.3% 4.1% 14% 28.3% 3.8% 22.2% 9.8% 14.4% Buying food on credit 40.3% 22.5% 10.5% 35% 3.8% 33.6% 20.9% 23.2% Eating wild foods / Culturally not allowed 5.6% 0.5% 6.7% 3.9% 3.1% 2.4% Consuming seeds 47.2% 6.6% 17.4% 0.5% 11.4% 21.5% 12.1% Begging for food or money to buy food 8.3% 0.5% 3.3% 1.8% 1.9% Send household members to eat elsewhere or to live with relatives or friends 15.3% 0.5% 3.5% 0.5% 4.2% 3.1% 3.1% Spend 1 or more days without eating 18.1% 3.5% 3.3% 0.5% 5.6% 3.7% 3.9% Have the children work 25% 1.2% 12.3% 16.9% 8% 10.4% Responses 270 153 112 102 92 791 246 1766 Observations 72 196 86 60 212 360 163 1149 Source: Household Survey. Households could report using more than one strategy. *Percentage of households using the strategy at least one Day per week. Bolded strategies are the strategies used to create the reduced index. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 67 | Page Figure 48: By Region, Percent* Using Coping Strategies (Midline MGD) Source: Household Survey. Households could report using more than one strategy. *Percentage of households using the strategy at least one day per week. The sample size was 1149. The first five strategies listed are the strategies used to create the reduced index. Figure 49: Percent of School Days during which Canteen Operated by Region Region Midline MGD Obs. Baseline MGD Obs. Bafing 100% 6 34.5% 6 Bagoué 93.6% 7 64% 8 Bounkani 97.4% 6 55.1% 7 Cavally 99% 5 29.7% 5 Gontougo 95% 16 66.2% 16 Poro 87.9% 18 68.7% 19 Tchologo 98% 5 6% 6 Total 93.1% 63 56.9% 67 Source: Canteen Survey. 0% 10% 20% 30% 40% 50% 60% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total Consuming less preferred foods for cheaper Depend on aid from relatives or friends Decrease the amount of food during meals Reduced quantities consumed by adult / parent for the benefit of young children Reduce the number of meals per day (skipping one or two meals in the day) Buying food on credit Eating wild foods / Culturally not allowed Consuming seeds Begging for food or money to buy food Send household members to eat elsewhere or to live with relatives or friends Spend 1 or more days without eating Have the children work Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 68 | Page Figure 50: By Region, Percent of School Days during which Canteen Operated Source: Canteen Survey. Figure 51: Parents’ Awareness of the School Canteen Project Mid MGD Base MGD Mid non-MGD Base non-MGD Parent is aware of the canteen program 94.1% 98.2% 30.7% 42.1% Obs. 1148 681 590 425 Source: Household Survey. Sample size was 1106 at baseline and 1752 at midline. Figure 52: Students Eating through the School Canteen Program Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total Student ate at the canteen 94.4% 93.3% 95.2% 98.3% 95.8% 88% 86.5% 91.6% Total 72 195 84 60 212 357 163 1143 Source: Household Survey. Figure 53: Frequency that Students Eat at the School Canteen Frequency Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total Never 5.6% 6.7% 4.8% 1.7% 4.3% 12% 13.5% 8.4% One day a week 6.9% 2.1% 6% 8.3% 1.4% 1.1% 0% 2.3% Two days a week 2.8% 16.4% 4.8% 3.3% 3.8% 2.5% 6.8% 6% Three days a week 9.7% 29.7% 27.4% 8.3% 10.9% 14% 7.4% 15.6% Four days a week 20.8% 19.5% 19.1% 11.7% 16% 10.4% 16.6% 15.2% Every day of the school week 54.2% 25.6% 38.1% 66.7% 63.7% 59.9% 55.8% 52.6% Total 72 195 84 60 212 357 163 1143 Source: Household Survey. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total Baseline MGD Midline MGD Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 69 | Page Figure 54: Foods Used to Prepare Canteen Meals (Midline Only) Source: Canteen Survey. The sample size was 67. Figure shows the probability that a given food group would be included in a meal in the region. This calculation was based on the number of days the food group was used in preparation of meals by each canteen and the total number of days that canteen operated. Figure 55: By Region, Canteen Managers Can Cite Three Health & Hygiene Practices Region Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total Midline 83.3% 75% 100% 40% 93.8% 94.7% 100% 87.9% Baseline 83.3% 75% 100% 100% 100% 94.7% 83.3% 92.5% Obs. 6 8 7 5 16 19 6* 67 Source: Canteen Survey. Only five schools were available in Tchologo at midline. Figure 56: By Region, Canteen Managers Can Cite Three Health & Hygiene Practices Source: Canteen Survey. Only five schools were available in Tchologo at midline. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Cereals Tubers/roots Pulses/nuts Dairy Meat, fish, giblets Eggs Vitamin A rich fruits and vegetables Other fruits and vegetables Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total Midline Baseline Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 70 | Page Figure 57: Teacher Training on Health Practices Source: School Survey. Sample size was 104 at baseline and midline. Figure 58: Nutrition-Related Trainings at School MGD Non-MGD Type of Training Midline Baseline Midline Baseline Teacher nutrition training 54.4% 48.5% 8.3% 5.6% Student nutrition training 19.4% 27.9% 0% 19.4% Vegetable garden training 7.4% 4.4% 0% 19.4% Observations 68 68 36 36 Source: School Survey. Sample size was 104 at baseline and midline. Figure 59: Type of Access to Water of the School Mid MGD Base MGD Mid non-MGD Base non-MGD Total No water 53% 50% 64% 69% 57% Tap 16% 10% 17% 3% 12% Village pump 19% 31% 17% 17% 22% Protected well 9% 6% 0% 3% 5% Unprotected well 3% 3% 3% 8% 4% Observations 68 68 36 36 208 Source: School Survey. Figure 60: Distance to School’s Water Source, Where Available (Midline MGD Only) Bafing Bagoué Boun. Cavally Gont. Poro Tcho. Total In the school grounds 17% 25% 29% 20% 35% 47% 17% 32% Less than 15 min walk 50% 63% 57% 80% 29% 32% 33% 43% Over 15 min walk 33% 13% 14% 0% 35% 21% 50% 25% Total 6 8 7 5 17 19 6 68 Source: School Survey. 0% 10% 20% 30% 40% 50% 60% 70% 80% Mid MGD Base MGD Mid non-MGD Base non-MGD Skills Based Health Education Deworming HIV/AIDS Prevention Malaria Prevention Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 71 | Page Figure 61: Most Common Problems with Water Source Mid MGD Base MGD Mid non￾MGD Base non￾MGD Broken pump 59% 59% 50% 37% Animals occupying water source 0% 0% 0% 5% Dried up water source 15% 26% 0% 26% Water is used for agriculture 0% 0% 0% 5% Other 11% 7% 40% 21% Distance 15% 7% 10% 5% Total 27 27 10 19 Source: School Survey. Figure 62: Access to and Quality of Sanitary Facilities Mid MGD Base MGD Mid non￾MGD Base non￾MGD Has sanitary facilities 59% 54% 50% 31% Separated by gender 31% 44% 25% 19% Separate for teachers 26% 29% 31% 14% Observations 68 68 36 36 Source: School Survey. Figure 63: Deworming Pills and Micronutrient Pills Distributed to Students Mid MGD Base MGD Mid non￾MGD Base non￾MGD Deworming treatments 64% 72% 56% 64% Micronutrient supplements 26% 53% 17% 14% Observations 68 68 36 36 Source: School Survey. The sample size was 104 at baseline and midline. Figure 64: Deworming Pills and Micronutrient Pills Distributed to Students Source: School Survey. The sample size was 104 at baseline and midline. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mid MGD Base MGD Mid non-MGD Base non-MGD Deworming treatments Micronutrient supplements Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 72 | Page Figure 65: Access to Food Storage and Equipment Midline Baseline Ventilated food storage 59% 63% Improved pallets 24% 22% Stock cards available 100% 65% Food prepared on-site 99% 96% Have a kitchen 90% 82% Source: School Survey. The sample size was 68 at baseline and midline. Figure 66: Access to Food Storage and Equipment Source: School Survey. The sample size was 68 at baseline and midline. Figure 67: Sufficient Food Preparation Equipment Source: Canteen Survey. Only 5 schools were available in Tchologo at midline. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Ventilated food storage Improved pallets Stock cards available Food prepared on-site Have a kitchen Midline Baseline 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total Midline Baseline Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 73 | Page Figure 68: Sufficient Food Storage Equipment Source: Canteen Survey. Only 5 schools were available in Tchologo at midline. Figure 69: Stove Type Midline Baseline None/Unknown 11.76 17.65 Traditional stove 39.71 73.53 Furnace without chimney 2.94 1.47 Stove with chimney 45.59 7.35 Observations 68 68 Source: Canteen Survey. The sample size was 68 at baseline and midline Figure 70: Stove Type by Region Bafing Bagoué Bounkani Cavally Gontougo Poro Tchologo Total None/Unknown 0% 13% 0% 40% 6% 21% 0% 12% Traditional stove 33% 25% 43% 0% 65% 47% 0% 40% Furnace without chimney 0% 0% 29% 0% 0% 0% 0% 3% Stove with chimney 67% 63% 29% 60% 29% 32% 100% 46% Observations 6 8 7 5 17 19 6 68 Source: Canteen Survey. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Bafing Bagoue Bounkani Cavally Gontougo Poro Tchologo Total Midline Baseline Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 74 | Page Figure 71: Reading Assessment Scores by Treatment Group, Phase, and Grade Mid MGD Base MGD Score CP1 CP2 CE1 CE2 CM1 CP1 CP2 CE1 CE2 CM1 0 7.9% 3.1% 1.8% 1.7% 0.4% 63.0% 35.3% 11.5% 7.3% 5.0% A 46.8% 25.2% 16.6% 10.7% 6.6% 32.4% 32.8% 30.1% 19.1% 10.8% B 31.8% 25.2% 13.6% 13.5% 6.8% 4.6% 13.1% 15.9% 14.6% 16.7% C 3.2% 8.8% 7.1% 4.5% 2.5% 4.9% 5.3% 7.3% 4.2% D 4.8% 12.6% 10.1% 7.3% 4.8% 5.7% 7.1% 11.8% 9.2% E 1.6% 3.1% 2.4% 1.7% 2.5% 1.6% 6.2% 9.2% F 4.0% 10.1% 17.2% 14.6% 23.4% 5.7% 15.0% 20.9% 23.3% G 7.6% 9.5% 9.6% 14.3% 0.8% 6.2% 9.1% 7.5% H 1.9% 3.0% 7.3% 5.8% 0.9% 3.6% 1.7% I 2.5% 8.3% 14.0% 12.4% 1.8% 2.7% 5.0% J 10.1% 9.0% 12.8% 2.7% 5.0% K 0.6% 6.2% 7.7% 0.9% 2.5% Obs. 126 159 169 178 517 108 122 113 110 120 Mid non-MGD Base non-MGD Score CP1 CP2 CE1 CE2 CM1 CP1 CP2 CE1 CE2 CM1 0 26.3% 10.6% 2.4% 0.5% 57.1% 40.3% 19.2% 1.5% 6.9% A 58.8% 46.0% 34.9% 18.1% 11.9% 39.3% 23.4% 39.7% 15.2% 10.3% B 12.3% 23.0% 18.1% 16.9% 16.6% 3.6% 14.3% 11.5% 24.2% 15.5% C 0.9% 3.5% 6.0% 3.6% 1.9% 6.5% 1.3% 3.0% D 0.9% 5.3% 10.8% 12.1% 7.1% 3.9% 6.4% 9.1% 5.2% E 0.9% 2.4% 4.8% 0.5% 2.6% 6.4% 6.1% 8.6% F 3.5% 8.4% 13.3% 21.3% 7.8% 11.5% 21.2% 32.8% G 2.7% 3.6% 12.1% 8.5% 1.3% 2.6% 6.1% 10.3% H 1.8% 2.4% 2.4% 3.3% 3.0% 5.2% I 1.8% 6.0% 9.6% 12.3% 10.6% 1.7% J 1.8% 3.6% 6.0% 10.4% 1.3% 3.5% K 1.2% 1.2% 5.7% Obs. 114 113 83 83 211 112 77 78 66 58 Source: Student Survey. Shaded areas are below grade-level reading proficiency. Unshaded areas are at or above grade-level reading proficiency. The sample size was 964 at baseline and 1753 at midline. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 75 | Page Figure 72: Food Consumption Score Group by Gender of Head of Household MGD Non-MGD Gender of Head of Household Food Consumption Score Group Midline Baseline Midline Baseline Man Poor 2.1% 0.8% 1.3% 0.5% Borderline 7.9% 2.8% 9.5% 4.3% Acceptable 90% 96.4% 89.2% 95.2% Observations 1062 602 547 376 Woman Poor 3.5% 0% 1.8% 6.1% Borderline 12.6% 3.8% 10.7% 2% Acceptable 83.9% 96.2% 87.5% 91.8% Observations 87 79 56 49 All Poor 2.2% 0.7% 1.3% 1.2% Borderline 8.3% 2.9% 9.6% 4% Acceptable 89.6% 96.3% 89.1% 94.8% Observations 1149 681 603 425 Source: Household Survey. Sample size was 1106 at baseline and 1752 at midline. Figure 73: Household Dietary Diversity Score by Gender of Head of Household MGD Non-MGD Gender of Head of Household Midline Baseline Midline Baseline Man 5.1 7.5 5 7.7 Observations 1061 602 547 376 Woman 5 7.4 4.6 7.1 Observations 87 79 56 49 All 5.1 7.5 5 7.6 Observations 1148 681 603 425 Source: Household Survey. The score is a sum of the number of food groups consumed during the previous day. It measures the dietary diversity available in the foods consumed in the home. Sample size was 1106 at baseline and 1751 at midline. Figure 74: Reduced Coping Strategies Index by Gender of Head of Household* MGD Non-MGD Gender of Head of Household Midline Baseline Midline Baseline Man 4.4 3.7 4.6 2.9 Observations 1062 602 547 376 Woman 5.4 4.4 4.7 2.9 Observations 87 79 56 49 Total 4.4 3.8 4.6 2.9 Observations 1149 681 603 425 * A higher index indicates more severe coping strategies necessary and higher levels of food insecurity. Source: Household Survey. The Reduced Coping Strategies Index is a context non-specific weighted sum of the number of days per week each household employs each of five strategies to cope with food scarcity. The maximum reduced index is 56. The reduced index is appropriate to measure less-severe coping measures, while the full index is more appropriate in more food-insecure contexts. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 76 | Page Annex 7: Project Results Framework MGD SO1: Improved Literacy of School Aged Children MGD 1.1: Improved Quality of Literacy Instruction MGD 1.3: Improved Student Attendance 1.1.2: Better Access to School Supplies and Teaching Materials 1.1.3: Improved Literacy Instructional Materials 1.3.1: Increased Economic and Cultural Incentives (decreased disincentive) 1.3.5: Increased Community Understanding of the Benefits of Education 1.3.4:Increased Student Enrollment Distribution: school supplies and materials (UNICEF) Distribution: school supplies and materials (WFP, UNICEF) Take-home rations (WFP) Provide school meals (WFP) Raising awareness on the importance of education (WFP, WCF) Raising awareness on the importance of education (WFP, WCF, UNICEF) MGD 1.2.1: Reduced Short-Term Hunger 1.2.1.1: Increased Access to Food (School Feeding) Provide school meals (WFP) 1.1.5: Increased Skills and Knowledge of Administrators Training: School Administrators (UNICEF) 1.1.4: Increased Skills and Knowledge of Teachers Training: teachers (UNICEF) Result supported throughpartner Result achieved by WFP WFP Côte d’Ivoire FY13-FY15 McGovern-Dole Proposal – Program-Level Results 1.3.3: Increased School Infrastructure Building/ Rehabilitation: Kitchens (DNC, WFP, WCF) Building/ Rehabilitation: Schools (UNICEF) Establish School Gardens (WFP) 1.3.2: Reduced Health Related Absences Distribution: de￾worming medication, vitamins and minerals (WFP, NPSUH) WFP activities Partner activities Key 1.1.1: More Consistent Teacher Attendance Training: Teachers (UNICEF) Promote teacher attendance (WFP / UNICEF) Establish Parent￾Teacher Associations (WCF) MGD 1.2: Improved Attentiveness Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 77 | Page MGD 1.4.1: Increased Capacity of Government Institutions (including schools) MGD 1.4.3: Increased Government Support MGD 1.4.4: Increased Engagement of Local Organizations and Community Groups MGD 1.4.2: Improved Policy and Regulatory Framework Training: Commodity Management (WFP) Develop Partnerships with Farmer Groups to supply food to schools (WFP, DNC) Capacity Building Local, regional and national level (WFP) Training: Parent-Teacher Associations (WCF, WFP) Establish Parent-Teacher Associations (WFP, WCF) SO1 Foundational Results Capacity Building Local, regional and national level (WFP) Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 78 | Page MGD 2.6: Increased Access to Requisite Food Preparation and Storage Tools and Equipment MGD SO2: Increased Use of Health and Dietary Practices MGD 2.4: Increased Access to Clean Water and Sanitation MGD 2.5 Increased Access to Preventative Health Interventions Building/ Rehabilitation: Latrines (UNICEF) Building/ rehabilitation: Wells and water stations/ systems (UNICEF) Building/Rehabilitation: Warehouses and Storerooms; Kitchens (WFP/WCF) Provide Energy Saving Stoves (WFP) Distribution: de-worming medication, vitamins and minerals (NPSUH, WFP) MGD 2.2: Increased Knowledge of Safe Food Prep and Storage MGD 2.3: Increased Knowledge of Nutrition Training: Good health and nutrition practices (DNC, WFP) MGD 2.1: Increased Knowledge of Health and Hygiene Practices MGD 2.7.1: Increased Capacity of Government Institutions MGD 2.7.3: Increased Government Support MGD 2.7.4: Increased Engagement of Local Organizations and Community Groups MGD 2.7.2: Improved Policy and Regulatory Framework Capacity Building Local, regional and national level (WFP) Capacity Building Local, regional and national level (UNICEF) Capacity Building Local, regional and national level (UNICEF) Develop Partnerships with Farmer Groups to supply food to schools (WFP, DNC) SO2 Foundational Results Result supported through partner Result achieved by WFP WFP activities Partner activities Key Training: Food Preparation and storage practices (DNC, WFP) Capacity Building Local, regional and national level (WFP, WCF) Establish School Gardens (WFP/FAO) Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 79 | Page Annex 8: Terms of Reference The terms of reference (TOR) for the “Support for the Integrated School Feeding Program" project decentralized mid-term evaluation are available in English and French on WFP’s website using the following link: https://www.wfp.org/content/cote-divoire-school￾feeding-mid-term-evaluation. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 80 | Page Annex 9: MGD Indicators RESULTS INDICATOR Baseline Midline Target MGD SO1: Literacy Learning of school children is improved Proportion of school-children who, at the end of two years of primary school, demonstrate that they can read and understand the meaning of the school grade text (boys). CP1= 5% CP2=16% CE1=25% CE2=11% CM1=8% CM2=8% CP1= 44% CP2=40% CE1=51% CE2=46% CM1=30% Proportion of school-children who, at the end of two years of primary school, demonstrate that they can read and understand the meaning of the school grade text (girls). CP1= 4% CP2=13% CE1=19% CE2=11% CM1=6% CM2=7% CP1= 47% CP2=36% CE1=46% CE2=27% CM1=17% CM2=27% Number of individuals benefiting directly from USDA-funded interventions (new). 0 Number of individuals benefiting directly from USDA-funded interventions (old). 0 Number of individuals benefiting directly from USDA-funded interventions (men). 0 Number of individuals benefiting directly from USDA-funded interventions (women). 0 Number of individuals benefiting directly from USDA-funded interventions. 0 MGD 1.1 Improved quality of literacy learning of school children Number of teachers in target schools demonstrating the use of new techniques or quality teaching tools as a result of USDA assistance. 0 MGD 1.1.1 More consistent teacher attendance Proportion of teachers in target schools who attend and teach at school regularly (at least 90% of school days) per school year. 94.2% 94.4% MGD 1.1.2 Better access to school supplies and materials Number of textbooks and other teaching and learning materials provided with USDA assistance. 0 MGD 1.1.3 Improved access to learning materials for reading Number of target schools in which school children have additional reading materials with USDA support. 0 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 81 | Page MGD 1.1.4 Increased skills and knowledge of teachers Number of teachers / teachers’ aides in targeted schools demonstrating the use of new and good techniques or teaching tools (per type, per gender). 0 Number of teachers / teachers’ aides trained or certified as a result of USDA assistance (per type, per sex). 0 MGD 1.1.5 Increased skills and knowledge of school administrators Number of targeted school administrators demonstrating new and good techniques or teaching tools (per type, per gender). 0 Number of leaders trained or certified as a result of USDA assistance (by gender). 0 MGD 1.2 Improved school children attentiveness Proportion of school children identified as being attentive in class by their teachers (per sex, per class). Girl = 78% Boy = 78% Girl = 68% Boy = 67% MGD 1.2.1 Reduced short￾term hunger Number of school lunches (breakfast, snack, lunch) offered to school children following USDA assistance. 0 Proportion of school children in targeted schools who ate a meal regularly before or during the school day (per gender). 0 MGD 1.2.1.1 /1.3.1.1 Increased access to school feeding Number of take-home rations provided as a result of USDA assistance. 0 Number of girls having received take-home rations as a result of USDA assistance. 0 Number of girls having received take-home rations as a result of USDA assistance (new). 0 Number of girls having received take-home rations as a result of USDA assistance (continue). 0 Number of school children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance (girls). 0 Number of school children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance (boys). 0 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 82 | Page Number of school children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance (new). 0 Number of school children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance (continue). 0 Number of daily school meals (breakfast, snack, lunch) provided to school-children following USDA assistance. 0 Proportion of households with acceptable food consumption per sex of the household head. Female = 96.2% Male = 96.3% Female = 83.9% Male = 90.0% Coping strategy index (average) per sex of the household head. Female = 4.1 Male = 2.9 Female = 5.4 Male = 4.4 Dietary diversity score per sex of the household. Female = 5.8 Male = 6 Female = 5.0 Male = 5.1 Number of social safety net beneficiaries participating in productive safety nets operations as a result of USDA assistance (ongoing). 0 Number of social safety net beneficiaries participating in productive safety nets operations as a result of USDA assistance (new). 0 Number of social safety net beneficiaries participating in productive safety nets operations as a result of USDA assistance (men). 0 Number of social safety net beneficiaries participating in productive safety nets operations as a result of USDA assistance (women). 0 MGD 1.3 Improved Student attendance Proportion of students regularly (80%) attending classes / schools supported by USDA (boys). 98% 100% Proportion of student regularly (80%) attending classes / schools supported by USDA (girls). 98.3% 100% MGD 1.3.1 Increased Economic and Cultural Incentives (Or Number of girls receiving take￾home rations as a result of USDA assistance (new). 0 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 83 | Page Decreased Disincentives) Number of girls receiving take￾home rations as a result of USDA assistance (ongoing). 0 MGD 1.3.2 Reduced Health Related Absences Proportion of school children who miss more than 10 days of school per year due to illness (boys). 2% 0.7% Proportion of school children who miss more than 10 days of school per year due to illness (girls). 1.7% 0.7% MGD 1.3.4 Increased Student Enrolment Number of school children enrolled in schools receiving USDA assistance (boys). 0 Number of school children enrolled in schools receiving USDA assistance (girls). 0 Gender Ratio, elementary school. 0.85 0.99 MGD 1.3.5 Increased Community Understanding of Benefits of Education Number of members of management committees and members of women production groups sensitized on the importance of education. 0 Value of Public and Private Investments creating a leverage effect due to USDA Assistance (Host Government). 0 Annual increase rate of the budget allocated by the Government to the Directorate of School Canteens. 0 Number of policies in the child health and nutrition sectors, regulations and administrative procedures, by level of development, due to USDA support (per stage). 0 Number of education sector policies, regulations and administrative procedures, by level of development, due to USDA assistance (per stage). 0 MGD 1.4.4 Increased Engagement of Local Number of parent-teacher associations or similar "school" governance structures supported as a result of USDA assistance. 0 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 84 | Page Organizations and Community Groups Number of public-private partnerships put in place as a result of USDA assistance (nutrition). 0 Number of public-private partnerships put in place as a result of USDA assistance (education). 0 Number of public-private partnerships put in place as a result of USDA assistance (health). 0 Number of public-private partnerships put in place as a result of USDA assistance (multi sectoral). 0 Number of public-private partnerships put in place as a result of USDA assistance (other). 0 MGD SO2 Increased Use of Health and Dietary Practices Proportion of school-aged children receiving a minimum acceptable diet (boys). 47% 83% Proportion of school-aged children receiving a minimum acceptable diet (girls). 47% 83% MGD 2.1 Improved Knowledge of Health and Hygiene Practices Proportion of School Management Committee members and canteen management staff who can identify at least three health and hygiene practices(men). 89.6% 88% Proportion of School Management Committee members and canteen management staff who can identify at least three health and hygiene practices (women). 89.6% 88% MGD 2.2 Increased Knowledge of Safe Food Preparation and Storage Practices Proportion of school management committee members and canteen management staff who can identify at least three safe food preparation and storage practices. .1% good food storage practice 72% good knowledge of safe food preparation practices 83% good food storage practice 82% good food preparation practices MGD 2.3 Increased Knowledge of Nutrition Number of individuals receiving training in child health and nutrition as a result of USDA assistance (men). 0 Number of individuals receiving training in child health and nutrition as a result of USDA assistance (women). 0 MGD 2.5 Number of school children who received deworming. Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 85 | Page Access to Preventative Health Interventions Number of school children who received deworming (boys). Number of school children who received deworming (girls). Proportion of schools using an improved water source. 48% has a water point 93% have an improved water source 54% has a water point 94% have an improved water source Proportion of schools with improved sanitation facilities. 53% 59% MGD 2.6 Increased Access to Requisite Food Prep and Storage Tools and Equipment Number of targeted schools with access to improved food preparation and storage equipment. MGD 1.4.1 /2.7.1 Increased Capacity of Government Institutions Number of government staff trained in food management, monitoring and evaluation. 0 MGD 1.4.2 /2.7.2 Improved Policy and regulatory framework Number of policies, regulations, and/or administrative procedures in the child health and nutrition sectors in each of the following stages of development as a result of USDA assistance (stage 1). 0 Number of child health and nutrition policies, regulations, and/or administrative procedures in each of the following stages of development as a result of USDA assistance (stage 2). 0 Number of child health and nutrition policies, regulations, and/or administrative procedures in each of the following stages of development as a result of USDA assistance (stage 5). 0 MGD 1.4.3 Increased government support Value of Public and Private Investments creating a leverage effect due to USDA Assistance (Host Government). 0 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 86 | Page Annual increase rate of the budget allocated by the Government to the Directorate of School Canteens. 0 MGD 1.4.4 Increased Engagement of Local of Local Organizations and Community Group Number of parent-teacher associations or similar "school" governance structures supported as a result of USDA assistance. 0 Number of public-private partnerships set up as a result of USDA assistance (women production groups). 0 Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 87 | Page Annex 10: Map of MGD School Feeding Program Mid-Term Evaluation Report: MGD Project in Côte d’Ivoire (2015-2020) 88 | Page Annex 11: List of Abbreviations AEA American Evaluation Association AFD Agence Française de Développement ANADER Agence Nationale d'Appui au Développement Rural AVSI Associazione Volontari Servizio Internazionale DCS Directorate of School Canteens DID Difference-in-difference FAO Food and Agriculture Organization FFE Food For Education FGD Focus group discussion IFAD International Fund for Agricultural Development IRB Institutional review board KII Key informant interview MAD Minimum acceptable diet MENETFP Ministère de l'Education Nationale, de l’Enseignement Technique, et de la Formation Professionnel MGD McGovern-Dole OEV Office of Evaluation SOP Standard Operating Procedure THR Take home rations TOR Terms of Reference UNDP United Nations Development Program UNEG United Nations Evaluation Group UNHCR United Nations High Commissioner for Refugees UNICEF United Nations International Children's Emergency Fund World Food Programme (WFP) Country Office in Côte d'Ivoire www.wfp.org