This publication was produced at the request of the United States Department of Agriculture. It was prepared independently by Abt Associates. Foreign Agricultural Service, United States Department of Agriculture McGovern-Dole Mozambique Impact Evaluation June 15, 2022 Submitted by: Amy Power Abt Associates 6130 Executive Boulevard Rockville, MD 20852 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. McGovern-Dole Mozambique Program Impact Evaluation The United States Department of Agriculture’s (USDA) McGovern-Dole International Food for Education and Child Nutrition (McGovern-Dole) Impact Evaluation in Mozambique was conducted by Abt Associates from July 2, 2021 to July 1, 2022. This report aims to evaluate the impact of two McGovern-Dole supported projects, one implemented by Planet Aid Inc. in Maputo province from 2015 to 2020 and one implemented by World Vision Inc. in Nampula province from 2015 to 2021. Agreement Number: GS-OOF-252CA/12314421F0453 Project Duration: July 2, 2021 to July 1, 2022 Implemented by: Abt Associates Evaluation Authored by: Abt Associates • Catherine Darrow • Daniel Litwok • Marius Meijerink • Feliciano Chimbutane • Jillian Ouellette • Megan Bogia • Adi Greif • Amy Power DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. CONTENTS ACKNOWLEDGEMENTS ........................................................................................................................ i LIST OF EXHIBITS................................................................................................................................... ii ABBREVIATIONS..................................................................................................................................... v EXECUTIVE SUMMARY ........................................................................................................................ 1 1. OVERVIEW OF EVALUATION AND KEY FINDINGS ........................................................ 6 2. BACKGROUND ............................................................................................................................ 7 2.1 Benefit of this Evaluation....................................................................................................... 7 2.2 Mozambique Context ............................................................................................................. 7 2.3 McGovern-Dole International Food for Education and Child Nutrition Program ................. 8 2.4 Summary of Other Literacy and School Feeding Projects Implemented in the Maputo and Nampula Provinces ......................................................................................................... 9 2.5 Baseline Assessment............................................................................................................ 10 2.6 Other Evaluations Conducted by Implementing Partners .................................................... 11 2.7 Prior Evidence on School Feeding in Mozambique (or in the Sub-Saharan Region).......... 11 2.8 Prior Evidence on Teacher Pedagogy, Training, and Remedial Education in Mozambique (or in the Sub-Saharan Region) ...................................................................... 12 2.9 Quasi-Experimental Impact Evaluation ............................................................................... 12 2.10 Differences Between Baseline and Endline Approaches ..................................................... 13 2.11 COVID-19 Effects on Schooling ......................................................................................... 14 3. EVALUATION APPROACH..................................................................................................... 15 3.1 Impact and Implementation Focus ....................................................................................... 15 3.2 Evaluation Questions ........................................................................................................... 15 4. METHODOLOGY ...................................................................................................................... 18 4.1 Sampling Frame ................................................................................................................... 18 4.2 Qualitative Sampling Design ............................................................................................... 18 4.3 Outcomes ............................................................................................................................. 19 4.4 Impact Analysis Models....................................................................................................... 21 4.5 Implementation Fidelity Analyses....................................................................................... 24 4.6 Analysis of Stakeholder Perspectives .................................................................................. 24 4.7 Analysis of Impacts in Adequate Fidelity Sites................................................................... 25 5. DATA COLLECTION................................................................................................................ 27 5.1 Overview .............................................................................................................................. 27 5.2 Participant Contact and Information Collection................................................................... 27 5.3 Enumerator Recruitment and Training ................................................................................. 27 5.4 Data Quality and Security .................................................................................................... 28 5.5 Data Management and Security ........................................................................................... 29 5.6 COVID-19 Specific Precautions.......................................................................................... 29 6. CHARACTERISTICS OF POPULATION............................................................................... 31 6.1 Sample Description.............................................................................................................. 31 6.2 Student Demographic Characteristics.................................................................................. 31 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. 6.3 Language of Literacy Instruction ......................................................................................... 32 6.4 Teacher Characteristics........................................................................................................ 32 6.5 School Characteristics.......................................................................................................... 33 6.6 Parent Focus Group Characteristics..................................................................................... 34 6.7 Key Official Characteristics................................................................................................. 35 7. MAIN FINDINGS........................................................................................................................ 36 7.1 Summary of Findings ........................................................................................................... 36 7.2 Impacts on Student Reading Performance ........................................................................... 36 7.3 Impacts on Student Attendance ............................................................................................ 38 7.4 Impacts on Student Attentiveness ........................................................................................ 40 7.5 Impacts for Student Subgroups ............................................................................................ 41 8. SECONDARY FINDINGS.......................................................................................................... 44 8.1 Impacts on Instructional Inputs.............................................................................................. 44 8.2 Impacts on Feeding Outcomes and Nutrition Knowledge ................................................... 45 8.3 Impacts on Student Knowledge and Use of Recommended Health Practices ..................... 47 9. FINDINGS ON IMPLEMENTATION...................................................................................... 49 9.1 Summary of Findings ........................................................................................................... 49 9.2 Determination of Implementation Adequacy ....................................................................... 49 9.3 Implementation of School Feeding ...................................................................................... 50 9.4 Implementation of Literacy Instruction................................................................................ 51 9.5 Implementation of Teacher Training ................................................................................... 51 10. FINDINGS ON IMPACT CONTROLLING FOR IMPLEMENTATION............................ 53 10.1 Summary of Findings ........................................................................................................... 53 10.2 Impact Findings for Schools with Combined Literacy and Feeding Interventions .............. 53 10.3 Impact Findings for Schools with Feeding Interventions Only ........................................... 53 11. PROJECT ADAPTATIONS RESULTING FROM COVID-19.............................................. 57 11.1 COVID-19 and School Operations ...................................................................................... 57 11.2 COVID-19 and School Feeding Interventions ..................................................................... 58 11.3 COVID-19 and Health Services ........................................................................................... 59 12. EVALUATION CONSTRAINTS AND LIMITATIONS......................................................... 60 12.1 School Closures and Gaps in Project Delivery .................................................................... 61 12.2 Sample Size Added Challenges to Detecting Impacts of School Feeding Only .................. 61 12.3 Multiple Comparisons.......................................................................................................... 61 13. CONSIDERATIONS WHEN CONDUCTING EVALUATIONS DURING PANDEMICS AND NATURAL DISASTERS....................................................................................... 62 14. CONSIDERATIONS FOR MCGOVERN-DOLE PROGRAM SUSTAINABILITY BY STAKEHOLDERS............................................................................................................................. 63 14.1 Considerations for School Feeding ...................................................................................... 63 14.2 Considerations for Literacy.................................................................................................. 64 14.3 Considerations on Sustainability.......................................................................................... 65 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. 15. RECOMMENDATIONS FOR FUTURE MCGOVERN-DOLE PROGrAMMING............ 66 15.1 Expand Awareness of McGovern-Dole Program Theory of Change and Impact Evaluation Criteria ............................................................................................................... 66 15.2 Improving Teachers’ Capacity to Support Student Development of Early Reading Skills .................................................................................................................................... 66 15.3 Language of Early-Grade Literacy Development ................................................................ 67 15.4 Increase Gender Sensitivity and Improve Instruction Focusing on Girls ............................ 67 15.5 Ensure the Sustainability of McGovern-Dole Programming ............................................... 68 16. REFERENCES............................................................................................................................. 69 Appendix A: Supporting Exhibits........................................................................................................... 72 Appendix B: Data Collection Instruments ............................................................................................ 132 Abt Associates June 2022 ▌i ACKNOWLEDGEMENTS This impact evaluation would not have been possible without the financial support of the United States Department of Agriculture – Foreign Agriculture Service (USDA/FAS). We extend special thanks to the USDA/FAS team including Delphine Hamlin, who served as our contracting officer; Traci Johnson, Amy Elhadi, and Molly Kairn, who provided ongoing technical guidance and program support; and Almeida Zacarias, who supported our team from the USDA office in Maputo. We extend our gratitude to the Ministry of Education and Human Development and its Provincial and District Directorates for their contributions and advice, specifically during the design phase of the study. Our data collection efforts would not have been successful without their support. We thank all the school management teams, community members, and local government officials for the support they extended to our team, with a special recognition to all the students and teachers who voluntarily participated in the study. We deeply appreciate the support of numerous staff at World Vision, Planet Aid Inc., ADPP, RTI, and Counterpart International who provided ongoing valuable insights and advice throughout the evaluation. We acknowledge the dedicated work of the entire Ernst & Young team led by José Diogo, who were responsible for data collection, using safe data collection practices during the COVID-19 pandemic. We greatly appreciate the work of Maria Boyle, Ben Johns, and Christine Ohresser-Joumard and all Abt staff who contributed to the evaluation. We are profoundly grateful for the technical expertise and guidance provided throughout the entire evaluation by Barbara Goodson and David Judkins, Senior Advisors with Abt Associates. Lastly, we extend thanks to Aster Charmine Sitoe for supporting the data collection process and special thanks to Feliciano Chimbutane for providing invaluable technical guidance, critical insights, and program support throughout every step of the evaluation. Abt Associates June 2022 ▌ii LIST OF EXHIBITS Exhibit 1. Evaluation Questions.................................................................................................................. 16 Exhibit 2. Score Ranges and Implementation Adequacy Thresholds ......................................................... 26 Exhibit 3. Staff Hired for Data Collection .................................................................................................. 27 Exhibit 4. Baseline and Endline Evaluation Sample ................................................................................... 31 Exhibit 5. School Characteristics, by Study Arm and Time ....................................................................... 34 Exhibit 6. Parent Focus Group Sample with Literacy and School Feeding Treatment............................... 34 Exhibit 7. Parent Focus Group Sample with School Feeding Treatment Only ........................................... 34 Exhibit 8. Summary of Impact on Average Mean Reading Performance, Attendance, and Attentiveness 36 Exhibit 9. Impacts on Average Mean Reading Comprehension Scores ..................................................... 37 Exhibit 10. Impacts on Student Attendance (Measured by Any Absence Last Week) ............................... 39 Exhibit 11. Impacts on Teacher-Reported Student Attentiveness............................................................... 40 Exhibit 12. Impacts on Student Reading Performance and Attendance for Boys and Girls ....................... 42 Exhibit 13. Impacts on Instructional Inputs ................................................................................................ 44 Exhibit 14. Impacts on Feeding Outcomes ................................................................................................. 46 Exhibit 15. Impacts on Student Knowledge and Use of Recommended Health Practices ......................... 47 Exhibit 16. Proportion of Adequate Implementing Schools Included in Analyses .................................... 50 Exhibit 17. Number of Schools Without a Feeding Intervention at Endline .............................................. 50 Exhibit 18. Average Fidelity Scores Across School by Intervention Component and by Treatment Arm . 51 Exhibit 19. Impacts on Average Mean Reading Comprehension Scores (Adequately Implementing Treatment Schools Only) ............................................................................................................................ 54 Exhibit 20. Student Attendance (Measured by Any Absence Last Week; Adequately Implementing Treatment Schools Only) ............................................................................................................................ 55 Exhibit 21. Impacts on Teacher-Reported Student Attentiveness (Adequately Implementing Treatment Schools Only).............................................................................................................................................. 56 Exhibit 22. Percentage of Treatment Schools Where Students Received a Daily Meal During COVID-19 .................................................................................................................................................................... 58 Exhibit 23. Percentage of Treatment Schools Where School Garden Was Cared For During COVID-19 58 Exhibit 24. Percentage of Schools Where Families Received Take-Home Rations During COVID-19 .... 59 Appendix A Exhibit A1. McGovern-Dole Results Framework (SO1)............................................................................ 72 Exhibit A2. McGovern-Dole Results Framework 2 ................................................................................... 73 Exhibit A3. Program logic model with representation of project implementation adequacy and impacts . 74 Exhibit A4. School Feeding Fidelity Index ................................................................................................ 75 Exhibit A5. Literacy Fidelity Index ............................................................................................................ 76 Exhibit A6. Head Teacher and Teacher Training Fidelity Index ................................................................ 77 Exhibit A7. Sample Size, Data Source, and Level of Analysis by Outcome Indicator .............................. 78 Exhibit A8. Student Responses on Household Demographic Characteristics ............................................ 79 Exhibit A9. Language of Literacy Instruction and Difference with Language Spoken at Home ............... 80 Exhibit A10. Teachers Characteristics........................................................................................................ 81 Exhibit A11. Reading Comprehension Estimates for School Feeding Only Schools ................................. 82 Exhibit A12. Reading Comprehension Estimates for School Feeding and Literacy Schools ..................... 82 Exhibit A13. Listening Comprehension Estimates for School Feeding Only Schools ............................... 82 Exhibit A14. Listening Comprehension Estimates for School Feeding and Literacy Schools ................... 83 Exhibit A15. Letter Identification Estimates for School Feeding Only Schools ........................................ 83 Exhibit A16. Letter Identification Estimates for School Feeding and Literacy Schools ............................ 83 Exhibit A17. Syllable Identification Estimates for School Feeding Only Schools..................................... 84 Exhibit A18. Syllable Identification Estimates for School Feeding and Literacy Schools ......................... 84 Exhibit A19. Oral Reading Fluency Estimates for School Feeding Only Schools ..................................... 84 Abt Associates June 2022 ▌iii Exhibit A20. Oral Reading Fluency Estimates for School Feeding and Literacy Schools ......................... 85 Exhibit A21. Word Dictation Estimates for School Feeding Only Schools ............................................... 85 Exhibit A22. Word Dictation Estimates for School Feeding and Literacy Schools ................................... 85 Exhibit A23. Impact Estimates on Student Attendance (Student Self-Reported Absence At least One Day Last Week) in School Feeding Only Schools ............................................................................................. 86 Exhibit A24. Impact Estimates on Student Attendance (Student Self-Reported Absence At least One Day Last Week) in School Feeding and Literacy Schools ................................................................................. 86 Exhibit A25. Impact Estimates on Student Attentiveness as Perceived by Teachers in School Feeding Only Schools ............................................................................................................................................... 86 Exhibit A26. Impact Estimates on Student Attentiveness as Perceived by Teachers in School Feeding and Literacy Schools.......................................................................................................................................... 86 Exhibit A27. Teachers' Perception on Student Attentiveness..................................................................... 87 Exhibit A28. Impact on Teacher-Reported Instructional Input in School Feeding Only Schools .............. 87 Exhibit A29. Impact on Teacher-Reported Instructional Input in School Feeding and Literacy Schools .. 88 Exhibit A30. Impact on Feeding Outcomes in School Feeding Only Schools ........................................... 89 Exhibit A31. Impact on Feeding Outcomes in School Feeding and Literacy Schools ............................... 89 Exhibit A32. Impact on Student Knowledge and Health Practices in School Feeding Only Schools ........ 90 Exhibit A33. Impact on Student Knowledge and Health Practices in School Feeding and Literacy Schools .................................................................................................................................................................... 90 Exhibit A34. Impact on Students’ Reading Comprehension in School Feeding Only Schools, by Grade and Gender .................................................................................................................................................. 91 Exhibit A35. Impact on Students’ Reading Comprehension in School Feeding and Literacy Schools, by Grade and Gender ....................................................................................................................................... 92 Exhibit A36. Impact of Adequate Implementing Programs on Student Reading Comprehension ............. 93 Exhibit A37. Impact of Adequate Implementing Programs on Oral Comprehension ................................ 93 Exhibit A38. Impact of Adequate Implementing Programs on Letter Identification.................................. 94 Exhibit A39. Impact of Adequate Implementing Programs on Syllable Identification.............................. 95 Exhibit A40. Impact of Adequate Implementing Programs on Oral Reading Fluency .............................. 95 Exhibit A41. Impact of Adequate Implementing Programs on Word Dictation......................................... 96 Exhibit A42. Impact of Adequate Implementing Programs on Student Attendance .................................. 96 Exhibit A43. Impact of Adequate Implementing Programs on Student Attentiveness ............................... 96 Exhibit A44. Impact on Student Reported Access to School Supplies and Study Materials ...................... 97 Exhibit A45. Impact on Teacher Access to Supplies and Learning Materials ............................................ 98 Exhibit A46. Percentage of Teachers Reporting Access to School Supply Items ...................................... 98 Exhibit A47. Impact on Head Teacher Reporting Access to Library/Book Banks and Materials .............. 99 Exhibit A48. Percentage of Head Teachers Reporting Availability of Specific Types of Print Materials . 99 Exhibit A49. Impact on Teacher Reporting on Teacher Training ............................................................ 100 Exhibit A50. Teachers Reporting on Teacher Training ............................................................................ 101 Exhibit A51. Teacher Knowledge and Skills ............................................................................................ 102 Exhibit A52. Impact on Head Teacher Reported Teacher Training ......................................................... 103 Exhibit A53. Head Teacher Knowledge and Skills .................................................................................. 104 Exhibit A54. Percentage of Head Teachers Reporting Teachers Attended Literacy Training and WI or PAI Training ............................................................................................................................................. 104 Exhibit A55. Impact on Head Teacher Reported Number of Male and Female Teachers who Attended Literacy Training ...................................................................................................................................... 105 Exhibit A56. Impact on Head Teachers’ Reported Reading Coaches Training and Supervising Teachers, and Number of Trainings Reading Coaches Provided .............................................................................. 106 Exhibit A57. Percentage of Head Teachers Reporting on Frequency Reading Coaches Supervise Teachers .................................................................................................................................................................. 106 Exhibit A58. Impact of Head Teachers’ Reported Teacher Attendance and Head Teacher Attendance.. 107 Exhibit A59. Head Teacher Reporting on How Class is Handled when Teacher is Absent..................... 107 Exhibit A60. Impact on Students Reporting School Serving Schools Meals, Current Hunger, and Household Hunger .................................................................................................................................... 108 Abt Associates June 2022 ▌iv Exhibit A61. Percentage of Students Reporting that School Served School Meals Yesterday, and Frequency of Household Hunger .............................................................................................................. 109 Exhibit A62. Head teacher Reporting on School Feeding Program ......................................................... 109 Exhibit A63. Impact on Teacher-Reported Nutrition and Hygiene Teaching .......................................... 110 Exhibit A64. Teachers' Perception of Student Knowledge of Nutrition, Hygiene, and Sanitation........... 110 Exhibit A65. Impact on Head Teacher Reported Teacher Training on Nutrition, Hygiene, and Health .. 111 Exhibit A66. Head Teacher Reported Frequency of Nutrition, Hygiene, and Sanitation Lessons ........... 111 Exhibit A67. Impact on Students Receiving Deworming Medication ...................................................... 112 Exhibit A68. Head Teacher Reported Cleanliness of latrines ................................................................... 112 Exhibit A69. Cooks Reporting on School Infrastructure .......................................................................... 113 Exhibit A70. Cook Training and Knowledge ........................................................................................... 114 Exhibit A71. Percentage of Schools that Adequately Implemented Literacy, Training, and Feeding Program Items........................................................................................................................................... 115 Exhibit A72. Impact on Reading Comprehension by Literacy Item Adequately Implemented by Schools in the Treatment Arms.............................................................................................................................. 116 Exhibit A73. Impact on Reading Comprehension by Training Item Adequately Implemented by Schools in the Treatment Arms.............................................................................................................................. 117 Exhibit A74. Impact on Reading Comprehension by Feeding Item Adequately Implemented by Schools in the Treatment Arms .................................................................................................................................. 118 Exhibit A75. Impact on Student Attendance by Literacy Item Adequately Implemented by Schools in the Treatment Arms........................................................................................................................................ 120 Exhibit A76. Impact on Student Attendance by Training Item Adequately Implemented by Schools in the Treatment Arms........................................................................................................................................ 121 Exhibit A77. Impact on Student Attendance by Feeding Item Adequately Implemented by Schools in the Treatment Arms........................................................................................................................................ 122 Exhibit A78. Impact on Student Attentiveness by Literacy Item Adequately Implemented by Schools in the Treatment Arms .................................................................................................................................. 124 Exhibit A79. Impact on Student Attentiveness by Training Item Adequately Implemented by Schools in the Treatment Arms .................................................................................................................................. 125 Exhibit A80. Impact on Student Attentiveness by Feeding Item Adequately Implemented by Schools in the Treatment Arms .................................................................................................................................. 126 Exhibit A81. Outcome Levels at Baseline and Endline by Treatment Arm for Main Outcomes............. 129 Exhibit A82. Outcome Levels at Baseline and Endline by Treatment Arm for Instructional Inputs ....... 129 Exhibit A83. Outcome Levels at Baseline and Endline by Treatment Arm for Feeding Outcomes and Nutrition Knowledge ................................................................................................................................ 130 Exhibit A84. Outcome Levels at Baseline and Endline by Treatment Arm for Student Knowledge and Use of Recommended Health Practices........................................................................................................... 130 Abt Associates June 2022 ▌v ABBREVIATIONS ADPP Ajuda de Desenvolviemento de Povo para Povo BL Baseline CapED Capacity Development for Education Programme C Control Group Schools EGRA Early Grade Reading Assessment EL Endline EY Ernst & Young FAS Foreign Agriculture Service IFP Teacher Training Institution McGovern-Dole McGovern-Dole International Food for Education and Child Nutrition Program MINEDH Ministry of Education and Human Development PAI Planet Aid Inc. PFG Parent Focus Group PRONAE Programa Nacional de Alimentação Escolar SF School Feeding SF+L School Feeding and Literacy TTC Teacher Training College UNESCO United Nations Educational, Scientific, and Cultural Organization USAID United States Agency for International Development USDA United States Department of Agriculture WASH Water, Sanitation, and Hygiene WFP United Nations World Food Programme WV World Vision Abt Associates June 2022 ▌1 EXECUTIVE SUMMARY The United States Department of Agriculture’s (USDA) McGovern-Dole International Food for Education and Child Nutrition Program (McGovern-Dole) uses commodities grown by American farmers to support education, child development, and food security in low-income, food-deficit countries around the globe. McGovern-Dole projects support activities to address two critical issues for children in Mozambique: (1) food insecurity and hunger and (2) inadequate literacy skills for school achievement and life success. In an effort to reduce hunger and improve literacy among primary school children in Mozambique, McGovern-Dole, together with implementing partner organizations, provides food in schools, teacher training in effective literacy instruction, and other resources. To ensure sustainability of these project activities, as well as the targeted literacy, health, and nutrition outcomes, McGovern-Dole works to build the capacity of participant communities to sustain food sources and instructional quality on their own or by working with government agencies and local organizations. McGovern-Dole Projects in Mozambique USDA/Foreign Agricultural Service (FAS) supported a first-of-its-kind evaluation of the effectiveness of the McGovern-Dole interventions across multiple contexts within Mozambique. The impact evaluation, conducted by Abt Associates Inc., focused on schools in six districts: four in Maputo province (Magude, Manhiça, Matutuine, and Moamba) and two in Nampula province (Muecate, Nacaroa). The evaluation focuses on two McGovern-Dole projects in Mozambique, one implemented by Planet Aid, Inc. in Maputo province from 2015 to 2020 and one implemented by World Vision, Inc. in Nampula province from 2015 to 2021. These projects were the focus of a baseline assessment conducted by RTI International in 2017. McGovern-Dole supported activities expected to result in improved student performance in school. The McGovern-Dole projects supported increased school success by providing food at school and through nutrition and health interventions. Reducing hunger and food insecurity were expected to increase student attendance and student ability to pay attention during school. Some project schools also participated in interventions designed to increase student success directly, through higher quality literacy instruction, by providing teacher and school administrator training, curricular support, and access to school supplies. Improved attendance and attentiveness were expected to result in students having greater exposure to the higher-quality instruction. In Mozambique, the McGovern-Dole projects consisted of two different sets of interventions: schools either received school feeding plus literacy or they received school feeding only. In the four districts in Maputo province, both sets of interventions were implemented in the schools; in the two districts in Nampula province, only the school feeding plus literacy interventions were implemented. The decisions about which interventions were implemented in which districts and schools was made in 2012, before the decision to conduct an impact evaluation was made. Assignment to the interventions was therefore non-random. Evaluation Objective The purpose of this evaluation was to provide USDA/FAS with findings on the impacts of the two McGovern-Dole projects in Mozambique upon project completion. Through this impact evaluation, the study team was able to understand the role of different strategies used for McGovern-Dole interventions; strategies including community engagement, access to clean Abt Associates June 2022 ▌2 water and sanitation services, supportive community-based literacy, teacher training and support, and feeding activities. The findings in this report can be used to better understand the implementation of school feeding and literacy projects in Mozambique and will inform future McGovern-Dole programming in Mozambique, and, potentially, programming more broadly across countries. Evaluation design This evaluation assesses the impact of the two sets of interventions of the McGovern-Dole projects after four years of implementation. The evaluation focused on the outcomes for students in third grade (early-grade students), who had the opportunity for up to two years of exposure to the intervention(s), and students in fifth grade (mid-grade students) who had the opportunity for up to four years of exposure to the intervention(s). The impact evaluation was conducted across two projects in two provinces and was not designed to compare impacts by province or implementing partner. The impact evaluation compared average school-level performance at RTI International’s baseline assessment to average school-level performance at endline, four years later. Abt Associates used the same evaluation design, instruments, key outcomes, and research questions as in the baseline report to compare changes on key indicators from the project activities. The evaluation is quasi-experimental because schools were not randomly assigned to a version of the intervention. The ramification of this design versus a randomized design is that we cannot be certain that differences between intervention and control schools is entirely due to the difference in programming but may also be affected by other differences that might be related to the outcomes, some of which could be unplanned and unobservable, such as varying COVID-19 transmission rates across school communities or differential changes in school staff composition brought about by COVID-related turnover. The primary impact evaluation questions for the endline evaluation are about the effects of the McGovern-Dole projects in Mozambique on student outcomes: literacy, attendance, and attentiveness. Secondary evaluation questions ask about effects on teachers and schools. These questions are answered by separately comparing the students and teachers in schools in each of the two intervention arms, with the control arm. There are also exploratory impact evaluation questions about (1) intervention effects in a subset of children attending adequately implementing schools and (2) intervention effects for younger versus older students and for boys versus girls. Key findings on student outcomes • Effect on reading comprehension: Students in schools implementing the combined literacy and school feeding intervention improved their average reading comprehension scores relative to students in comparison schools. Early-grade students experienced an increase of 4.8 percentage points and mid-grade students experienced an increase of 7.6 percentage points in their reading comprehension. Students in schools implementing the school feeding only intervention did not improve average mean reading comprehension relative to students in comparison schools. Impact analyses using only students attending adequately implementing treatment schools found stronger impacts on student reading comprehension for both early-grade (0.6 percentage point increase) and mid-grade (3.3 percentage point increase) students compared to impacts in all treatment schools. Abt Associates June 2022 ▌3 • Effect on student attendance: No impact was observed on student attendance (absences) for either early-grade or mid-grade students who experienced either the combined school feeding and literacy intervention or the school feeding intervention alone. Unlike impacts on reading comprehension, adequately implementing schools in either treatment conditions (school feeding plus literacy and school feeding alone) did not improve average mean student attendance relative to comparison schools. • Effect on student attentiveness: Neither school feeding only nor school feeding and literacy improved student attentiveness for either early- or mid-grade students relative to comparison schools. For mid-grade students in school feeding only schools, teacher-reported student attentiveness declined by 7.2 percentage points relative to comparison schools. Adequately implementing schools in either treatment conditions (school feeding plus literacy and school feeding alone) did not improve average mean student attentiveness relative to comparison schools. Implementation findings • On average, McGovern-Dole projects were implemented at adequate levels of fidelity across intervention components, although there was variation in the fidelity scores across the treatment schools. • Overall, the schools in Nampula were rated as having better implementation of their interventions, compared to the schools in Maputo. Nampula schools implementing school feeding and literacy interventions received the highest scores on the evaluation’s implementation fidelity scales. • There was evidence that alternative programming of a similar nature to the McGovern-Dole program was concurrently being implemented in comparison schools. Because McGovern￾Dole schools had comparatively higher implementation scores, the contrast between treatment and comparison schools was detectable. More robust implementation in McGovern-Dole schools could produce a larger contrast to alternative programming, greater improvements in student outcomes, and stronger impacts overall. Considerations The positive findings on student literacy are very promising. Despite the diminished exposure on students and weakened training and supports for teachers around literacy instruction caused by COVID-related disruptions, the evaluation detected some positive findings. This is evident by the substantial positive impacts detected for teacher training on the fundamentals of reading instruction (15.3 percentage points for teachers of early-grade students and 27.9 percentage points for teachers of mid-grade students). These results suggest that teacher training is a plausible mechanism for explaining the impacts on student reading performance. Even in times of great disruption for schools as was the case between 2020 and 2021, training may be associated with changes in teacher practice, which is linked to increases in student literacy development. The lack of effects on attendance and attentiveness suggests that the impact on literacy was not because students were in school more often or more attentive but that the instruction was more effective. It is possible that the effects on student literacy would have been larger if the feeding intervention had resulted in improving these aspects of students’ readiness to learn. As discussed below, there were limitations affecting our ability to estimate the combined effectiveness of the two interventions delivered in combination. Abt Associates June 2022 ▌4 Limitations Interpretation of the evaluation findings is limited by several key factors. These include factors related to outcome measures, precision of the impact estimates, gaps in implementation, and disproportionate sample size. Outcome measures: Two of the primary outcome measures were based on survey responses that are subject to potential bias. Student attendance was measured with self-reported data from young children, and student attentiveness was measured using teacher recall in the teacher survey. In both cases, these measures could suffer from bias due to recall and measurement error. However, given that the survey included a substantial number of students and teachers, the effect of the expected errors could be minimized. For future evaluations, it is advisable to consider alternative measures of outcomes like student attentiveness to capture impacts at the student level more precisely. Relatedly, teacher-reported levels of student attentiveness were very high (greater than 90 percent in all arms) at baseline. This implies there is limited room for growth in this measure at endline. Precision of impact estimates: The evaluation interprets the estimates from the quasi￾experimental difference-in-differences analysis as the causal impact of the McGovern-Dole projects. The difference-in-differences design assumes the observed trend in the comparison schools can be used to project what the outcome would have been in treatment schools in the absence of treatment. This typically implies comparison schools do not experience any treatment. However, our assessment of implementation fidelity found that many comparison schools had some evidence of comparable feeding, literacy, and teacher training activities, although levels of implementation were determined to be low. For these schools, the implied comparison is not of McGovern-Dole interventions to no intervention, but of McGovern-Dole interventions to other programs being implemented. This could either attenuate or exacerbate the treatment effect and complicates interpretation of the average effect; however, if the conditional parallel trends assumption holds, it does not threaten the causal interpretation of the findings. Gaps in implementation: In Maputo, Planet Aid was contracted to implement the McGovern￾Dole project from 2015 to 2020. Counterpart was then awarded the contract in Maputo from 2020 to 2025. However, at the time of endline data collection in October 2021, Counterpart was still preparing to implement the project. School feeding and literacy interventions were not yet in place, creating a situation in which the study team was evaluating an intervention that had phased out a year before. Schools themselves were also functioning at a reduced level. When reopening in March 2021, after a year of COVID-19-related closures, many students were going to school only two or three times a week. Across provinces, a prevailing issue with the impact evaluation is that timing of data collection in 2021 proved problematic in detecting delivery of school feeding and other project elements. Even if project delivery such as the provision of school meals, supplemental role of school gardens, access to clean water, and use of literacy materials was well established up until school closures in March 2020, their presence diminished in Nampula and disappeared almost entirely in Maputo by endline. Because of this, intervention impacts may be difficult to detect, especially when capturing outcomes of interest like access to meals and student short-term hunger. When conducting a fair test of a project’s effectiveness, as was the goal of this evaluation, it is clear that project implementation matters. The lack of project implementation makes it difficult to understand impacts, as a lack of impact may be more likely due to the absence of the project rather than a flawed theory of change or ineffective project design. In this evaluation, given the Abt Associates June 2022 ▌5 unprecedented disruptions caused by the COVID-19 pandemic, we suspect that intervention impacts were likely weakened by project disruption and less than ideal implementation. Disproportionate sample sizes. Because it was only tested in Maputo, the school feeding only treatment arm had a smaller quantitative sample than the literacy and school feeding arm or the comparison group. This smaller sample implies impacts had to be larger in that treatment arm than in the literacy and school feeding arm to achieve statistical significance. Recommendations Given the interest in understanding the effects of the McGovern-Dole projects and the aim of sustaining project implementation, the study team offers the following recommendations: Expand stakeholder awareness of the McGovern-Dole program theory of change and engagement in identifying student and school outcomes of interest. Understanding the final goal of the McGovern-Dole program and the interconnection among the different components of the projects including desired outcomes would help ministry officials, school managers, teachers, and parents better monitor progress toward that goal. This understanding at the local level could encourage parents to advocate for stronger implementation and collaborate with school leaders and other stakeholders in meeting those goals. Improve teachers’ capacity to support student development of early reading skills. The study team recommends that teacher training increase focus on early-grade literacy instruction, allowing teachers to better understand the casual relationship between lower- and high-order reading skills and enhance their capacity to help children develop these two levels of reading growth. We also suggest strengthening the connection between the Pedagogical University and teacher training activities with local schools. Expand dual language of early-grade literacy development. McGovern-Dole projects could further support the scaling of dual language early literacy instruction inclusive of languages familiar to the child (their home languages). McGovern-Dole projects may help local education authorities to expand the bilingual program to more schools or to cover more classes in schools where it is already in place. Increase gender sensitivity and improve instruction focusing on girls. McGovern-Dole projects work to improve the environment that allows girls to have equal opportunities for learning in schools, homes, and larger communities. We suggest looking more closely at the implementation of programming for girls to determine if community members understand the importance of girls’ education, and training teachers on gender-sensitive classroom interaction and reading practices. We also suggest enhancing the gender-sensitivity approach with the production of teaching and learning materials and in the selection and hiring practices for school leadership positions. Ensure the sustainability of McGovern-Dole programming. The study team encourages USDA/FAS, implementing partners, ministry officials, and others to put in place strong exit strategies to allow sustainability of the McGovern-Dole interventions after the end of designated projects, including through increased involvement of local education authorities and establishment of linkages with similar government programs/interventions. Abt Associates June 2022 ▌6 1. OVERVIEW OF EVALUATION AND KEY FINDINGS The United States Department of Agriculture – Foreign Agriculture Service (USDA/FAS) supported an evaluation of McGovern-Dole International Food for Education and Child Nutrition (McGovern-Dole) in Mozambique. This evaluation, conducted by Abt Associates Inc., focuses on impacts from 2015 to 2020 in four districts of Maputo province (Magude, Manhiça, Matutuine, and Moamba) and two districts of Nampula province (Muecete and Nacaroa). These projects were the focus of a baseline evaluation conducted by RTI International in 2017. The report aims to evaluate the impact of two McGovern-Dole interventions: literacy and school feeding interventions combined and a school feeding intervention alone. The evaluation estimated impacts at endline, after four years of implementation, and reports program impacts between baseline and endline. The evaluation focused on the outcomes for students in grade 3 (early-grade students) who had two years of exposure to the intervention and students in grade 5 (mid-grade students) who had experienced four years of the intervention. The evaluation was conducted in six districts: four districts in Maputo province, where both interventions were implemented, and two in Nampula province, where only the school feeding intervention was implemented. Key findings of this evaluation include: • Effect on reading comprehension: Students in schools implementing the combined literacy and school feeding intervention improved their average reading comprehension scores relative to students in comparison schools. Early-grade students experienced an increase of 4.8 percentage points and mid-grade students experienced an increase of 7.6 percentage points in their reading comprehension. Students in schools implementing school feeding alone did not improve average mean reading comprehension relative to students in comparison schools. • Effect on student attendance: No impact was observed on student attendance (absences) for either early-grade or mid-grade students who experienced either the combined school feeding and literacy intervention or the school feeding intervention alone. • Effect on student attentiveness. Neither school feeding nor school feeding and literacy improved student attentiveness for either early- or mid-grade students relative to comparison schools. For mid-grade students in school feeding only schools, teacher-reported student attentiveness declined by 7.2 percentage points relative to comparison schools. • Impacts for schools with adequate fidelity of implementation. On average, McGovern￾Dole projects were implemented at adequate levels of fidelity, although there was variation in the fidelity scores across the treatment schools. Impact analyses using only adequately implementing treatment schools found stronger impacts on student reading comprehension for both early grade (0.6 percentage point increase) and mid-grade (3.3 percentage point increase) students compared to impacts in all treatment schools. • Impacts on teacher training. Substantial positive impacts were detected for teacher training on the fundamentals of reading instruction (15.3 percentage points for teachers of early-grade students and 27.9 percentage points for teachers of mid-grade students) suggesting that teacher training is a plausible mechanism for explaining the impacts on student reading performance. Abt Associates June 2022 ▌7 2. BACKGROUND 2.1 Benefit of this Evaluation The purpose of this evaluation is to provide USDA/FAS with findings on the impacts of two of its McGovern-Dole projects in Mozambique upon project completion. Through this endline impact evaluation, the study team was able to understand the role of different strategies used for McGovern-Dole activities; strategies including community engagement, access to clean water and sanitation services, supportive community-based literacy, teacher training and support, and feeding activities. The findings in this report can be used to better understand the implementation of school feeding and literacy projects in Mozambique and will inform future McGovern-Dole programming in Mozambique, and, potentially, programming more broadly across countries. 2.2 Mozambique Context Mozambique has made progress over the past decade in improving both the literacy and nutrition of its population. Nevertheless, rates of illiteracy and malnutrition remain high, particularly in rural areas and among women. In addition, during the period of the McGovern-Dole projects being assessed by this impact evaluation (2015–2021), the country was hit with major events, including the COVID-19 pandemic and Tropical Cyclone Eloise. These events had the potential to negatively influence progress toward improving literacy and nutrition of school-age children. Education in Mozambique In most provinces in Mozambique, schools are overcrowded, student/teacher ratios are high, and reading and math test scores are low. One study on school operations found that teacher absenteeism, limited instructional time, and other factors limited Mozambican schools to, on average, 30 days of actual instructional time per the 193-day school year in 2010 (USAID, 2021a). Furthermore, the study found that 59 percent of third grade students in the 49 schools studied could not read a single word per minute nor recognize letters, and that students who could read were able to read, on average, only five words per minute. More than one-third of students drop out before Grade 3 and less than half complete primary school (Global Partnership, 2021). Mozambique’s overall adult literacy rate is 47 percent but with a significant gap between male (60 percent) and female (28 percent) rates (USAID, 2021a). Nutrition in Mozambique Many Mozambicans, particularly children, suffer from food insecurity, including poor diet quality. Diets in Mozambique consist mainly of starchy staple foods, which results in limited intake of key micronutrients. This contributes to high levels of stunting. In rural areas in the northern part of the country – which also has the highest levels of stunting – the World Food Programme (WFP) found that households consume mostly maize, with additional food groups such as oils/fats and green leafy vegetables consumed as infrequently as 2–3 days per week. In 6 out of Mozambique’s 11 provinces, more than 40 percent of children under 5 are stunted (USAID, 2021b), and Nampula province has the highest level of stunting, at 55 percent (USAID, 2014). Water, Sanitation and Hygiene (WASH) in Mozambique Mozambique has made significant strides in terms of access to water, sanitation and hygiene (WASH) services, although the level of deprivation continues to be a matter of concern. Only 61% of Mozambicans have access to improved water supply and only 36% have improved sanitation facilities (WHO/UNICEF, 2021). Access to water supply improved by 12 percentage points between 2015 and 2020 and access to improved sanitation facilities by 8 percentage points Abt Associates June 2022 ▌8 in the same period. According to World Bank (2018: 27), “about 20 percent of the substantial disease burden in Mozambique is linked to poor sanitation and unsafe water.” WASH-related diseases include diarrhea, dysentery, cholera, respiratory infections, and parasitic infections, which are responsible for most of the deaths in Mozambique, particularly among children. Assuming that WASH is an integral part of learning environment and key to school achievement, the McGovern Dole Program includes initiatives that aim at improving WASH infrastructures and practices in target schools in Mozambique. 2.3 McGovern-Dole International Food for Education and Child Nutrition Program The McGovern-Dole program is structured around two strategic objectives. Strategic Objective 1 represents a theory of change linking the improvement of student literacy to improvements in reading instruction, student attentiveness in class, and student attendance at school (Exhibit A1). Improvements in teacher behaviors, instructional materials, student hunger, economic incentives, and school infrastructure are some program components hypothesized to be causally linked to student literacy, attentiveness, and attendance. Strategic Objective 2 proposes use of health and dietary practices as a pathway to reduce health-related absences, which improves school attendance (Exhibit A2). As a result, the McGovern-Dole program takes steps to improve student literacy by (1) enhancing teacher and school administrator training, curricular support, and access to school supplies; (2) reducing short-term hunger and improving student attendance and attentiveness by ensuring greater access to food; (3) supporting student attendance and attentiveness through nutrition and health interventions. Projects in Maputo and Nampula provinces were constructed around this theory of change represented in the strategic objectives. In Maputo province, where the project was implemented by Planet Aid, Inc. (PAI) in partnership with Ajuda de Desenvolviemento de Povo para Povo (ADPP), four districts were included in the McGovern-Dole project. For literacy instruction, PAI/ADPP conducted early-grade reading and literacy interventions, including providing 63 schools materials in either Rhonga or Changana, to help the schools provide improved instruction in local languages. For attentiveness and attendance, PAI/ADPP provided daily meals, school gardens, safe water supply, and nutrition education activities. In Nampula province, where the project was implemented by World Vision, two districts were included in the McGovern-Dole project. World Vision’s literacy activities emphasized the use of literacy assessments and curriculum analysis; access to materials such as children’s books and other supplies; and enhanced community engagement in literacy activities, such as reading clubs. For attentiveness and attendance, World Vision provided health and nutrition training, supported the construction of water infrastructure, helped with medical delivery and services, and constructed savings and lending groups to invest in school feeding activities. To reduce short-term hunger in students, World Vision provided school meals. World Vision activities also included training of teachers, school administrators/education officers, construction of classrooms and latrines, formation and training of farmer groups, and formation of Savings and Loans Associations. This evaluation offers a robust assessment of the hypothesis above – that is, whether these interventions indeed improved conditions for literacy development and students’ literacy in general. We use a mixed methods design to quantitatively examine the causal impact of the intervention on desired outcomes, as well as qualitatively understand the “what” and “why” of implementation and outcomes in schools during the intervention period. This provides a clearer window into the effects on literacy development, not only because it isolates the causal impact in general, but also because this analysis allows us to examine the effects of school feeding interventions alone – for instance, food or public health programming (in the school feeding only Abt Associates June 2022 ▌9 arm) – on literacy development, as well as the effects of combining this programming with more direct literacy support (in the school feeding plus literacy arm). Exhibit A3 presents the associated logic model with inclusion of project-specific components. This model incorporates project implementation adequacy with resulting impacts on key outcomes of interest. This evaluation focused on the Phase II interventions of both projects, which were implemented from 2015 to 2020 in Maputo and 2015 to 2021 in Nampula. As such, it is also the first of its kind to evaluate the impact of McGovern-Dole projects across multiple contexts within Mozambique. 2.4 Summary of Other Literacy and School Feeding Projects Implemented in the Maputo and Nampula Provinces In August 2021, the study team conducted a desk review of existing documentation and conducted stakeholder interviews to better understand both the needs and support projects currently in the Mozambique region. Multiple projects are implemented in the region; some are associated with McGovern-Dole while others have alternative host agencies and funding sources. The review and interviews revealed a few other literacy and school feeding projects in the target provinces. The initial delivery and sustainment of services is a benefit to the local communities but may complicate the detectable contrast between McGovern-Dole schools and comparison schools involved in this evaluation. These alternative projects include the USAID-funded Vamos Ler! project that was implemented in the provinces of Nampula and Zambezia, the Capacity Development for Education (CapED) Programme, under UNESCO, that is being piloted in Mozambique to develop a new curriculum for primary education for youths and adults, and the United Nations WFP take-home rations program. These initiatives are described below. Vamos Ler! (Let’s Read!) was a five-year project (2016–2021) funded by USAID to develop bilingual education pedagogical tools and activities, improve national early grade literacy policies and delivery and monitoring systems, enhance school leadership and increase parental and community engagement in early grade literacy. Let’s Read! assisted the government in delivering a quality bilingual education program for first, second and third grade students in the provinces of Nampula and Zambezia. The project worked alongside government counterparts to develop instructional tools and learning materials in the local languages (Emakhuwa, Elomwe, and Echwabu) with oral language development and transition to Portuguese reading and writing in the fourth grade. Creative Associates International implemented the program in partnership with World Education, Inc., American Institutes for Research, Overseas Strategic Consulting, and Blue Tree Group. CapED is working alongside the government to develop a new curriculum for primary education for youths and adults and builds the capacities of teacher-trainers who will go on to instruct teachers of non-literate adults and youths. CapED is also supporting the Family Learning Programme, which was originally developed by the UNESCO Malala Fund for Girls’ Right to Education. CapED enriched the program by developing the content of its learning manual, and supported the training of literacy teachers to use this manual. The program aims to equip non￾literate families with literacy, numeracy, and life skills, such as hygiene techniques, nutrition, and parenting. The program also aims to address the scarcity of early childhood education in Mozambique, as only 4 percent of under 5-year-olds receive this type of education. The program therefore equips parents with the skills they need to support the early development of their children and prepare them for primary school education through joint learning. The Family Learning Program is currently being rolled out in two provinces, Nampula and Maputo. Abt Associates June 2022 ▌10 The WFP’s take-home rations program has reached over 100,000 beneficiaries in Nampula with take-home rations in the districts of Malema, Nacala-a-Velha, and Ribaue. This assistance is part of a five-years partnership WFP established with the company Nacala Logistics to expand school feeding in the province. The baskets are directed at the students from these schools and their families as an adaptation to the closure of schools. WFP works closely with implementers of school feeding in Mozambique and the Ministry of Education and Human Development (MINEDH) to discuss the sustainability of its national home-grown school feeding program (Programa Nacional de Alimentação Escolar - PRONAE) through the creation of specific legislation and a budget line from the national budget. 2.5 Baseline Assessment In 2016, USDA contracted RTI International to design an independent impact evaluation of McGovern-Dole Mozambique and to conduct a baseline assessment. The baseline assessment provides a snapshot of the situation in 2017 after some programming began and was not an evaluation of project impact. Rather, the baseline assessment was designed to support this quasi￾experimental impact evaluation which is intended to capture changes in student and school outcomes between 2017 (baseline) and 2021 (endline). Its results were intended to inform decision makers about the situation in Nampula and Maputo and share with the implementing partners a wider vision of the overall status of student reading ability, student attendance, student nutrition practices, student nutrition and health knowledge, and teacher and head teacher attendance in these provinces. This quasi-experimental evaluation only captures impacts between baseline and endline; a resulting limitation is that any impacts of programming that occurred prior to the baseline assessment are not captured in this evaluation’s impact estimates. The school-based survey used to obtain the results was conducted in the third trimester of the school year to allow for adequate exposure of the students to standard education and school meals. The instruments were developed to address the components of the theory of change, calculate the relevant indicators, address the research questions for the evaluation, and assess the similarities and differences between the intervention and comparison schools at baseline. To assess literacy, enumerators administered an Early Grade Reading Assessment (EGRA) tool, a 15-minute oral assessment that measures the basic skills a child must have to eventually be able to read fluently and with comprehension. A student survey was conducted that included questions on student attendance, school feeding activities, and nutrition and hygiene knowledge and practices. Teachers, head teachers, and school cooks were interviewed to obtain information on teacher attendance, school amenities and the school feeding activities. The survey was paired with qualitative data collection, which provides supplemental insight from parents about McGovern-Dole. In February 2018, the baseline report was completed. It included student reading ability, student attendance, student nutrition practices, student nutrition and health knowledge and teacher and head teach attendance. The baseline report included the following recommendations for future evaluations: • Continue tracking activities in both intervention and comparison schools within sampled districts • Invest additional effort in matching intervention and comparison schools, students, and teachers in future analyses • Conduct a meta-analysis to account for variance across implementers and provinces Abt Associates June 2022 ▌11 • Use the proposed difference-in-difference methodology to account for the significant differences between the comparison and intervention arms at baseline and track the rate of progress of these schools over time 2.6 Other Evaluations Conducted by Implementing Partners Implementing partners have conducted evaluations of the McGovern-Dole projects in Mozambique with markedly promising results. Evaluations of phases I and II of the McGovern￾Dole projects in Nampula province conducted by COSDER Consultants (Phase I) and International Food Policy Research Institute (Phase II), for instance, examined World Vision’s implementation of the McGovern-Dole projects in 2012–2015 and 2015–2021, respectively. They found that the participation in McGovern-Dole resulted in a 6.5 percentage point increase in third-graders’ performance in the Portuguese language. Moreover, Dr. Simone Doctors evaluated PAI’s implementation of the McGovern-Dole project from 2015 to 2020 in Maputo (Doctors, 2021). Again, by midline, over one-quarter of students met the literacy benchmark. Most recently, Counterpart International is implementing the McGovern-Dole project in Maputo from 2020 to 2025, with results to be assessed by Maraxis. World Vision also implemented the Local and Regional Food Procurement Program in Mozambique from 2017 to 2019. While this did not assess literacy outcomes, it was reported that “the program was able to achieve most of its output targets on an aggregate level” (ICED, 2019). All four evaluations used a similar mixed methods approach to this study – most typically, a combination of semi-structured interviews, focus groups, quantitative surveys, and EGRA scores. However, they differ in analysis. International Food Policy Research Institute and Maraxis’s evaluations both use multi-level modelling when evaluating EGRA scores, allowing them to assess causality on assessment outcomes, whereas other evaluations did not. The midline report for Phase II of World Vision’s implementation notes, for instance, that while the results may be positive, “student performance is not only explained by [the program] but also by other variables” not accounted for the model. (p. 44). Phase I of World Vision’s implementation of the McGovern-Dole project was evaluated through cross tabulation and correlations, rather than regression models that could control for other factors. The remainder – including the Local and Regional Food Aid Procurement Program and Food for Education Phase II – are only evaluated through comparing endline outcomes across certain indicators, but not necessarily their statistical significance. 2.7 Prior Evidence on School Feeding in Mozambique (or in the Sub-Saharan Region) The endorsement of PRONAE, the national school feeding program, by the Mozambican Council of Ministers in 2013 represented a key stage in the institutionalization of school feeding and local food purchases in the country’s education system (Milhorance de Castro, 2018). PRONAE’s pillars emphasized the responsibilities of the education sector, with less emphasis placed on coordination with agricultural production and local purchases. These pillars included: (1) improving the nutritional and health status of students; (2) food and nutrition education in schools; and iii) developing students’ agricultural production skills. The assessment of the pilot phase of PRONAE conducted in 2018 indicated that the introduction of a school feeding program in Nampula province appeared to have a positive influence on the increased enrollment, attendance, and retention of primary school learners. Schools without school feeding programs failed to retain learners (Sitao, 2018). The McGovern-Dole projects in Mozambique began in late 2012 supporting projects in Maputo and Nampula provinces. The first phase of the project was from 2012 to 2015 in Nampula and Abt Associates June 2022 ▌12 2012 to 2016 in Maputo. The evaluation report on the project in Nampula done at the end of Phase I reported that school feeding contributed to increased enrollment and retention rates and improved health and dietary practices (World Vision Mozambique, 2016). As a result of the COVID-19 pandemic, the final evaluations for the second phase (2015–2021) of the McGovern-Dole projects in Maputo and Nampula provinces encountered challenges collecting data and interpreting results. However, the mid-term evaluations for both projects reported the projects were beginning to have impact. The project in Nampula reported notable achievements in increasing literacy, enhancing student and teacher performance, and reducing hunger at the point of the midline evaluation (Diogo, 2018; Doctors, 2019). School feeding programs in sub-Saharan Africa have been in place for decades and are currently embedded in national policies. The Global School Feeding Sourcebook, developed in 2016, includes case studies on several school feeding programs in sub-Saharan Africa including Botswana, Cote d’Ivoire, Kenya, Mali, Namibia, and Nigeria. Although quantitative data were not collected for evaluating the impact of most of these programs, anecdotal evidence from stakeholders indicated growth in enrollments and school attendance rates that are highly associated with the availability of food at school. During the past five years, evaluations have been conducted on McGovern-Dole projects in sub-Saharan Africa, including Tanzania (Vyamanam, Ntunda, and Dulla, 2017), Malawi (Tirivayi, 2019), Rwanda (Ravesloot et al., 2021), and Mali (Sarfarha et al., 2020). Although not all the evaluations were able to assess improved literacy, attentiveness, and attendance, most of the evaluations were able to establish that project implementation contributed to ongoing student enrollment and attendance. 2.8 Prior Evidence on Teacher Pedagogy, Training, and Remedial Education in Mozambique (or in the Sub-Saharan Region) According to the midline results of the USAID-funded Vamos Ler! program, implemented from 2016 to 2021 in Nampula and Zambezia provinces, the program improved student reading skills (Turney et al., 2020). Compared to the program baseline, student performance improved substantially for all EGRA subtasks. This was true for all three Mozambican languages assessed (Emakhuwa, Elomwe, and Echwabu). The improvements were in large part due to a substantial reduction in the percentage of students registering zero scores on the EGRA subtasks. The largest improvements were seen in letter name and letter sound identification subtasks, with more modest gains in word and text reading. The midline results also noted positive reception of bilingual education in classrooms and communities. Prior evidence in Mozambique showed that bilingual education has been transformative. Chimbutane (2011, 2015) shows that using local languages for teaching and learning has enabled classroom interaction and contributed to bridging the gap between rural bilingual schools and pupils’ communities in Mozambique. Consistent with the United Nations’ Sustainable Development Goal 4, improving literacy outcomes in sub-Saharan Africa has been a central focus of national governments, donors, and nongovernmental organizations. Although significant international development funding has been devoted to teacher training in the region, little evidence is available on how teachers improve their literacy instruction in practice (Lee, J., D'sa, N., and Zuilkowski, S. S. (2020). 2.9 Quasi-Experimental Impact Evaluation In late June 2021, USDA/FAS contracted with Abt Associates to conduct an endline quasi￾experimental design impact evaluation of McGovern-Dole projects in Mozambique. Abt Abt Associates June 2022 ▌13 Associates did not have a role in the implementation of the McGovern-Dole projects. Abt Associates mirrored the evaluation conducted at the baseline in order to compare changes on key indicators from the project activities. The evaluation design, instruments, key outcomes measured, and research questions remained the same as they were at baseline. However, the endline evaluation design contains modifications to better reflect and capture the actual project activities and the adequacy of the implementation of activities as planned. 2.10 Differences Between Baseline and Endline Approaches Target grades The initial evaluation design was to assess the reading performance of second grade and fourth grade students, four years after the start of implementation. At that point, the students in second grade would have been in their schools for two of the four years and the students in fourth grade would have been in their respective schools for all four years. The COVID-related school closures shut down the schools from March 2020 to March 2021. This meant that the second grade students experienced only one year and two months of exposure while the students in fourth grade experienced only three years and two months of exposure. With agreement from USDA, the Abt research team assessed students one year older (third grade), who had been in their schools for an additional year for a full two years of exposure. Instead of assessing fourth graders for the endline assessment, the Abt team assessed students one year older (fifth grade), who had been in their schools for an additional year. Language of Assessment Mozambique has both monolingual and bilingual education, depending on the school and district, and in some cases, Portuguese does not become the language of instruction until third grade. This was not the case during the baseline assessment, when all teaching was done in Portuguese. By the end of the implementation activities, all districts in Nampula province continued to have language instruction in Portuguese. However, in some districts in Maputo province, students were offered bilingual education in both local languages and Portuguese; for the midline evaluation, the independent evaluator conducted EGRA assessments in local languages or Portuguese, as was most relevant for the intervention. In each school where bilingual instruction has been identified, we assessed a random selection of eight third grade students in the appropriate local language (Rhonga or Changana). We assessed a random selection of eight fifth grade students in Portuguese. Ultimately, we tested 441 students in a local language in 31 Maputo schools across 445 sample schools. All but eight of these students were in the third grade. Most local language assessments were conducted in Changana (323) compared to Rhonga (118). Instrument Enhancements To maintain consistency across baseline and endline evaluations, the study team used baseline instruments. To capture more details on implementation including and beyond the effects of COVID-related school closures, we added several open-ended questions to head teacher and cook questionnaires as well as adding items to parent focus group (PFG) discussions and the interviews with the MINEDH. These supplementary items focused primarily on the implementation of McGovern-Dole and contextual factors that may have influenced implementation. Abt Associates June 2022 ▌14 2.11 COVID-19 Effects on Schooling The planned implementation of the project activities over the four-year period from 2017 through 2021 was disrupted by school closures related to COVID-19. These closures affected nearly 70 percent of students in over 160 countries worldwide.1 In Mozambique, over 6.9 million primary school students were shut out of in-person schooling for a year (March 2020–March 2021) (UNICEF, 2020). The closures meant that students attending the study schools will have had fewer years of exposure to their study condition than was expected. Furthermore, for most students in the two mostly rural provinces in this evaluation, alternative delivery of instruction via online education was not a possibility in the country due to lack of access to computers and, even more basic, to electricity. 1 Education: From disruption to recovery (unesco.org) Abt Associates June 2022 ▌15 3. EVALUATION APPROACH The evaluation design is informed by the McGovern-Dole Results Frameworks (Exhibit A1 and Exhibit A2), which includes two strategic objectives. Strategic Objective 1 identifies improved literacy, quality of literacy instruction, improved attentiveness, and improved attendance as interrelated primary objectives. Strategic Objective 2 identifies use of health and dietary practices as a pathway to reduce health-related absences, which improves school attendance. 3.1 Impact and Implementation Focus The impact evaluation used a quasi-experimental difference-in-difference design, comparing average school outcomes at project beginning to average school outcomes at project end, four years later. Alternative impact analyses are conducted on a subset of schools with adequate fidelity; this analysis is based on an assessment of the level of implementation in each school, based on descriptive information on level of fidelity of implementation collected as part of the evaluation. This approach allows us to generate exploratory evidence on both the “what” and “why” of implementation and outcomes in schools during the intervention time period. This approach also supports a clear and robust understanding of what happened as planned in schools during the intervention time period, what did not happen as planned, and why. The mixed methods approach takes into account the importance of the quality of the implementation when testing the impact of an intervention. The results of our approach provide detailed qualitative data to help to interpret the data from the impact evaluation, which is important given the four￾year time period from baseline to endline. Conducting the impact evaluation across two projects in two provinces allows this report to describe common lessons learned across provinces in Mozambique. This report provides information on how to estimate effects on student literacy performance, such as the effect of COVID-19, which is especially relevant since many project-level evaluations will face challenges implementing literacy testing as planned. 3.2 Evaluation Questions The primary impact evaluation questions for the endline impact evaluation are about the effects of the McGovern-Dole projects on improved literacy of students and increased student attendance and attentiveness. Secondary evaluation questions ask about effects on teachers and schools (Exhibit 1). These questions will be answered by separately comparing the students, teachers, and schools in each of the two intervention arms with the control arm. There are also exploratory impact evaluation questions about how the effects vary as a function of level of implementation of the McGovern-Dole projects. These questions will be addressed by conducting exploratory analyses estimating impacts of the two treatment arms including only the subset of schools showing adequate fidelity of implementation of their assigned project(s). Finally, there are also descriptive evaluation questions about implementation, parent perceptions, as well as sustainability and recommendations that will be answered descriptively, using qualitative data. Abt Associates June 2022 ▌16 Exhibit 1. Evaluation Questions Primary Evaluation Questions about Impacts on Students How does reading ability differ for Grade 3 and 5 students who attend: 1. schools with the McGovern-Dole Literacy and School Feeding activities, 2. schools with the McGovern-Dole School Feeding Only activities, and 3. schools without any McGovern-Dole activities (Comparison)? How does attendance differ for Grade 3 and 5 students in: 1. schools with the McGovern-Dole Literacy and School Feeding activities, 2. schools with the McGovern-Dole School Feeding Only activities and 3. schools without any McGovern-Dole projects (Comparison)? How does attentiveness differ for Grade 3 and 5 students in: 1. schools with the McGovern-Dole Literacy and School Feeding activities, 2. schools with the McGovern-Dole School Feeding Only activities, and 3. schools without any McGovern-Dole projects (Comparison)? Secondary Impact Evaluation Questions about Mediators Factors relating to student hunger and diet How does the level of short-term hunger differ for Grade 3 and 5 students who attend: 1. schools with a McGovern-Dole School Feeding project, and 2. schools without any McGovern-Dole projects (Comparison)? How does dietary intake/access to food differ for Grade 3 and 5 students who attend: 1. schools with a McGovern-Dole School Feeding project, and 2. schools without any McGovern-Dole projects (Comparison)? How does access to food preparation and storage facilities differ for: 1. schools with a McGovern-Dole School Feeding project, and 2. schools without any McGovern-Dole projects (Comparison)? Is there a reduction in health-related absences in: 1. McGovern-Dole schools and 2. non McGovern-Dole schools. Factors relating to student reading ability How does availability of literacy materials and resources differ for Grade 3 and 5 students who attend: 1. schools with the McGovern-Dole Literacy and School Feeding activities, 2. schools with the McGovern-Dole School Feeding Only activities and 3. schools without any McGovern-Dole projects (Comparison)? How does teacher training availability and attendance differ for Grade 3 and 5 teachers in: 1. schools with McGovern-Dole Literacy and School Feeding activities, 2. schools with the McGovern-Dole School Feeding Only activities and 3. schools without any McGovern-Dole projects (Comparison)? How does teacher attendance of Grade 3 and 5 teachers differ across: 1. schools with the McGovern-Dole Literacy and School Feeding activities, 2. schools with the McGovern-Dole School Feeding Only activities and 3. schools without any McGovern-Dole projects (Comparison)? Factors Related to Student Knowledge/Use of Recommended Health and Nutrition Practices Abt Associates June 2022 ▌17 How does knowledge of recommended health and nutrition practices differ for Grade 3 and 5 students who attend: 1. schools with a McGovern-Dole School Feeding project, and 2. schools without any McGovern-Dole projects (Comparison)? How does use of recommended health and nutrition practices differ for Grade 3 and 5 students who attend: 1. schools with a McGovern-Dole School Feeding project, and 2. schools without any McGovern-Dole projects (Comparison)? Implementation Evaluation Questions To what extent were McGovern-Dole project activities in schools in the intervention arms implemented as planned? To what extent were school operations interrupted by external school events? • How did these interruptions affect the project implementation (i.e., especially COVID-19) that may have affected a change in dosage of intervention received by schools in the two intervention arms? What are parents’ perceptions of the McGovern-Dole School feeding and literacy programs? • How did the project affect what parents did at home, if at all? Were the issues or concerns identified by parents at baseline addressed by the project and were there changes in parental perceptions by endline? When project activities were modified, to what extent were these adapted activities considered successful by project participants? In what ways could provincial and local efforts aid in program sustainability? Differential Impacts by Level of Implementation Are impacts greater among the subset of schools assessed as having adequate fidelity of implementation of their assigned McGovern-Dole programs? • Among students attending schools assigned to implement the McGovern-Dole School Feeding only, are there differences in the reading ability, attentiveness, and attendance among Grade 3 and 5 students who attend schools with high implementation adequacy and students who attend schools with low implementation adequacy? • Among students attending schools assigned to implement the McGovern-Dole Literacy and School Feeding activities, are there differences in the reading ability, attentiveness, and attendance among Grade 3 and 5 students who attend schools with high implementation adequacy and students who attend schools with low implementation adequacy? Abt Associates June 2022 ▌18 4. METHODOLOGY 4.1 Sampling Frame The sampling frame for the treatment arms was developed based on school and enrollment data provided by the implementing partners of the McGovern-Dole projects. The sample of schools in the evaluation were selected at the time of the baseline evaluation. At the time of the baseline assessment, the McGovern-Dole projects were implemented in 273 schools across four districts in Maputo and in 150 schools across two districts in Nampula (RTI Baseline Report, 2018). This evaluation uses a three-stage stratified clustered sampling design. At baseline, the provinces and districts were selected in a non-random process to represent treatment arms based on services schools were receiving. Within the province of Maputo and Nampula, schools were stratified by treatment arm and sampled with equal allocation. As noted in the RTI Baseline Report (2018), the use of equal allocation rather than proportional allocation was used because school enrollment data were missing for many of the schools in the frame and the enrollment data were believed to be inaccurate after comparison with other sources. The sampling frame for the comparison group was developed using Education Management Information System data. The selection of comparison districts was based on a number of criteria including: location in the province, no known literacy intervention, geographically close to districts with intervention schools, similar urban-rural divide as the intervention districts; similar number of students and schools; similar languages spoken at home. The schools in the districts that met these criteria were automatically selected into the comparison group. Note, the city of Maputo was not selected because it is primarily urban (RTI Baseline Report, 2018). Classrooms and students within schools were sampled as part of the data collection in schools. Within each school, one second grade and one fourth grade classroom were selected. Selection of the study classrooms was random if the school had more than one classroom in either second grade or fourth grade. Because of COVID-19-related school closures from March 2020 to March 2021, students missed a year of instruction. With agreement from USDA, the Abt research team collected endline data from third grade students for the impact evaluation. By October 2021, third grade students had been in their schools for an additional year, for a full two years of exposure. Abt also collected endline data from fifth grade students for impact evaluation. These students had also been in their schools for an additional year, mitigating the loss of a year of schooling. At endline, we used the same approach to select eight students in third grade classrooms and eight students in fifth grade classrooms from available daily rosters. 4.2 Qualitative Sampling Design The endline evaluation used the same qualitative sample for PFG discussions that was selected at baseline. Schools where PFG discussions were conducted were selected from a subset of schools in the quantitative sample where survey data collection was also completed. Schools were stratified based on the number of second grade classes (as a proxy for school size), head teacher satisfaction with parental involvement, and rural and urban locations. Within these strata, schools were randomly selected within each province (RTI Baseline Report, 2018). We used purposive sampling to select key stakeholders at the MINEDH for interviews. The rest of this report refers generally to these respondents as “key officials”; the group includes representatives from the district, provincial, and national levels. Abt Associates June 2022 ▌19 4.3 Outcomes Primary Student Outcomes Primary outcomes of interest include student reading ability, student attentiveness, and student attendance (absences). Reading Ability We used the EGRA tool to determine whether students in the early and middle grades are developing the fundamental reading skills. The EGRA tests students’ competency on a number of subtasks, including oral comprehension, letter and syllable identification, word dictation, oral reading fluency, and reading comprehension. The ability to read and understand a simple text is one of the most fundamental skills a student can learn. In this report, we discuss the impact on the reading comprehension subtask in Section 7: Main Findings, and have included exhibits with the impacts for the other subtasks in Appendix A. Below we briefly describe the subtasks and how we measured student scores. Reading ability was measured with six subtasks: • Oral comprehension: measures the ability to understand basic oral vocabulary. The enumerator reads the student a short text and when finished asks the student a few questions to test their understanding of the text. In the Portuguese EGRA, the enumerator asks the student five questions about the text; in the local language EGRA, the enumerator asks the student four questions about the text. Outcome measure: percent zero scores, percent of questions correctly answered. • Letter identification: assesses the ability to provide the names of the letters of the alphabet. The student is given a letter card with 100 letters (10 letters on 10 rows with letters in uppercase and lowercase) and is asked to name as many letters on the card as they can within one minute. The order of letters differed between the Portuguese and local language EGRAs, and the local language EGRA contained a number of different letters. Outcome measures: percent zero scores, percent of letters correctly identified, a score that measures the correct letters read per minute. • Syllable identification: assesses the ability to correctly pronounce syllables. The student is given a syllable card with 50 syllables (five syllables on 10 rows) and asked to pronounce as many syllables on the card as they could within one minute. The order of the syllables differed between the Portuguese and local language EGRAs. Given that the syllable structure of the local languages is different, some of the syllables differed from the Portuguese EGRA. Outcome measures: percent zero scores, percent of syllables correctly pronounced, a score that measures the correct syllables read per minute. • Word dictation: measures the ability to correctly write words that are dictated. The enumerator reads five words and after each word the student is asked to write the word on a piece of paper. Portuguese EGRA has different words than the local language EGRA. Outcome measure: percent zero score, percent of words correctly written. • Oral reading fluency: measures the ability of students to read passages fluently. In this subtask, the students were given a short story and were asked to read the story aloud in one minute. The story in the local language EGRA was a little shorter than the store in the Portuguese EGRA. Outcome measure: percent zero score, percent of words correctly read. • Reading comprehension: measures the student’s textual understanding. After the student finished reading the passage in the oral reading fluency subtask, students were orally asked Abt Associates June 2022 ▌20 five questions about the text in the Portuguese EGRA and four questions in the local language EGRA. Outcome measure: percent zero score, percent of questions correctly answered. Student attendance To estimate student attendance, we asked students whether they were absent at least one day last week. Students answered this question with a simple yes or no response. The results from this question are presented in this report. Findings from other variables that measured student attendance (e.g., teacher perception on average percent of students absent in a classroom) is also included in this report. Student attentiveness To measure how student attentiveness has changed, we asked teachers about their perception on whether students are attentive in class. Teachers were asked to answer a question with a simple yes or no response. In this report, we show the results from this question. Findings from other variables that provide insight into how student attentiveness changed (e.g., student self-reported attentiveness, teacher perception on approximate number of students who are attentive in class) are also included in this report. Secondary Outcomes Impacts on instructional input • Teacher-reported receipt of training on the fundamentals of reading instruction: teachers were asked if they ever received training on the fundamentals of reading instruction, which they could answer with yes or no response. • Teacher-reported absence of at least one day last week: teachers were asked if they were absent from school any day last week. Teachers answered this question with a yes or no response. • Teacher has access to school supplies: to measure access to teaching and learning materials, we asked teachers if they have access to school supplies which was answered with a yes or no response. Feeding outcomes • Short-term hunger: to measure the McGovern-Dole impact on short-term hunger, we asked students how hungry they felt during the time of the interview. Students could choose from the following response options: very hungry, somewhat hungry, just right, somewhat full, or full. We report on use a binary outcome that shows the percentage of students who felt hungry (very hungry or somewhat hungry). • Dietary intake: as McGovern-Dole aimed to improve access to school meals and use of dietary practices, we measure the impact on student’s receiving a minimum acceptable diet. The minimum acceptable diet indicator is based on both the minimum meal frequency and minimum dietary diversity. Minimum meal frequency is defined as three or more meals per day. Minimum dietary diversity is defined as the daily consumption of four or more of seven food groups (grains, roots, and tubers; legumes and nuts; dairy products; flesh foods; eggs; vitamin A-rich fruits and vegetables; and other fruits and vegetables). If a student meets the Abt Associates June 2022 ▌21 minimum feeding frequency and minimum dietary diversity for their age group, then they are considered to receive a minimum acceptable diet.2 • Student nutrition knowledge score: students were asked eight questions to test their knowledge of nutrition based on the curriculum provided by the implementer partners. These nutrition questions focused on student’s knowledge of food groups, nutrients, and consumption of a balanced diet. • Food preparation: cooks were also surveyed about the conditions and supplies available for food preparation at the school, including the availability of pots and utensils, access to clean water, and the presence of a kitchen. Cooks were also asked a series of questions to evaluate their knowledge of food safety and hygiene. Student knowledge and use of recommended health practices • Hygiene and sanitation knowledge score: students were asked seven questions to test their knowledge of nutrition based on the curriculum provided by the implementer partners. The hygiene and sanitation questions tested students’ knowledge of basic hygienic and sanitation practices, including prevention of diarrhea and when and how long they should wash their hands. • Health-related absence: students who said they were absent from school any day last week, were also asked for the reason they were absent. To assess the impact on health-related absences, we show the percentage of students who self-reported that they were absent due to illness. • Improved latrines: head teacher were asked to report on whether the school has improved latrines. • Tippy-taps or hand-washing stations: head teacher reports that school has tippy taps or hand￾washing stations. • Tippy taps or hand-washing stations have ash or soap: head teacher reports that tippy taps or hand-washing stations have ash or soap. For information on the sample sizes by treatment arm for each of the primary and secondary outcomes, we refer Exhibit A7. 4.4 Impact Analysis Models The quasi-experimental estimates use data on the same outcomes for schools at baseline and endline to support difference-in-differences analyses. Difference-in-differences analysis implicitly projects what the endline outcome for schools that received the intervention in question would have been “absent treatment” using the observed difference between baseline and endline in comparison schools. The difference between the observed endline outcome for treatment schools and this “absent treatment” projection is an estimate of the treatment effect. Specifically, our estimates of impact for outcomes based on student or teacher survey responses come from the following regression model: 2 The Minimum Acceptable Diet (MAD) for children 6–23 months old, is one of the core indicators for assessing infant and young child feeding (IYCF) practices developed by the World Health Organization (WHO, 2008). The minimum acceptable diet indicator is not validated for school-age children. Abt Associates June 2022 ▌22 The variables in the model are as follows: • represents the outcome measure for student/teacher i in grade g in school s in province p in year t. • is a binary variable indicating whether the outcome is for students in Grade 4 at baseline and Grade 5 at endline (and equal to zero if the outcome is for students in Grade 2 at baseline and Grade 3 at endline). • is a binary variable indicating whether the year is 2021 (and equal to zero if the year is 2017). • 1 is a binary variable indicating whether school s received the school feeding intervention only (and equal to zero if the school is either a comparison school or received the combined school feeding and literacy interventions). • 2 is a binary variable indicating whether school s received the combined school feeding and literacy interventions (and equal to zero if the school is either a comparison school or received the school feeding intervention alone). • 4 is the effect of the school feeding intervention by itself for students in who entered Grade 1 in 2019 (and were thus in Grade 3 in 2021). • 5 is the effect of the combined intervention for students who entered Grade 1 in 2019. • 6 is the additional effect of the school feeding intervention by itself for students who entered Grade 1 in 2017 and therefore had two additional years of exposure to the intervention. • 7 is the additional effect of the combined intervention for students who entered Grade 1 in 2017 and therefore had two additional years of exposure to the intervention. • is a school fixed effect.3 • is a school-by-grade fixed effect.4 • is a province-by-time fixed effect. • is the error term. Unbiased estimation of the impacts requires us to assume “conditional parallel trends” between the treatment and comparison schools (Lechner, 2011). Conditional parallel trends implies that, net of the factors we control for in the model, the trend between baseline and endline in comparison schools is parallel to what would have been observed in treatment schools had they 3 The evaluation design plan described as a school random effect. We revised this to a school fixed effect to relax assumptions about the nature of school-specific effects. 4 “Grade” refers to either early-grade students (in grade 2 at baseline and grade 3 at endline) or mid-grade students (in grade 4 at baseline and grade 5 at endline). = 0 + 1 + 2 + 3 × + 4( 1 × ) + 5( 2 × ) + 6 1 × × + 7 2 × × + + + + Abt Associates June 2022 ▌23 not been treated. To make this assumption more plausible, the model includes a series of fixed effects. The fixed effects aim to control for unobserved factors that are unchanged between baseline and endline. For instance, school fixed effects control for unobserved factors common to schools, such as school size or quality of school leadership. School-by-grade fixed effects control for any unobserved factors common to early grades and middle grades within schools that are unchanged between baseline and endline, such as curriculum or teacher quality for those grades. The province-by-time fixed effects control for unobserved factors common to the province and year such as hurricane damage and COVID closures.5 We used software that calculates standard errors and associated p-values in a manner that reflects the extra uncertainty caused by testing eight students per school instead of spreading the students out across the entire province.6 After fitting the model, we also used software to predict regression-adjusted average outcomes at endline for both intervention groups and the comparison group. These predictions are calculated for both treatment groups and the comparison group by summing together weighted baseline outcomes and estimates from the regression model. For outcomes based on head teacher and cook survey responses, where there is only one response from each school, the regression model is modified as follows: The new variables in this model are as follows: • represents the outcome measure for head teacher/cook i in school s in province p in year t. • 4 is the effect of the school feeding intervention by itself for head teachers/cooks. • 5 is the effect of the combined intervention for head teachers/cooks. • is a province-by-time fixed effect. • is the error term. As with students and teachers, we calculated standard errors and associated p-values in a manner that reflects the extra uncertainty caused by testing a random subset of schools.7 5 With data only analyzed at baseline and endline, formal testing for conditional parallel trends was not feasible. We explored the possibility of placebo testing using outcomes that, in theory, were unlikely to be influenced by the treatment; but no such outcomes were available for analysis. Given the careful selection of comparison schools (RTI Baseline Report, 2018) as well as our series of fixed effects to control for unobserved factors, we felt comfortable with the conditional parallel trends assumption. 6 This clustering makes for far more efficient data collection, but it also results in higher standard errors than would be seen on an evenly dispersed student sample of the same total size. We adjust the standard errors by using regression software that allows for survey analysis of the data. We declare provinces as strata and schools as primary sampling units. We also apply weights that reflect the inverse probability of respondents’ selection for the survey (these probabilities are different for students and teachers). 7 We declare provinces as strata and schools as primary sampling units. We also apply weights that reflect the inverse probability of school selection for the survey. = 0 + 1 + 2 1 + 3 2 + 4( 1 × ) + 5( 2 × ) + + Abt Associates June 2022 ▌24 The relative size of the quantitative sample across treatment arms has important implications for statistical power. Because it was only tested in Maputo, the school feeding only treatment arm has a smaller quantitative sample (178 schools across baseline and endline) than the literacy and school feeding arm (360 schools) or the comparison group (355 schools). The impact analyses separately compare literacy and school feeding to comparison and school feeding only to comparison. The smaller quantitative sample in the school feeding only treatment arm implies impacts must be larger in that treatment arm than in the literacy and school feeding arm to achieve statistical significance. Conducting statistical tests of many hypotheses creates a “multiple comparisons” problem and increases the likelihood of a false discovery. To mitigate this concern, the study team specified a single primary outcome in each of three separate domains: reading comprehension, attendance, and attentiveness. The key findings presented in the report are based on the results for these three primary outcomes. Impacts for secondary evaluation questions use the same methodological approach and are included in the report but are considered to be exploratory. 4.5 Implementation Fidelity Analyses Given the scope and length of the McGovern-Dole projects in Mozambique, it is important to identify variation in implementation across schools, provinces, treatment arms, and project components. A deeper understanding of implementation variation has implications for effectiveness, adaptations, and sustainability. In order to identify implementation variation, we calculated a quantified level of fidelity for every study school, including those served by one of the two McGovern-Dole projects as well as comparison schools that may have had school feeding and literacy interventions. The study team created an implementation fidelity index informed by survey items within the head teacher, teacher, and cook instruments. An index and calculated score represent each of the three primary components: (1) School feeding, (2) Literacy instruction, and (3) Head teacher and teacher training. Exhibits A4, A5, and A6 represent the fidelity indices and scoring. 4.6 Analysis of Stakeholder Perspectives Qualitative data and analysis allowed the study team to provide additional details on McGovern￾Dole intervention activities in order to contextualize findings from the impact evaluation. In collecting open-ended responses from stakeholders, the study team can provide illustrative examples to complement the quantitative findings. Qualitative data were collected using two main instruments, the PFG discussion guide and the key officials’ semi-structured interviews. Data from the open-response questions from the head teacher and cook/warehouse manager surveys were also used but were not as prevalent in the qualitative analysis. This section provides detail on the qualitative data sources and methodology. Findings generated from qualitative data are presented throughout this report. Qualitative Data Sources The study team received translated notes of responses from the PFG discussions and transcripts from the key officials semi-structured interviews. Copies of the guides are included in Appendix B. PFG discussions were held at baseline and endline, but the study team added the key officials interviews for the impact evaluation at endline. For both sets of data, interviews and notes were transcribed and translated by the in-country data collection firm and submitted to Abt Associates for analysis. Abt Associates June 2022 ▌25 Parent Focus Group discussions: At baseline, treatment schools were randomly selected for inclusion in the sample for PFG discussions. At endline, the data collectors used the same sample of schools to conduct the PFG discussions. At endline, some questions were added to capture parent and family perspectives on events that occurred during the project implementation, specifically parent perceptions on the impact of COVID-19. Parent participation in the discussions was voluntary. The discussions usually lasted about 60 minutes. An audio recording and notes were collected during each discussion. The discussions covered a common set of topics including involvement at the school, satisfaction and expectations with the school, positive and negative aspects of the McGovern-Dole project, at￾home activities, as well as sustainability and recommendations. Ministry of Education and Human Development interviews: Key officials’ interviews were not completed at baseline. The study team for the endline evaluation proposed adding interviews with “key informant stakeholders” to provide an administrative and management perspective on implementation of the McGovern-Dole projects. There were several levels of perspective provided, described in the sample section below. Participation in the interviews was voluntary. Interviews usually lasted between 60 and 90 minutes. Interviewers used a conversational approach to allow probing. The interviews were semi-structured and covered a common set of topics: involvement in intervention activities, positive and negative aspects of the McGovern-Dole project, goal attainment, the influence of COVID-19, as well as sustainability and recommendations. Qualitative Analysis Methodology The study team received translated notes of responses from the PFG discussions and transcripts from the interviews with the key officials. The files were uploaded to Nvivo, a qualitative analysis software. The team created a “codebook” of categories using the qualitative evaluation questions and based on the major topics covered in the different data collection protocols (e.g., types of activities, experiences with the project, and the influence of COVID-19). Text segments associated with specific broad categories were coded and analyzed using an inductive thematic approach (Guest, Namey, and Mitchell, 2013). Transcripts were analyzed to provide contextual understanding of the various stakeholders’ experience. Though responses to the broad, open-ended questions provide important contextual information to complement the closed-ended responses, the respondents that mentioned particular themes or topics cannot be assumed to represent all stakeholders. 4.7 Analysis of Impacts in Adequate Fidelity Sites In order to more accurately estimate the size of treatment-control contrast, the team captured implementation of similar project elements in all schools, across treatment and comparison groups. The team then calculated for each school and for each major component (i.e., literacy instruction, school feeding, and teacher training) a score on the adequacy of implementation. Based on these scores, each school was identified as either an adequate or inadequate implementer for each component. A school is defined as an “adequate implementer” if it scored above the adequacy threshold on all three components or a “low implementer” if that score is below the threshold. Threshold for each component were set at 50 percent of the total possible (Exhibit 2). This determination is the basis of an alternative impact analysis using only the subset of schools where implementation of the project activities was assessed to be adequate. Abt Associates June 2022 ▌26 Exhibit 2. Score Ranges and Implementation Adequacy Thresholds Inadequate Adequate School feeding 0-7 8-15 Literacy instruction 0-5 6-11 Teacher training 0-5 6-12 For these analyses, we used the same regression model as that used to estimate the primary impacts described above. However, we limited the sample to all comparison schools and only McGovern-Dole schools identified as an adequate implementer. Abt Associates June 2022 ▌27 5. DATA COLLECTION 5.1 Overview For this evaluation, the Abt team used baseline data collected in 2017 and new endline (post￾project) data. The endline data collection processes were designed to closely match the baseline data collection processes. In country, direct data was collected by Ernst & Young (EY) Limitada located in Maputo, Mozambique. 5.2 Participant Contact and Information Collection The mobilization strategy consisted of two phases. In the first phase, EY submitted the permissions and authorizations to MINEDH’s Health and Nutrition Directorate and Primary Education Directorate. After receiving the necessary permissions and authorizations, the EY mobilization team visited the District Services of Education, Youth and Technology, and the selected schools in advance to inform them of the work plan and methodology to be used. The mobilization strategy was key to the success of the evaluation. The selected schools were involved and understood how relevant the work that Abt Associates and EY were implementing. This involvement and understanding allowed the work to be conducted under the best conditions possible. 5.3 Enumerator Recruitment and Training EY recruited the data collectors and prepared their training. EY prepared additional background materials and lists of references for the data collection managers and team leads to familiarize senior staff with the content. At that time, EY made modifications to the Training Guide/Field Manual for Enumerators that Abt had developed. Enumerator Recruitment EY recruited and hired data collectors and data quality staff with backgrounds in early child development, education, psychology, and social welfare (Exhibit 3). Additionally, EY recruited staff with prior experience in national surveys such as Demographic and Health Survey or Multiple Indicator Cluster Survey. All data collectors were required to have at least completed required schooling, and supervisors were required to have a degree in social sciences. Data collectors were also required to be fluent in Portuguese and the local languages spoken in the select districts of Maputo and Nampula. All hired staff were familiar with using electronic devices for data collection. Exhibit 3. Staff Hired for Data Collection Staff Role Maputo Nampula Enumerators 36 24 Facilitators 2 2 Note takers 1 1 Data controllers 15 15 Mobilizers 3 5 Abt Associates June 2022 ▌28 Enumerator Training The training ensured that data collectors were familiar with not only the techniques for successful interviewing, but also the content of the tools. The training spanned five days in October 2021 and was conducted in person by EY using COVID-19 safety precautions. The training was divided into four parts: 1. Objectives and scope of the project 2. Training on the questionnaires 3. Fieldwork testing 4. Fieldwork results and administrative issues with the enumerators In terms of training methods, EY conducted the testing of the tools in simulation form (enumerators interviewing each other) and also in real conditions. This method allowed interview techniques to be assimilated more effectively. Pilot Testing EY performed pilot testing to ensure that all staff and respondents understood the data collection tools. The pilot testing process allowed the team to test abilities of the enumerators in real circumstances. As a result of this exercise, refinements and/or adjustments of the tools were made based on the situations reported by the field team during the pilot testing. All questionnaires and data collection tools were tested. Data collected during the pilot tests were not included in the evaluation sample. There were two phases of pilot testing. The first phase was conducted in two days in September and took place in four schools in Maputo. The EGRA instrument and student questionnaire was tested with 10 students. The head teacher, teacher, and cook questionnaires were tested with four head teachers, four teachers, and four cooks and warehouse managers, respectively. The PFG discussion guide was also tested with seven parents. The second phase was conducted in one day in October and took place in three schools across Maputo and Nampula. Each enumerator pilot tested student assessment with the EGRA, alongside one of Abt’s in-country consultants. After the pilot data collection, the data were shared with the Abt study team for analysis and improvement of the scripts. The data from the pilot were analyzed to identify any implementation issues. After any needed resolution, EY integrated the revisions into the tools and manuals. 5.4 Data Quality and Security Checks of Accuracy of Sample Backchecks were conducted as internal control method to verify if the enumerators effectively visited the correct schools and interviewed the correct populations. Additional Training Topics • Background information • General information for the enumerators • Objectives and nature of baseline evaluation • The prescribed questionnaires and listing schedules • Procedures to be followed • Analysis of the tools • Introduction to data entry software • Practical work • Editing • Administrative instructions for enumerators Abt Associates June 2022 ▌29 The backcheck questionnaire was implemented by data quality controllers through phone calls to teachers, head teachers, and cooks across all districts within the scope of work. A total 218 backcheck phone calls were conducted: • 57 backchecks for cooks (37 in Maputo and 20 in Nampula) representing 22 percent of the total cooks interviewed • 53 backchecks for head teachers (30 in Maputo and 23 in Nampula) representing 12 percent of the total head teachers interviewed • 108 backchecks for teachers (70 in Maputo and 38 in Nampula) representing 13 percent of the total teachers interviewed. It was found that all schools contacted for the backchecks are on the list and the teachers, head teachers, and cooks confirmed that were interviewed. 5.5 Data Management and Security Mobilization Strategy Relevant information was collected on the target schools. The team contacted the government and other relevant stakeholders at central, province, and district levels. The team subsequently elaborated on the mobilization plan based on these meetings. Data Collection Logistics Data collection logistics were managed internally, including transportation, support to field team, supply of materials, and other required goods and services. Team members also printed and photocopied questionnaires for a back-up option for enumerators to use in the case of survey device failure while with a respondent. Data Collection Quantitative data were collected using the EGRA instrument and questionnaires. Any problems were resolved as they arose throughout the data collection process. The key informant interviews and PFG transcripts were translated into English. The data collection team regularly updated Abt Associates on the progress of data collection. Data entry was done through the mobile data collection platform. For this project the team used “ODK” software, which in addition to typing, the platform allows the verification of the insertion to ensure consistency in data entry. In addition, ODK software allows data transfer to different statistical analysis packages. The program verifies the consistency of entered data. Violations of these checks lead to an alert about data inconsistency or violations. 5.6 COVID-19 Specific Precautions Since the beginning of the state of emergency imposed to limit the spread of COVID-19, EY Mozambique implemented a set of measures to protect the health and safety of employees and the communities. The entire EY team received a summary of these protocols prior to beginning fieldwork, and during training. Prior to data collection staff completed COVID-19 forms. Reminder messages of the protocols were sent to staff throughout the data collection period. All data collection team members were required to comply with the following measures and other measures established by the Government of Mozambique: • Mandatory use of face masks • Frequent hand hygiene using hand sanitizers or soap and water • Practice of physical distance, keeping a distance of two meters apart Abt Associates June 2022 ▌30 • Holding interactions, such as interviews with respondents, outdoors or in open spaces with ventilation • Daily self-monitoring of symptoms (fever or chills; cough; shortness of breath or difficulty breathing; fatigue from their data collection team members each morning) • Daily collection of COVID self-screening checklists by Field Work leaders • Avoiding the sharing of tools, phones, or other objects when possible • Use of photos to confirm compliance with protocols during events like enumerator training and travel • Use of privately contracted vehicles and drivers for data collection team member travel Abt Associates June 2022 ▌31 6. CHARACTERISTICS OF POPULATION This section describes the realized sample used in this evaluation and details the demographic characteristics of the students, teachers, and parents who participated in the evaluation. The description of characteristics provides context to the findings discussed in the sections below and confirms that there are no substantial differences in sample characteristics between base- and endline and across treatment arms. 6.1 Sample Description The endline evaluation revisited the same schools as sampled at baseline. At baseline, data were collected at 448 schools, while at endline, data were collected at 445 schools (three schools that were permanently closed in the implementation period were lost to the analysis sample). 8 The goal for the endline data collection was a sample that matched the baseline sample in terms of numbers of respondents of each type. The endline samples represent an average of 88 percent of the number of participants in the baseline samples (Exhibit 4). Exhibit 4. Baseline and Endline Evaluation Sample Treatment Arm Phase Evaluation Schools Head Teachers Cooks Teachers Students Grade 2/3 Grade 4/5 Grade 2/3 Grade 4/5 Maputo Literacy and school feeding BL 90 84 62 83 78 661 638 EL 90 89 68 84 88 690 696 School feeding only BL 90 85 70 82 81 666 616 EL 88 89 81 85 81 670 645 Comparison BL 89 86 n/a 86 83 704 681 EL 88 88 27 90 79 689 676 Nampula Literacy and school feeding BL 90 85 82 79 72 707 685 EL 90 88 88 85 85 717 710 Comparison BL 89 89 n/a 82 80 690 676 EL 89 89 n/a 85 78 712 664 Total sample baseline 448 429 214 412 394 3428 3296 Total sample endline 445 443 264 429 411 3478 3391 Total sample (base- and endline) 893 872 478 841 805 6906 6687 Expected sample (base- and endline) 898 898 538 898 898 7184 7184 Overall response rate 99.4% 97.1% 88.8% 93.7% 89.6% 96.1% 93.1% Note: The expected sample at endline was based on the realized sample at baseline. BL indicates baseline, and EL indicates endline. 6.2 Student Demographic Characteristics The average student at endline is six to nine months older than the average student interviewed at baseline. This reflects third grade and fifth grade students being interviewed at endline while at baseline students in second grade and fourth grade were interviewed. On average, students in the 8 Three schools in the Maputo province that were visited at baseline were found to be permanently closed at endline: two schools in the school feeding only group and one school in the comparison group. Abt Associates June 2022 ▌32 school feeding only intervention were a little younger than the average student in the comparison group and the literacy and school feeding intervention. Note that students self-reported their age. Key findings on student characteristics and differences across treatment arms are outlined below. Exhibit A8 presents student responses on household demographic characteristics. • Approximately half of the students interviewed were female despite random sampling that did not take account of gender. As a proxy for the student’s family household wealth, students were asked about their family’s access to water and ownership of household assets. A smaller percentage of students at endline said their family sources water from a river, stream, or lake, or a well or borehole, while a larger percentage of students indicated that their family has a communal tap or a water pipe or tap at home. A larger share of the families of students in the comparison group have access to a water pipe or have a tap in their house (between 39 and 49 percent), compared to students in the other treatment arms (between 13 and 28 percent). Families of students in the literacy and school feeding intervention have on average less access to a water pipe or a tap at home. • Students reported on their family’s ownership of nine household assets (asset index). Compared with baseline, the average number of household assets in student’s homes at endline was slightly lower. For mid-grade students in the comparison group, however, the average number of household assets was a little higher compared to baseline. Families of mid-grade students in the comparison group on average also own the largest number of household assets (four household assets out of nine). Families of students in the literacy and school feeding intervention have on average the lowest number of household assets. 6.3 Language of Literacy Instruction As expected, nearly all teachers responded that Portuguese is the language of literacy instruction. Local languages are also used as a language of literacy instruction and to facilitate the teaching of Portuguese. Rhonga and Changana, languages that are primarily spoken in the south, are used by teachers in Maputo province. Few teachers, however, said they used Rhonga to teach literacy; a larger percentage of teachers indicated they used Changana. In school feeding only schools, Changana is used by approximately 35 to 40 percent of teachers, compared to 12 to 22 percent of teachers in comparison and literacy and school feeding schools at endline. About 22 to 43 percent of teachers at comparison and literacy and school feeding schools in Nampula province said they used Macua, a language primarily spoken in the North. Students often speak languages other than the language of instruction at home. Between 13 and 31 percent of students speak a different language at home than the language used for instruction (Exhibit A9). This difference is largest among students in the school feeding only intervention. 6.4 Teacher Characteristics At endline, just over 60 percent of teachers in the comparison group were female whereas a little over 40 percent of teachers at schools in the school feeding only intervention and school feeding and literacy intervention were female. Compared to baseline, the percentage of teachers in comparison schools and the school feeding only schools who were female was slightly higher. In the school feeding and literacy schools, fewer female teachers were interviewed at endline compared to baseline. Teachers at schools in the school feeding only intervention more often teach multi-grade classes. While close to 10 percent of teachers in the comparison schools and the literacy and school feeding schools say they teach multi-grade classes, roughly a third of teachers in the school Abt Associates June 2022 ▌33 feeding only schools say they teach a multi-grade class. Compared to baseline, the percentage of teachers teaching classes with students from two or more grades has decreased. PAI had supported teacher training in 11 teacher training colleges throughout Mozambique, essentially facilitating the development of new teachers who would soon be placed in rural primary schools located in several provinces including Maputo and Nampula. Nearly all teachers say they attended and finished a teacher training college (TTC). The majority of teachers attended the Primary Teacher Training Institute. A relatively small percentage of teachers were trained at ADPP and the Pedagogical University with no noteworthy differences between base￾and endline. Similar to baseline, the percentage of teachers who attained a certification was near 100 percent. The question about teacher’s highest level of education differed between baseline and endline. The study team suspects that a literal approach to the English to Portuguese conversion of the baseline teacher survey failed to take into account education classifications commonly used in Mozambique. Baseline reports of teacher qualifications are unclear; therefore, responses are reported separately in the table A.10. Most teachers interviewed at baseline said that the certificate was their highest level of education (65 to 71 percent). At endline, most teachers responded that middle school (equivalent to high school in other contexts) was their highest level of education (61 to 73 percent). A smaller percentage of teachers said that secondary education (12 to 17 percent) or a bachelor’s degree or higher (14 to 23 percent) was their highest level of education (Exhibit A10). Given the comparability of percentages, despite the erroneous classification, we feel confident that teachers’ levels of education from baseline to endline did not vary dramatically. Not all teachers received specific training on how to teach reading and writing to students in primary grades during pre-service training. At baseline, between 58 and 61 percent of teachers received this training; at endline between 58 and 71 percent of teachers did. 6.5 School Characteristics At endline, schools in the school feeding only intervention had the fewest teachers on average (6.5 teachers), whereas schools in the comparison group have on average the largest number of teachers (14.8 teachers). The number of teachers in the treatment schools remained more or less the same between base- and endline, but comparison schools had on average two more teachers at endline compared to baseline. Treatment schools have on average fewer female teachers than male teachers. Comparison schools, however, have more female than male teachers and showed an increase in the number of female teachers when compared to the number of female teachers at baseline. Most schools in the sample have slightly more teachers teaching early-grade students than teachers teaching mid-grade students. The number of early-grade and mid-grade classes (streams) the school had mirrored the number of teachers teaching early-grade and mid-grade students. Differences between base- and endline in the number of teachers teaching early- and mid-grade students and number of early- and mid-grade classes are minimal (Exhibit 5). Abt Associates June 2022 ▌34 Exhibit 5. School Characteristics, by Study Arm and Time ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Total number of teachers 12.7 14.8 2.2 6.8 6.5 -0.3 10.0 10.3 0.3 Total number of male teachers 6.1 6.9 0.8 4.5 4.2 -0.3 5.8 5.8 0.0 Total number of female teachers 6.7 8.1 1.4 2.4 2.4 0.0 5.1 4.5 -0.6 Number of teachers currently teaching early-grade students 2.3 2.1 -0.3 1.2 1.2 0.0 1.9 1.7 -0.2 Number of teachers currently teaching mid-grade students 1.8 1.9 0.2 1.1 1.2 0.1 1.5 1.5 0.0 Number of early-grade classes (streams) 2.3 2.1 -0.1 1.4 1.2 -0.2 1.9 1.7 -0.2 Number of mid-grade classes (streams) 1.7 1.9 0.2 1.3 1.1 -0.1 1.4 1.5 0.1 Note: BL indicates baseline, and EL indicates endline. 6.6 Parent Focus Group Characteristics At baseline and endline, the goal was to complete 30 PFG discussions. Exhibit 6 and Exhibit 7 below provide details on the number and average size of PFG discussions per province and district. The sample planned to include a larger number of schools from Maputo to achieve adequate representation from both the school feeding only and the literacy and school feeding schools. At baseline, 30 PFG discussions were conducted. Exhibit 6. Parent Focus Group Sample with Literacy and School Feeding Treatment Baseline Endline Province/District Number of Parent Focus Groups Mean Number of Parents in the Focus Group Number of Parent Focus Groups Mean Number of Parents in the Focus Group Maputo Magude 1 7 1 7 Manhiça 4 7 3 10 Matutuine 3 8 2 9 Moamba 3 7 3 11 Nampula Muecate 6 11 6 10 Nacaroa 6 12 6 10 TOTAL 23 9 21 9 to 10 Exhibit 7. Parent Focus Group Sample with School Feeding Treatment Only Baseline Endline Province/District Number of Parent Focus Groups Mean Number of Parents in the Focus Group Number of Parent Focus Groups Mean Number of Parents in the Focus Group Maputo Magude 3 11 3 7 Manhiça 2 8 3 10 Matutuine - - 1 9 Moamba 2 12 2 11 Nampula Muecate - - - - Nacaroa - - - - TOTAL 7 10 9 10 Abt Associates June 2022 ▌35 There were 289 unique parent respondents (285 at baseline), with an average of 9–10 participants in each discussion at baseline and endline. The average age of the parents was about 39 (ranging from 17 to 74 years). The PFG discussions mainly included females (68 percent at endline, 57 percent at baseline). The average number of children that the parents had was 2.2 (3.4 at baseline). On average, PFG discussions were composed of slightly more female parents at endline, and the average number of children was slightly lower for families represented at endline compared with those who participated at baseline. At endline, parents were asked how they were involved at the school. Roughly two-fifths (41 percent) of the parents said that they did not have any role or involvement at the school. Parents were most likely involved as a cook at the school (26 percent of parents), followed by 16 percent of parents serving on the school council. Parents also reported being involved with the school by helping with the garden, construction, or cleaning. 6.7 Key Official Characteristics At endline, 28 interviews were completed, including 17 representatives from Maputo province, nine from Nampula province, and with national representatives. Nearly half (46 percent) of the interview respondents held positions in the Department for General Education. Another quarter (25 percent) were District Services of Education, Youth and Technology directors, and the remaining six (including the national representatives) varied in their positions. We have not provided the more detailed breakdown of the respondents by district to protect the privacy of the respondents. Abt Associates June 2022 ▌36 7. MAIN FINDINGS 7.1 Summary of Findings This section of the report provides a review of the effects on the three main outcomes: reading performance, attendance, and attentiveness. Evidence indicates that the school feeding and literacy intervention had a substantial effect on students’ reading performance. Effects on the other two outcomes were not statistically significant (Exhibit 8).9 Exhibit 8. Summary of Impact on Average Mean Reading Performance, Attendance, and Attentiveness ........................................................ School Feeding Only School Feeding and Literacy Impact Impact Early-Grade Students Student reading performance (Average Reading Comprehension Scores) 1.2 4.8** Student reading performance (Percent Zero Reading Comprehension Scores) -10.6 -10.0** Student attendance -4.6 -5.7 Student attentiveness -3.5 -4.1 Mid-Grade Students Student reading performance (Average Reading Comprehension Scores) -1.8 7.6* Student reading performance (Percent Zero Reading Comprehension Scores) -6.5 -14.4** Student attendance 4.8 3.9 Student attentiveness -7.2* -3.0 Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. 7.2 Impacts on Student Reading Performance In schools implementing both literacy and school feeding interventions, the average student reading comprehension scores for both early-grade and mid-grade students showed significantly more improvement over time than did scores in comparison schools. Early-grade students experienced an increase of 4.8 percentage points (p<.01, effect size=.16), and mid-grade students experienced an increase of 7.6 percentage points (p<.05, effect size=.26). There was no parallel difference in the improvement in reading comprehension for students in schools implementing only school feeding intervention versus students in comparison schools (Exhibit 9). 10 This finding held up when the analysis models included covariates describing the difference in (1) 9 See Appendix A for outcome levels at baseline and endline by treatment arm for all main and secondary outcomes. 10 Exhibit 9 shows the average reading comprehension score across all treatment conditions at baseline separately for each condition and student age group. At endline, the reported reading comprehension score is the sum of the baseline average and the analysis model’s predicted change in reading comprehension for each condition and student age group. The difference-in￾differences estimate from the analysis model can be calculated by subtracting the average change in reading comprehension in the intervention group under study from the average change in reading comprehension for the comparison group. Abt Associates June 2022 ▌37 language of instruction and the child’s home language and (2) the language of assessment and the child’s home language.11 Exhibit 9. Impacts on Average Mean Reading Comprehension Scores Notes: “Early-Grade Students” were in grade 2 at baseline and grade 3 at endline. “Mid-Grade Students” were in grade 4 at baseline and grade 5 at endline. “C” denotes comparison schools; “SF” denotes school feeding only schools; “SF+L” denotes school feeding and literacy schools. Exhibit reports the unadjusted, weighted average outcome for each condition at baseline. The depicted outcome at endline is the sum of that average at baseline and estimates from the regression model. Y-axis for this exhibit (not shown) runs from 0 to 40 percent. In addition to improving average mean scores, another way to look at performance on the reading comprehension task is to score students on whether they answered at least one test question correctly. Combining literacy and school feeding reduced the share of early-grade students who could not answer any reading comprehension question correctly by 10.0 percentage points (p<.01, effect size = -.21) and of mid-grade students by 14.4 percentage points (p<.01, effect size = -.31) (Exhibit A12). As with average mean reading comprehension scores, school feeding alone did not impact the share of early-grade or mid-grade students who could not answer any reading comprehension question correctly. The magnitude of the impact estimates for school feeding alone suggest a reduction in the share of early-grade and mid-grade students 11 A potential concern for the analysis of reading comprehension is a lack of correspondence between the language spoken at home, the language of instruction, and the language of assessment. This potential concern is not addressed by school fixed effects, as there is variation in these variables within schools. To test the robustness of the findings to this concern, we re￾estimated the regression model with two covariates—one indicating that the language of instruction differed from the language spoken at home and a second indicating that the language of assessment differed from the language spoken at home. Results were robust to including these covariates (see Exhibit A7). Abt Associates June 2022 ▌38 who could not answer any reading comprehension question correctly, but the larger standard errors imply these estimates are not statistically significant.12 While it is a primary outcome, reading comprehension is one of six subtasks of the EGRA. The other five EGRA subtasks evaluated include oral comprehension, letter identification, syllable identification, oral reading fluency, and word dictation. However, findings were consistent across all subtasks, with early- and mid-grade students demonstrating statistically significant gains across most subtasks (Exhibit A13 though Exhibit A22). Stakeholder Perception on Student Learning Respondents mentioned themes that may contextualize the learning environment or educational progress. A common theme was that the teachers felt that their training helped them with planning strategies and teaching methods. For example, the training helped teachers work with children according to their individual needs. One head teacher described that when bilingual teaching was introduced, the children were more at ease. Several key official respondents felt similarly that expanding bilingual education was important, and they described noticing stronger pedagogical performance. 7.3 Impacts on Student Attendance Neither school feeding nor school feeding and literacy had an impact on average student attendance rate (absences) relative to average attendance in comparison schools. The magnitude of the impact estimates suggests opposite trends in attendance rates from baseline to endline for early-grade students (improved attendance) and for mid-grade students (decreased attendance) (Exhibit 10). However, none of the estimates is statistically significant (Exhibit A23 and Exhibit A24). 12 Given that the school feeding only program was only implemented in the Maputo province, the number of schools sampled into this treatment arm is smaller than the number of schools in the comparison group and the literacy and school feeding group. Hence, the probability that we find a statistically significant difference between the school feeding only program and the comparison group, if such a difference would exist, is smaller. “…there is nothing better than learning in your own language… I didn't have that opportunity… but if I had learned in that way I would have been much better.” - Key Official (Maputo) “Those who were in the bilingual system and those who were in the monolingual system had a slight difference, so I concluded that learning in their own language was better… children who have had literacy with bilingual reinforcement, they have better results... We were able to reinforce reading, writing and basic math in those children, but we could have reached more children.” - Key Official (Maputo) Abt Associates June 2022 ▌39 Exhibit 10. Impacts on Student Attendance (Measured by Any Absence Last Week) Notes: “Early-Grade Students” were in grade 2 at baseline and grade 3 at endline. “Mid-Grade Students” were in grade 4 at baseline and grade 5 at endline. “C” denotes comparison schools; “SF” denotes school feeding only schools; “SF+L” denotes school feeding and literacy schools. Exhibit reports the unadjusted, weighted average student-reported absence (at least one day last week) for each condition at baseline. The depicted outcome at endline is the sum of that average at baseline and estimates from the regression model. Y-axis for this exhibit (not shown) runs from 0 to 40 percent. Stakeholder Perception of Student Attendance During focus groups and interviews, many parents and key officials reported that they noticed an impact on attendance because of the project, specifically because of the school feeding intervention. Parents noted that their children were more excited to attend classes because they were being fed. The lack of corroborating quantitative evidence that the McGovern-Dole project improved attendance for early￾grade and mid-grade students may suggest that reduction in attendance was in large part due to COVID-19-related school closures. Key official interview respondents provided a broader perspective and felt that the projects impacted attendance, as well as enrollment into school. The respondents also observed greater retention of students in school because of the school feeding intervention or framed the improvement in terms of a decrease in dropouts. A few interviews specifically mentioned an improvement in the enrollment and retention of girls. “When there is a meal at school, the children are motivated and adhere to school… That program contributes directly to the reduction of dropouts, the children look at that in positive way.” - Key Official (Maputo) “We had higher retention because [students] knew that by going to school they were not only going to learn, but they were also going to get a meal. So, they had more time in school, less truancy, and less lateness… the reinforcement of their learning abilities was through literacy and also through bilingual education.” - Key Official (Maputo) Abt Associates June 2022 ▌40 Some of the officials described that the parent and community involvement at the school, or parent participation in literacy centers, helped parents see the value of education for their children. One PFG participant mentioned that getting involved at the school motivated them to make sure their children were attending school and leaving on time for school. A few officials also mentioned that students’ positive experience in school, including getting meals, encouraged their siblings to attend. 7.4 Impacts on Student Attentiveness The improvement in the average level of student attentiveness was not significantly better in treatment schools than in comparison schools (Exhibit 11). Although there were declines in teacher-reported student attentiveness for students across grades and conditions, the impact estimates are not statistically significant (Exhibit A25 and Exhibit A26). Levels of teacher-reported student attentiveness were very high (greater than 90 percent in all arms) at baseline. As a result, there is limited room for growth in this measure at endline. Indeed, the apparent decline in student attentiveness is partly explained by the growth in attentiveness in comparison schools. It would not have been possible for attentiveness to increase in treatment schools at the same rate, as this would have resulted in rates of attentiveness greater than 100 percent. Exhibit 11. Impacts on Teacher-Reported Student Attentiveness Notes: “Early-Grade Students” were in grade 2 at baseline and grade 3 at endline. “Mid-Grade Students” were in grade 4 at baseline and grade 5 at endline. “C” denotes comparison schools; “SF” denotes school feeding only schools; “SF+L” denotes school feeding and literacy schools. Exhibit reports the unadjusted, weighted average outcome for each condition at baseline. The depicted outcome at endline is the sum of that average at baseline and estimates from the regression model. Y-axis for this exhibit (not shown) runs from 80 to 105 percent. “The younger ones, who were not yet of school age, were already looking forward to [school] when their classes would start, because they always followed their [siblings] to school and [saw their] meals.” - Key Official (Nampula) Abt Associates June 2022 ▌41 Stakeholder Perceptions of Student Attentiveness Concerns about attentiveness were supported by the head teacher and key official responses, with some respondents mentioning challenges with student attentiveness related to the timing of the school meal. Similar to the findings above, there were some challenges reported with attentiveness directly after students received a meal. Head teachers suggested that meals should be carefully scheduled so they do not interfere with learning activities. Respondents generally felt that the net benefit of students having a school meal was positive and that it improved the academic environment or attentiveness. A theme that emerged from head teachers’ open-ended responses was that the intervention improved the learning environment. 7.5 Impacts for Student Subgroups As described earlier, research suggests that literacy rates vary between women and men in Mozambique, with 60 percent of males identified as literate compared to only 28 percent of women (USAID, 2021a). Given the disparities in literacy and a growing focus on improving the educational experiences of girls, it is important to highlight impacts by gender. Impacts on student reading comprehension scores did not vary in meaningful ways between boys and girls. Average reading comprehension scores improved for both early-grade and mid-grade boys in schools implementing the literacy and school feeding intervention (by 5.1 percentage points for early-grade boys and 11.1 percentage points for mid-grade boys). While the magnitude of impacts for girls is smaller and not statistically significant, the difference in impacts between boys and girls is also not statistically significant. As in the assessment of overall impacts, there was no difference in the improvement in reading comprehension for boys or girls in schools implementing the school feeding only intervention versus boys or girls in comparison schools (see Exhibit 12). Similarly, impacts on the percentage of students who could not answer any reading comprehension question correctly did not vary for boys and girls. Combining literacy and school feeding reduced the share of early-grade boys who could not answer any reading comprehension question correctly by 9.5 percentage points and reduced the share of early-grade girls who could not answer any reading comprehension question correctly by 10.5 percentage points. Combining literacy and school feeding also reduced the share of mid-grade boys who could not answer any reading comprehension question correctly by 17.9 percentage points and reduced the share of mid-grade girls who could not answer any reading comprehension question correctly by 12.9 percentage points. The differences in these impacts between boys and girls were not statistically significant. As in the assessment of overall impacts, school feeding alone did not impact the share of early-grade or mid-grade boys or girls who could not answer any reading comprehension question correctly (Exhibit 12). “The children progressed a lot, and the children's desire to learn was greater. It was possible for the children to retain more of what they learned.” - Head Teacher (Maputo) Abt Associates June 2022 ▌42 Exhibit 12. Impacts on Student Reading Performance and Attendance for Boys and Girls13 ........................................................ School Feeding Only School Feeding and Literacy Impact Impact Early-Grade Students Boys Student reading performance (Average Reading Comprehension Scores) 2.9 5.1* Student reading performance (Percent Zero Reading Comprehension Scores) -12.8 -9.5* Student attendance -9.3 -14.8**† Girls Student reading performance (Average Reading Comprehension Scores) -1.3 4.1 Student reading performance (Percent Zero Reading Comprehension Scores) -6.9 -10.5** Student attendance -1.4 0.6† Mid-Grade Students Boys Student reading performance (Average Reading Comprehension Scores) -0.7 11.1* Student reading performance (Percent Zero Reading Comprehension Scores) -3.3 -17.9** Student attendance 3.7 -1.9 Girls Student reading performance (Average Reading Comprehension Scores) -2.2 5.5 Student reading performance (Percent Zero Reading Comprehension Scores) -9.3 -12.9* Student attendance 7.0 9.9 Sample sizes for analysis of student reading performance: Boys: 2,745 comparison; 1,238 school feeding only; 2,737 school feeding and literacy. Girls: 2,729 comparison; 1,338 school feeding only; 2,767 school feeding and literacy. Sample sizes for analysis of student attendance: Boys: 2,743 comparison; 1,236 school feeding only; 2,731 school feeding and literacy. Girls: 2,730 comparison; 1,338 school feeding only; 2,760 school feeding and literacy. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Crosses indicate statistical significance of difference in impact estimates between boys and girls. †† p<0.01; † p<0.05. Impacts on student attendance in schools implementing the literacy and school feeding intervention varied between boys and girls. Early-grade boys experienced a reduction in 13 Subgroup analysis of attentiveness is not included as that outcome is based on teacher report and is not able to be disaggregated by sex. Abt Associates June 2022 ▌43 attendance of 14.8 percentage points while attendance for girls increased by 0.6 percentage points (the difference in these impacts is statistically significant). Although not statistically significant, the magnitude of impacts for mid-grade students in these schools had a similar pattern (the difference between the -1.9 percentage point impact for boys and 9.9 percentage point impact for girls had a p-value of 0.0504). There was no evidence that impact on attendance varied for boys and girls in schools that implemented only school feeding interventions. Abt Associates June 2022 ▌44 8. SECONDARY FINDINGS Analysis of secondary evaluation questions generates evidence to further explore the mechanisms through which impacts on primary evaluation questions did (or did not) operate. The subsections that follow report estimates of impact for secondary evaluation questions pertaining to instructional inputs, feeding, and student knowledge and use of recommended health practices, and food preparation facilities. The analysis of each secondary evaluation question is limited to key secondary outcome measures; estimates of other related outcomes appear in the appendix. 8.1 Impacts on Instructional Inputs Teacher Training on Reading Instruction Training teachers on the fundamentals of reading instruction is a direct avenue through which literacy activities might influence student reading performance. Teachers of both early-grade and mid-grade students in school feeding and literacy schools experienced sizeable positive impacts in receipt of training on the fundamentals of reading instruction (15.3 percentage points for teachers of early-grade students and 27.9 percentage points for teachers of mid-grade students) ( Exhibit 13). The positive impacts are strongly consistent with the primary impacts on student reading performance, which were both positive and only apparent in school feeding and literacy schools. This suggests that teacher training on the fundamentals of reading instruction is a plausible mechanism for explaining the impacts on student reading performance. Exhibit 13. Impacts on Instructional Inputs ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Teacher-reported receipt of training on the fundamentals of reading instruction 0.3 14.5 14.3 15.5 15.3* (9.3) (7.0) Teacher-reported absence of at least one day last week -0.6 -12.2 -11.6 -1.7 -1.1 (6.7) (6.6) Teacher has access to school supplies 11.5 0.2 -11.3** 1.5 -10.0** (3.3) (3.7) Mid-Grade Students Teacher-reported receipt of training on the fundamentals of reading instruction -8.9 -5.0 3.9 19.0 27.9** (9.1) (7.5) Teacher-reported absence of at least one day last week -1.2 -3.4 -2.2 1.4 2.6 (6.7) (5.8) Teacher has access to school supplies 8.7 3.3 -5.5 0.7 -8.1* (4.1) (3.9) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 625 Comparison; 310 School feeding only; 632 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌45 Resources for Reading Instruction The literacy intervention also provided resources to schools to support reading instruction. No statistical difference was shown in access to instructional resources for teachers in schools implementing literacy interventions and comparison schools or schools implementing only feeding interventions (Exhibit 13). Teachers of early-grade students in both treatment arms experienced a reduction of 10.0 percentage points in access to school supplies, such as pens/pencils, notebooks, chalk, and cardboard, and teachers of mid-grade students in school feeding and literacy schools experienced a reduction of 8.1 percentage points in access to school supplies. That these findings operate in a different direction of the impacts on student reading performance suggests that provision of instructional resources was not a key mechanism for the improvement in reading performance. Stakeholder Perception on School Materials Several of the key officials discussed the emphasis during teacher training on the benefits of using materials. Overall, key officials thought that materials such as notebooks, writing utensils, books, and teacher materials promoted a positive learning atmosphere. However, some of the key officials mentioned that access to school materials may have been a challenge for the schools. Further, a theme emerged that materials provided by donors or grants only last so long, and that the material resources provided were negligible. One of the key officials pointed out that trainings for teachers should challenge teachers to use other types of resources. On the other hand, 16 PFGs mentioned that they had some materials at home for their children to use. They were more likely to have writing materials versus drawing materials. A few mentioned that the students brought materials back and forth between school. Teacher Absence Another hypothesis is that the feeding and literacy interventions may have increased teacher attendance and, as a correlate, student instructional time. However, no statistical difference was shown in average level of teacher absence in schools in either treatment arm versus level of teacher absence in comparison schools (Exhibit 13). 8.2 Impacts on Feeding Outcomes and Nutrition Knowledge The McGovern-Dole program theory of change hypothesizes that the school feeding intervention will improve student attendance/health outcomes and attentiveness, which, in turn, may function as mediators of or contributors to improvement in reading performance. There were no significant impacts on average levels of student hunger and dietary sufficiency in treatment schools versus comparison schools (Exhibit 14). Although non-significant, there were consistent negative trends in both treatment arms and both student age groups. Indeed, some feeding outcomes were worse in treatment schools than comparison schools. In school feeding only schools, the share of mid-grade students reporting current hunger at the time of survey completion increased by 14.3 percentage points. Across both treatment arms, early-grade students experienced a reduction in minimum meal frequency (34.2 percentage points in school feeding only schools and 14.7 percentage points in school feeding and literacy schools) and minimum acceptable diet (13.8 percentage points in both school feeding only schools and school feeding and literacy schools). Mid-grade students experienced similar reductions in dietary outcomes. In school feeding “in the trainings I feel that there is [effort] to make teachers see that they should not expect everything from the project. You can recycle cardboard, cans, and make lots of things. So, it is not possible for the project to offer everything.” - Key Official (Nampula) Abt Associates June 2022 ▌46 only schools, they experienced a reduction in minimum meal frequency of 16.8 percentage points; in school feeding and literacy schools, they experienced a reduction of 9.8 percentage points in minimum dietary diversity; and in both treatment arms they experienced reductions in minimum acceptable diet of 13.9 and 13.1 percentage points, respectively. Nearly all PFGs stated their household diets and meals mostly stayed the same, regardless of the school feeding activity. Nearly half (n=14) of PFGs said that they cooked fewer meals as a result of the school feeding activities, and eight PFGs stated that the children ate fewer meals at home. While the feeding intervention was unable to improve direct measures of hunger and diet, it succeeded in improving students’ nutrition knowledge. Early-grade students across both treatment arms experienced improvements in nutrition knowledge of 0.5 and 0.9, with effect sizes of 0.35 and 0.51 respectively. Similarly, mid-grade students in school feeding and literacy schools experienced improvement of 0.8 units (with effect sizes of 0.17 and 0.09). Exhibit 14. Impacts on Feeding Outcomes ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Student reports feeling hungry now -8.2 0.9 9.1 -15.2 -6.9 (6.2) (4.9) Student achieved minimum meal frequency 7.0 -27.2 -34.2** -7.7 -14.7** (6.2) (5.1) Student achieved minimum dietary diversity -28.8 -26.5 2.3 -28.0 0.8 (6.2) (4.7) Student achieved minimum acceptable diet -7.0 -20.8 -13.8* -20.8 -13.8** (5.8) (4.4) Student's nutrition knowledge score -0.4 0.0 0.5** 0.4 0.9** (0.1) (0.1) Mid-Grade Students Student reports feeling hungry now -2.1 12.2 14.3* -2.0 0.1 (6.7) (4.6) Student achieved minimum meal frequency 1.4 -15.4 -16.8** -2.5 -3.9 (4.9) (4.2) Student achieved minimum dietary diversity -24.6 -31.2 -6.7 -34.3 -9.8* (7.3) (4.6) Student achieved minimum acceptable diet -9.2 -23.1 -13.9* -22.2 -13.1** (6.7) (4.2) Student's nutrition knowledge score -0.5 -0.1 0.4 0.3 0.8** (0.2) (0.2) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,476 Comparison; 2,576 School feeding only; 5,504 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌47 8.3 Impacts on Student Knowledge and Use of Recommended Health Practices Students’ knowledge of recommended health practices and their translation of that knowledge into use of those practices are considered to be potential mediators of improved reading performance because good health directly corresponds to school attendance and attentiveness, both of which could influence student achievement. There were no significant differences in student knowledge and use of health practices across grade levels and treatment arms (Exhibit 15). There were improvements in hygiene and sanitation knowledge among both early-grade students and mid-grade students but the difference from baseline to endline was not statistically significant. Relatedly, another series of outcomes captured access to clean water and sanitation. Head teachers reported whether schools had improved latrines, access to hand-washing stations, and access to ash or soap. Neither treatment arm had a favorable impact on access to clean water and sanitation, and the school feeding only intervention reduced access to hand-washing stations as well as to ash or soap (by 27.7 and 21.0 percentage points, respectively). Exhibit 15. Impacts on Student Knowledge and Use of Recommended Health Practices ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Student's hygiene and sanitation knowledge score 0.8 1.1 0.3 1.6 0.9** (0.3) (0.2) Student reports absence last week as a result of illness 0.3 -17.8 -18.1 13.2 13.0 (10.2) (8.2) Mid-Grade Students Student's hygiene and sanitation knowledge score 1.0 0.9 -0.0 1.3 0.3* (0.2) (0.1) Student reports absence last week as a result of illness 1.7 -2.1 -3.8 10.6 9.0 (12.1) (9.1) All Students Head teacher reports school has improved latrines 11.4 11.8 0.3 8.6 -2.8 (7.5) (5.0) Head teacher reports school has tippy taps or hand-washing stations 29.4 1.7 -27.7** 19.6 -9.8 (9.0) (6.1) Head teacher reports all tippy taps or hand-washing stations have ash or soap 26.7 5.7 -21.0** 21.0 -5.7 (7.0) (4.2) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,471 Comparison; 2,576 School feeding only; 5,503 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌48 Stakeholder Perception on Student Knowledge and Health Practices Parents shed some light on what students were learning and what health practices they were applying at home. Parents perceptions of the health education practices were positive. Most PFG participants mentioned that their children learned the importance of washing their hands at school. The next common themes that parents described were that their children learned personal hygiene (including bathing, cutting their fingernails, and brushing their hair) and oral hygiene (including brushing their teeth). Eight of the 30 PFG participants described that their children shared their knowledge with the parents or their siblings. Parents also described that the children’s lessons supported the parent’s responsibilities, as students would take more initiative with household chores, including washing dishes, sweeping, and cleaning their clothes. Two parent focus groups mentioned specific cleaning practices, such as that the children learned to sweep with the wind, so that they do not inhale the dust. Nearly half of the PFG participants mentioned they noticed their children washing their hands before or after eating. “…they are applying what they have learnt at school. Because we can see that what he is doing he has not learnt at home but at school… we ask ‘where did you learn that?’ and he says ‘Teacher taught me about hygiene.’” - Parent Focus Group (Nampula) Abt Associates June 2022 ▌49 9. FINDINGS ON IMPLEMENTATION 9.1 Summary of Findings Capturing implementation in October 2021, which for some schools was over a year after services were provided, has challenges. Whenever possible, we asked participants to describe programming prior to COVID-19, but respondents may have mingled descriptions of programming in October 2021 and how it was implemented prior to COVID-19 when implementing partners were more active. We believe the overall results, across provinces, are useful in understanding intervention receipt concurrent with the measure of intervention effectiveness. When considering implementation across treatment schools and provinces, results of the fidelity analyses suggest that the school feeding and literacy interventions were adequately implemented. The availability of similar feeding and literacy activities in comparison schools was also examined. Although there was evidence that alternative programming of a similar nature to McGovern-Dole was concurrently being implemented in comparison schools,14 McGovern-Dole schools had higher implementation scores. This suggests that there was a contrast between treatment and comparison schools that could lead to greater improvements in the treatment schools. On average, the schools in Nampula were rated as having better implementation of their interventions than the schools in Maputo. Nampula schools implementing school feeding and literacy programming received the highest scores. There is evidence that implementation in Maputo schools was severely limited largely due to endline data collection occurring when services were disrupted, and the implementing partner was no longer operating in the districts. Although we collected data at the same chronological time, the provinces were not at comparable stages of implementation. This is likely a key factor in the differences in fidelity scores between the two provinces. A closer examination of implementation fidelity at the school level reveals that several schools were no longer receiving intervention components at endline in 2021. In Maputo, 68 percent of treatment schools no longer received the school feeding intervention. This is likely due to COVID-related school closures and transitions in implementing partners. 9.2 Determination of Implementation Adequacy Across the treatment arms, 32 percent of schools implemented intervention elements with an adequate level of fidelity. Schools with feeding and literacy interventions had the highest percentage of adequate implementing schools. It should be noted, however, that 7 percent of comparison schools were scored as adequate implementers on the collective elements of school feeding, literacy instruction, and teacher training (Exhibit 16). This may be an indication that alternative programming was being delivered in comparison schools and the treatment￾comparison contrast was smaller than anticipated. Existence of alternative programming in comparison schools, though not uncommon in real-life evaluations, can diminish our ability to understand intervention effectiveness. 14 In particular, comparison schools across Maputo were implementing literacy instruction and teacher training activities that were comparable to the McGovern-Dole although overall less intensive. Abt Associates June 2022 ▌50 Exhibit 16. Proportion of Adequate Implementing Schools Included in Analyses Treatment Condition Inadequate Adequate Missing Comparison 165 (93%) 12 (7%) 0 (0%) Feeding only 51 (58%) 37 (42%) 0 (0%) Feeding + literacy 84 (47%) 95 (53%) 1 (1%) Total 300 (67%) 144 (32%) 1 (<1%) 9.3 Implementation of School Feeding On average, the school feeding intervention was delivered with adequate fidelity in schools in Nampula implementing the school feeding and literacy interventions (Exhibit 18). In Maputo, the feeding intervention was not being implemented with adequate fidelity at the time of endline data collection regardless of whether the feeding intervention was implemented alone or in combination with the literacy intervention. In this case, it is better to interpret these results as an indication of sustainability and conclude that programming was not sustained in Maputo after the withdrawal of implementing partners and upon school reopening in March of 2021. The lack of adequate implementation of the feeding intervention may be part of why there were no positive impacts on secondary feeding outcomes, for example, on student short-term hunger, which is expected to improve attentiveness. The majority of schools in Maputo (both treatment and comparison) and Nampula (comparison only) did not have a school feeding intervention in place during endline data collection (Exhibit 17). Schools served by World Vision in Nampula reported that feeding activities continued in the 2021 school year once school reopened and were in place when school staff and students were surveyed. Exhibit 17. Number of Schools Without a Feeding Intervention at Endline Study Arm # Schools % Schools Maputo comparison 61 of 89 68.5% Maputo food & literacy 65 of 90 72.2% Maputo food only 56 of 89 62.9% Nampula comparison 89 of 89 100.0% Nampula food & literacy 0 of 89 0.0% Note: Head teacher survey item: Does the school have a school feeding program? [Answer=No] Stakeholder Perception of School Feeding Implementation In the cook survey, when asked what changes they observed since they started working at the school as a cook, 12 cooks from Maputo and 8 from Nampula described that there were not enough kitchen supplies at endline; more specifically they mentioned that there were fewer dishes or kitchen utensils. Similarly, there were some, but not many, instances of cooks describing “improvements” or construction to the kitchen infrastructure. Sixteen cooks from Maputo and nine from Nampula mentioned improvements; some said that the cooking space (or food storage space) “we know that [the school meal prep] work is not easy, [as time went on we realized] that they should also have some incentive, right? At the end of the month they should have a part of the amount of flour, right? This ended up encouraging the community itself to continue doing that kind of work.” - Key Official (Nampula) Abt Associates June 2022 ▌51 was created or constructed due to the project, or that the cooking area was given more space over time. Key officials described other challenges with implementing the school feeding intervention. Namely, some reported that it was difficult to convince parents or volunteers to prepare the school meals if there was a weak incentive to do so. Overall, as described in Section 7, key officials still felt that when school feeding was implemented, there was an impact on student attendance and attentiveness. 9.4 Implementation of Literacy Instruction On average, literacy instruction was delivered with adequate fidelity across schools in Maputo and Nampula (Exhibit 18). Using the same implementation fidelity measure, there is evidence that comparison schools located in Maputo can be considered to have received a literacy intervention that met the criterion for adequate fidelity (although the fidelity score, on average, was lower than for the McGovern-Dole schools in the province). In comparison schools in Nampula that were implementing a literacy intervention, it was delivered with low fidelity (average fidelity score across Nampula comparison schools = score 5.7 of 12). 9.5 Implementation of Teacher Training Since teacher training is a potential mechanism through which students can be exposed to higher quality literacy instruction, which could lead to increases in literacy, we explored the degree to which teacher training and support was implemented with fidelity. Analyses suggest that teachers in treatment schools in Nampula and Maputo received training that was delivered with adequate fidelity. Nampula McGovern-Dole schools experienced the highest level of implementation fidelity (average fidelity score across World Vision treatment schools = score 10.80 of 12). PAI delivered the literacy intervention with adequate fidelity to McGovern-Dole schools in Maputo (average fidelity score across PAI food and literacy treatment schools = score 9.83 of 12; food only = score 9.15 of 12). Using the same fidelity measure, there is evidence that comparison schools located in Maputo experienced a lower degree of training than treatment schools (average score across Maputo comparison schools = score 5.98 of 12). Comparisons schools in Nampula also received inadequate literacy programming (average score across Nampula comparison schools = score 5.03 of 12). Exhibit 18. Average Fidelity Scores Across School by Intervention Component and by Treatment Arm Study Arm Average Feeding Implementation Scores (0-15) Average Literacy Implementation Scores (0-12) Average Training Implementation Scores (0-12) Maputo comparison 3.32 7.08 5.98 Maputo food & literacy 6.14 7.83 9.83 Maputo food only 7.34 7.70 9.15 Nampula comparison 0.11 5.70 5.03 Nampula food & literacy 8.72 9.53 10.80 Note: Intervention elements are considered to have been implemented with “adequate” fidelity if they delivered at least 50 percent of the measured items (e.g., received a score of 5 and above out of 10). Abt Associates June 2022 ▌52 Stakeholder Perception of Implementation Reported by parents and key officials, qualitative analysis found that community engagement, specifically local buy-in and parent participation, was important to the success in implementing the school feeding intervention. On the other hand, some key official respondents noted that the lack of buy-in sometimes hurt the implementation. They attributed this to an initial lack of understanding about the benefits of the intervention. “For parents, school feeding is a great opportunity to participate in school activities. The parents were giving their contribution in terms of labor for the construction of small infrastructures such as the kitchen, but the biggest contribution was in the preparation of meals and productive activities.” - Key Official (Maputo) “…in the early years, people sometimes refused to go to prepare the porridge, so the ministry had to go down there to find out what was happening, and we ended up discovering that it was because we had trusted the school council and not the local leadership… but [the leader] managed to mobilize the community... So, we have to have this perspective that if we need to introduce something new, the first people to be informed are the leaders.” - Key Official (Maputo) Abt Associates June 2022 ▌53 10. FINDINGS ON IMPACT CONTROLLING FOR IMPLEMENTATION 10.1 Summary of Findings Analyses of the impacts on reading performance were conducted using the subset of treatment schools with adequate implementation of their assigned interventions. For the schools implementing combined feeding and literacy interventions, the positive and significant impacts were stronger for the adequately implementing treatment schools. For the schools implementing only feeding interventions, the results were no different for adequately implementing schools compared to analyses using all schools. 10.2 Impact Findings for Schools with Combined Literacy and Feeding Interventions For average mean reading comprehension scores, early-grade students in adequately implementing schools experienced an increase of 5.4 percentage points (p<.05) and mid-grade students experienced an increase of 10.9 percentage points (p<.01) (Exhibit 19). When including all treatment schools, regardless of implementation adequacy, average mean reading comprehension scores for early-grade students increased by 4.8 percentage points (p<.01) and for mid-grade students by 7.6 percentage points (p<.05) (see Exhibit A12 for more detailed findings on reading comprehension). Limiting this analysis to only adequate implementers demonstrates that higher implementation is associated with stronger intervention impacts on student reading comprehension. Below, we also visually depict this finding using the average regression-adjusted mean reading comprehension score across all adequately implementing schools in treatment conditions at baseline and graphically depicting the change over time separately for the same schools in each condition at endline (Exhibit 19). Though reading comprehension was the main indicator of interest, we also evaluated student performance on the other five EGRA subtasks related to literacy (Exhibit A36 through Exhibit A41). 10.3 Impact Findings for Schools with Feeding Interventions Only As was the case for impact analyses using all treatment schools implementing feeding interventions, analyses using only adequately implementing schools with school feeding alone did not improve average mean reading comprehension relative to comparison schools. While it is a primary outcome, reading comprehension is one of six subtasks of the EGRA. Exhibit A36 through Exhibit A41 report findings for the other five EGRA subtasks (oral comprehension, letter identification, syllable identification, oral reading fluency, and word dictation). Abt Associates June 2022 ▌54 Exhibit 19. Impacts on Average Mean Reading Comprehension Scores (Adequately Implementing Treatment Schools Only) Notes: “Early-Grade Students” were in grade 2 at baseline and grade 3 at endline. “Mid-Grade Students” were in grade 4 at baseline and grade 5 at endline. “C” denotes comparison schools; “SF” denotes school feeding only schools; “SF+L” denotes school feeding and literacy schools. Exhibit reports the unadjusted, weighted average outcome for each condition at baseline. The depicted outcome at endline is the sum of that average at baseline and estimates from the regression model. Y-axis for this exhibit (not shown) runs from 0 to 40 percent. Unlike impacts on reading comprehension, adequately implementing schools in either treatment conditions (school feeding plus literacy and school feeding alone) did not improve average mean student attendance relative to comparison schools (Exhibit 20). At baseline, roughly a quarter of students were absent at least once in the prior week. That proportion of absenteeism remained relatively stable – if not decreasing slightly – at endline. These findings mirror the lack of impacts on attendance (absences) across all treatment schools regardless of implementation adequacy (Exhibit A23 and Exhibit A24). Abt Associates June 2022 ▌55 Exhibit 20. Student Attendance (Measured by Any Absence Last Week; Adequately Implementing Treatment Schools Only) Notes: “Early-Grade Students” were in grade 2 at baseline and grade 3 at endline. “Mid-Grade Students” were in grade 4 at baseline and grade 5 at endline. “C” denotes comparison schools; “SF” denotes school feeding only schools; “SF+L” denotes school feeding and literacy schools. Exhibit reports the unadjusted, weighted average outcome for each condition at baseline. The depicted outcome at endline is the sum of that average at baseline and estimates from the regression model. Y-axis for this exhibit (not shown) runs from 0 to 40 percent. Unlike impacts on reading comprehension, adequately implementing schools in either treatment conditions (school feeding plus literacy and school feeding alone) did not improve average mean student attentiveness relative to comparison schools (Exhibit 21). These findings mirror the lack of impacts on attentiveness across all treatment schools regardless of implementation adequacy (Exhibit A25 and Exhibit A26). Abt Associates June 2022 ▌56 Exhibit 21. Impacts on Teacher-Reported Student Attentiveness (Adequately Implementing Treatment Schools Only) Notes: “Early-Grade Students” were in grade 2 at baseline and grade 3 at endline. “Mid-Grade Students” were in grade 4 at baseline and grade 5 at endline. “C” denotes comparison schools; “SF” denotes school feeding only schools; “SF+L” denotes school feeding and literacy schools. Exhibit reports the unadjusted, weighted average outcome for each condition at baseline. The depicted outcome at endline is the sum of that average at baseline and estimates from the regression model. For mid-grade students, this depiction results in a rate of student attentiveness above 100 percent in school feeding and literacy schools. This estimate should not be interpreted as implying that greater than 100 percent of teachers reported their students were attentive at endline, but instead is useful for reproducing the model-based difference-in-differences estimate via subtraction. Y-axis for this exhibit (not shown) runs from 80 to 105 percent. Abt Associates June 2022 ▌57 11. PROJECT ADAPTATIONS RESULTING FROM COVID-19 Parents, key officials, head teachers, and cooks all provided details on challenges and barriers related to the COVID-19 pandemic and how the McGovern-Dole projects were adapted in response to these. In the endline interviews, surveys, and PFG discussion respondents were asked about the impact of the COVID-19. Often, before data collectors even specifically mentioned the pandemic, it was clearly “front of mind” for the respondents. The COVID-19 interruptions and modifications are categorized as those affecting school operations, school feeding programming, and school health services. 11.1 COVID-19 and School Operations In March 2020, with the closure of in-person teaching, schools switched to remote learning, including the use of community radio and television, video conferencing, and WhatsApp to conduct remote classes. A few key officials raised concerns because of unequal access to the technologies, such as internet access. They also noted unequal assistance from parents. Some students have parents at home to help them with schoolwork, while others do not. When school was the primary source of learning, there was previously more equality of learning opportunities, since all students had access to support from teachers. Some key officials felt that the inconsistent schooling hurt early grades the hardest and argued that “face-to￾face” time with teachers was fundamental to student learning. Some referenced that with breaks in their learning, students were not as motivated to continue with at-home school or to return to school after COVID-19 restrictions were lifted. Another role that parents were expected to play at some schools was coming to school to obtain materials for children to complete at home, such as exercise sheets. Some key officials and teachers discussed challenges at these schools with parents not picking up materials for their children, or the school not being able to produce enough materials for all students. For example, one respondent mentioned that the school made an effort to copy the materials for all students but that there were not enough handouts for students to take home. Parents also detailed changes in the patterns of their students attending school and changes in their motivation. Some attributed this to students prioritizing agricultural work over schoolwork, which some parents preferred. One PFG explained witnessing differences in the teaching style throughout the pandemic, which has interrupted student’s studies and progress. However, the parents were hopeful that “The handouts were not enough… the parents didn't even take them to heart, but the teachers, the principals did mobilization work, so that the parents came to school to pick up the handouts to give their children.” - Key Official (Nampula) “…[children working from home] when the network is poor…How do you solve the problem of exclusion?” - Key Official “…[first and second graders have] classes twice a week, and the rest of the days they are at home…this negatively affects the [learning] process… in these initial classes the physical presence of the teacher to follow up is indispensable... there was no shortage of appeals to urge teachers to develop [take￾home exercises] from March of last year until January of this year. This process did not work very well, there was a setback.” - Key Official (Nampula) “This disease has killed the school, they are not studying at all.” - Parent Focus Group (Nampula) Abt Associates June 2022 ▌58 getting back to school daily would re-motivate the students. 11.2 COVID-19 and School Feeding Interventions School closures halted the usual implementation of the McGovern-Dole feeding intervention, resulting in fewer students at endline reporting that they were receiving a meal from their school. Key officials, cooks, and PFG respondents referenced stopping other activities within the feeding intervention, including the school gardens and agricultural classes. According to the school cooks from McGovern-Dole schools, most (87 percent) of the schools did not provide a daily meal while the schools were shut down (Exhibit 22). Additionally, a majority of the cooks reported that the school garden activity was halted (79 percent) (Exhibit 23). Exhibit 22. Percentage of Treatment Schools Where Students Received a Daily Meal During COVID-19 While schools were shut down during COVID-19,* did students still receive daily meal? Yes No # % of respondents # % of respondents Maputo (n=149) 5 2.1% 144 60.7% Nampula (n=88) 25 10.5% 63 26.6% Total (n=237) 30 12.6% 207 87.3% Source: Cook survey * In the surveys, “during COVID-19” refers to the period from March 2020 to March 2021. Exhibit 23. Percentage of Treatment Schools Where School Garden Was Cared For During COVID-19 While schools were shut down during COVID-19, did anyone take care of the school gardens? Yes No # % of respondents # % of respondents Maputo (n=149) 38 16.0% 111 4.6% Nampula (n=88) 10 4.2% 78 32.9% Total (n=237) 48 20.3% 189 79.7% Source: Cook survey During school closures, students and families may have had access to take-home rations, but families were more likely to receive rations in the following academic year, when schools reopened with adjustments (Exhibit 24).15 Some parents mentioned receiving cornmeal and soybeans to take home, but in most cases it was once, at the beginning of the pandemic when schools first closed. Of the 122 head teachers who reported students were given take-home rations, 111 (90.9 percent) of them reported that they were distributed only once, early in the shutdown. 15 Exhibit 24 is based on responses to the head teacher survey. In Nampula, World Vision, with agreement from USDA, provided parents with take-home rations for all students. “The food was packaged in plastic bags and parents were called in small groups to receive the food according to the number of children they had at school, and the bags were already identified with the names of the beneficiaries.” - Key Official (Maputo) “After school closures the children did not receive this kit, but this year when the schools reopened, each child received a kit with five kilos of cornmeal” - Parent Focus Group (Nampula) Abt Associates June 2022 ▌59 Exhibit 24. Percentage of Schools Where Families Received Take-Home Rations During COVID-19 While schools were shut down during COVID-19 did students receive take-home rations? Yes No # % of respondents # % of respondents Maputo (n=178) 51 19.2% 127 47.7% Nampula (n=88) 71 26.7% 17 6.4% Total (n=266) 122 45.9% 144 54.1% Source: Head teacher survey Some key officials described that stopping school would mean negative impacts on enrollment, attendance and attentiveness. They linked receiving meals at school as a positive aspect that drew in students to be more engaged with school. 11.3 COVID-19 and Health Services Key officials and PFGs talked about sanitary practices implemented in schools to minimize the impact of the COVID-19 pandemic and allow schools to reopen. This included people wearing masks, promoting hand washing, and sanitizing spaces and providing hand sanitizer. One key official emphasized the importance of parents’ help implementing new sanitation guidelines and getting access to the proper materials. Some interviews mentioned a lack of resources or materials to effectively sanitize learning spaces, especially as time went on. A few key officials connected the access to sanitation materials with their relationships with partners, such as Terra dos Homens and UNIESCO. One key official described that after a while they ran out of sanitation material, and that while support for buying sanitation materials was helpful, there was never enough. Parents were asked about whether their children received deworming medication (albendazole or mebendazole) during school closures. A third of the PFG respondents answered “yes,” that their children received the medication. Most of the other two-thirds said that the children did not receive the medication. Three PFGs were split (with some saying “yes” and others saying “no”) or were unsure. “…[in many schools] that had no masks suddenly in a short time already had masks… helped to clean the school, to create conditions for hand washing, all this the parents and guardians contributed… they clean, they weed… all this to facilitate the literacy of their children. The participation of parents and guardians is admirable.” - Key Official (Maputo) “The children already had that rhythm of going to school to have their meals, and I think that interrupted the desire they had to stay in school.” - Key Official (Maputo) Abt Associates June 2022 ▌60 12. EVALUATION CONSTRAINTS AND LIMITATIONS Interpretation of the evaluation findings is limited by several key factors. These include factors related to outcomes measure, internal validity of the impact estimates, timing of baseline, and precision of the impact estimates. Two of the primary outcome measures were based on survey responses that are subject to potential bias. Student attendance was measured with self-reported data from young children, and student attentiveness was measured using teacher recall in the teacher survey. In both cases, these measures could suffer from bias due to recall and measurement error. However, given that the survey included a substantial number of students and teachers, the effect of the expected errors could be minimized. For future evaluations, it is advisable to consider alternative measures of outcomes like student attentiveness to capture impacts at the student level more precisely. The evaluation interprets the estimates from the quasi-experimental difference-in-differences analysis as the causal impact of the McGovern-Dole projects. A strength of the evaluation design is that it is based on similar data collection efforts at baseline and endline in the same schools. Despite that strength, causal interpretation requires the conditional parallel trends assumption. The series of fixed effects included in the difference-in-differences model control for unobserved factors at the school, school-by-grade, and province-by-time levels such that conditional parallel trends assumption plausible. However, those fixed effects do not capture all unobserved factors, including any other unobserved shock at endline that differentially affects treatment and comparison schools. The existence of such “unobservables” would threaten the conditional parallel trends assumption, and therefore the internal validity of the evaluation. The likely contamination of the comparison group complicates interpretation of the evaluation findings. The difference-in-differences design assumes the observed trend in the comparison schools can be used to project what the outcome would have been in treatment schools in the absence of treatment. This typically implies comparison schools do not experience any treatment. However, our assessment of implementation fidelity found that many comparison schools had some evidence of comparable feeding, literacy, and teacher training activities, although levels of implementation were determined to be low. For these schools, the implied comparison is not of McGovern-Dole projects to no intervention, but of McGovern-Dole projects to other programs. This could either attenuate or exacerbate the treatment effect and complicates interpretation of the average effect; however, if the conditional parallel trends assumption holds, it does not threaten the causal interpretation of the findings. Because program activities began prior to baseline, an evaluation limitation is that the impact estimates do not capture the impact of all programming. The impact estimates only capture changes between baseline and endline. If the activities that occurred prior to baseline measurement were very impactful, this would have the overall effect of biasing our impact estimates downward. However, in most cases it seems unlikely that this downward bias would be of sufficient magnitude to change our inference. Future evaluations should consider improving alignment between the timing of baseline and implementation of program activities. Three elements of the methodology for the evaluation merit discussion because of their potential to reduce the precision of the impact estimates: • School closures and gaps in project delivery • Sample size concerns for the impact analyses of the school feeding intervention Abt Associates June 2022 ▌61 • Limitations when conducting multiple comparisons 12.1 School Closures and Gaps in Project Delivery In Maputo, PAI was contracted to implement the McGovern-Dole program from 2015 to 2020. Counterpart was then awarded the contract from 2020 to 2025. However, at the time of endline data collection in October 2021, Counterpart still was preparing to implement the program. School feeding and literacy interventions were not yet in place creating a situation in which the study team was evaluating a project that had phased out a year before. Schools themselves were also functioning at a reduced level. When reopening in March 2021, after a year of COVID-19- related closures, many students were going to school only two or three times a week. The combination of COVID-19-related closures, partial school reopening, and the ending of PAI’s implementation in late 2020 could have posed even greater risks to sustainability, making it more difficult to capture implementation and effectiveness during endline data collection in October 2021. The fact that despite these disruptions the evaluation detects some positive impacts reflects positively on the sustainability of the McGovern-Dole program. Nampula schools and World Vision implementation was also affected by the COVID-19 shutdowns. But since the World Vision contract started in 2016 and ended in September 2021 through a no-cost extension, the implementer was still connected to the study schools when schooling resumed in March 2021. Across provinces, a prevailing issue with the evaluation is that timing of data collection in 2021 proved problematic in detecting delivery of school feeding and other program elements. Even if program delivery such as the provision of school meals, supplemental role of school gardens, access to clean water, and use of literacy materials was well established up until school closures in March 2020, their presence diminished in Nampula and disappeared almost entirely in Maputo by endline. Because of this, impacts may be difficult to detect, especially when capturing outcomes of interest like access to meals and student short-term hunger. 12.2 Sample Size Added Challenges to Detecting Impacts of School Feeding Only Because it was only tested in Maputo, the school feeding only treatment arm had a smaller quantitative sample than the literacy and school feeding arm or the comparison group. This smaller sample implies impacts had to be larger in that treatment arm than in the literacy and school feeding arm to achieve statistical significance. 12.3 Multiple Comparisons In this evaluation, we conducted a large number of tests by estimating impacts on several outcomes and subgroups. Studies that employ multiple treatment arms and either explore treatment effect heterogeneity or use multiple estimators of the treatment effect may choose to adjust for these multiple comparisons, as some significance across outcomes is likely to arise by chance. We considered multiple comparison adjustments for the primary impact estimates on student reading performance, attendance, attention, and hunger. We ultimately decided that for these analyses, multiple comparison adjustments were not essential because the primary outcomes represent different domains (i.e., academic performance, behavior, and attendance). Many of the findings are from exploratory analyses that are meant to help provide context for understanding the main effects. Abt Associates June 2022 ▌62 13. CONSIDERATIONS WHEN CONDUCTING EVALUATIONS DURING PANDEMICS AND NATURAL DISASTERS This evaluation of the McGovern-Dole projects in Mozambique accounts for project implementation across a five-year span of time, beginning in 2015 and extending through 2021. At the time that the baseline evaluation was designed in 2016, there was no way of predicting the extent to which a global pandemic would cause widespread school closures and extensive gaps in delivery. When contracted to conduct the endline evaluation in 2021, the world was over a year into the COVID-19 pandemic and Mozambique schools were reopening to students for the first time in a year. Given these realities, Abt’s study team members considered several enhancements and modifications to the evaluation approach. These approaches were developed in response to COVID-19 but could be applicable to evaluations conducted during a variety of health- and environment-related disasters. Due to travel restrictions, Abt Associates’ United States-based staff were unable to travel to Mozambique for meetings and presentations typically held in person. With the direct connection to the focus communities severed, we understood the heightened need for a trusted and expert in￾country local consultant and the importance of quicky establishing an in-person and virtual hybrid approach. The role of the consultant was expanded as necessary throughout the contract to carry out tasks that would have otherwise been handled by the Abt study team. During the endline data collection process, we benefitted from the participation of the in-country consultant in establishing and maintaining communication with implementing partners, providing guidance and relaying supports to the Mozambique-located data collection during enumerator training and direct data collection, reviewing and participating in data analysis and interpretation, and attending dissemination events in Maputo and Nampula provinces with stakeholders at the conclusion of the study. For this impact evaluation, the contracted data collection firm traveled to schools and conducted in-person student assessments, school staff and student interviews, as well as PFGs. At this point, COVID-19 transmission rates were high and vaccines were not readily available. Person-to￾person proximity and contact remained dangerous to both the data collection team and study participants. The period of time between baseline and endline data collection was filled with many challenges. When endline data were collected in 2021, Mozambican schools were working to reopen and were not yet operating at full capacity. In retrospect, a midline evaluation in 2019 with robust data collection mirroring the baseline approach could have captured the state of implementation and any changes in student and school outcomes at that point. These findings, collected prior to pandemic-related school closures, would have informed the endline data collection and analysis and helped interpret intervention effectives. Regardless of health crises and natural disasters, interim and systematic data collection efforts guard against challenges at endline and provide additional evidence of intervention impacts. Abt Associates June 2022 ▌63 14. CONSIDERATIONS FOR MCGOVERN-DOLE PROGRAM SUSTAINABILITY BY STAKEHOLDERS Parents, key officials, head teachers, as well as cooks were all asked about their thoughts on program sustainability as well as on recommendations they have for consideration. The study team documented a preliminary set of themes based on stakeholder responses. 14.1 Considerations for School Feeding Meal and nutrition quality: Despite parents largely having positive perceptions of the school feeding intervention, parents from half of the PFG discussions raised meal quality concerns and recommended that the school meals should provide a greater variety. Some respondents noted that sometimes the children did not like the food and wasted it. More than one PFG participant suggested alternating what was served “porridge today, rice tomorrow.” Another suggested that a variation in food, like “rice, xima [cornmeal porridge], beans and other foods” would provide more nutritional benefit. From a broader perspective, a key official interviewed felt there was a contradiction between the nutrition training and educational component of the project compared to the nutrition of the school meals being provided. Food access: A few PFG discussions made specific recommendations related to the supply of food. A few discussants mentioned a need for improved storage and warehouse conditions, noting that the soy or flour they received was spoiled. Another PFG participant mentioned it would be helpful to have a protocol for what should be purchased with the revenue generated from the sale of products. A few respondents mentioned needing stronger connections or associations with farms and other agricultural stakeholders. A respondent described that a school farm is helpful, but there are challenges if it is only seasonal and cannot be a stable supplier for school meals. School gardens, water, and other agricultural inputs: Despite reported challenges with school gardens, key officials wanted to continue them and had several recommendations. Needing help from partners was acknowledged, especially when it comes to access to inputs required for making gardens successful. Respondents noted that many inputs were necessary along the production chain, including seeds, water, fertilizer, garden tools, machinery (such as motor pumps), infrastructure (such as fences to keep out animals), and human resources for planning and labor. “School vegetable gardens should exist in every school. The goal is not only to produce for the children to eat, but also to teach the children to know how to do it at home and for the future. We would need partners to support the School Vegetable Garden Program, because we already have the land, some schools have water, we would need reinforcements in inputs and other materials (e.g., electric water pump).” - Key Official (Maputo) “They learned a lot about eating habits, how they should eat, how to improve their diet, [and] the variety of foods. But there was a contradiction, because they prepared soy every day, and this aspect is linked to the curriculum, so ADPP provided us with some nutrition books/manuals that were used as support in the teaching of [on] food groups… the child learned that he should vary the foods to improve the diet, but at school they ate soy every day.” - Key Official (Maputo) Abt Associates June 2022 ▌64 Many respondents from PFG discussions and key official interviews cited issues with access to water for school gardens. Several PFG discussions recommended the installation of irrigation systems, or a borehole, to help extend the growing season. A few also noted that the lack of water at schools also prevented the preparation of school meals. Several PFG discussants recommended needing a greater variety of seeds to provide a greater variety of food for school meals. One pointed out that access to the seeds must coincide with seasonal changes and planting timelines. Role of the cooks: Several respondents mentioned formalizing the role of the cooks. Respondents recommended doing this through providing stronger incentives (including paying/hiring cooks) and formalizing the management of the cooks by providing a clear schedule or a role description with clear expectations. One key official noted that using volunteers was not always reliable, and “calls into question the expected results and more than that, compliance.” Some felt that paying the cooks would strengthen their commitment to the helping at the school. 14.2 Considerations for Literacy Training: Many respondents, including parents and key officials, thought that more training was necessary to maintain the knowledge and sustain the project. A few elaborated that there could be stronger follow-up monitoring of the training, potentially to identify needs for refresher trainings. Respondents recognized the value in training to provide teachers with methodologies to improve their teaching. School materials: Respondents linked materials and training to greater success and sustainability of the project. They noted that teachers needed training, but also the material resources (i.e., books) to implement what they are learning. Respondents suggested that materials could be recycled and used for future students. Bilingual teaching: As described earlier, several respondents recommended expanding the bilingual education. One key official detailed that he saw the value in bilingual, local language education, citing that he didn’t have that experience in school, but he thought he would be much better off now if he had had that experience at a young age. Further, he noted observing a difference in performance between schools where education was monolingual versus bilingual. Literacy teacher/coach incentives: Key official respondents felt that teachers should be provided stronger incentives and benefits for teaching and for attending trainings. “We have had difficulties in getting real seeds, sometimes they are brought at inappropriate time, to sow while others have already sown … [we’d like the donors] to allocate us early seeds.” - Parent Focus Group (Nampula) “The following year they should be trained again… [Partners] must continue to give these trainings because there are those with weak memories and they don’t take long to forget.” - Parent Focus Group (Nampula) “[Teachers] need water, juice… the project always gives them water, some allowance so that the teachers can travel from the countryside to the session location without having to sacrifice their pockets, or take out debts to a stall to get fuel for their motorcycle. These resources have to exist, and I would like the project to continue to provide them as much as possible.” - Key Official (Nampula) Abt Associates June 2022 ▌65 14.3 Considerations on Sustainability At least 12 respondents specifically mentioned challenges with continuity of the project, suggesting some frustration with the end of the project, and wishing that there was a better plan for sustainability. Local buy-in, support, and involvement: A prominent theme across parent and key official respondents was the importance of bolstering the local community for the sustainability and success of the project. Key officials especially felt it was necessary to mobilize the community, potentially by including them more in the planning, or by making the goals of the project clearer from the start. A key official noted that training for, and support from, the local government could be especially influential given that they can promote and describe the purpose of the project to others. A few respondents felt that incorporating local feedback into the design of the project was important. For example, one key official noted that implementation seemed to be “based on studies carried out in a generalized and non-specific way.” They gave the example that the enriched porridge served pleased a group of children and parents but not other groups. Training and capacity building: In relation to the literacy intervention, several key officials reported that training should continue in order to promote sustainability. In some interviews, respondents also felt that training for cooks, and people helping in the gardens or on school farms should also continue to get training. Some respondents told us that, beyond training, projects and stakeholders should provide “testimonies” documenting successes and the value of the project and how to carry it forward. A few specifically recommended incorporating stronger capacity building plans to do so. “The small leaders have the responsibility to monitor the activities and make sure they are successful, so I think there should be training for them to explain what this is, [and the] goal to achieve... In everything we do, we have to understand the "why" and "what for" so that we can work towards a common goal. The idea is very good, but we have to involve everyone, and divide up the responsibilities in order to have the effect, the impact of our project.” - Key Official (Maputo) “It's important that somehow more research is done at the community level to reach a consensus… I cannot dictate an individual's diet based on the experience of another.” - Key Official (Maputo) Abt Associates June 2022 ▌66 15. RECOMMENDATIONS FOR FUTURE MCGOVERN-DOLE PROGRAMMING 15.1 Expand Awareness of McGovern-Dole Program Theory of Change and Impact Evaluation Criteria The McGovern-Dole program posits that improved literacy instruction paired with regular school meals will lead to both improved literacy and improved health and dietary practices of students via various program impact pathways. Evaluations of McGovern-Dole projects in Mozambique have shown that while provision of school meals and associated reports of almost zero hunger in beneficiary schools as well as availability of WASH (Water, Sanitation, and Hygiene) infrastructures and reading materials are the most visible components of the program, the impact of these inputs on the students’ literacy skills has not been that substantial. In fact, from interviews with key stakeholders, in particular parents and other community members, the school feeding component of the program is the most referenced program component, but not its impact on students’ learning, or more specifically on literacy development. This seems to suggest that, while implementers and education authorities are aware of McGovern-Dole program theory of change and impact evaluation criteria, most of the other groups of stakeholders are not, including school managers, teachers, parents, and other members of the beneficiary communities. Recommendation 1: The study team recommends involving stakeholders in understanding and engaging in the McGovern-Dole program theory of change and identified outcomes. By using non-technical registers tailored to different audiences, USDA and McGovern-Dole leaders can enhance stakeholder capacity to assess the progress and impact of the projects. Parents and other community members, particularly through school councils, have been playing a key role in monitoring school management, and teacher behavior and performance. Knowing the overarching goal of the McGovern-Dole program, the criteria for impact evaluation, and the interconnection among the different components of the program would help them better monitor progress toward that goal. The same applies to school managers, education authorities, and local government authorities. 15.2 Improving Teachers’ Capacity to Support Student Development of Early Reading Skills Despite the positive impact of the McGovern-Dole programming on literacy development, EGRA scores also show that most students assessed do not have the foundational skills needed to become fluent readers and use their reading ability to learn, even by fourth grade. Studies show that there is correlation between lower-order (letter knowledge, vocabulary, familiar words reading, oral listening comprehension) and high-order reading skills (oral reading fluency and reading comprehension) in such a way that the former predict the latter. Moreover, there is also a positive relationship between oral reading fluency and reading comprehension: the speed and accuracy at which a person reads correlates closely with reading comprehension (Daane et al., 2005; Abadzi, 2011). Therefore, the overall weak performance in reading comprehension uncovered in this study can be explained by the poor results on ‘letter knowledge,’ ‘familiar words reading,’ ‘oral listening comprehension,’ and ‘oral reading fluency.’ Moreover, although the study found that teachers in treatment schools in Nampula and Maputo received training in literacy instruction that was delivered with adequate fidelity, it is important to consider that only about 60 percent of the teachers surveyed reported that, during pre-service training, they received specific training on how to teach reading and writing to students in the primary grades. This suggests that teachers need continuous training in literacy instruction and highlights the study Abt Associates June 2022 ▌67 team’s recommendation to strengthen the connection between teacher training programs and local schools. Recommendation 2: The study team recommends that teacher training increase focus on early￾grade literacy instruction, allowing teachers to better understand the casual relationship between lower- and high-order reading skills and enhance their capacity to help children develop these two levels of reading growth. We are not suggesting transforming language arts classes into sessions for preparing students to succeed in EGRA tests (i.e., “teaching to the test”), but to include in such classes opportunities for them to reflect and practice pre-reading and actual reading skills (reading fluency and reading comprehension). Teachers and students must be aware that the goal of reading acquisition is to be able to construct meaning out of what one reads and use the reading ability to learn. 15.3 Language of Early-Grade Literacy Development Study results indicate that between 13 and 31 percent of students were instructed in a language different from the one they speak at home (Exhibit A9), which may have a bearing on the overall weak EGRA results obtained. In fact, studies have shown that early literacy development is best achieved in the child’s home language (e.g., Bamgbose, 2000; Fafunwa, 1990). When a child learns how to read in a language different from the one spoken at home, that child is faced with two levels of challenges – to learn the basics of the new language at the same time that he or she learns how to read. The co-occurring challenges diminish his or her chances to develop reading skills (e.g., Bamgbose, 2000). In interviews with key stakeholders, in particular with parents and education authorities, respondents frequently called for the introduction or expansion of bilingual education inclusive of local African languages. Recommendation 3: The study team recommends support for the scaling up of dual language early literacy instruction inclusive of languages familiar to the child (their home languages). Mozambique is already implementing mother tongue-based bilingual education in selected schools throughout the country, using 17 African languages for early instruction. The McGovern-Dole program may help local education authorities to expand the bilingual program to more schools or to cover more classes in schools where it is already in place. 15.4 Increase Gender Sensitivity and Improve Instruction Focusing on Girls About half of the students (47 percent) in the school feeding only intervention who benefited from McGovern-Dole were female and about half of the students interviewed in this study were female. Enrollment and data collection efforts confirm that there is gender parity in access to education in the schools targeted by McGovern-Dole programing. However, literacy outcomes show that boys performed better than girls. Average reading comprehension scores improved for both early-grade and mid-grade boys in schools implementing both literacy and school feeding interventions. Impacts on reading for girls is smaller and not statistically significant, although the difference in impacts between boys and girls is also not statistically significant. This reading achievement gap is consistent with the general pattern in Mozambique, particularly in rural settings (MINEDH, 2014, 2017; Bassi et al., 2019). Studies suggest that this pattern has to do, at least in part, with rural communities devoting less attention to the education of girls compared to boys, lack of gender sensitivity in materials and teaching practices, which tend to privilege males, and scarcity of female role models among teaching and school management staff (Bagnol et al., 2015). Less than a quarter of head teachers involved in this evaluation are female. McGovern-Dole programing is already addressing some of these issues, especially through Abt Associates June 2022 ▌68 community involvement in education and gender sensitive teacher training interventions, but those can still be improved and expanded. Recommendation 4: The study team recommends improving the environment to allow girls to have equal opportunities for learning in schools, homes, and larger communities. We suggest meeting this objective by sensitizing communities about the importance of girls’ education, training teachers on gender-sensitive classroom interaction and reading practices, and incorporating a gender-sensitivity approach in the production of teaching and learning materials and in the selection and hiring practices for school leadership positions. 15.5 Ensure the Sustainability of McGovern-Dole Programming When the evaluation took place in Maputo province, the McGovern-Dole program supported ADPP/PAI Food for Knowledge project had been phased out a year prior. It is important to note that none of the activities implemented by ADPP/PAI remained in place, including activities like teacher training in literacy instruction that could have been implemented by education authorities with certain modifications to mitigate the spread of COVID-19. Children who had been accustomed to receiving at least one meal at school suddenly found themselves without that kind of motivation to attend school. School fields and gardens lay fallow. These conditions created a situation in which outcomes gained over five years disappeared when the Food for Knowledge project ended. The situation in Nampula also suggests that at least the arrangements made for continuing the project initiatives, in particular school feeding, may not be sustainable. Farmer groups are challenged by relying on rain to water their crops and on very traditional agriculture technics. Consequently, they do not have the capacity to supply enough food to schools and throughout the school calendar. Recommendation 5: The study team encourages USDA/FAS, implementing partners, ministry officials and others to put in place strong exit strategies to allow sustainability of the USDA/FAS McGovern-Dole program after the end of designated projects, including through increased involvement of local education authorities and establishment of linkages with similar government programs. For example, Mozambique has the PRONAE national school feeding program in place, and it could be expanded to schools immediately after the McGovern-Dole projects end. The same can be applied to other project interventions such as literacy and WASH. Abt Associates June 2022 ▌69 16. REFERENCES Abadzi, H. (2011) Reading Fluency Measurements in EFA Partner Countries: Outcomes and Improvement Prospects, EFA, FTI Secretariat Update September 2011. https://documents.worldbank.org.pdf Bagnol, B., De Sousa, L., Fátima, F., and Cabral, Z. (2015). As barreiras à educação da rapariga no ensino primário na Zambézia. Maputo: IBIS. Bamgbose, A. (2000). Language and Exclusion: The Consequences of Language Policies in Africa. Hamburg: Lit Verlag Munster. Bassi, M., Medina, O., and Nhampossa, L. (2019). Education Service Delivery in Mozambique: A Second Round of the Service Delivery Indicators Survey. http://documents1.worldbank.org/curated/en/811891562864504006/pdf/Education-Service￾Delivery-in-Mozambique-A-Second-Round-of-the-Service-Delivery-Indicators-Survey.pdf. Chimbutane, F. (2011). Rethinking Bilingual Education in Postcolonial Contexts. Multilingual Matters. Chimbutane, F. (2015). Bilingual education: Enabling classroom interaction and bridging the gap between schools and rural communities in Mozambique. International Journal of Educational Development in Africa, 2(1), 101-120. https://doi.org/10.25159/2312-3540/19 Daane, M.C., Campbell, J.R., Grigg, W.S., Goodman, M. J., and Oranje, A. (2005) Fourth-grade Students Reading Aloud: NAEP 2002 Special Study of Oral Reading (NCES 2006-469). U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. Washington, DC: Government Printing Office. https://nces.ed.gov/nationsreportcard/pdf/studies/2006469.pdf. Diogo, J. (2018). Mid-Term Evaluation of the Mozambique McGovern Dole International Food for Education and Child Nutrition Educating Children Together - Phase II (ECT2). USDA. Doctors, S. (2019). Mozambique McGovern-Dole Food for Education Project Phase 2: Midterm Evaluation Report. USDA. https://pdf.usaid.gov/pdf_docs/PA00XF2T.pdf Doctors, S. (2021). Mozambique McGovern-Dole Food for Education Project: Final Evaluation. USDA. Fafunwa, A.B. (1990). Using national languages in education: A challenge to African educators. In UNESCO/UNICEF (eds) African Thoughts on the Prospects of Education for All (pp. 97– 110). Dakar: UNESCO Regional Office for Education in Africa. Guest, G., Namey, E.M., and Mitchell, M.L. (2013). Collecting Qualitative Data: A Field Manual for Applied Research. SAGE Publications. Global Partnership. (2021, September 10). Mozambique. https://www.globalpartnership.org/where-we-work/mozambique The International Centre for Evaluation and Development. (2019). World Vision Mozambique: Final Evaluation Report. Lechner, M. (2011). The estimation of causal effects by difference-in-difference methods. Foundations and Trends in Econometrics, 4, 165–224. Abt Associates June 2022 ▌70 Lee, J., D’sa, N., and Zuilkowski, S. S. (2020). Integrating new knowledge into everyday practices: Teacher pedagogy in early grade literacy in rural Mozambique. Literacy, 54(3), 111-122. https://doi.org/10.1111/lit.12221 Milhorance de Castro, C. (2018). PAA Africa’s Contributions to the Consolidation of PRONAE in Mozambique (Working Paper 170). Food and Agriculture Organization of the United Nations. https://agritrop.cirad.fr/597856/1/WP170_PAA_Africa_s_contributions_to_the_consolidatio n_of_PRONAE_in_Mozambique%281%29.pdf MINEDH. (2017). Avaliação Nacional da 3a Classe. Maputo: MINEDH. MINEDH. (2014). Avaliação Nacional da 3a Classe. Maputo: MINEDH. Ravesloot, B., Downen, J., Tuyiringire, J., Muteteri, D., Mujawase, J., Kayiraba, C., Mueller, M., Banwart, L., and Kaelin, J. (2021). WFP’s USDA McGovern-Dole International Food for Education and Child Nutrition Program’s Support in Rwanda 2016-2021: Endline Decentralized Evaluation. USDA. https://dec.usaid.gov/dec/GetDoc.axd?ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjktZT cxMjM2NDBmY2Uy&rID=NTk0MDM0&pID=NTYw&attchmnt=True&uSesDM=False&r Idx=MzEzNjA2&rCFU= RTI International. (2018). McGovern-Dole Mozambique Impact Evaluation and Literacy Assessment: Baseline Report. USDA. Safarha, E., Hoffman, V., Zaas, D., Gulemetova, M., Taube, A., Li, J., and Gulati, K. (2020). USDA McGovern-Dole International Food for Education and Child Nutrition Program Phase III in Mali Endline Evaluation Report. USDA. Sitao, V.J.R. (2018). An assessment of school feeding programme - Pilot phase and its relationship with enrolment attendance retention and the local agricultural production in Nampula province in Mozambique [Doctoral thesis, University of Pretoria]. UPSpace. http://hdl.handle.net/2263/67908 Tirivayi, N. (2019). Final Evaluation of the School Meals Programme in Malawi with financial support from United States Department of Agriculture (USDA)'s McGovern￾Dole Food for Education (Vol. II). USDA. https://pdf.usaid.gov/pdf_docs/PA00XX3F.pdf Turney, A., Noyes, D., Magaia, F., Dhliwayo, T.M., Machkasov, Y., Zacarias, E.C.D., and Hua, H. (2020). Effectiveness Evaluation: Early Grade Reading Assessment & Supplementary Tools. USAID. https://pdf.usaid.gov/pdf_docs/PA00X21D.pdf UNICEF. (2020). The Impacts of COVID-19 on Children in Mozambique. COVID-19 Policy Note. https://www.unicef.org/esa/media/6536/file/The%20Impacts%20of%20COVID￾19%20on%20Children%20in%20Mozambique.pdf USAID. (2021a, December 1). Education – Mozambique. https://www.usaid.gov/mozambique/education USAID. (2021b, May). Mozambique – Nutrition Profile. https://www.usaid.gov/sites/default/files/documents/tagged_Mozambique-Nutrition￾Profile.pdf USAID. (2014, June). Mozambique – Nutrition Profile. https://www.usaid.gov/sites/default/files/documents/1864/USAID-Mozambique-Profile.pdf Abt Associates June 2022 ▌71 Vyamana, V., Ntunda, P., and Dulla, R. (2017). End-Line Evaluation report for FEE II Project. USDA. https://dec.usaid.gov/dec/GetDoc.axd?ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjktZT cxMjM2NDBmY2Uy&rID=NTgwMTY4&pID=NTYw&attchmnt=True&uSesDM=False&r Idx=MzA0NjUy&rCFU= World Bank Group. (2018). Findings of the Mozambique Water Supply, Sanitation, and Hygiene Poverty Diagnostic. Washington, DC. World Health Organization. (2008). Indicators for assessing infant and young child feeding practices: conclusions of a consensus meeting held 6–8 November 2007 in Washington D.C., USA. World Health Organization (WHO) and the United Nations Children’s Fund (UNICEF). (2021). Progress on household drinking water, sanitation and hygiene 2000-2020: Five years into the SDGs. Geneva. World Vision Mozambique. (2016). End of Program Evaluation Report: Educating Children Together Project I. USDA. https://pdf.usaid.gov/pdf_docs/PA00XKT4.pdf Abt Associates June 2022 ▌72 APPENDIX A: SUPPORTING EXHIBITS MCGOVERN_DOLE RESULTS FRAMEWORKS Exhibit A1. McGovern-Dole Results Framework (SO1) Abt Associates June 2022 ▌73 Exhibit A2. McGovern-Dole Results Framework 2 Abt Associates June 2022 ▌74 Exhibit A3. Program logic model with representation of project implementation adequacy and impacts Abt Associates June 2022 ▌75 FIDELITY INDEX Exhibit A4. School Feeding Fidelity Index Data Source Survey Item Per Respondent Rescoring Across Respondent Scoring Decision Head Teacher Survey Does the school have a school feeding program? Yes = 1; No = 0 1 = 100% responded yes How often do students receive the school meals? (1) Daily, (2)A few times a week = 1; else=0 1 = 100% reported at least a few times a week Does the school have a garden? Yes = 1; No = 0 1 = 100% responded yes How often is the garden used to supply food for school meals? (1) Every day, (2) 3-4 times per week, (3) 1-2 times per week = 1; else =0 1 = 100% reported at least 1-2 times a week Cook Survey Does the school have a kitchen and storeroom for the school feeding program? Yes = 1; No = 0 1 = 100% responded yes Are cooking pots and cooking utensils (such as stirrers or scoops) available? Yes = 1; No = 0 1 = 100% responded yes Do you supplement the corn-soy blend with other foods? Yes = 1; No = 0 1 = 100% responded yes Does the school have bowls and spoons for students? Yes = 1; No = 0 1 = 100% responded yes Is there soap available to clean the bowls, spoons, pots, and cooking utensils? Yes = 1; No = 0 1 = 100% responded yes Is there a supply of clean water (from a borehole or rainwater tank) available at the school? Yes = 1; No = 0 1 = 100% responded yes Is there a place for children to wash their hands before eating? Yes = 1; No = 0 1 = 100% responded yes Is there soap available for children to wash their hands before eating? Yes = 1; No = 0 1 = 100% responded yes Did you receive any training for being a school cook? Yes = 1; No = 0 1 = 100% responded yes Is there a place for cooks to wash their hands? Yes = 1; No = 0 1 = 100% responded yes Is there soap for cooks to wash their hands? Yes = 1; No = 0 1 = 100% responded yes School Feeding Fidelity Score Range per school 0-15 Abt Associates June 2022 ▌76 Exhibit A5. Literacy Fidelity Index Data Source Survey Item Per Respondent Rescoring Across Respondent Scoring Decision Head Teacher Survey Do you have a school literacy program? Yes = 1; No = 0 1 = 100% responded yes Does the school have a library/book bank? Yes = 1; No = 0 1 = 100% responded yes Does the school have a literacy afterschool program? Yes = 1; No = 0 1 = 100% responded yes Are there other types of afterschool learning clubs? Yes = 1; No = 0 1 = 100% responded yes Teacher Survey Do you have a Teacher Guide for reading instruction? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Do you use the Teacher Guide for reading instruction? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Do each of your students have exercise books that can be used during reading instruction? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Do your students use the exercise books during reading instruction? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Do you have access to school supplies? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Do you use any school supplies in your classroom during reading instruction? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Is there an afterschool literacy club in this school? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Do your students attend an afterschool literacy club at this school? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Literacy Fidelity Score Range per school 0-12 Abt Associates June 2022 ▌77 Exhibit A6. Head Teacher and Teacher Training Fidelity Index Data Source Survey Item Per Respondent Rescoring Across Respondent Scoring Decision Head Teacher Survey Did you attend the teacher training held this academic year? Yes = 1; No = 0 1 = 100% responded yes (1); 0= less than 100% said yes (1) Did teachers at the school attend the literacy training this year? Yes = 1; No = 0 1 = 100% responded yes (1); 0= less than 100% said yes (1) Do reading coaches supervise teachers at the school? Yes = 1; No = 0 1 = 100% responded yes (1); 0= less than 100% said yes (1) Have you received training on the fundamentals of reading instruction? Yes = 1; No = 0 1 = 100% responded yes (1); 0= less than 100% said yes (1) Have you received training on school management? Yes = 1; No = 0 1 = 100% responded yes (1); 0= less than 100% said yes (1) Teacher Survey During your pre-service training, did you receive any specific training on how to teach reading and writing to students in the primary grades? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) This year, have you attended any in￾service training on how to teach reading in the primary grades? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Have you attended the WV/PAI training? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Have you received training on the fundamentals of reading instruction? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Does the Head Teacher or Assistant Head Teacher check your lesson plans? Yes = 1; No = 0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Since the beginning of the school year, how frequently does the Assistant Head Teacher or Head Teacher observe your teaching? (1) Quarterly, (2) Monthly, (3) weekly = 1; else=0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Since the beginning of the school year, how frequently does the reading coach visit you? (1) Quarterly, (2) Monthly, (3) weekly = 1; else=0 1 = 50% or more responded yes (1); 0= less than 50% said yes (1) Head teacher and teacher training Fidelity Score Range per school 0-12 Abt Associates June 2022 ▌78 SAMPLE SIZE BY OUTCOME INDICATOR Exhibit A7. Sample Size, Data Source, and Level of Analysis by Outcome Indicator Outcome Data Source Level of Analysis Literacy and School Feeding School Feeding Only Comparison Main Impacts Student reading performance1 EGRA assessment Student 5,504 2,576 5,474 Student attendance Student survey Student 5,491 2,574 5,473 Student attentiveness Teacher survey Teacher 634 311 631 Secondary Impacts Teacher-reported receipt of training on the fundamentals of reading instruction Teacher survey Teacher 634 311 631 Teacher-reported absence of at least one day last week Teacher survey Teacher 634 311 632 Teacher has access to school supplies Teacher survey Teacher 631 310 631 Student reports feeling hungry now Student survey Student 5,504 2,576 5,476 Student achieved minimum meal frequency Student survey Student 5,405 2,543 5,421 Student achieved minimum dietary diversity Student survey Student 5,502 2,576 5,476 Student achieved minimum acceptable diet Student survey Student 5,427 2,545 5,429 Student’s nutrition knowledge score Student survey Student 5,503 2,576 5,471 Student’s hygiene and sanitation knowledge score Student survey Student 5,503 2,576 5,471 Student reports absence last week as a result of illness Student survey Student 5,491 2,574 5,473 Head teacher reports school has improved latrines Head teacher survey Head teacher 346 174 352 Head teacher reports school has tippy taps or hand￾washing stations Head teacher survey Head teacher 346 174 352 Head teacher reports all tippy taps or hand-washing stations have ash or soap Head teacher survey Head teacher 346 174 352 1 Student reading performance includes average reading comprehension scores and percent zero reading comprehension scores Abt Associates June 2022 ▌79 SAMPLE POPULATION CHARACTERISTICS Exhibit A8. Student Responses on Household Demographic Characteristics ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Early-Grade Students Student age (years) 8.5 9.2 0.7 8.1 8.6 0.5 8.7 9.4 0.7 Female students (%) 50.3 49.7 -0.6 50.0 46.9 -3.1 49.5 50.3 0.8 Household source of water River, stream, or lake (%) 12.7 10.3 -2.4 15.4 9.6 -5.7 20.0 14.6 -5.4 Well or borehole (%) 29.8 21.1 -8.7 40.3 23.5 -16.8 40.9 20.8 -20.2 Communal tap (%) 21.5 27.6 6.1 24.9 36.8 11.9 23.4 39.3 15.9 Water pipe / tap in home (%) 31.6 39.0 7.4 17.4 26.2 8.8 13.9 24.0 10.1 Water truck or tank (%) 2.1 0.1 -2.0 1.9 2.4 0.5 1.1 0.3 -0.8 Other (%) 2.2 1.6 -0.6 0.2 0.9 0.7 0.7 1.0 0.3 Household asset index Radio (%) 57.9 50.0 -7.9 61.2 53.0 -8.2 58.9 45.3 -13.6 Mobile phone (%) 73.3 74.8 1.5 86.8 95.5 8.7 66.7 72.3 5.6 Television (%) 51.7 51.6 -0.1 56.1 46.8 -9.2 34.4 30.5 -3.9 Computer (%) 20.6 18.0 -2.5 24.7 15.5 -9.2 12.5 10.2 -2.3 Refrigerator (%) 41.6 36.7 -4.9 35.8 26.2 -9.5 21.9 20.9 -1.0 Bicycle (%) 40.8 30.5 -10.3 44.4 37.3 -7.1 48.9 36.9 -12.0 Motorbike (%) 18.1 17.1 -1.0 16.2 14.5 -1.7 19.1 16.9 -2.2 Car/truck (%) 24.5 15.2 -9.3 33.1 22.6 -10.5 14.9 11.2 -3.7 Electricity (%) 57.9 56.4 -1.5 47.4 45.3 -2.1 35.6 32.2 -3.4 Asset index (9 items) 3.8 3.5 -0.3 4.0 3.6 -0.4 3.1 2.7 -0.4 Mid-Grade Students Student age (years) 10.9 10.9 -0.1 10.4 10.8 0.4 11.3 11.4 0.1 Female students (%) 51.2 50.4 -0.7 50.2 52.2 2.0 50.3 50.9 0.6 Household source of water River, stream, or lake (%) 12.7 9.8 -2.9 19.4 13.4 -6.0 17.5 14.1 -3.4 Well or borehole (%) 28.6 15.5 -13.2 35.6 24.7 -10.9 43.4 22.3 -21.0 Communal tap (%) 20.8 24.4 3.6 23.3 28.3 5.0 21.2 36.0 14.8 Water pipe / tap in home (%) 33.5 48.5 15.0 18.9 28.5 9.6 16.0 26.0 10.0 Water truck or tank (%) 2.5 0.6 -1.9 2.7 1.9 -0.8 0.7 0.7 0.1 Other (%) 1.8 1.2 -0.6 0.1 2.6 2.4 1.3 0.9 -0.4 Household asset index Radio (%) 57.0 50.7 -6.2 57.0 60.7 3.7 59.7 50.0 -9.6 Mobile phone (%) 76.1 85.5 9.4 90.6 94.4 3.8 72.3 76.7 4.4 Television (%) 53.6 62.6 9.0 53.0 46.8 -6.2 32.9 36.9 4.1 Computer (%) 18.5 21.6 3.1 12.7 14.4 1.6 10.1 12.6 2.5 Refrigerator (%) 38.9 47.3 8.4 31.6 30.4 -1.2 21.3 27.5 6.2 Bicycle (%) 43.3 30.2 -13.1 50.7 43.0 -7.7 50.9 36.9 -14.0 Motorbike (%) 17.9 14.2 -3.7 14.0 12.7 -1.2 20.5 19.8 -0.7 Car/truck (%) 22.1 23.3 1.2 30.1 22.6 -7.5 14.5 12.6 -1.9 Electricity (%) 56.1 67.2 11.1 44.9 43.9 -1.0 30.5 37.8 7.3 Asset index (9 items) 3.8 4.0 0.2 3.8 3.7 -0.2 3.1 3.1 0.0 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌80 Exhibit A9. Language of Literacy Instruction and Difference with Language Spoken at Home ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Early-Grade Students Teacher-reported language of instruction Portuguese (%) 99.6 100.0 0.4 98.5 99.0 0.6 99.1 99.1 0.0 Rhonga (%) 10.6 3.4 -7.2 1.8 0.0 -1.8 1.4 1.4 0.0 Changana (%) 13.9 12.2 -1.8 58.6 40.4 -18.2 25.3 21.7 -3.7 Macua (%) 37.5 36.0 -1.5 0.0 0.0 0.0 50.5 45.1 -5.4 Other (%) 0.0 0.3 0.3 0.0 12.4 12.4 0.7 10.1 9.4 Language student speaks at home Portuguese (%) 38.0 52.5 14.5 21.8 40.0 18.2 12.1 22.4 10.4 Rhonga (%) 0.7 0.1 -0.6 1.2 0.5 -0.7 0.9 1.6 0.7 Changana (%) 17.7 7.4 -10.2 74.7 59.1 -15.6 31.8 20.8 -11.0 Macua (%) 43.3 38.6 -4.7 0.0 0.0 0.0 54.8 54.8 -0.0 Other (%) 0.4 1.4 0.9 2.3 0.3 -1.9 0.5 0.4 -0.1 Students whose language spoken at home differs from language of instruction (%) 18.5 14.5 -3.9 26.6 28.2 1.7 17.8 21.3 3.6 Mid-Grade Students Teacher-reported language of instruction Portuguese (%) 99.6 100.0 0.4 99.5 98.8 -0.7 99.4 100.0 0.6 Rhonga (%) 4.5 7.1 2.6 2.4 0.8 -1.6 0.8 3.1 2.2 Changana (%) 10.4 12.9 2.4 61.4 35.9 -25.5 21.1 17.5 -3.6 Macua (%) 35.5 23.4 -12.1 0.0 0.0 0.0 48.2 43.5 -4.7 Other (%) 0.1 0.2 0.1 0.0 0.1 0.1 1.7 0.7 -1.0 Language student speaks at home Portuguese (%) 40.1 65.2 25.1 23.8 41.2 17.3 10.7 28.2 17.5 Rhonga (%) 0.9 0.1 -0.7 0.6 0.6 -0.0 1.1 0.7 -0.4 Changana (%) 16.3 5.8 -10.5 75.5 58.2 -17.3 34.4 26.2 -8.3 Macua (%) 42.1 27.8 -14.4 0.0 0.0 0.0 53.1 44.5 -8.6 Other (%) 0.7 1.1 0.4 0.1 0.1 -0.1 0.7 0.4 -0.3 Students whose language spoken at home differs from language of instruction (%) 20.3 13.2 -7.1 27.3 31.0 3.7 23.8 23.3 -0.6 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌81 Exhibit A10. Teachers Characteristics ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Female teachers (%) 60.1 62.5 2.4 37.2 41.9 4.8 47.8 44.8 -3.0 Teach in multi-grade class (%) 16.0 9.8 -6.2 41.0 31.8 -9.2 13.2 9.4 -3.8 Attendance at Teacher Training College (TTC) Did not attend TTC (%) 1.8 0.4 -1.4 1.8 0.0 -1.8 1.4 0.0 -1.4 Still attends TTC (%) 1.9 0.5 -1.3 3.0 0.0 -3.0 3.0 0.9 -2.1 Attended and finished TTC (%) 96.3 99.1 2.7 95.2 100.0 4.8 95.5 99.1 3.5 TTC teacher attended Primary Teacher Training Institute (%) 75.8 80.9 5.1 85.6 83.3 -2.3 75.4 80.3 4.9 ADPP (%) 13.1 9.0 -4.1 9.6 10.6 1.0 15.6 11.5 -4.1 Pedagogical University (%) 9.1 8.0 -1.0 2.4 1.0 -1.4 5.1 4.7 -0.4 Attainment of certification (%) 99.5 98.5 -1.0 98.8 99.5 0.7 97.1 99.8 2.7 Highest level of education (baseline) Form 6 (Basic) (%) 19.0 23.4 25.0 Certificate (%) 64.6 71.3 69.3 Diploma (%) 1.4 0.0 2.1 Advanced Diploma (%) 15.1 5.4 3.4 Other (%) 0.0 0.0 0.2 Highest level of education (endline) Secondary Education (%) 15.8 12.2 16.8 Middle School (%) 61.3 72.6 69.2 Bachelor’s Degree or Higher (%) 22.9 15.2 13.9 During pre-service training, the teacher received specific training on how to teach reading and writing to students in the primary grades (%) 61.4 66.5 5.1 60.5 58.1 -2.4 58.9 70.5 11.6 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌82 SUPPLEMENTAL FINDINGS RELATED TO PRIMARY IMPACTS Exhibit A11. Reading Comprehension Estimates for School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores -7.3 -17.9 -10.6 6.0 0.08 5,474 2,576 Average mean scores (percentage correct) 6.5 7.7 1.2 2.9 0.69 5,474 2,576 Mid-Grade Students Percent zero scores -8.5 -15.0 -6.5 6.6 0.32 5,474 2,576 Average mean scores (percentage correct) 10.1 8.4 -1.8 4.6 0.70 5,474 2,576 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A12. Reading Comprehension Estimates for School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores -7.3 -17.4 -10.0** 3.6 0.00 5,474 5,504 Average mean scores (percentage correct) 6.5 11.3 4.8** 1.9 0.01 5,474 5,504 Mid-Grade Students Percent zero scores -8.5 -23.0 -14.4** 4.4 0.00 5,474 5,504 Average mean scores (percentage correct) 10.1 17.7 7.6* 3.3 0.02 5,474 5,504 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A13. Listening Comprehension Estimates for School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores -10.8 -38.5 -27.6** 4.3 0.00 5,476 2,575 Average mean scores (percentage correct) 14.3 30.7 16.4** 3.5 0.00 5,476 2,575 Mid-Grade Students Percent zero scores -9.5 -24.7 -15.2** 3.6 0.00 5,476 2,575 Average mean scores (percentage correct) 17.5 25.8 8.3** 3.0 0.01 5,476 2,575 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌83 Exhibit A14. Listening Comprehension Estimates for School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores -10.8 -38.3 -27.4** 4.7 0.00 5,476 5,503 Average mean scores (percentage correct) 14.3 37.0 22.8** 3.3 0.00 5,476 5,503 Mid-Grade Students Percent zero scores -9.5 -22.8 -13.3** 3.0 0.00 5,476 5,503 Average mean scores (percentage correct) 17.5 26.8 9.3** 3.2 0.00 5,476 5,503 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A15. Letter Identification Estimates for School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores 5.4 -20.6 -26.0** 5.2 0.00 5,476 2,576 Average mean scores (percentage correct) 13.3 14.6 1.3 3.9 0.74 5,476 2,576 Average mean scores per minute 17.5 17.2 -0.3 4.4 0.95 5,456 2,575 Mid-Grade Students Percent zero scores 1.6 1.9 0.2 2.3 0.92 5,476 2,576 Average mean scores (percentage correct) 19.1 17.1 -2.0 3.1 0.52 5,476 2,576 Average mean scores per minute 28.8 21.3 -7.5 4.4 0.09 5,456 2,575 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A16. Letter Identification Estimates for School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores 5.4 -9.4 -14.8** 4.5 0.00 5,476 5,504 Average mean scores (percentage correct) 13.3 16.3 2.9 2.6 0.26 5,476 5,504 Average mean scores per minute 17.5 18.6 1.2 3.3 0.73 5,456 5,488 Mid-Grade Students Percent zero scores 1.6 0.1 -1.5 2.2 0.50 5,476 5,504 Average mean scores (percentage correct) 19.1 25.9 6.8* 2.8 0.02 5,476 5,504 Average mean scores per minute 28.8 29.4 0.6 4.5 0.89 5,456 5,488 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌84 Exhibit A17. Syllable Identification Estimates for School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores -30.9 -45.0 -14.2 7.2 0.05 5,474 2,573 Average mean scores (percentage correct) 9.1 12.8 3.7 4.9 0.45 5,474 2,573 Average mean scores per minute 6.6 8.0 1.4 2.9 0.63 5,445 2,556 Mid-Grade Students Percent zero scores 48.0 37.8 -10.2* 4.6 0.03 5,474 2,573 Average mean scores (percentage correct) 11.5 10.7 -0.8 3.9 0.85 5,474 2,573 Average mean scores per minute 11.6 9.1 -2.6 3.4 0.45 5,445 2,556 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A18. Syllable Identification Estimates for School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores -30.9 -21.0 9.9 5.3 0.06 5,474 5,502 Average mean scores (percentage correct) 9.1 15.5 6.4* 2.8 0.02 5,474 5,502 Average mean scores per minute 6.6 9.5 2.9 1.9 0.13 5,445 5,464 Mid-Grade Students Percent zero scores 48.0 38.5 -9.5 4.8 0.05 5,474 5,502 Average mean scores (percentage correct) 11.5 22.8 11.3** 3.6 0.00 5,474 5,502 Average mean scores per minute 11.6 14.9 3.3 4.2 0.43 5,445 5,464 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A19. Oral Reading Fluency Estimates for School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores 23.4 -7.0 -30.4** 6.5 0.00 5,474 2,576 Average mean scores (percentage correct) 6.8 10.5 3.7 3.6 0.30 5,474 2,576 Mid-Grade Students Percent zero scores 3.5 -1.6 -5.1 4.1 0.22 5,474 2,576 Average mean scores (percentage correct) 13.3 12.3 -1.0 4.5 0.83 5,474 2,576 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌85 Exhibit A20. Oral Reading Fluency Estimates for School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores 23.4 6.0 -17.4** 4.8 0.00 5,474 5,504 Average mean scores (percentage correct) 6.8 11.9 5.1* 2.1 0.02 5,474 5,504 Mid-Grade Students Percent zero scores 3.5 -1.0 -4.5 4.4 0.30 5,474 5,504 Average mean scores (percentage correct) 13.3 23.3 10.0** 3.3 0.00 5,474 5,504 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A21. Word Dictation Estimates for School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores 20.3 6.8 -13.5** 4.8 0.01 5,476 2,576 Average mean scores (percentage correct) 2.7 10.9 8.2 5.9 0.16 5,476 2,576 Mid-Grade Students Percent zero scores 7.3 8.1 0.8 2.8 0.76 5,476 2,576 Average mean scores (percentage correct) 7.4 5.6 -1.7 3.8 0.64 5,476 2,576 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A22. Word Dictation Estimates for School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Percent zero scores 20.3 9.5 -10.8* 4.2 0.01 5,476 5,504 Average mean scores (percentage correct) 2.7 11.5 8.8** 3.3 0.01 5,476 5,504 Mid-Grade Students Percent zero scores 7.3 6.4 -0.9 3.6 0.80 5,476 5,504 Average mean scores (percentage correct) 7.4 14.2 6.9* 3.1 0.03 5,476 5,504 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌86 Exhibit A23. Impact Estimates on Student Attendance (Student Self-Reported Absence At least One Day Last Week) in School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Grade 3 -1.8 -6.4 -4.6 5.8 0.43 5,473 2,574 Grade 5 -5.9 -1.1 4.8 4.5 0.28 5,473 2,574 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A24. Impact Estimates on Student Attendance (Student Self-Reported Absence At least One Day Last Week) in School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Grade 3 -1.8 -7.5 -5.7 3.6 0.11 5,473 5,491 Grade 5 -5.9 -2.0 3.9 3.8 0.31 5,473 5,491 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A25. Impact Estimates on Student Attentiveness as Perceived by Teachers in School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Grade 3 5.4 1.9 -3.5 4.1 0.40 632 311 Grade 5 6.1 -1.1 -7.2* 3.3 0.03 632 311 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A26. Impact Estimates on Student Attentiveness as Perceived by Teachers in School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Grade 3 5.4 1.3 -4.1 3.5 0.24 632 634 Grade 5 6.1 3.0 -3.0 2.9 0.29 632 634 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌87 SUPPLEMENTAL FINDINGS RELATED TO SECONDARY IMPACTS Exhibit A27. Teachers' Perception on Student Attentiveness ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Approximate number of students who are generally attentive in this class Less than half 7.1 8.8 1.7 4.2 2.1 -2.1 8.7 8.6 -0.1 Half 18.7 17.2 -1.5 5.4 10.8 5.4 11.5 16.9 5.4 More than half 57.6 57.0 -0.6 64.7 65.5 0.8 60.0 62.0 2.0 All 7.9 17.0 9.2 22.2 21.6 -0.5 15.6 12.5 -3.1 Ways teacher gauges if students are attentive in class Students are seated facing and making eye contact with the teacher 55.7 54.8 -0.9 59.3 53.5 -5.7 47.9 59.5 11.6 Students actively listen and act upon teacher directions 38.4 47.6 9.2 41.3 46.0 4.6 37.4 60.1 22.7 Students are working independently, in pairs, or in small groups on the current learning activity 14.4 7.6 -6.9 15.0 5.6 -9.4 13.1 17.5 4.4 Students respond to questions posed by the teacher 43.4 52.7 9.3 43.1 53.5 10.4 43.1 60.2 17.1 Students ask the teacher or another student for help related to the current learning activity 14.4 7.5 -6.9 10.2 7.1 -3.1 18.7 16.6 -2.1 Students ask the teacher questions related to the current learning activity 16.7 18.9 2.2 13.8 13.1 -0.6 13.0 17.3 4.3 Students are reading 2.3 13.8 11.5 8.4 2.0 -6.4 6.3 13.1 6.9 Students are writing 4.4 13.1 8.7 4.2 1.5 -2.7 4.2 15.1 10.9 Students are listening to a story 2.2 8.4 6.2 0.6 0.0 -0.6 3.5 5.4 1.9 Other 9.1 3.7 -5.4 1.8 8.1 6.3 6.1 3.8 -2.4 Student attentiveness in relation to the school meal Students are more attentive after the school meal 20.8 44.7 23.9 36.5 72.6 36.1 34.4 54.8 20.4 Students are less attentive after the school meal 10.3 17.3 7.0 19.8 12.3 -7.4 21.3 13.2 -8.1 Students show the same attentiveness before and after lunch 17.6 26.0 8.4 40.7 15.1 -25.7 38.2 27.5 -10.7 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Exhibit A28. Impact on Teacher-Reported Instructional Input in School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Teacher-reported receipt of training on the fundamentals of reading instruction 0.3 14.5 14.3 9.3 0.13 625 310 Teacher-reported absence of at least one day last week -0.6 -12.2 -11.6 6.7 0.08 632 311 Teacher has access to school supplies 11.5 0.2 -11.3** 3.3 0.00 631 310 Mid-Grade Students Abt Associates June 2022 ▌88 Teacher-reported receipt of training on the fundamentals of reading instruction -8.9 -5.0 3.9 9.1 0.67 625 310 Teacher-reported absence of at least one day last week -1.2 -3.4 -2.2 6.7 0.74 632 311 Teacher has access to school supplies 8.7 3.3 -5.5 4.1 0.18 631 310 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A29. Impact on Teacher-Reported Instructional Input in School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Teacher-reported receipt of training on the fundamentals of reading instruction 0.3 15.5 15.3* 7.0 0.03 625 632 Teacher-reported absence of at least one day last week -0.6 -1.7 -1.1 6.6 0.87 632 634 Teacher has access to school supplies 11.5 1.5 -10.0** 3.7 0.01 631 631 Mid-Grade Students Teacher-reported receipt of training on the fundamentals of reading instruction -8.9 19.0 27.9** 7.5 0.00 625 632 Teacher-reported absence of at least one day last week -1.2 1.4 2.6 5.8 0.66 632 634 Teacher has access to school supplies 8.7 0.7 -8.1* 3.9 0.04 631 631 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌89 Exhibit A30. Impact on Feeding Outcomes in School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Student reports feeling hungry now -8.2 0.9 9.1 6.2 0.14 5,476 2,576 Student achieved minimum meal frequency 7.0 -27.2 -34.2** 6.2 0.00 5,421 2,543 Student achieved minimum dietary diversity -28.8 -26.5 2.3 6.2 0.71 5,476 2,576 Student achieved minimum acceptable diet -7.0 -20.8 -13.8* 5.8 0.02 5,429 2,545 Student's nutrition knowledge score -0.4 0.0 0.5** 0.1 0.00 5,471 2,576 Mid-Grade Students Student reports feeling hungry now -2.1 12.2 14.3* 6.7 0.04 5,476 2,576 Student achieved minimum meal frequency 1.4 -15.4 -16.8** 4.9 0.00 5,421 2,543 Student achieved minimum dietary diversity -24.6 -31.2 -6.7 7.3 0.36 5,476 2,576 Student achieved minimum acceptable diet -9.2 -23.1 -13.9* 6.7 0.04 5,429 2,545 Student's nutrition knowledge score -0.5 -0.1 0.4 0.2 0.06 5,471 2,576 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A31. Impact on Feeding Outcomes in School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Student reports feeling hungry now -8.2 -15.2 -6.9 4.9 0.16 5,476 5,504 Student achieved minimum meal frequency 7.0 -7.7 -14.7** 5.1 0.00 5,421 5,405 Student achieved minimum dietary diversity -28.8 -28.0 0.8 4.7 0.86 5,476 5,502 Student achieved minimum acceptable diet -7.0 -20.8 -13.8** 4.4 0.00 5,429 5,427 Student's nutrition knowledge score -0.4 0.4 0.9** 0.1 0.00 5,471 5,503 Mid-Grade Students Student reports feeling hungry now -2.1 -2.0 0.1 4.6 0.98 5,476 5,504 Student achieved minimum meal frequency 1.4 -2.5 -3.9 4.2 0.35 5,421 5,405 Student achieved minimum dietary diversity -24.6 -34.3 -9.8* 4.6 0.03 5,476 5,502 Student achieved minimum acceptable diet -9.2 -22.2 -13.1** 4.2 0.00 5,429 5,427 Student's nutrition knowledge score -0.5 0.3 0.8** 0.2 0.00 5,471 5,503 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌90 Exhibit A32. Impact on Student Knowledge and Health Practices in School Feeding Only Schools ........................................................ School Feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Student's hygiene and sanitation knowledge score 0.8 1.1 0.3 0.3 0.25 5,471 2,576 Student reports absence last week as a result of illness 0.3 -17.8 -18.1 10.2 0.08 1,325 566 Mid-Grade Students Student's hygiene and sanitation knowledge score 1.0 0.9 -0.0 0.2 0.84 5,471 2,576 Student reports absence last week as a result of illness 1.7 -2.1 -3.8 12.1 0.76 1,325 566 All Students Head teacher reports school has improved latrines 11.4 11.8 0.3 7.5 0.97 352 174 Head teacher reports school has tippy taps or hand-washing stations 29.4 1.7 -27.7** 9.0 0.00 352 174 Head teacher reports all tippy taps or hand-washing stations have ash or soap 26.7 5.7 -21.0** 7.0 0.00 352 174 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Exhibit A33. Impact on Student Knowledge and Health Practices in School Feeding and Literacy Schools ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Student's hygiene and sanitation knowledge score 0.8 1.6 0.9** 0.2 0.00 5,471 5,503 Student reports absence last week as a result of illness 0.3 13.2 13.0 8.2 0.11 1,325 1,219 Mid-Grade Students Student's hygiene and sanitation knowledge score 1.0 1.3 0.3* 0.1 0.02 5,471 5,503 Student reports absence last week as a result of illness 1.7 10.6 9.0 9.1 0.33 1,325 1,219 All Students Head teacher reports school has improved latrines 11.4 8.6 -2.8 5.0 0.57 352 346 Head teacher reports school has tippy taps or hand-washing stations 29.4 19.6 -9.8 6.1 0.11 352 346 Head teacher reports all tippy taps or hand-washing stations have ash or soap 26.7 21.0 -5.7 4.2 0.18 352 346 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌91 GENDER DISAGGREGATED IMPACT OF READING COMPREHENSION Exhibit A34. Impact on Students’ Reading Comprehension in School Feeding Only Schools, by Grade and Gender ........................................................ School feeding Only C Diff T Diff Impact SE p-value C N T N Early-Grade Students Boys Student reading performance (Average Reading Comprehension Scores) 5.9 8.8 2.9 3.1 0.35 2,745 1,238 Student reading performance (Percent Zero Reading Comprehension Scores) -7.2 -20.1 -12.8 6.6 0.05 2,745 1,238 Student attendance 1.2 -8.1 -9.3 6.8 0.16 2,743 1,236 Girls Student reading performance (Average Reading Comprehension Scores) 8.0 6.7 -1.3 3.5 0.72 2,729 1,338 Student reading performance (Percent Zero Reading Comprehension Scores) -8.6 -15.5 -6.9 6.2 0.27 2,729 1,338 Student attendance -2.6 -4.0 -1.4 8.6 0.87 2,730 1,338 Mid-Grade Students Boys Student reading performance (Average Reading Comprehension Scores) 10.6 9.9 -0.7 5.2 0.89 2,745 1,238 Student reading performance (Percent Zero Reading Comprehension Scores) -10.1 -13.4 -3.3 8.0 0.68 2,745 1,238 Student attendance -5.3 -1.6 3.7 6.0 0.54 2,743 1,236 Girls Student reading performance (Average Reading Comprehension Scores) 8.3 6.1 -2.2 6.1 0.72 2,729 1,338 Student reading performance (Percent Zero Reading Comprehension Scores) -4.8 -14.1 -9.3 7.7 0.23 2,729 1,338 Student attendance -6.4 0.6 7.0 5.5 0.21 2,730 1,338 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌92 Exhibit A35. Impact on Students’ Reading Comprehension in School Feeding and Literacy Schools, by Grade and Gender ........................................................ School Feeding and Literacy C Diff T Diff Impact SE p-value C N T N Early-Grade Students Boys Student reading performance (Average Reading Comprehension Scores) 5.9 11.0 5.1* 2.2 0.02 2,745 2,737 Student reading performance (Percent Zero Reading Comprehension Scores) -7.2 -16.8 -9.5* 4.4 0.03 2,745 2,737 Student attendance 1.2 -13.6 -14.8** 4.9 0.00 2,743 2,731 Girls Student reading performance (Average Reading Comprehension Scores) 8.0 12.1 4.1 2.3 0.08 2,729 2,767 Student reading performance (Percent Zero Reading Comprehension Scores) -8.6 -19.1 -10.5** 3.9 0.01 2,729 2,767 Student attendance -2.6 -2.0 0.6 4.7 0.90 2,730 2,760 Mid-Grade Students Boys Student reading performance (Average Reading Comprehension Scores) 10.6 21.7 11.1* 4.3 0.01 2,745 2,737 Student reading performance (Percent Zero Reading Comprehension Scores) -10.1 -28.0 -17.9** 5.6 0.00 2,745 2,737 Student attendance -5.3 -7.2 -1.9 4.7 0.69 2,743 2,731 Girls Student reading performance (Average Reading Comprehension Scores) 8.3 13.8 5.5 4.1 0.19 2,729 2,767 Student reading performance (Percent Zero Reading Comprehension Scores) -4.8 -17.7 -12.9* 5.5 0.02 2,729 2,767 Student attendance -6.4 3.6 9.9 5.1 0.05 2,730 2,760 Note: “C Diff” denotes difference in the comparison group. “T Diff” denotes difference in the treatment group. “Impact” denotes the difference in differences. “SE” denotes standard error. “C N” denotes sample size in the comparison group. “T N” denotes sample size in the treatment group. Abt Associates June 2022 ▌93 SUPPLEMENTAL FINDINGS RELATED TO MEDIATING IMPACTS OF IMPLEMENTATION Exhibit A36. Impact of Adequate Implementing Programs on Student Reading Comprehension ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Percent zero scores -7.2 -21.2 -14.0 -17.0 -9.8* (10.6) (4.5) Average mean scores (percentage correct) 6.3 8.9 2.6 11.7 5.4* (4.0) (2.3) Mid-Grade Students Percent zero scores -8.3 -15.9 -7.5 -27.2 -18.9** (7.4) (5.3) Average mean scores (percentage correct) 9.8 7.4 -2.4 20.7 10.9** (5.2) (3.8) Note: “Early-Grade Students” were in grade 2 at baseline and grade 3 at endline. “Mid-Grade Students” were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,476 Comparison; 1,054 School Feeding Only; 2,959 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference in differences. Exhibit A37. Impact of Adequate Implementing Programs on Oral Comprehension ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Percent zero scores -11.4 -38.5 -27.1** -41.8 -30.4** (6.6) (6.4) Average mean scores (percentage correct) 14.5 29.3 14.9** 42.3 27.8** (3.8) (4.3) Mid-Grade Students Percent zero scores -9.9 -23.7 -13.8** -23.7 -13.8** (4.6) (3.7) Average mean scores (percentage correct) 17.5 25.9 8.4* 30.5 13.0** (3.8) (3.9) Note: Standard errors appear in parentheses. Sample sizes: 5,476 Comparison; 1,053 School feeding only; 2,959 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌94 Exhibit A38. Impact of Adequate Implementing Programs on Letter Identification ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Percent zero scores 5.9 -17.9 -23.8** -8.9 -14.8** (7.1) (5.3) Average mean scores (percentage correct) 13.5 15.0 1.5 18.1 4.6 (5.2) (2.9) Average mean scores per minute 17.8 17.0 -0.8 20.7 2.9 (5.5) (3.8) Mid-Grade Students Percent zero scores 2.1 3.6 1.4 -0.3 -2.4 (3.0) (2.7) Average mean scores (percentage correct) 19.3 14.7 -4.6 30.6 11.3** (4.2) (3.4) Average mean scores per minute 29.1 18.6 -10.4 34.3 5.2 (5.4) (5.2) Note: Standard errors appear in parentheses. Sample sizes: 5,476 Comparison; 1,054 School feeding only; 2,959 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌95 Exhibit A39. Impact of Adequate Implementing Programs on Syllable Identification ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Percent zero scores -33.0 -42.9 -9.9 -21.9 11.1 (9.4) (6.1) Average mean scores (percentage correct) 9.1 15.7 6.6 16.2 7.1* (6.9) (3.0) Average mean scores per minute 6.6 9.3 2.7 9.7 3.1 (3.7) (2.1) Mid-Grade Students Percent zero scores 45.6 37.2 -8.3 40.9 -4.7 (6.2) (6.2) Average mean scores (percentage correct) 11.4 5.2 -6.2 28.4 17.0** (4.3) (4.2) Average mean scores per minute 11.7 5.7 -6.0 18.3 6.6 (3.6) (4.9) Note: Standard errors appear in parentheses. Sample sizes: 5,474 Comparison; 1,053 School feeding only; 2,958 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A40. Impact of Adequate Implementing Programs on Oral Reading Fluency ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Percent zero scores 23.7 -11.1 -34.8** 4.5 -19.2** (10.2) (5.7) Average mean scores (percentage correct) 6.7 12.8 6.1 11.2 4.5 (5.2) (2.6) Mid-Grade Students Percent zero scores 3.9 -0.8 -4.6 -6.3 -10.2* (4.6) (5.1) Average mean scores (percentage correct) 13.2 10.0 -3.3 26.2 13.0** (5.0) (4.0) Note: Standard errors appear in parentheses. Sample sizes: 5,476 Comparison; 1,054 School feeding only; 2,959 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌96 Exhibit A41. Impact of Adequate Implementing Programs on Word Dictation ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Percent zero scores 20.6 12.0 -8.6 7.9 -12.7** (6.7) (4.6) Average mean scores (percentage correct) 2.6 14.3 11.6 12.0 9.4** (7.9) (3.5) Mid-Grade Students Percent zero scores 7.7 8.7 1.0 4.1 -3.6 (3.4) (4.4) Average mean scores (percentage correct) 7.3 3.5 -3.7 16.5 9.3* (3.6) (4.0) Note: Standard errors appear in parentheses. Sample sizes: 5,476 Comparison; 1,054 School feeding only; 2,959 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A42. Impact of Adequate Implementing Programs on Student Attendance Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students -1.5 6.9 8.4 -7.6 -6.1 (6.1) (3.7) Mid-Grade Students -5.5 -4.0 1.5 -5.3 0.1 (7.1) (4.1) Note: “Early-Grade Students” were in grade 2 at baseline and grade 3 at endline. “Mid-Grade Students” were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,473 Comparison; 1,052 School feeding only; 2,956 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference in differences estimates. ** p<0.01; * p<0.05. Exhibit A43. Impact of Adequate Implementing Programs on Student Attentiveness Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students 5.2 3.7 -1.4 -2.5 -7.7 (5.4) (4.0) Mid-Grade Students 5.9 3.0 -2.9 3.9 -1.9 (4.7) (3.0) Note: “Early-Grade Students” were in grade 2 at baseline and grade 3 at endline. “Mid-Grade Students” were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 632 Comparison; 125 School feeding only; 336 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌97 SUPPLEMENTAL FINDINGS RELATED TO SECONDARY IMPACTS Exhibit A44. Impact on Student Reported Access to School Supplies and Study Materials ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Student has supplementary reader available today -23.0 -10.1 12.9 -8.6 14.5** (7.5) (5.6) Student has Portuguese/Local Language textbook available today -38.9 -47.2 -8.3 -51.0 -12.1* (6.8) (4.9) Student has exercise notebook available today 16.5 15.6 -0.9 9.4 -7.1* (3.4) (2.8) Student reads books daily in the classroom or school library -9.0 -11.0 -2.0 -8.1 0.9 (6.7) (5.1) Mid-Grade Students Student has supplementary reader available today -23.7 -11.8 11.9 -18.6 5.1 (7.2) (5.7) Student has Portuguese/Local Language textbook available today -37.8 -48.0 -10.2 -54.0 -16.3** (6.7) (5.1) Student has exercise notebook available today 4.7 7.3 2.6 4.7 0.0 (2.5) (1.8) Student reads books daily in the classroom or school library -12.8 -11.2 1.6 -2.5 10.3* (6.5) (5.0) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,473 Comparison; 2,574 School feeding only; 5,491 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌98 Exhibit A45. Impact on Teacher Access to Supplies and Learning Materials ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Teacher has a teacher guide for reading instruction 5.4 14.4 9.0 14.7 9.3 (8.0) (7.2) Students have exercise books that can be used during reading instruction 16.1 -6.7 -22.8** 3.6 -12.6* (7.0) (6.0) Teacher has access to school supplies 11.5 0.2 -11.3** 1.5 -10.0** (3.3) (3.7) Mid-Grade Students Teacher has a teacher guide for reading instruction 3.3 4.5 1.3 -4.1 -7.3 (8.4) (7.4) Students have exercise books that can be used during reading instruction 39.8 17.0 -22.8** 31.8 -8.0 (8.0) (6.1) Teacher has access to school supplies 8.7 3.3 -5.5 0.7 -8.1* (4.1) (3.9) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 628 Comparison; 309 School feeding only; 623 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A46. Percentage of Teachers Reporting Access to School Supply Items ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Pens/pencils 92.2 95.8 3.6 93.0 88.7 -4.3 88.3 95.6 7.3 Notebooks 89.4 95.0 5.6 88.6 86.7 -1.9 83.8 92.8 9.0 Chalk 84.5 95.9 11.4 75.3 90.8 15.5 84.3 86.6 2.3 Cardboard 73.3 49.7 -23.7 77.8 58.5 -19.4 66.7 60.8 -5.9 Paint 7.2 11.3 4.1 31.0 27.2 -3.8 13.9 18.3 4.4 Markers 9.5 20.9 11.4 24.7 45.6 21.0 21.9 35.9 14.1 Chart paper 26.8 24.4 -2.4 44.9 52.3 7.4 40.2 38.4 -1.7 Other 12.6 12.4 -0.2 10.1 16.4 6.3 16.5 13.6 -2.9 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌99 Exhibit A47. Impact on Head Teacher Reporting Access to Library/Book Banks and Materials ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) HTs who report that the school has a library/book bank 5.7 -15.6 -21.3** 7.0 1.2 (7.9) (5.2) HTs who report that the print material in the library/book bank is available in Portuguese 5.7 -14.1 -19.8* 5.3 -0.4 (7.8) (5.2) HTs who report that the print material in the library/book bank is available in the mother tongue 0.6 2.9 2.2 12.8 12.1** (4.0) (3.7) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 352 Comparison; 174 School feeding only; 346 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A48. Percentage of Head Teachers Reporting Availability of Specific Types of Print Materials ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Flipbooks 19.1 59.0 39.9 36.0 31.3 -4.7 18.6 63.8 45.2 Storybooks 61.8 70.5 8.7 84.0 68.8 - 15.2 84.1 79.4 -4.6 Primers/student manuals 70.2 27.8 - 42.4 84.0 31.3 - 52.8 57.9 53.0 -5.0 Fables 39.6 36.5 -3.1 40.0 31.3 -8.8 50.3 52.9 2.6 Leveled books 63.5 30.9 - 32.7 52.0 18.8 - 33.3 53.5 34.9 -18.6 Supplementary readers 50.2 33.0 - 17.2 40.0 62.5 22.5 40.8 56.2 15.4 Guidelines for parents 6.2 2.4 -3.8 0.0 6.3 6.3 1.2 21.0 19.8 Reading posters 29.8 46.2 16.4 28.0 12.5 - 15.5 20.8 44.4 23.6 Other 2.3 14.9 12.7 8.3 18.8 10.4 4.3 9.9 5.7 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌100 Exhibit A49. Impact on Teacher Reporting on Teacher Training ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students This year, the teacher attended any in-service training on how to teach reading in the primary grades -8.5 -12.0 -3.5 -12.2 -3.7 (7.6) (5.8) Teacher received training on the fundamentals of reading instruction -3.3 14.0 17.3 15.7 19.0** (9.4) (7.1) Teacher can name the core reading skills 16.6 18.8 2.2 3.8 -12.8* (8.0) (5.3) Mid-Grade Students This year, the teacher attended any in-service training on how to teach reading in the primary grades 12.5 -9.7 -22.2** 6.0 -6.5 (8.4) (6.4) Teacher received training on the fundamentals of reading instruction -7.1 -5.4 1.7 19.9 26.9** (9.0) (7.5) Teacher can name the core reading skills 0.5 10.3 9.8 14.8 14.3* (7.6) (6.2) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 631 Comparison; 311 School feeding only; 634 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌101 Exhibit A50. Teachers Reporting on Teacher Training ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Percentage of teacher attended WV/PAI training 10.9 . . 63.5 64.3 0.8 70.0 71.3 1.3 Frequency of the reading coach visiting the teacher (this academic school year) Never 76.0 65.5 -10.5 66.5 56.1 -10.4 59.0 35.5 -23.5 Quarterly 10.4 9.5 -1.0 13.2 13.1 -0.0 15.4 29.0 13.7 Monthly 4.7 2.1 -2.6 6.6 7.1 0.5 12.2 9.0 -3.1 Weekly 0.7 1.2 0.5 6.6 8.1 1.5 10.0 14.2 4.2 Types of activities teacher participates in during in-service training Analysis of curriculum and syllabus 34.9 46.3 11.4 46.7 55.9 9.2 25.9 46.8 20.9 Analysis of teacher guide 23.0 12.1 -11.0 28.1 18.6 -9.5 19.5 34.3 14.8 Practice teaching lessons from the teacher guide 23.6 20.0 -3.5 21.6 28.8 7.3 19.5 38.6 19.1 Analysis of the student exercise book 36.1 26.7 -9.4 61.1 28.8 -32.3 42.3 36.2 -6.1 How to plan and prepare for a lesson 64.8 45.4 -19.4 80.8 55.9 -24.9 67.1 56.8 -10.4 Improvisation and use of teaching and learning aids 35.4 32.6 -2.8 55.1 28.8 -26.3 42.0 44.5 2.6 How to assess students during the lesson 42.1 23.8 -18.2 64.7 35.6 -29.1 54.8 45.4 -9.4 How to use assessments to adjust instruction 23.3 11.9 -11.4 40.1 18.6 -21.5 30.7 21.6 -9.1 How to give students feedback 22.8 17.6 -5.2 43.1 13.6 -29.6 28.8 22.5 -6.4 How to use supplementary readers/decodable texts 25.4 1.5 -23.9 28.7 15.3 -13.5 22.0 20.1 -1.9 How to apply new strategies for reading instruction 27.0 35.1 8.1 32.9 44.1 11.1 28.2 46.2 18.0 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌102 Exhibit A51. Teacher Knowledge and Skills ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff The reading skills the teacher can name Letter knowledge 65.0 94.0 29.0 58.1 87.3 29.3 66.6 94.9 28.3 Phonemic awareness 34.5 39.7 5.2 24.6 56.6 32.1 34.1 71.1 37.0 Fluency 22.9 17.3 -5.6 14.4 41.0 26.6 21.3 57.1 35.8 Vocabulary 35.4 58.9 23.5 36.5 57.2 20.7 51.5 68.9 17.4 Comprehension 43.5 57.0 13.5 36.5 60.2 23.7 35.3 71.3 36.0 Grade level at which the teacher expects children to read text in the target language fluently Grade 1 7.1 1.5 -5.5 6.0 3.5 -2.5 4.4 0.4 -4.0 Grade 2 36.9 16.4 -20.5 31.1 11.1 -20.0 21.2 11.3 -9.9 Grade 3 25.0 45.9 20.9 27.5 33.8 6.3 40.9 40.3 -0.6 Grade 4 or higher 29.6 35.6 6.0 35.3 50.0 14.7 30.1 47.2 17.1 Grade level at which the teacher expects students to write a coherent and comprehensible short story correctly Grade 1 4.2 2.7 -1.5 3.6 0.0 -3.6 2.5 0.5 -1.9 Grade 2 22.2 13.2 -8.9 21.0 4.0 -16.9 16.6 7.9 -8.8 Grade 3 27.8 36.6 8.8 29.3 29.3 -0.0 34.4 32.2 -2.2 Grade 4 or higher 43.1 47.1 4.0 45.5 64.6 19.1 44.1 57.7 13.5 Ways the teacher measures students' reading and writing progress Written tests 63.8 77.2 13.4 61.1 65.2 4.1 70.0 81.2 11.2 Oral evaluations 66.9 74.4 7.5 66.5 81.3 14.8 62.1 78.6 16.5 Observation 40.9 29.0 -11.9 34.7 25.3 -9.5 40.6 44.4 3.8 Portfolios and other projects 4.1 4.0 -0.1 6.0 1.5 -4.5 4.4 6.7 2.3 Homework 36.6 38.7 2.0 31.7 32.8 1.1 34.9 44.5 9.5 Classwork 30.9 42.8 11.9 30.5 49.0 18.5 32.4 42.0 9.6 End-of-unit evaluation 6.0 17.9 11.9 5.4 15.2 9.8 5.5 24.3 18.8 End-of-term evaluation 2.6 17.4 14.8 3.0 9.6 6.6 5.8 26.7 20.9 Ways the teacher checks for student understanding during the lesson Ask questions of individual students 57.4 72.3 14.9 56.3 62.1 5.8 56.0 82.7 26.7 Ask questions of whole class 46.5 49.7 3.3 40.7 48.0 7.3 38.9 58.1 19.2 Ask questions of students in groups 25.5 17.4 -8.1 25.1 21.2 -3.9 25.6 32.7 7.2 Give students a task and correct the written responses before the end of the lesson 32.8 45.1 12.3 44.3 51.5 7.2 36.3 53.8 17.5 Give students a task and correct the written responses after the end of the lesson 20.9 35.3 14.4 35.9 25.8 -10.2 25.6 29.3 3.7 Ask students to put thumbs up or down 2.8 2.2 -0.6 3.6 1.5 -2.1 5.2 6.4 1.2 Other 12.0 4.8 -7.2 7.8 4.0 -3.7 14.5 9.5 -5.0 Ways the teacher uses the results of students' oral and written assessments in instruction Grade students 53.7 59.1 5.4 47.9 64.1 16.2 46.9 69.9 22.9 Evaluate students' understanding of subject matter 59.5 0.0 -59.5 56.3 1.0 -55.3 53.5 2.2 -51.3 Plan teaching and learning activities 23.5 39.8 16.3 27.5 25.3 -2.3 29.1 45.2 16.1 Adapt teaching to better suit students' needs 21.5 0.0 -21.5 37.1 0.0 -37.1 21.4 0.0 -21.4 Other 16.3 11.4 -4.9 7.2 10.6 3.4 11.7 18.4 6.7 Techniques teacher uses to teach a new vocabulary word Pictures 40.9 56.9 16.0 42.5 57.6 15.1 55.0 68.5 13.5 Abt Associates June 2022 ▌103 ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Realia 40.5 50.1 9.7 49.1 49.5 0.4 47.1 49.9 2.8 Gestures 26.3 26.4 0.1 24.0 22.2 -1.7 20.4 35.4 15.0 Verbal explanation 55.3 76.8 21.5 49.7 73.2 23.5 51.9 74.7 22.8 Other 15.8 3.0 -12.8 13.8 8.1 -5.7 12.9 6.3 -6.6 Frequency of teacher switching between official language of instruction and a vernacular language during the teaching and learning process Never 14.3 19.6 5.3 6.6 10.6 4.0 3.7 5.0 1.3 Occasionally 58.6 59.7 1.2 62.3 55.6 -6.7 67.5 56.6 -10.9 Often 21.2 15.0 -6.2 22.2 20.2 -2.0 18.8 30.3 11.5 All of the time 5.6 5.5 -0.2 7.8 13.6 5.9 9.7 7.6 -2.1 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Exhibit A52. Impact on Head Teacher Reported Teacher Training ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) % of HTs who attended WV or PAI training -6.4 -10.5 -4.1 -8.0 -1.6 (4.7) (2.8) % of HTs who received training on the fundamentals of reading instruction 2.3 27.8 25.5** 28.1 25.7** (9.4) (6.9) % of HTs who report receiving training on school management 4.5 -2.4 -6.9 17.0 12.5* (7.7) (6.2) % of HTs who report receiving training about the importance of access to and use of school supplies and materials -2.9 8.0 10.9 11.2 14.0* (8.9) (7.1) Head teacher can name the core reading skills 12.3 17.0 4.7 12.7 0.4 (10.9) (7.0) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 352 Comparison; 174 School feeding only; 346 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌104 Exhibit A53. Head Teacher Knowledge and Skills ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff The reading skills the head teacher can name Letter knowledge 5.9 92.3 86.3 4.8 85.7 81.0 21.4 91.0 69.6 Phonemic awareness 10.2 54.9 44.7 8.3 58.9 50.6 30.9 84.2 53.2 Fluency 5.9 35.9 30.0 9.5 44.6 35.1 21.8 71.3 49.4 Vocabulary 15.2 63.4 48.3 17.9 57.1 39.3 30.9 80.4 49.5 Comprehension 17.1 61.3 44.2 15.5 78.6 63.1 22.2 80.8 58.6 Ways the head teacher identifies academic progress Classroom observation 58.3 61.8 3.5 48.8 61.4 12.6 62.5 75.6 13.1 Monitor student results on tests given by teachers 55.9 58.1 2.2 51.2 48.9 -2.3 48.1 56.8 8.7 Evaluate students orally myself 32.2 41.8 9.5 29.8 39.8 10.0 41.5 51.2 9.7 Check students' assignments or homework 32.7 20.4 -12.3 33.3 23.9 -9.5 24.8 30.4 5.5 Teachers provide progress reports 35.9 36.1 0.3 35.7 27.3 -8.4 26.3 41.3 15.0 End-of-term evaluations 37.9 42.0 4.1 28.6 38.6 10.1 32.6 42.5 9.9 Feedback from parents 7.0 11.3 4.3 0.0 2.3 2.3 1.2 8.2 6.9 Feedback from school counselors 5.3 9.5 4.2 3.6 4.5 1.0 0.0 11.5 11.5 Feedback from school committees 1.1 8.9 7.7 1.2 3.4 2.2 0.6 5.8 5.2 Other 14.8 9.5 -5.2 16.7 8.0 -8.7 12.4 4.5 -7.9 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Exhibit A54. Percentage of Head Teachers Reporting Teachers Attended Literacy Training and WI or PAI Training ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff HTs who report that teachers in the school attended literacy training this academic year 12.8 49.6 36.8 7.1 100.0 92.9 13.9 100.0 86.1 HTs who report the teachers in the school attended WV or PAI training (of the HTs who reported teachers attended literacy training) 1.1 0.0 -1.1 6.0 81.8 75.9 44.0 92.8 48.9 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌105 Exhibit A55. Impact on Head Teacher Reported Number of Male and Female Teachers who Attended Literacy Training ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Mean number of female teachers in the school who attended literacy training this academic year 1.0 -0.1 -1.2** 0.8 -0.3 (0.2) (0.3) Mean number of male teachers in the school who attended literacy training this academic year 0.5 0.9 0.4* 1.1 0.6** (0.2) (0.2) Mean number of early-grade teachers who attended literacy training this academic year 0.7 0.4 -0.3 1.2 0.4* (0.2) (0.2) Mean number of female early￾grade teachers who attended literacy training this academic year 0.5 0.0 -0.4* 0.5 0.1 (0.2) (0.1) Mean number of male early-grade teachers who attended literacy training this academic year 0.3 0.4 0.1 0.6 0.3** (0.1) (0.1) Mean number of mid-grade teachers who attended literacy training this academic year 0.6 0.4 -0.2 0.9 0.3 (0.2) (0.1) Mean number of female mid-grade teachers who attended literacy training this academic year 0.4 -0.0 -0.4** 0.4 -0.0 (0.1) (0.1) Mean number of male mid-grade teachers who attended literacy training this academic year 0.3 0.5 0.2* 0.6 0.3** (0.1) (0.1) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 352 Comparison; 174 School feeding only; 346 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Note that at baseline teachers were asked if they attended literacy training this academic year; at endline teachers were asked if they attended literacy training between 2017 and 2019. Abt Associates June 2022 ▌106 Exhibit A56. Impact on Head Teachers’ Reported Reading Coaches Training and Supervising Teachers, and Number of Trainings Reading Coaches Provided ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) % of HTs who report that reading coaches train teachers at this school 11.9 17.5 5.6 33.0 21.1** (9.6) (6.2) % of HTs who report that reading coaches supervise teachers at this school 11.9 17.5 5.6 33.0 21.1** (9.6) (6.2) Mean number of trainings provided by reading coaches this academic year if there is training by reading coaches 19.7 26.0 6.3 40.3 20.6** (8.5) (5.8) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 352 Comparison; 174 School feeding only; 346 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A57. Percentage of Head Teachers Reporting on Frequency Reading Coaches Supervise Teachers ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Never 45.2 27.4 -17.7 41.2 32.7 -8.5 11.2 13.6 2.3 Once a year 15.3 54.1 38.8 11.8 24.5 12.7 3.4 45.5 42.1 Once per term 21.5 7.6 -13.9 8.8 14.3 5.5 27.4 17.2 -10.2 Once a month 12.8 7.1 -5.7 11.8 26.5 14.8 30.5 23.1 -7.5 More than once a month 5.2 3.8 -1.4 26.5 2.0 -24.4 27.5 0.6 -26.8 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌107 Exhibit A58. Impact of Head Teachers’ Reported Teacher Attendance and Head Teacher Attendance ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Average number of teachers absent yesterday or last day school was in session 0.5 -0.0 -0.5 -0.3 -0.8 (0.7) (0.6) Teacher attendance register available -39.2 -45.9 -6.8 -60.3 -21.1** (8.1) (5.6) HT absent from school any day last week 12.5 13.8 1.3 2.9 -9.6 (8.9) (6.8) HT late to school any day last week 15.5 27.1 11.6 4.2 -11.4 (7.7) (6.0) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 231 Comparison; 110 School feeding only; 182 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A59. Head Teacher Reporting on How Class is Handled when Teacher is Absent ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Let class proceed without teacher 2.7 5.3 2.6 1.2 3.4 2.2 0.6 2.9 2.3 Allocate class to another teacher 54.7 76.6 21.9 61.9 71.6 9.7 70.9 80.8 9.9 Join all students in one class 26.7 18.8 -7.9 15.5 8.0 -7.5 14.8 20.3 5.5 Dismiss students for the day 0.5 5.5 4.9 1.2 0.0 -1.2 1.1 6.0 4.9 Distribute students among other classes 6.6 5.8 -0.8 11.9 3.4 -8.5 7.4 4.0 -3.4 Other 8.3 14.9 6.6 8.3 21.6 13.3 4.0 10.1 6.1 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌108 Exhibit A60. Impact on Students Reporting School Serving Schools Meals, Current Hunger, and Household Hunger ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students School serves school meals -0.1 -28.9 -28.7** -26.4 -26.3** (7.8) (3.7) Eats fewer meals at home when receives a school meal -1.8 -45.5 -43.7** -27.5 -25.6** (7.8) (4.0) Feels less hungry on days when receives a school meal -0.3 -22.9 -22.6** -17.6 -17.3** (8.7) (4.5) Feels hungry now -8.2 0.9 9.1 -15.2 -6.9 (6.2) (4.9) Any household hunger during school year 10.3 13.2 2.9 -3.1 -13.4** (4.8) (3.8) Mid-Grade Students School serves school meals 2.4 -39.8 -42.2** -29.0 -31.4** (6.5) (4.0) Eats fewer meals at home when receives a school meal -2.5 -45.5 -42.9** -37.5 -35.0** (6.4) (3.5) Feels less hungry on days when receives a school meal 2.6 -33.9 -36.5** -18.7 -21.3** (6.3) (4.4) Feels hungry now -2.1 12.2 14.3* -2.0 0.1 (6.7) (4.6) Any household hunger during school year 6.6 12.4 5.8 1.1 -5.5 (4.5) (4.5) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,473 Comparison; 2,574 School feeding only; 5,488 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌109 Exhibit A61. Percentage of Students Reporting that School Served School Meals Yesterday, and Frequency of Household Hunger ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff At schools that serve meals: School meal served yesterday 4.2 3.6 -0.6 77.4 1.1 -76.3 71.3 41.8 -29.5 Every month 28.6 61.7 33.1 15.0 27.6 12.6 16.5 24.5 8.1 During the rainy season 20.3 10.6 -9.7 12.3 7.5 -4.8 15.7 23.7 8.0 Other 51.1 27.8 - 23.3 72.8 64.9 -7.9 67.8 51.7 -16.1 At schools that serve meals: School meal served yesterday 5.4 5.0 -0.3 72.6 1.0 -71.6 68.9 35.1 -33.7 Every month 26.5 53.7 27.1 12.2 11.8 -0.4 33.3 23.3 -10.0 During the rainy season 13.1 3.7 -9.5 15.2 11.5 -3.8 14.3 19.8 5.4 Other 60.3 42.7 - 17.7 72.6 76.7 4.2 52.3 56.9 4.6 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Exhibit A62. Head teacher Reporting on School Feeding Program ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff School has a school feeding program 5.3 14.3 8.9 98.8 37.5 -61.3 100.0 64.7 -35.3 Organization supporting school feeding (if program is present) PAI/ADPP 0.0 0.0 0.0 100.0 100.0 0.0 46.1 18.7 -27.4 WV 0.0 0.0 0.0 0.0 0.0 0.0 51.0 79.6 28.7 Community support 0.0 59.3 59.3 0.0 0.0 0.0 0.0 0.0 0.0 Other 100.0 40.7 -59.3 0.0 0.0 0.0 4.7 1.7 -3.0 Frequency of school meals (if program is present) Daily 100.0 0.0 -100.0 91.6 0.0 -91.6 92.3 0.0 -92.3 A few times per week 0.0 78.3 78.3 4.8 100.0 95.2 2.9 98.1 95.2 Whenever food is available 0.0 21.7 21.7 2.4 0.0 -2.4 4.8 1.9 -3.0 Issues with school feeding program (if program is present) No issues 10.0 33.3 23.3 30.1 45.5 15.3 18.1 26.7 8.5 Food is not always available 10.0 18.5 8.5 2.4 12.1 9.7 4.1 17.8 13.7 Kitchen/warehouse not constructed or in disrepair 30.0 18.5 -11.5 1.2 0.0 -1.2 4.2 9.8 5.6 Not enough pots, utensils, or dishes 40.0 3.7 -36.3 20.5 6.1 -14.4 31.5 24.2 -7.3 Not enough volunteer cooks 0.0 14.8 14.8 7.2 15.2 7.9 5.9 14.9 9.0 Other 70.0 25.9 -44.1 53.0 42.4 -10.6 68.3 47.2 -21.1 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌110 Exhibit A63. Impact on Teacher-Reported Nutrition and Hygiene Teaching ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Early-Grade Students Teacher had training on teaching nutrition -3.8 -19.8 -16.0* -25.9 -22.1** (7.1) (4.7) Teacher has nutrition lesson plans -17.8 -3.5 14.3* -19.4 -1.7 (6.5) (5.8) Teacher had training on teaching hygiene and sanitation -3.7 -0.2 3.5 -2.1 1.6 (7.3) (6.9) Teacher has hygiene and sanitation lesson plans -18.9 -3.0 15.8* -2.5 16.4** (6.4) (4.9) Mid-Grade Students Teacher had training on teaching nutrition -8.2 0.6 8.8 -11.6 -3.4 (7.0) (4.9) Teacher has nutrition lesson plans -25.2 4.6 29.8** -15.3 9.9 (6.5) (5.3) Teacher had training on teaching hygiene and sanitation 20.4 3.8 -16.6* 11.3 -9.1 (7.6) (6.4) Teacher has hygiene and sanitation lesson plans -13.3 0.3 13.6 -9.6 3.7 (7.6) (6.5) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 632 Comparison; 311 School feeding only; 634 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A64. Teachers' Perception of Student Knowledge of Nutrition, Hygiene, and Sanitation ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Teacher rating of student knowledge of nutrition Weak 38.5 29.7 -8.9 18.6 18.9 0.3 41.2 22.1 -19.1 Average 51.8 49.5 -2.3 72.4 72.1 -0.4 51.3 63.6 12.3 Strong 9.7 20.9 11.2 9.0 9.0 0.0 7.5 14.3 6.8 Teacher rating of student knowledge of hygiene and sanitation Weak 22.1 9.2 -13.0 15.2 4.0 -11.2 28.6 6.7 -21.8 Average 56.3 55.7 -0.5 64.6 77.3 12.8 56.1 69.5 13.4 Strong 21.6 35.1 13.5 20.3 18.7 -1.6 15.3 23.8 8.4 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌111 Exhibit A65. Impact on Head Teacher Reported Teacher Training on Nutrition, Hygiene, and Health ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Teachers trained on teaching nutrition -58.2 -40.0 18.2 -115.1 -56.9 (64.0) (45.1) Teachers trained on teaching hygiene and sanitation 10.9 38.3 27.4 -3.4 -14.3 (70.7) (52.0) Teachers have materials for teaching nutrition 5.4 17.5 12.2 1.5 -3.9 (8.2) (6.1) Teachers have materials for teaching hygiene and sanitation 22.7 21.6 -1.1 19.2 -3.5 (9.0) (6.8) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 279 Comparison; 171 School feeding only; 321 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A66. Head Teacher Reported Frequency of Nutrition, Hygiene, and Sanitation Lessons ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff Frequency of teaching nutrition lessons Once per week 15.3 12.9 -2.4 64.1 62.2 -1.9 60.5 43.0 -17.5 Once every two weeks 6.1 5.4 -0.7 7.7 11.0 3.3 9.1 8.5 -0.7 Once per month 7.7 8.1 0.4 12.8 9.8 -3.1 14.2 13.1 -1.0 Once per term 38.6 0.0 -38.6 9.0 0.0 -9.0 6.1 0.0 -6.1 Never 32.3 52.4 20.1 6.4 7.3 0.9 10.1 21.7 11.6 Frequency of teaching hygiene and sanitation lessons Once per week 35.0 29.3 -5.8 69.6 59.3 -10.3 66.5 56.4 -10.1 Once every two weeks 8.6 4.5 -4.1 6.3 8.1 1.8 8.5 8.7 0.2 Once per month 6.8 13.0 6.2 5.1 8.1 3.1 7.2 11.4 4.2 Once per term 26.3 0.0 -26.3 6.3 0.0 -6.3 7.1 0.0 -7.1 Never 23.3 29.3 6.0 12.7 8.1 -4.5 10.7 7.0 -3.7 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌112 Exhibit A67. Impact on Students Receiving Deworming Medication ........................................................ Comparison School Feeding Only School Feeding and Literacy Difference Difference Difference in Differences Difference Difference in Differences (a) (b) (b)-(a) (c) (c)-(a) Students are given deworming medication -15.6 -1.6 14.0 0.3 15.9** (7.3) (5.9) Students received deworming medication at least once during this school year -39.8 -37.1 2.7 -22.2 17.6** (8.5) (6.2) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 352 Comparison; 174 School feeding only; 346 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A68. Head Teacher Reported Cleanliness of latrines ........................................................ Comparison School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff BL EL Diff All latrines are dirty with feces on the slabs 16.8 5.3 -11.5 11.6 4.5 -7.0 13.0 8.8 -4.3 Some latrines are dirty 55.2 27.1 -28.1 44.9 28.4 -16.5 45.4 37.8 -7.7 All latrines are clean with no feces on the slabs 28.0 45.7 17.8 43.5 65.9 22.4 41.5 52.9 11.4 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌113 Exhibit A69. Cooks Reporting on School Infrastructure ........................................................ School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff School has a school feeding log book 92.3 52.5 -39.8 78.6 68.1 -10.5 Log book is filled out and up to date 84.6 81.0 -3.6 71.4 83.2 11.8 School has a kitchen and storeroom for school feeding program 98.7 97.5 -1.2 98.2 100.0 1.8 Cooking pots and cooking utensils (such as stirrers or scoops) are reported to be available 92.3 82.5 -9.8 86.2 88.6 2.4 Cooking pots and cooking utensils (such as stirrers or scoops) are reported to be sufficient 75.6 50.0 -25.6 57.2 54.2 -3.0 School has bowls and spoons for students 78.3 89.7 11.4 80.2 89.6 9.4 Number of bowls and spoons for students sufficient for each meal session 60.9 69.2 8.3 52.1 68.4 16.3 School has ever had a gap in the supply of corn-soy blend 37.2 62.1 24.9 50.0 77.3 27.3 Condition of the kitchen and storeroom Not adequate (holes in roof or walls, no door, and/or no lock) 16.7 9.0 -7.7 19.7 9.6 -10.1 Moderate (complete building but needs repairs) 59.0 56.4 -2.6 52.4 58.3 5.9 Very good (complete building with no repairs needed) 23.1 34.6 11.5 28.0 32.1 4.1 School has clean water available 59.4 72.5 13.1 67.2 64.7 -2.5 Soap is available to clean bowls, spoons, pots, and cooking utensils 60.9 79.3 18.4 36.0 48.1 12.1 School has a place for cooks to wash their hands 73.9 73.8 -0.1 59.0 76.6 17.6 Soap is available for cooks to wash their hands 59.4 75.0 15.6 28.3 54.4 26.1 School has a place for children to wash their hands before eating 72.5 72.5 0 64.7 78.5 13.8 Soap is available for children to wash their hands before eating 44.9 72.5 27.6 18.9 53.9 35 School meals are served on time Never 5.1 0.0 -5.1 3.8 1.9 -1.9 Sometimes 47.4 41.4 -6.0 47.3 41.3 -6.0 Always 47.4 58.6 11.2 48.9 56.8 7.9 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌114 Exhibit A70. Cook Training and Knowledge ........................................................ School Feeding Only School Feeding and Literacy BL EL Diff BL EL Diff School cook training received 40.6 70.0 29.4 47.0 80.8 33.8 Timing of last school cook training Less than 6 months ago 18.8 0.0 -18.8 13.0 46.6 33.6 6- 11 months ago 7.2 3.6 -3.6 8.6 7.4 -1.2 12 or more months ago 14.5 96.4 81.9 26.2 46.0 19.8 Cook knowledge of food safety and hygiene Cook's knowledge of food safety (8 items) 3.4 5.2 1.8 3.8 5.5 1.7 Cook's knowledge of hygiene (5 items) 2.3 2.8 0.5 2.7 3.2 0.5 Note: ""BL" indicates baseline; "EL" indicates endline; "Diff" indicates difference. Abt Associates June 2022 ▌115 SUPPLEMENTAL FINDINGS RELATED TO IMPLEMENTATION BY ITEMS Exhibit A71. Percentage of Schools that Adequately Implemented Literacy, Training, and Feeding Program Items ...................................................................................................................... ............. Comparison School Feeding Only School Feeding and Literacy Literacy Items Do you have a school literacy program? 36.2 56.8 71.8 Does the school have a library/book bank? 22.0 18.2 57.1 Does the school have a literacy afterschool program? 14.1 39.8 50.3 Are there other types of afterschool learning clubs? 14.1 39.8 50.3 Do you have a Teacher Guide for reading instruction? 70.6 77.3 80.0 Do you use the Teacher Guide for reading instruction? 71.8 80.7 81.7 Do each of your students have exercise books that can be used during reading instruction? 93.8 89.8 91.7 Do your students use the exercise books during reading instruction? 97.7 97.7 97.2 Do you have access to school supplies? 100.0 98.9 99.4 Do you use any school supplies in your classroom during reading instruction? 99.4 100.0 100.0 Is there an afterschool literacy club in this school? 10.2 39.8 51.7 Do your students attend an afterschool literacy club at this school? 8.5 31.8 41.1 Training Items Did you attend the teacher training held this academic year? 0.0 85.2 90.6 Did teachers at the school attend the literacy training this year? 0.0 81.8 92.8 Do reading coaches supervise teachers at the school? 30.5 58.0 85.6 Have you received training on the fundamentals of reading instruction? 36.7 71.6 79.4 Have you received training on school management? 71.8 84.1 84.4 During your pre-service training, did you receive any specific training on how to teach reading and writing to students in the primary grades? 79.1 73.9 87.4 This year, have you attended any in-service training on how to teach reading in the primary grades? 46.3 46.6 67.8 Have you attended the WV/PAI training? 0.0 85.2 88.0 Have you received training on the fundamentals of reading instruction? 65.5 84.1 92.3 Does the Head Teacher or Assistant Head Teacher check your lesson plans? 100.0 100.0 100.0 Since the beginning of the school year, how frequently does the Assistant Head Teacher or Head Teacher observe your teaching? 96.6 98.9 98.9 Since the beginning of the school year, how frequently does the reading coach visit you? 23.7 45.5 72.7 Feeding Items Does the school have a school feeding program? 15.3 37.5 63.3 How often do students receive the school meals? 10.2 36.4 58.2 Does the school have a garden? 13.6 33.0 20.9 How often is the garden used to supply food for school meals? 1.1 3.4 2.3 Does the school have a kitchen and storeroom for the school feeding program? 81.5 97.5 100.0 Are cooking pots and cooking utensils (such as stirrers or scoops) available? 96.3 82.5 88.5 Do you supplement the corn-soy blend with other foods? 3.7 8.8 1.3 Does the school have bowls and spoons for students? 77.8 32.5 60.3 Is there soap available to clean the bowls, spoons, pots, and cooking utensils? 92.6 28.8 32.7 Is there a supply of clean water (from a borehole or rainwater tank) available at the school? 85.2 72.5 65.4 Abt Associates June 2022 ▌116 ...................................................................................................................... ............. Comparison School Feeding Only School Feeding and Literacy Is there a place for children to wash their hands before eating? 100.0 72.5 78.8 Is there soap available for children to wash their hands before eating? 92.6 72.5 54.5 Did you receive any training for being a school cook? 44.4 70.0 80.1 Is there a place for cooks to wash their hands? 85.2 73.8 76.9 Is there soap for cooks to wash their hands? 96.3 75.0 55.1 Exhibit A72. Impact on Reading Comprehension by Literacy Item Adequately Implemented by Schools in the Treatment Arms ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Early-Grade Students Do you have a school literacy program? 2.1 4.4* (3.5) (2.0) Does the school have a library/book bank? 4.6 4.1* (6.0) (1.9) Does the school have a literacy afterschool program? 4.1 6.6** (4.4) (2.4) Are there other types of afterschool learning clubs? 4.1 6.6** (4.4) (2.4) Do you have a Teacher Guide for reading instruction? 2.3 4.7* (3.1) (1.9) Do you use the Teacher Guide for reading instruction? 0.5 4.7* (3.2) (1.9) Do each of your students have exercise books that can be used during reading instruction? 1.6 5.8** (3.1) (1.9) Do your students use the exercise books during reading instruction? 1.7 4.8* (3.0) (1.9) Do you have access to school supplies? 1.4 4.9** (2.9) (1.9) Do you use any school supplies in your classroom during reading instruction? 1.2 4.8** (2.9) (1.9) Is there an afterschool literacy club in this school? 2.8 2.5 (4.6) (2.2) Do your students attend an afterschool literacy club at this school? -2.8 5.0 (3.0) (2.6) Mid-Grade Students Do you have a school literacy program? -3.7 7.1* (4.6) (3.5) Does the school have a library/book bank? -4.2 9.0* (7.1) (3.5) Does the school have a literacy afterschool program? -4.3 8.6* (5.3) (3.9) Are there other types of afterschool learning clubs? -4.3 8.6* (5.3) (3.9) Do you have a Teacher Guide for reading instruction? -1.4 9.8** (4.3) (3.2) Do you use the Teacher Guide for reading instruction? -4.9 9.8** (4.8) (3.2) Abt Associates June 2022 ▌117 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Do each of your students have exercise books that can be used during reading instruction? -2.5 8.5* (4.7) (3.4) Do your students use the exercise books during reading instruction? -2.3 7.6* (4.6) (3.4) Do you have access to school supplies? -1.7 7.7* (4.6) (3.3) Do you use any school supplies in your classroom during reading instruction? -1.8 7.6* (4.6) (3.3) Is there an afterschool literacy club in this school? -2.3 12.5** (5.4) (3.3) Do your students attend an afterschool literacy club at this school? 1.0 12.6** (4.5) (3.5) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,474 Comparison; 1,436 School feeding only; 3,951 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A73. Impact on Reading Comprehension by Training Item Adequately Implemented by Schools in the Treatment Arms ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Early-Grade Students Did you attend the teacher training held this academic year? 0.5 4.7* (3.1) (1.9) Did teachers at the school attend the literacy training this year? 3.7 4.9* (3.1) (1.9) Do reading coaches supervise teachers at the school? 1.5 4.6* (3.1) (1.9) Have you received training on the fundamentals of reading instruction? 2.8 3.5 (3.0) (1.8) Have you received training on school management? 1.6 5.3** (2.7) (2.0) During your pre-service training, did you receive any specific training on how to teach reading and writing to students in the primary grades? -0.7 5.4** (3.1) (2.0) This year, have you attended any in-service training on how to teach reading in the primary grades? -3.4 6.9** (2.5) (2.0) Have you attended the WV/PAI training? 0.5 5.4** (3.2) (1.9) Have you received training on the fundamentals of reading instruction? 1.0 5.1** (3.2) (1.9) Does the Head Teacher or Assistant Head Teacher check your lesson plans? 1.2 4.8** (2.9) (1.9) Since the beginning of the school year, how frequently does the Assistant Head Teacher or Head Teacher observe your teaching? 1.3 4.3* (3.0) (1.8) Abt Associates June 2022 ▌118 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Since the beginning of the school year, how frequently does the reading coach visit you? 3.7 4.5* (4.0) (1.9) Mid-Grade Students Did you attend the teacher training held this academic year? -1.7 7.3* (4.9) (3.4) Did teachers at the school attend the literacy training this year? 0.1 7.0* (4.3) (3.3) Do reading coaches supervise teachers at the school? 1.1 7.3* (4.1) (3.4) Have you received training on the fundamentals of reading instruction? 1.7 7.4* (3.9) (3.5) Have you received training on school management? 3.6 8.0* (3.8) (3.5) During your pre-service training, did you receive any specific training on how to teach reading and writing to students in the primary grades? -2.0 8.5* (5.1) (3.4) This year, have you attended any in-service training on how to teach reading in the primary grades? 2.3 11.8** (4.2) (3.2) Have you attended the WV/PAI training? -2.0 8.7* (4.9) (3.4) Have you received training on the fundamentals of reading instruction? -2.1 7.3* (4.9) (3.4) Does the Head Teacher or Assistant Head Teacher check your lesson plans? -1.8 7.6* (4.6) (3.3) Since the beginning of the school year, how frequently does the Assistant Head Teacher or Head Teacher observe your teaching? -1.6 7.4* (4.6) (3.3) Since the beginning of the school year, how frequently does the reading coach visit you? 0.1 10.2** (5.6) (3.4) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,474 Comparison; 2,174 School feeding only; 4,919 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A74. Impact on Reading Comprehension by Feeding Item Adequately Implemented by Schools in the Treatment Arms ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Early-Grade Students Does the school have a school feeding program? -0.9 4.9* (4.1) (2.1) How often do students receive the school meals? -0.7 4.5* (4.2) (2.1) Does the school have a garden? 2.3 6.4 (4.8) (4.4) How often is the garden used to supply food for school meals? -5.2 21.1** Abt Associates June 2022 ▌119 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences (3.8) (4.4) Does the school have a kitchen and storeroom for the school feeding program? 0.7 4.7* (3.1) (2.0) Are cooking pots and cooking utensils (such as stirrers or scoops) available? 0.1 5.2* (3.4) (2.0) Do you supplement the corn-soy blend with other foods? -6.7* 3.6 (3.0) (2.3) Does the school have bowls and spoons for students? 2.9 5.9** (4.9) (2.1) Is there soap available to clean the bowls, spoons, pots, and cooking utensils? -0.6 3.8 (4.9) (2.9) Is there a supply of clean water (from a borehole or rainwater tank) available at the school? -0.4 5.3* (3.2) (2.3) Is there a place for children to wash their hands before eating? 1.8 4.8* (3.5) (2.1) Is there soap available for children to wash their hands before eating? 0.4 3.0 (3.6) (2.3) Did you receive any training for being a school cook? 0.2 5.1** (3.8) (1.9) Is there a place for cooks to wash their hands? 2.0 4.7* (3.6) (2.0) Is there soap for cooks to wash their hands? 0.8 3.5 (3.5) (2.3) Mid-Grade Students Does the school have a school feeding program? -7.2 12.1** (6.2) (2.9) How often do students receive the school meals? -7.0 12.4** (6.3) (2.9) Does the school have a garden? -4.8 7.3 (6.1) (6.9) How often is the garden used to supply food for school meals? 14.5 10.7* (10.0) (4.2) Does the school have a kitchen and storeroom for the school feeding program? -1.4 7.9* (4.8) (3.3) Are cooking pots and cooking utensils (such as stirrers or scoops) available? -1.0 7.6* (5.3) (3.5) Do you supplement the corn-soy blend with other foods? 6.3 16.6** (7.3) (5.7) Does the school have bowls and spoons for students? -0.3 13.5** (6.4) (2.8) Is there soap available to clean the bowls, spoons, pots, and cooking utensils? -7.5 14.8** (7.1) (3.6) Is there a supply of clean water (from a borehole or rainwater tank) available at the school? -4.1 6.2 (5.1) (3.8) Is there a place for children to wash their hands before eating? -3.0 7.7* (5.2) (3.6) Is there soap available for children to wash their hands before eating? -3.1 8.9* (5.4) (4.3) Did you receive any training for being a school cook? -3.2 11.2** (5.7) (3.2) Abt Associates June 2022 ▌120 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Is there a place for cooks to wash their hands? -2.4 8.0* (5.4) (3.6) Is there soap for cooks to wash their hands? -2.1 7.9 (5.3) (4.4) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,474 Comparison; 950 School feeding only; 3,479 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A75. Impact on Student Attendance by Literacy Item Adequately Implemented by Schools in the Treatment Arms ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Early-Grade Students Do you have a school literacy program? 3.0 -5.4 (5.8) (3.8) Does the school have a library/book bank? 3.3 -4.6 (9.5) (3.9) Does the school have a literacy afterschool program? -0.2 -6.0 (7.3) (4.3) Are there other types of afterschool learning clubs? -0.2 -6.0 (7.3) (4.3) Do you have a Teacher Guide for reading instruction? 1.2 -7.4* (5.5) (3.5) Do you use the Teacher Guide for reading instruction? -3.3 -7.3* (6.7) (3.5) Do each of your students have exercise books that can be used during reading instruction? -3.3 -5.9 (6.2) (3.7) Do your students use the exercise books during reading instruction? -3.6 -6.3 (6.0) (3.6) Do you have access to school supplies? -4.6 -5.7 (5.9) (3.6) Do you use any school supplies in your classroom during reading instruction? -4.6 -5.7 (5.8) (3.6) Is there an afterschool literacy club in this school? -1.1 -10.2** (5.5) (3.9) Do your students attend an afterschool literacy club at this school? -3.7 -10.6* (6.6) (4.2) Mid-Grade Students Do you have a school literacy program? 3.8 2.6 (6.1) (4.2) Does the school have a library/book bank? 12.9* 0.8 (5.6) (4.5) Does the school have a literacy afterschool program? 7.0 1.4 (5.1) (4.7) Are there other types of afterschool learning clubs? 7.0 1.4 (5.1) (4.7) Do you have a Teacher Guide for reading instruction? 1.1 2.8 Abt Associates June 2022 ▌121 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences (5.3) (4.0) Do you use the Teacher Guide for reading instruction? 2.7 3.0 (4.9) (4.0) Do each of your students have exercise books that can be used during reading instruction? 4.7 2.1 (4.7) (3.7) Do your students use the exercise books during reading instruction? 4.5 4.4 (4.6) (3.8) Do you have access to school supplies? 4.7 3.7 (4.5) (3.8) Do you use any school supplies in your classroom during reading instruction? 4.8 3.9 (4.5) (3.8) Is there an afterschool literacy club in this school? 7.4 -1.4 (5.0) (4.1) Do your students attend an afterschool literacy club at this school? 5.0 0.2 (5.3) (4.3) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,473 Comparison; 1,435 School feeding only; 3,941 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A76. Impact on Student Attendance by Training Item Adequately Implemented by Schools in the Treatment Arms ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Early-Grade Students Did you attend the teacher training held this academic year? -7.9 -5.3 (6.0) (3.7) Did teachers at the school attend the literacy training this year? -2.1 -5.9 (5.5) (3.6) Do reading coaches supervise teachers at the school? -4.1 -5.5 (6.4) (3.6) Have you received training on the fundamentals of reading instruction? -6.9 -5.9 (5.6) (3.5) Have you received training on school management? -0.4 -5.6 (5.7) (3.8) During your pre-service training, did you receive any specific training on how to teach reading and writing to students in the primary grades? -7.5 -5.9 (6.4) (3.7) This year, have you attended any in-service training on how to teach reading in the primary grades? 4.3 -3.2 (5.3) (3.5) Have you attended the WV/PAI training? -3.7 -5.3 (6.3) (3.6) Have you received training on the fundamentals of reading instruction? -5.4 -5.8 (6.2) (3.6) Does the Head Teacher or Assistant Head Teacher check your lesson plans? -4.6 -5.7 (5.8) (3.6) Abt Associates June 2022 ▌122 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Since the beginning of the school year, how frequently does the Assistant Head Teacher or Head Teacher observe your teaching? -5.0 -4.5 (5.9) (3.5) Since the beginning of the school year, how frequently does the reading coach visit you? 4.3 -7.3* (4.8) (3.5) Mid-Grade Students Did you attend the teacher training held this academic year? 5.8 1.8 (4.7) (3.7) Did teachers at the school attend the literacy training this year? 6.3 3.8 (4.1) (3.9) Do reading coaches supervise teachers at the school? 0.1 3.0 (5.7) (3.9) Have you received training on the fundamentals of reading instruction? 5.8 1.9 (5.8) (4.1) Have you received training on school management? 6.8 2.6 (4.1) (3.8) During your pre-service training, did you receive any specific training on how to teach reading and writing to students in the primary grades? 3.2 3.5 (4.9) (3.9) This year, have you attended any in-service training on how to teach reading in the primary grades? -1.1 -0.1 (7.6) (4.1) Have you attended the WV/PAI training? 4.5 4.3 (4.7) (3.8) Have you received training on the fundamentals of reading instruction? 4.0 4.2 (4.8) (3.9) Does the Head Teacher or Assistant Head Teacher check your lesson plans? 4.8 3.9 (4.5) (3.8) Since the beginning of the school year, how frequently does the Assistant Head Teacher or Head Teacher observe your teaching? 4.5 3.8 (4.5) (3.9) Since the beginning of the school year, how frequently does the reading coach visit you? -2.8 1.7 (7.4) (4.0) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,473 Comparison; 2,172 School feeding only; 4,907 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A77. Impact on Student Attendance by Feeding Item Adequately Implemented by Schools in the Treatment Arms ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Early-Grade Students Does the school have a school feeding program? -3.8 -6.8* (8.9) (3.4) How often do students receive the school meals? -3.1 -7.1* (9.0) (3.5) Abt Associates June 2022 ▌123 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Does the school have a garden? 1.9 -11.3* (6.8) (5.5) How often is the garden used to supply food for school meals? -6.8 -34.5** (17.9) (6.0) Does the school have a kitchen and storeroom for the school feeding program? -3.1 -5.9 (6.2) (3.7) Are cooking pots and cooking utensils (such as stirrers or scoops) available? -4.3 -5.8 (6.5) (3.7) Do you supplement the corn-soy blend with other foods? -12.4 -21.9 (9.9) (21.0) Does the school have bowls and spoons for students? 2.9 -8.1* (6.8) (3.5) Is there soap available to clean the bowls, spoons, pots, and cooking utensils? -6.5 -6.4 (9.0) (3.9) Is there a supply of clean water (from a borehole or rainwater tank) available at the school? -6.0 -4.4 (6.8) (4.1) Is there a place for children to wash their hands before eating? -3.1 -5.3 (7.0) (3.7) Is there soap available for children to wash their hands before eating? -4.6 -4.0 (6.9) (3.9) Did you receive any training for being a school cook? -3.7 -7.4* (7.3) (3.6) Is there a place for cooks to wash their hands? -1.3 -5.0 (7.1) (3.6) Is there soap for cooks to wash their hands? -2.9 -4.0 (6.9) (3.9) Mid-Grade Students Does the school have a school feeding program? 6.0 -1.9 (6.4) (4.0) How often do students receive the school meals? 6.4 -2.0 (6.4) (4.1) Does the school have a garden? 11.7* 6.9 (5.1) (7.0) How often is the garden used to supply food for school meals? 33.4** 2.7 (8.5) (5.7) Does the school have a kitchen and storeroom for the school feeding program? 4.1 2.3 (4.6) (3.7) Are cooking pots and cooking utensils (such as stirrers or scoops) available? 4.1 1.3 (5.0) (3.8) Do you supplement the corn-soy blend with other foods? 13.5* 18.2 (5.8) (19.2) Does the school have bowls and spoons for students? 2.5 -5.1 (8.7) (3.8) Is there soap available to clean the bowls, spoons, pots, and cooking utensils? 3.4 -4.2 (7.3) (4.8) Is there a supply of clean water (from a borehole or rainwater tank) available at the school? 1.9 2.4 (5.1) (4.1) Is there a place for children to wash their hands before eating? 2.7 4.1 (5.1) (4.1) Is there soap available for children to wash their hands before eating? 4.6 4.6 Abt Associates June 2022 ▌124 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences (5.2) (4.4) Did you receive any training for being a school cook? 3.5 0.4 (5.5) (3.9) Is there a place for cooks to wash their hands? 3.6 2.1 (5.1) (4.0) Is there soap for cooks to wash their hands? 4.3 4.5 (5.0) (4.6) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 5,473 Comparison; 948 School feeding only; 3,467 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A78. Impact on Student Attentiveness by Literacy Item Adequately Implemented by Schools in the Treatment Arms ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Early-Grade Students Do you have a school literacy program? -7.3 -5.7 (4.1) (3.6) Does the school have a library/book bank? -7.4* -1.4 (3.6) (3.2) Does the school have a literacy afterschool program? -7.3 -2.1 (4.7) (3.7) Are there other types of afterschool learning clubs? -7.3 -2.1 (4.7) (3.7) Do you have a Teacher Guide for reading instruction? -5.4 -3.7 (4.3) (3.7) Do you use the Teacher Guide for reading instruction? -5.5 -3.7 (4.2) (3.7) Do each of your students have exercise books that can be used during reading instruction? -3.1 -4.5 (4.2) (3.6) Do your students use the exercise books during reading instruction? -3.4 -4.0 (4.2) (3.5) Do you have access to school supplies? -3.5 -4.1 (4.1) (3.5) Do you use any school supplies in your classroom during reading instruction? -3.5 -4.1 (4.1) (3.5) Is there an afterschool literacy club in this school? -3.9 -6.7 (5.9) (4.3) Do your students attend an afterschool literacy club at this school? -3.3 -3.3 (7.3) (4.2) Mid-Grade Students Do you have a school literacy program? -10.6** -2.6 (3.4) (3.0) Does the school have a library/book bank? -0.9 -2.4 (6.0) (3.1) Does the school have a literacy afterschool program? -8.1** -2.1 (2.9) (3.0) Abt Associates June 2022 ▌125 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Are there other types of afterschool learning clubs? -8.1** -2.1 (2.9) (3.0) Do you have a Teacher Guide for reading instruction? -5.2 -3.0 (3.4) (2.9) Do you use the Teacher Guide for reading instruction? -7.1* -3.6 (3.6) (2.9) Do each of your students have exercise books that can be used during reading instruction? -5.5 -3.2 (3.3) (2.9) Do your students use the exercise books during reading instruction? -7.2* -2.9 (3.4) (2.9) Do you have access to school supplies? -7.2* -3.0 (3.3) (2.9) Do you use any school supplies in your classroom during reading instruction? -7.2* -3.0 (3.3) (2.9) Is there an afterschool literacy club in this school? -3.0 -3.5 (4.5) (2.9) Do your students attend an afterschool literacy club at this school? -8.1** -2.6 (3.0) (3.0) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 632 Comparison; 177 School feeding only; 448 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A79. Impact on Student Attentiveness by Training Item Adequately Implemented by Schools in the Treatment Arms ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Early-Grade Students Did you attend the teacher training held this academic year? -3.1 -6.0 (4.3) (3.5) Did teachers at the school attend the literacy training this year? -2.6 -4.0 (4.4) (3.5) Do reading coaches supervise teachers at the school? -2.6 -4.4 (4.5) (3.6) Have you received training on the fundamentals of reading instruction? -3.8 -2.3 (4.6) (3.3) Have you received training on school management? -4.5 -5.6 (4.3) (3.5) During your pre-service training, did you receive any specific training on how to teach reading and writing to students in the primary grades? -2.4 -3.6 (4.3) (3.6) This year, have you attended any in-service training on how to teach reading in the primary grades? -1.8 -3.0 (5.2) (3.4) Have you attended the WV/PAI training? -3.2 -3.3 (4.3) (3.6) Have you received training on the fundamentals of reading instruction? -5.7 -3.9 (4.2) (3.5) Abt Associates June 2022 ▌126 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Does the Head Teacher or Assistant Head Teacher check your lesson plans? -3.5 -4.1 (4.1) (3.5) Since the beginning of the school year, how frequently does the Assistant Head Teacher or Head Teacher observe your teaching? -3.5 -3.9 (4.1) (3.5) Since the beginning of the school year, how frequently does the reading coach visit you? -4.3 -4.4 (5.4) (3.7) Mid-Grade Students Did you attend the teacher training held this academic year? -5.7 -3.7 (3.3) (2.9) Did teachers at the school attend the literacy training this year? -7.0 -3.4 (3.6) (2.9) Do reading coaches supervise teachers at the school? -7.4* -3.2 (2.9) (2.8) Have you received training on the fundamentals of reading instruction? -7.3 -2.6 (3.8) (3.0) Have you received training on school management? -7.5* -3.9 (3.5) (2.9) During your pre-service training, did you receive any specific training on how to teach reading and writing to students in the primary grades? -9.5** -3.2 (3.2) (2.9) This year, have you attended any in-service training on how to teach reading in the primary grades? -3.4 -2.5 (4.4) (2.9) Have you attended the WV/PAI training? -5.7 -3.1 (3.3) (2.9) Have you received training on the fundamentals of reading instruction? -7.2* -3.4 (3.5) (2.9) Does the Head Teacher or Assistant Head Teacher check your lesson plans? -7.2* -3.0 (3.3) (2.9) Since the beginning of the school year, how frequently does the Assistant Head Teacher or Head Teacher observe your teaching? -5.8 -2.9 (3.2) (2.9) Since the beginning of the school year, how frequently does the reading coach visit you? -3.9 -4.2 (4.0) (2.9) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 632 Comparison; 265 School feeding only; 562 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Exhibit A80. Impact on Student Attentiveness by Feeding Item Adequately Implemented by Schools in the Treatment Arms ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences Early-Grade Students Does the school have a school feeding program? -6.9 -4.4 (4.5) (4.0) Abt Associates June 2022 ▌127 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences How often do students receive the school meals? -7.1 -6.7 (4.5) (3.9) Does the school have a garden? -1.0 -8.4 (5.0) (6.0) How often is the garden used to supply food for school meals? -7.8* -6.9 (3.9) (3.6) Does the school have a kitchen and storeroom for the school feeding program? -4.3 -4.1 (4.2) (3.6) Are cooking pots and cooking utensils (such as stirrers or scoops) available? -3.9 -3.4 (4.5) (3.7) Do you supplement the corn-soy blend with other foods? -7.9* -1.0 (3.9) (2.4) Does the school have bowls and spoons for students? -7.2 -3.2 (3.8) (4.7) Is there soap available to clean the bowls, spoons, pots, and cooking utensils? -6.9 -2.2 (3.8) (6.4) Is there a supply of clean water (from a borehole or rainwater tank) available at the school? -3.9 -8.0* (4.7) (3.7) Is there a place for children to wash their hands before eating? -2.0 -6.0 (4.5) (3.7) Is there soap available for children to wash their hands before eating? -2.1 -4.8 (4.5) (4.2) Did you receive any training for being a school cook? -3.9 -1.1 (4.8) (3.4) Is there a place for cooks to wash their hands? -2.0 -6.0 (4.5) (3.7) Is there soap for cooks to wash their hands? -2.2 -4.9 (4.5) (4.1) Mid-Grade Students Does the school have a school feeding program? -3.5 -2.9 (4.2) (3.1) How often do students receive the school meals? -3.6 -2.7 (4.3) (3.1) Does the school have a garden? -8.3** -5.3 (3.1) (3.2) How often is the garden used to supply food for school meals? -8.6** -6.9* (3.1) (2.9) Does the school have a kitchen and storeroom for the school feeding program? -7.1* -3.1 (3.5) (2.9) Are cooking pots and cooking utensils (such as stirrers or scoops) available? -7.1 -3.3 (3.7) (2.9) Do you supplement the corn-soy blend with other foods? -8.6** -1.7 (3.1) (3.4) Does the school have bowls and spoons for students? -1.9 -1.7 (5.3) (3.4) Is there soap available to clean the bowls, spoons, pots, and cooking utensils? -1.5 -1.3 (5.4) (3.3) Is there a supply of clean water (from a borehole or rainwater tank) available at the school? -8.1** -4.2 (2.9) (2.9) Is there a place for children to wash their hands before eating? -5.1 -3.2 Abt Associates June 2022 ▌128 ................................................................................................................................... School Feeding Only School Feeding and Literacy Difference in Differences Difference in Differences (3.6) (2.9) Is there soap available for children to wash their hands before eating? -5.2 -2.6 (3.6) (3.1) Did you receive any training for being a school cook? -7.6 -1.7 (3.9) (3.0) Is there a place for cooks to wash their hands? -5.2 -3.3 (3.6) (2.9) Is there soap for cooks to wash their hands? -5.2 -3.0 (3.6) (3.0) Note: "Early-Grade Students" were in grade 2 at baseline and grade 3 at endline. "Mid-Grade Students" were in grade 4 at baseline and grade 5 at endline. Standard errors appear in parentheses. Sample sizes: 632 Comparison; 120 School feeding only; 391 Literacy and school feeding. Sample sizes vary for certain outcomes due to missing data. Asterisks indicate statistical significance of difference-in-differences estimates. ** p<0.01; * p<0.05. Abt Associates June 2022 ▌129 OUTCOME LEVELS AT BASELINE AND ENDLINE BY TREATMENT ARM Exhibit A811. Outcome Levels at Baseline and Endline by Treatment Arm for Main Outcomes ........................................................ Baseline Endline Comparison School Feeding Only School Feeding and Literacy Comparison School Feeding Only School Feeding and Literacy Early-Grade Students Student reading performance (Average Reading Comprehension Scores) 4.9 2.0 2.2 10.8 12.2 13.1 Student reading performance (Percent Zero Comprehension Scores) 88.0 93.2 92.6 81.6 73.4 75.0 Student attendance 28.7 30.1 25.2 27.1 20.8 18.5 Student attentiveness 92.2 95.6 95.4 96.9 99.0 96.7 Mid-Grade Students Student reading performance (Average Reading Comprehension Scores) 26.7 23.1 14.2 41.6 35.1 33.3 Student reading performance (Percent Zero Comprehension Scores) 52.3 57.8 70.9 38.4 40.6 45.1 Student attendance 27.9 27.1 24.7 22.1 24.5 21.7 Student attentiveness 91.2 98.7 96.9 96.2 96.8 98.1 Note: Table reports weighted average outcome by treatment arm at baseline and endline. Difference-in-differences calculated by simple subtraction will differ from impact estimates because impacts are based on the impact analysis models described in Section 3.4. Exhibit A822. Outcome Levels at Baseline and Endline by Treatment Arm for Instructional Inputs ........................................................ Baseline Endline Comparison School Feeding Only School Feeding and Literacy Comparison School Feeding Only School Feeding and Literacy Early-Grade Students Teacher-reported receipt of training on the fundamentals of reading instruction 57.6 53.8 67.2 53.4 71.8 84.0 Teacher-reported absence of at least one day last week 22.6 27.5 25.0 23.8 13.6 24.6 Teacher has access to school supplies 86.5 97.8 96.4 98.0 99.0 97.4 Mid-Grade Students Teacher-reported receipt of training on the fundamentals of reading instruction 55.2 61.8 58.7 51.4 60.0 78.4 Teacher-reported absence of at least one day last week 17.4 27.6 23.2 16.5 26.3 24.4 Teacher has access to school supplies 89.0 92.0 94.6 97.3 97.9 95.2 Note: Table reports weighted average outcome by treatment arm at baseline and endline. Difference-in-differences calculated by simple subtraction will differ from impact estimates because impacts are based on the impact analysis models described in Section 3.4. Abt Associates June 2022 ▌130 Exhibit A833. Outcome Levels at Baseline and Endline by Treatment Arm for Feeding Outcomes and Nutrition Knowledge ........................................................ Baseline Endline Comparison School Feeding Only School Feeding and Literacy Comparison School Feeding Only School Feeding and Literacy Early-Grade Students Student reports feeling hungry now 55.0 39.7 51.4 47.8 38.1 37.9 Student achieved minimum meal frequency 42.7 55.3 57.6 49.9 41.9 48.0 Student achieved minimum dietary diversity 67.8 53.5 71.8 38.2 34.6 43.2 Student achieved minimum acceptable diet 34.1 30.5 46.0 26.5 20.1 23.8 Student's nutrition knowledge score 0.7 0.4 0.4 0.3 0.6 0.8 Mid-Grade Students Student reports feeling hungry now 46.0 30.7 39.8 43.5 41.8 37.8 Student achieved minimum meal frequency 56.9 57.4 62.4 62.0 54.7 59.0 Student achieved minimum dietary diversity 71.8 56.0 76.3 50.0 29.9 40.7 Student achieved minimum acceptable diet 45.2 36.3 49.6 39.1 21.2 26.5 Student's nutrition knowledge score 1.3 1.0 0.7 0.8 1.1 1.0 Note: Table reports weighted average outcome by treatment arm at baseline and endline. Difference-in-differences calculated by simple subtraction will differ from impact estimates because impacts are based on the impact analysis models described in Section 3.4. Exhibit A844. Outcome Levels at Baseline and Endline by Treatment Arm for Student Knowledge and Use of Recommended Health Practices ........................................................ Baseline Endline Comparison School Feeding Only School Feeding and Literacy Comparison School Feeding Only School Feeding and Literacy Early-Grade Students Student's hygiene and sanitation knowledge score 2.9 2.6 2.5 3.6 3.5 4.2 Student reports absence last week as a result of illness 59.3 66.4 55.8 55.7 51.7 70.2 Mid-Grade Students Student's hygiene and sanitation knowledge score 3.5 3.5 3.1 4.6 4.3 4.5 Student reports absence last week as a result of illness 58.3 57.3 57.5 64.5 55.7 67.3 All Students Head teacher reports school has improved latrines 39.5 82.1 86.0 51.9 90.9 95.4 Head teacher reports school has tippy taps or hand-washing stations 12.6 33.3 38.8 36.9 55.7 54.2 Abt Associates June 2022 ▌131 Head teacher reports all tippy taps or hand-washing stations have ash 1.1 8.3 3.5 24.0 29.5 21.4 or soap Note: Table reports weighted average outcome by treatment arm at baseline and endline. Difference-in-differences calculated by simple subtraction will differ from impact estimates because impacts are based on the impact analysis models described in Section 3.4. APPENDIX B: DATA COLLECTION INSTRUMENTS Data Collection Instruments 1. Student Survey (English) 2. Head Teacher Questionnaire (English) 3. Teacher Questionnaire (English) 4. Cook/Warehouse Manager Questionnaire (English) 5. Ministry of Education Interview (English) 6. Parent Focus Groups & Discussion Guide (English) Abt Associates June 2022 ▌132 1. Student Questionnaire (English) 1. Is the student a girl? 2. How old are you? 3. What class are you in right now? a. 3rd Grade b. 5th Grade 4. What class were you in last year? a. 1st Grade b. 2nd Grade c. 3rd Grade d. 4th Grade e. 5th Grade f. Don’t Know/Refuse to Respond 5. Did you go to pre-primary? 6. I would like to see what schoolbooks you have with you today. Please show me your: a. School Books b. Supplementary reader c. textbook d. exercise book 7. During the teaching of reading, what language does your teacher use to teach you and your classmates? a. Português b. Ronga c. Changana d. Macua e. Other 8. What does the teacher normally do when you are unable to answer a question or you answer a question incorrectly? a. Teacher rephrases/explains the question b. Teacher provides an example c. Teacher encourages the student to try again d. Teacher asks another student e. Teacher asks again f. Teacher corrects the student Abt Associates June 2022 ▌133 g. Teacher assigns a student helper h. Other i. Don’t Know/Refuse to Respond 9. Do you have someone at home who helps you with TPC? a. No b. Yes c. I don’t have TPC d. Don’t Know/Refuse to Respond 10. Were you absent from school any day last week? Why were you absent? a. No, was not absent from school last week b. Yes, illness c. Yes, because there was other work at home d. Yes, because I had to take care of a family member e. Yes, no transportation f. Yes, because of bad weather g. Yes, because of an emergency h. Yes, because getting to school was too dangerous i. Yes, because I woke up late j. Yes, because I could not find my uniform, or because my uniform was not ready on time in the morning k. Yes, because I am treated poorly by teachers at school l. Yes, because I am treated poorly by other students at school m. Other n. Don’t Know/Refuse to Respond 11. Do you read books in your classroom or in your school library every day? 12. Do you attend an afterschool learning club? 13. What type (s) of afterschool learning club do you attend? a. Afterschool Reading/Writing Club b. Help with Homework Club c. Math Learning Club d. Science Learning Club e. School Garden Learning Club f. Other 14. Do you bring home reading books from your classroom? Abt Associates June 2022 ▌134 15. If you bring home reading books from your classroom, what language are they in? a. Português b. Ronga c. Changana d. Macua e. Other 16. Do you bring home books from the school library/book bank? 17. If you bring home reading books from the school library/book bank, what language are they in? a. Português b. Ronga c. Changana d. Macua e. Other 18. What language do you normally speak with your family at home? a. Português b. Ronga c. Changana d. Macua e. Other 19. In addition to school books, do you also read newspapers or other materials at home? 20. How often do you read on your own at home? Never, sometimes, or every day? a. Never b. Sometimes c. Every day d. Don’t Know/Refuse to Respond 21. How many times do you read aloud for everyone to hear when you're at home? Never, sometimes or every day? a. Never b. Sometimes c. Every day d. Don’t Know/Refuse to Respond 22. How many times do you have someone read to you at home? Never, sometimes or every day? a. Never b. Sometimes Abt Associates June 2022 ▌135 c. Every day d. Don’t Know/Refuse to Respond 23. When you're at home, how often do you tell stories to others? Never, sometimes or 24. every day? a. Never b. Sometimes c. Every day d. Don’t Know/Refuse to Respond 25. When you're at home, how often do they tell you stories? Never, sometimes or every day? a. Never b. Sometimes c. Every day d. Don’t Know/Refuse to Respond 26. Do you have drawing or writing material at home? 27. Do you usually draw or write when you're at home? 28. How often do you draw or write? Never, sometimes or every day? a. Never b. Sometimes c. Every day d. Don’t Know/Refuse to Respond 29. Do you tend to be very attentive in classes? 30. What are you doing in the classroom to show that you are always paying attention? a. Sit facing the teacher and looking them in the eye when they’re speaking. b. Listen attentively and follow the teacher’s explanation c. Work alone, in pairs, or in small groups during learning activities. d. Answer all the questions that the teacher asks e. I ask the teacher or aide when I don’t understand what was explained. f. I ask questions about the explanation given g. Read h. Write i. Listen to the story being told. j. Other 31. Which phrase best matches your level of attention in the classroom? a. My attention is greater before school lunch Abt Associates June 2022 ▌136 b. My attention is much higher after school lunch c. My level of attention before and after school lunch is almost always the same d. Don’t Know/Refuse to Respond 32. Is there a certificate or award in school to celebrate progress in reading? 33. Have you ever received a distinction or diploma to celebrate your success in reading? 34. Did you eat any food before you arrived at school today? a. No b. Yes, at home c. Yes, on the way to school d. Other, specify e. Don’t Know/Refuse to Respond 35. How hungry are you right now a. Very hungry b. Somewhat hungry c. Just right d. Somewhat full e. Very full 36. Does your school serve school meals? 37. Was a school meal served yesterday or the last day you were at school? 38. Did you eat the school meal yesterday or the last day you were at school?? a. No b. Yes, breakfast c. Yes, lunch d. Don’t Know/Refuse to Respond 39. What type of food was served during the school meal yesterday or the last day you were at school? a. Porridge only b. Porridge plus other foods, specify types of other foods c. Other foods only, specify types of other foods d. Don’t Know/Refuse to Respond 40. On the days when you eat a meal at school, do you usually get: a. Same number of meals at home b. Fewer number of meals at home 41. On the days when you eat a meal at school, do you feel: Abt Associates June 2022 ▌137 a. More hungry than usual b. Less hungry than usual c. Same amount hungry as usual 42. Please describe everything you ate yesterday during the day or night, whether at home or outside the home. 43. Starting from the morning, what was the first thing you ate after waking up? Please tell me everything you ate at that time. 44. PROBE IF MIXED DISH: What ingredients were in the dish? Anything else? 45. Yesterday during the day or at night, did you eat or drink: a. Any foods made from grains, like rice, maize, spaghetti, sorghum, bread, porridge, or nsima? b. Any vegetables or roots that are orange-colored inside, like pumpkin, carrots, or sweet potatoes that are orange inside? c. Any white roots and tubers or plantains, such as white potatoes (batatas reno), manioc/cassava, plantains, or other roots that are white inside? d. Any dark green leafy vegetables, such as chinese cabbage, cassava leaves, mustard greens, rape, or dark green lettuce e. Any fruits that are dark yellow or orange inside, like ripe mango or ripe papaya? f. Any other fruits like banana, maca, lemon, orange, tangerine, guava, grapes, or apple? g. Any other vegetables like tomato, onion, cauliflower, eggplant, mushroom, cucumber, or cabbage? h. Any meat made from animal organs, such as gizzards, liver, kidney, heart or other organ meats i. Any other types of meat or poultry, like beef, pork, lamb, goat, chicken, or duck? j. Any eggs from chicken, duck, or other birds? k. Any fish or seafood, whether fresh or dried fish, prawns, shrimp, crab, clams, or other seafood? l. Any beans or peas, such as beans, lentils, or peas m. Any nuts or seeds, like groundnuts, almonds, pumpkin seeds, or sesame seeds n. Any milk or milk products, such as fresh milk, powdered milk, yogurt, or cheese o. Insects, such as termites, snails, or spiders? p. Oils and fats, such as vegetable oil, butter, or margarine q. Fortified foods, such as corn-soy porridge eaten at school? 46. Yesterday during the day and at night, how many times did you eat, including meals and snacks? Abt Associates June 2022 ▌138 47. How many food groups do you know in Mozambique? 48. What are the names of the Mozambican food groups? a. Staple Foods (base foods) b. Foods to grow (protein rich foods) c. Foods to protect (protein-rich foods) d. Energy-rich foods (high in calories) 49. Name one type of food that has a lot of protein (or is a food to grow) a. Fruit b. Vegetables c. Maize d. Bread e. Meat f. Fish g. Eggs h. Other (specify)_______________________ i. Don’t Know 50. What are nutrients? (Mark the first response given by the student) a. The organs of animals b. The four food groups c. The part of food that provides nutrition for the body d. Others (specify) e. Don’t Know 51. What is a balanced diet? (Mark the first response given by the student) a. Eating from the four food groups b. Eating a variety of different types of foods c. Others (specify) d. Don’t Know 52. Why is it important to eat a balanced diet? (Mark the first response given by the student) a. To make meals more interesting b. To stay healthy and grow well c. To have enough energy to study or play d. To get all the nutrientes your body needs e. Other (specify) f. Don’t Know Abt Associates June 2022 ▌139 53. What should you wash your hands with? (Mark the first response given by the student) a. Water b. Water and soap or ash c. Other (specify) d. Don’t Know 54. How long should you take to wash your hands? (Mark the first response given by the student) a. At least 20 seconds b. At least 2 minutes c. At least 5 minutes d. Other (specify) e. Don’t Know 55. When should you wash your hands? a. Before preparing meals b. Before eating c. After using the toilet 56. How often do you wash your hands with soap or ash before eating your school meal? a. Never b. Sometimes c. Always d. Don’t Know 57. Why is it important to wash your hands? (Mark the first response given by the student) a. To prevent germs from spreading b. To prevent diarrhea c. To stay healthy d. Because my parents or teachers told me to e. Other (specify) f. Don’t know 58. What can you do to prevent diarrhea? (Mark the first response given by the student) a. Wash your hands before preparing food, before eating, and after using the toilet b. Drink clean water from a protected well c. Use a latrine d. Make sure food is well cooked (hot) before you eat it e. Cover food to keep flies off it. f. Other (specify) Abt Associates June 2022 ▌140 g. Don’t Know 59. What should you do to prevent malaria? a. Sleep in the mosquito net b. Avoid mosquito bites(use long pants and long sleeves at night and sunset) c. Other(specify) d. Don’t Know 60. Since the beginning of the school year, were there times when you and your family did not have enough food to eat? 61. How often did this happen? a. Every Month b. During Rainy season c. Other (specify) 62. Does your house have electricity? 63. Where do you normally get your water from at home? a. River, stream or lake b. Well or borehole c. Communal tap d. Water pipe / tap in your home e. Water truck or tank f. Other g. Don’t Know/Refuse to Respond 64. What do you cook food with in your house? a. Firewood b. Coal stove c. Oil stove d. Gas stove e. Electric stove f. Other g. Don’t Know/Refuse to Respond 65. What type of toilet does your family use at your home? a. No toilet b. Pit toilet (including shared and communal) c. Flush/eastern toilet outside your home d. Flush/eastern toilet inside your home Abt Associates June 2022 ▌141 e. Other f. Don’t Know/Refuse to Respond 66. Where does your family get water that you use at home? a. Stream, river, pond, lake b. Unprotected well c. Protected (borehole) well) d. Piped water to yard e. Piped water into home 67. Does your family have the following items in your home? 68. Point to appropriate pictograms. a. Radio b. Mobile phone c. Television d. Computer e. Refrigerator f. Bicycle g. Motorbike h. Car/truck 69. Do you sleep in the mosquito net? 70. What language do you usually speak at home? a. Português b. Rhonga c. Changana d. Macua e. Other Abt Associates June 2022 ▌142 2. Head Teacher Questionnaire (English) 1. What is your position at the school? a. Head Teacher? b. Assistant Head Teacher? c. Other? d. Don’t know / Refuse. 2. In what year did you start your position at this school? 3. Is the Head Teacher / Assistant Head Teacher female? 4. What date did classes start this year? 5. Between 2017 and 2019, did you attend any WV trainings? 6. Between 2017 and 2020, did you attend any PAI trainings? 7. Between 2017 and 2020, did you attend any teacher trainings? 8. Between 2017 and 2019, did you modify any aspects of the WV meals or literacy programs? 9. Between 2017 and 2019, did you modify any aspects of the PAI meals or literacy programs? 10. Do you believe these adaptations were successful at meeting the needs of your teachers and students? 11. How many teachers are currently employed at this school? a. Number of teachers? b. Male? c. Female? d. Don’t know / Refuse 12. Do you have a school literacy program? 13. If yes, in which language? a. Portuguese? b. Rhonga? c. Changana? d. Macua? e. Other (specify)? 14. Besides the ministry of education, what other organization supports the school’s literacy program? a. PAI/ADPP? b. World Vision? c. USAID? d. Other (specify)? Abt Associates June 2022 ▌143 e. Don’t know/Refuse 15. Did teachers at the school attend the literacy training this year? 16. Have they attended the WV training? 17. Have they attended the PAI training? 18. Of these teachers, how many are currently teaching Standard 2? a. Total number of teachers? b. Male? c. Female? d. Don’t know / Refuse 19. Of these teachers, how many are currently teaching Standard 3? a. Total number of teachers? b. Male? c. Female? d. Don’t know / Refuse 20. Of these teachers, how many are currently teaching Standard 4? a. Total number of teachers? b. Male? c. Female? d. Don’t know / Refuse 21. Of these teachers, how many are currently teaching Standard 5? a. Total number of teachers? b. Male? c. Female? d. Don’t know / Refuse 22. How many teacher training sessions are expected to be held this academic year? 23. How many teacher training sessions have been held this year up until today? 24. How many streams each are there for Standard 3 and 5? a. Number of Standard 3 streams at the school? b. Number of Standard 5 streams at the school? 25. How many teachers are currently trainees expected to graduate from ADPP-supported Teacher Training Colleges? a. Total number of teachers? b. Male? c. Female? Abt Associates June 2022 ▌144 d. Don’t know / Refuse 26. How many teachers have already graduated from ADPP-supported Teacher Training Colleges? a. Total number of teachers? b. Male? c. Female? d. Don’t know / Refuse 27. Could I please see your teacher attendance register? a. How many teachers were absent yesterday (or on the last day school was in session) across the school? b. Record unavailable? c. Don’t know / Refuse 28. What do you do with a class whose teacher is absent? a. Let the class proceed without a teacher? b. Allocate that class to another teacher? c. Join all students in one class? d. Bring in a teacher from outside? e. Dismiss students for the day? f. Send students to the playground? g. Distribute students among other classrooms? h. Other? i. Don’t know / Refuse 29. Is it easy to find and retain good teachers at this school? 30. Between 2017 and 2019, did reading coaches train teachers at your school? 31. Between 2017 and 2019, how many trainings did reading coaches provide? 32. Between 2017 and 2019, did reading coaches supervise teachers at the school? 33. Between 2017 and 2019, how often did reading coaches supervise teachers at the school? a. Once in the year? b. Once per term? c. Once a month? d. More than once a month? e. Don’t know / Refuse 34. Have you received training on the fundamentals of reading instruction? 35. Can you name the core reading skills? 36. Please name as many as you can. Abt Associates June 2022 ▌145 a. Letter knowledge. b. Phonemic awareness. c. Fluency. d. Vocabulary. e. Comprehension. 37. Have you received training on school management? 38. How do you know whether students are progressing academically? a. Classroom observation b. Monitor pupil results on tests given by teachers c. Evaluate students orally myself d. Check students’ assignments or homework e. Teachers provide progress reports f. End-of-term evaluations g. Feedback from parents h. Feedback from school counselors i. Feedback from school committees j. Other k. Don’t know/Refuse 39. Are there special events held at the school to celebrate students’ progress in reading? 40. Are there special events held at the school in recognition of effective reading instruction? 41. Are you generally satisfied with parents’ involvement in the school? a. Strongly agree b. Agree c. Disagree d. Strongly Disagree 42. What ways have parents evidenced support for this school? a. None b. Visiting the school c. Actively seeking information about school matters d. Following up on academic progress e. Volunteering time f. Volunteering in school feeding program g. Volunteering resources h. Participating in school committees/meetings Abt Associates June 2022 ▌146 i. Other j. Don’t know/Refuse 43. How frequently are these types of supports received from parents? a. Once in the year b. Once a quarter c. Once a month d. More than once a month e. Don’t know/Refuse 44. Between 2017 and 2019, did you receive training to learn about the importance of access and use of school supplies and materials? 45. Do teachers have access to a Teacher's Handbook for teaching reading? 46. Do teachers have access to school supplies? a. No b. Yes c. Don’t know/Refuse d. Pens/pencils e. Notebooks f. Chalk g. Cardboard h. Paint i. Markers j. Chart paper k. Other (Specify) __________________ 47. Do students receive materials that are used in teaching and learning reading? a. No b. Yes c. Don’t know/Refuse d. Student exercise book e. Student notebook f. Supplementary reading books / decodeable texts g. Notebook h. Pens / Pencils i. Other (Specify) __________________ 48. Does the school have a library/book bank? Abt Associates June 2022 ▌147 a. No b. Yes c. Don’t know/Refuse d. Flipbooks e. Storybooks f. Primers g. Fables h. Leveled books i. Supplementary readers/decodable texts j. Guidelines for parents k. Reading posters l. Other 49. Is the print material in the library/book bank available in Portuguese? 50. Is the print material in the library/book bank available in the mother tongue? 51. This year, how many books or other printed materials have students ordered from the library/book bank? a. Printed material consulted by 3rd grade: b. Printed material consulted by 5th grade: 52. Are there other types of afterschool learning clubs? a. No b. Yes c. Don’t know/Refuse d. Help with Homework Club e. Math Learning Club f. Science Learning Club g. School Garden Learning Club h. Other 53. Were you absent from school any day last week? 54. Were you late to school any day last week? 55. How many teachers are currently teaching Standard 2? 56. How many teachers are currently teaching Standard 4? 57. Between 2017 and 2019, How many teachers at this school received training on teaching nutrition? 58. Does/do the teacher/teachers have materials for teaching nutrition? Abt Associates June 2022 ▌148 59. How often do they teach lessons on nutrition to each class? a. Never b. Once per week c. Once every two weeks d. Once per month e. Once per term 60. How many teachers at this school received training on teaching hygiene and sanitation? 61. Does/do the teacher/teachers have materials for teaching hygiene and sanitation? 62. How often do they teach lessons on hygiene and sanitation to each class? a. Once per week b. Once every two weeks c. Once per month d. Once per term e. Never 63. Does the school have a school feeding program? 64. What organization supports the school feeding program? a. PAI/ADPP b. World Vision c. Government of Mozambique d. Other (specify) e. Don’t know 65. Are students on a three days per week schedule, or other schedule less than five days per week? a. No b. Yes c. Depends on which month this academic year 66. Do students receive school meals only on days they attend school? 67. How often do students receive the school meals? a. Daily b. A few times per week c. Whenever food is available 68. Please describe any issues this school has faced related to the school feeding program. a. No issues b. Food is not always available c. Kitchen/warehouse not constructed or in disrepair Abt Associates June 2022 ▌149 d. Not enough pots, utensils, or dishes e. Not enough volunteer cooks f. Meals not prepared on time g. Other (specify) ______________________ 69. Does the school have a garden? (Answered based on surveyor observations) 70. Does the garden operate year-round or during the rainy season only? a. Rainy season only b. Year round 71. How often is the garden used to supply food for school meals? a. No school feeding program b. Every day c. 3-4 times per week d. 1-2 times per week e. Never f. During the rainy season g. Only when school garden products are available 72. Are the students in the school provided with deworming medication (albendazole or mebendazole)? 73. If yes, have students in the school received deworming medication at least once during this school year? 74. Does the school have improved latrines? (Answered based on surveyor observations) 75. How many improved latrines does the school have? (Answered based on surveyor observations) 76. How clean are the latrines? (Answered based on surveyor observations) a. All latrines are dirty with feces on the slabs b. Some latrines are dirty c. All latrines are clean with no feces on the slabs 77. Are there tippy-taps or another type of hand washing station near the latrines? (Answered based on surveyor observations) 78. Do the tippy-taps or hand washing stations have ash or soap? (Answered based on surveyor observations) a. None have ash or soap b. Some have ash or soap c. All have ash or soap Abt Associates June 2022 ▌150 79. Due to COVID-19, in many schools students have missed a year or more of literacy. Are you implementing the curriculum from the prior standard to your students? (For example, are Standard 2 students studying the Standard 1 curriculum?) a. No b. Yes c. Depends on the grade d. Don’t know/Refuse 80. Did your school provide literacy learning during COVID-19? (For example, radio activities?) 81. While schools were shut down during COVID-19 [from March 2020 to March 2021], did students still receive deworming? 82. While schools were shut down during COVID-19 [from March 2020 to March 2021], did students receive take-home rations? 83. How frequently were take-home rations given to students between March 2020 and March 2021? a. One time, early in the shutdown b. Weekly c. Monthly d. Other (specify) ____________________ 84. Will students be able to catch up to their grade level by the end of the school year? How much of the year will be spent on catching up to this years’ curriculum? Please tell us your thoughts. 85. Comparing the school between 2017 to now, what has changed in terms of literacy activities, sanitation, and health and nutrition? 86. How much of these changes are because of help from World Vision? 87. Comparing the school between 2017 to now, what has changed in terms of literacy activities, sanitation, and health and nutrition? 88. How much of these changes are because of help from PAI? 89. Which of these changes do you think will stay over time? What do you need to maintain changes? 90. Our survey firm, Ernst and Young, may call some people back if there is anything we would like to clarify. Would you mind if we call you back if needed? If yes, what is your phone number? Abt Associates June 2022 ▌151 3. Teacher Questionnaire (English) 1. Identify the shift. a. All day b. Morning shift c. Afternoon shift 2. Was the teacher present in the classroom at the beginning of the shift? 3. Is the teacher female? 4. In what languages do you teach reading? a. Portuguese b. Rhonga c. Changana d. Macua 5. What is the primary (target) language you teach children to read and write in? a. Portuguese b. Rhonga c. Changana d. Macua 6. In what year did you start your position at this school? 7. What teacher training college did you attend? a. Did not attend a teacher training college b. Still attends a teacher training college c. Institute of Primary Teaching (IMAP) d. Primary Teaching Training Institute e. ADPP Teacher Training Schools for the Future f. Pedagogical University – UP g. Other (Specify) 8. Did you attain certification? 9. Are you currently a trainee expected to graduate from a Teacher Training College? 10. What teacher training college are you expected to graduate from? a. Institute of Primary Teaching (IMAP) b. Primary Teaching Training Institute c. ADPP Teacher Training Schools for the Future d. Pedagogical University – UP e. Other (Specify) Abt Associates June 2022 ▌152 11. Identify the grade in which you teach reading. a. Grade 2 b. Grade 4 12. What subjects do you teach in? a. Portuguese Reading b. Portuguese Writing c. Reading (Other language) d. Writing (Other language e. Health and Nutrition Education f. Games, Sport and Visual Arts g. Other (Specify) h. Don’t know/Refuse 13. What is your highest level of academic education? a. Secondary Education b. Middle School c. Bachelor d. Licensure e. Masters f. Other (Specify) 14. During your pre-service training, did you receive any specific training on how to teach reading and writing to students in the primary grades? 15. This year, have you attended any in-service training on how to teach reading in the primary grades? 16. What kinds of activities did you participate in during in-service training? a. Analysis of the curriculum and syllabus b. Analysis of the teacher guide c. Practice teaching lessons from the teacher guide d. Analysis of the student exercise book e. How to plan and prepare for a lesson f. Improvisation and use of teaching and learning aids g. How to assess students during the lesson h. How to use assessments to adjust instruction i. How to give students feedback j. How to use supplementary readers/decodable texts Abt Associates June 2022 ▌153 k. How to apply new strategies for reading instruction l. Don’t know/Refuse 17. Between 2017 and 2019, did you attend any teacher trainings? 18. Between 2017 and 2019, did you attend any WV trainings? 19. Between 2017 and 2020, did you attend any PAI trainings? 20. Have you received training on the fundamentals of reading instruction? 21. Can you name the core reading skills? 22. Please name as many core reading skills as you can. (Tick all that apply) a. Letter knowledge b. Phonemic awareness c. Fluency d. Vocabulary e. Comprehension 23. Is the class that you teach a multi-grade class? 24. Could I please see your student attendance register? a. Register was not available to be examined b. Attendance records were completed daily c. Attendance records were completed weekly d. Attendance records were completed biweekly e. Attendance records were completed monthly f. Other (Specify) g. Unclear how often the attendance records were completed 25. Record the date of the most recent attendance record entry. 26. How many boys are enrolled in this class? 27. How many girls are enrolled in this class? 28. How many boys in this class are repeaters? 29. How many girls in this class are repeaters? 30. On a typical day, how many students are absent? 31. On a typical day, how many students are late? We define “late” to be arriving at least 15 minutes after the start of the first class. 32. Does the Head Teacher or Assistant Head Teacher check your lesson plans? 33. How often during this school year have your lesson plans been checked? (Read the responses) a. Once every 2-3 months b. Once every month Abt Associates June 2022 ▌154 c. Once every two weeks d. Once every week e. Daily f. Don’t know/Refuse 34. When you need some help with your teaching, who do you consult? (Read the responses) a. Never need help b. There is no one to ask for help c. Discuss casually with other teachers d. Discuss at meetings with other teachers e. Reading Coach f. Assistant Head Teacher g. Head Teacher h. Don’t know/Refuse i. Other (Specify) 35. Since the beginning of the school year, how frequently does the Assistant Head Teacher or Head Teacher observe your teaching? (Read the responses) a. Never b. Quarterly c. Monthly d. Weekly e. Don’t know/Refuse 36. Does the Assistant Head Teacher or Head Teacher provide feedback on your teaching? 37. Since the beginning of the school year, how frequently does the reading coach visit you? (Read the responses) a. Never b. Quarterly c. Monthly d. Weekly e. Don’t know/Refuse 38. Do you have a Teacher Guide for reading instruction? 39. Do you use the Teacher Guide for reading instruction? 40. Do each of your students have exercise books that can be used during reading instruction? 41. Do your students use the exercise books during reading instruction? 42. Do you have access to school supplies? Abt Associates June 2022 ▌155 a. School supplies available: i. Pens/pencils ii. Notebooks iii. Chalk iv. Cardboard v. Paint vi. Markers vii. Chart paper viii. Other (Specify) 43. Do you use any school supplies in your classroom during reading instruction? a. School supplies available: i. Pens/pencils ii. Notebooks iii. Chalk iv. Cardboard v. Paint vi. Markers vii. Chart paper viii. Other (Specify) 44. How many books are registered for your class? 45. How frequently do you switch languages between the official language of instruction and a vernacular language during the teaching and learning process? (Read the responses) a. Never b. Occasionally c. Often d. All of the time e. Don’t know/Refuse 46. How would you rate the reading skills of your students in the target language of instruction? Weak, Average or Strong? 47. How would you rate the writing skills of your students in the target language of instruction? Weak, Average or Strong? 48. How do you measure your students’ reading and writing progress? (Do NOT READ the options) a. Written tests b. Oral evaluations Abt Associates June 2022 ▌156 c. Observation d. Portfolios and other projects e. Homework f. Classwork g. End of unit evaluation h. End-of-term evaluation i. Other (Specify) j. Don’t know/Refuse 49. How do you check for student understanding during the lesson? (Do NOT READ the options) a. Ask questions to individual students b. Ask questions to whole class c. Ask questions to students in groups d. Give students a task and correct the written responses e. before the end of the lesson f. Give students a task and correct the responses after g. the end of the lesson. h. Ask students to put thumbs up or down i. Other (Specify) j. Don’t know/Refuse 50. How do you use the results of students’ oral and written assessments in your teaching? (Do NOT READ the options) a. Grade students b. Evaluate students’ understanding of subject c. matter d. Plan teaching and learning activities e. Adapt teaching to better suit students’ needs f. Arrange students in ability groups g. Other (Specify) h. Don’t know/Refuse 51. What techniques do you use to teach a new vocabulary word? (Do NOT READ the options) a. Pictures b. Real objects c. Gestures d. Verbal explanation Abt Associates June 2022 ▌157 e. Don’t know/Refuse f. Other (Specify) 52. In your class, how many parents/guardians have reviewed the Students' TPC? None, some, most or all? (Read the options, mark ONLY one answer) a. None b. Some c. Majority d. All e. Don’t know/Refuse 53. Overall, are you satisfied with the involvement of parents and guardians in the school work of their students? 54. At what grade level do you expect children to read text in the target language fluently (accurately and to use punctuation marks correctly)? a. Grade 1 b. Grade 2 c. Grade 3 d. Grade 4 or higher e. Don’t know/Refuse 55. At what grade level do you expect children to write a coherent and comprehensible short story correctly? a. Grade 1 b. Grade 2 c. Grade 3 d. Grade 4 or higher e. Don’t know/Refuse 56. Are students attentive in class? 57. How many students are generally attentive in class? a. Less than half b. Half c. More than half d. All 58. How do you know if students are attentive in class? a. Students are seated facing the teacher and making eye contact with the teacher while the teacher is talking Abt Associates June 2022 ▌158 b. Students actively listen and act upon teacher directions c. Students are working independently, in pairs, or in small groups on the current learning activity d. Students respond to questions posed by the teacher e. Students ask the teacher or another student for help related to the current learning activity f. Students ask the teacher questions related to the current learning activity g. Students are reading h. Students are writing i. Students are listening to a story j. Other (Specify) 59. Are there special events held at this school to celebrate students’ progress in reading? 60. Have any of your students received a certificate or award in school to celebrate progress in reading? 61. Are there special events held at this school to celebrate students’ school attendance? 62. Have any of your students received a certificate or award in school to celebrate improvement in attendance? 63. Is there an afterschool literacy club in this school? 64. Do your students attend an afterschool literacy club at this school? 65. If yes, specify the name of the afterschool literacy club. 66. How many students in your class attend an afterschool literacy club at this school? a. None b. Half c. More than half d. All 67. Were you absent from school any day last week? 68. If so, why were you absent? a. No, I wasn't absent from school last week b. Yes, I was sick c. Yes, I went to collect my salary d. Yes, I did other work e. Yes, I did not receive/received too little, received f. irregular payment g. Yes, a lack of motivation h. Yes, I had family responsibilities Abt Associates June 2022 ▌159 i. Yes, I had no transportation j. Other (Specify) k. Don’t know/Refuse 69. Were you late from school any day last week? 70. If so, why were you late? a. No, I wasn't absent from school last week b. Yes, I was sick c. Yes, I went to collect my salary d. Yes, I did other work e. Yes, I did not receive/received too little, received irregular payment f. Yes, a lack of motivation g. Yes, I had family responsibilities h. Yes, I had no transportation i. Other (Specify) j. Don’t know/Refuse 71. Is there anything at this school that you identify as an issue, or is missing? a. If so, explain. b. Issues with the structure of the school building (walls, c. windows, roof, etc.) d. Issues with the surrounding area e. Lack of clean drinking water f. Issues with latrines for boys g. Issues with latrines for girls h. Issues with latrines for teachers i. Poor relationships with parents/community j. Bullying at school k. Other (Specify) l. Don’t know/Refuse 72. Did you receive training on teaching nutrition? 73. Do you have lesson plans or other materials that you can use to teach lessons on nutrition? 74. May I see the lesson plans or materials? (Do NOT READ the options) a. Has all the nutrition lesson plans provided by the b. implementing organization c. Has some of the nutrition lesson plans Abt Associates June 2022 ▌160 d. Has no nutrition lesson plans e. Has other materials, like picture cards or posters f. Has no other materials 75. How often do you do lessons on nutrition? a. Daily b. A few times per week c. Once per week d. A few times per month e. Once per month f. Less than once per month 76. What classes do you teach nutrition classes in? a. Grade 1 b. Grade 2 c. Grade 3 d. Grade 4 e. Grade 5 77. How would you rate your students’ knowledge of nutrition? a. Weak b. Average c. Strong d. Don’t know/Refuse 78. Have you received training on teaching about hygiene and sanitation? 79. Have you received training on hygiene and sanitation practices? 80. Do you have lesson plans or other materials that you can use to teach lessons on hygiene and sanitation? 81. May I see the lesson plans or materials? (Do NOT READ the options) a. Has all the nutrition lesson plans provided by the b. implementing organization c. Has some of the nutrition lesson plans d. Has no nutrition lesson plans e. Has other materials, like picture cards or posters f. Has no other materials 82. How often do you do lessons on hygiene and sanitation? a. Daily Abt Associates June 2022 ▌161 b. A few times per week c. Once per week d. A few times per month e. Once per month f. Less than once per month 83. What classes do you teach hygiene and sanitation classes in? a. Grade 1 b. Grade 2 c. Grade 3 d. Grade 4 e. Grade 5 84. How would you rate your students’ knowledge of hygiene and sanitation? a. Weak b. Average c. Strong d. Don’t know/Refuse 85. Does the school serve meals? 86. Which statement most accurately reflects the level of student attentiveness in your class? a. The students show the same level of attentiveness after the school meal b. The students are less attentive after school meal c. The students show the same level of attention before and after school meals d. Don’t know/Refuse 87. Did you eat a school meal yesterday or on the last day that you were at school? 88. Do you receive corn-soy flour from the school meal program to take home with you? 89. How many kilograms of corn-soy blend do you receive per month? 90. How do you think the school feeding program impacts your students’ learning? a. School meals distract students b. School meals make students more attentive c. School meals don’t affect student learning d. Other (Specify) 91. Did you provide literacy instruction to your students during COVID-19? (For example, radio activities?) 92. Will students be able to catch up to their grade level by the end of the school year? Abt Associates June 2022 ▌162 93. How much of the year will be spent on catching up to this years’ curriculum? Please tell us your thoughts. 94. Our survey firm, Ernst and Young, may call some people back if there is anything we would like to clarify. Would you mind if we call you back if needed? If yes, what is your phone number? Abt Associates June 2022 ▌163 4. Cook/Warehouse Manager Questionnaire (English) 1. Is there a log book for this school year showing how many children were present and how many meals were served? 2. May I see the log book? a. Log book is not filled out or up to date b. Log book is filled out and up to date 3. Check the attendance record for the previous school day (with the head teacher) and cross check the number of school meals prepared that day. a. Number present and number of meals cooked are the same b. Number present is lower than number of meals cooked c. Number present is higher than number of meals cooked 4. Does the school have a kitchen and storeroom for the school feeding program? (Answered based on surveyor observations) 5. What is the condition of the kitchen/storeroom for school feeding? (Answered based on surveyor observations) a. Not adequate (e.g., holes in roof or walls, no door, no lock on storeroom, etc.) b. Moderate (complete building, but could use some repairs) c. Very good (complete building, no repairs needed) 6. Do you ever run out or have no supply of corn-soy blend? a. For part of each month b. Once every 2-3 months c. Once in the school year 7. Are cooking pots and cooking utensils (such as stirrers or scoops) available? (Answered based on surveyor observations) 8. Is the number of pots and cooking utensils sufficient? 9. What do you do if you are lacking materials (corn-soy blend, pots, utensils) for preparing the school meals? a. Nothing b. Inform the head teacher c. Inform the project d. Other (specify) 10. At what time or times of day do you serve the school meal? a. Time 1: ____________________ b. Time 2: ____________________ Abt Associates June 2022 ▌164 11. At what time do students in grade 2 eat? a. Time:________________ 12. At what time do students in grade 4 eat? a. Time:________________ 13. How often would you say that the school meals are served on time? a. Never b. Sometimes c. Always 14. How many grams of corn-soy blend should a child receive during the school lunch? 15. How many cooks are present today? (Answered based on surveyor observations) 16. How many of these cooks are parents with at least one child attending the school? 17. Do you supplement the corn-soy blend with other foods? 18. How often is the meal supplemented with other foods? a. Every day b. 3-4 times per week c. 1-2 times per week d. Never e. During the rainy season f. Only when school garden products are g. available 19. What types of foods do you supplement the corn-soy blend with? a. Fresh maize b. Vegetables c. Fruits d. Other/specify ______________ 20. Does the school have bowls and spoons for students? (Answered based on surveyor observations) 21. Are there enough bowls and spoons for students for each meal session? 22. Are the bowls and spoons cleaned after each meal? 23. Is there soap available to clean the bowls, spoons, pots, and cooking utensils? (Answered based on surveyor observations) 24. Is there a supply of clean water (from a borehole or 25. rainwater tank) available at the school? (Answered based on surveyor observations) 26. Is there a place for children to wash their hands before 27. eating? (Answered based on surveyor observations) Abt Associates June 2022 ▌165 28. Is there soap available for children to wash their hands 29. before eating? (Answered based on surveyor observations) 30. How many days per week do you cook at the school? 31. In what year did you start as a cook? 32. Did you receive any training for being a school cook? 33. When was the last time you received training related to being a school cook? a. Less than 6 months ago b. 6-11 months ago c. 12 or more months ago 34. How can you get rid of bacteria and other germs in food? a. Wash it with clean water b. Cook it c. Keep it in a safe place away from animals 35. What do you do to make sure the food you serve to the children at school is safe? a. Cook it for long enough b. Wash hands with soap before preparing the food c. Wash bowls and spoons with clean water and soap d. Wash cooking pots and utensils with clean water and soap e. Have children wash hands before eating 36. Is there a place for cooks to wash their hands? (Answered based on surveyor observations) 37. Is there soap for cooks to wash their hands? (Answered based on surveyor observations) 38. What should you wash your hands with? a. Water b. Water and soap or ash c. Without water 39. How long should you take to wash your hands? a. Less than 2 minutes b. 2 minutes c. More than 2 minutes 40. When should you wash your hands? a. Before preparing food b. Before eating food c. After using the toilet 41. Do you usually eat the school meal when you are at the school cooking? Abt Associates June 2022 ▌166 42. What do you do with leftover corn-soy blend at the end of the meal? a. Give more to students b. Give it to siblings/other children at c. school who are not students d. Give it to teachers e. Cooks eat it f. Cooks take it home g. Other/specify ______________ h. Don’t know 43. What do you receive in return for cooking the food? a. Nothing b. Payment c. Corn soy blend d. Don’t know/No answer e. Other/specify ______________ 44. While schools were shut down during COVID-19 [from March 2020 to March 2021], did anyone take care of the school gardens? 45. What changes have you seen in terms of health and nutrition since you started working as a cook, if any? [probe: is the cooking space larger than before? More cookware? More knowledge of nutrition? Changes due to COVID-19?] 46. What changes have you seen in terms of health and nutrition since you started working as a cook, if any? [probe: is the cooking space larger than before? More cookware? More knowledge of nutrition? Changes due to COVID-19?] Abt Associates June 2022 ▌167 5. Ministry of Education Interview (English) 1. What is your involvement in the program? 2. What are your opinions about the literacy program? a. Probe: Positive aspects, innovations on what was new b. Challenges i. Working with parents? ii. Working with teachers? iii. Working with donors? 3. How did external factors such as COVID-19 influence program implementation? 4. What goals do you think were or were not achieved? 5. Are there any aspects of the program that you intend to continue? Why or why not? a. Probe: is there any budget set aside for this? What resources and personnel are necessary? Why are they or are they not being provided? 6. What are your opinions about the school feeding program? a. Probe: Positive aspects b. Challenges i. Working with parents? ii. Working with teachers? iii. Working with donors? 7. What are your opinions about the school garden program? a. Probe: Positive aspects b. Challenges i. Working with parents? ii. Working with teachers? iii. Working with donors? 8. How did external factors such as COVID-19 influence program implementation? 9. What goals do you think were or were not achieved? 10. How did COVID-19 influence goals? a. Probe: are most students now behind a grade level? Learning at a slower rate due to coming to school fewer days a week? 11. Are there any aspects of the program that you intend to continue? Why or why not? a. Probe: is there any budget set aside for this? Abt Associates June 2022 ▌168 6. Parent Focus Groups & Discussion Guide 1. How many children do each of you have at this school? 2. What is your involvement in the school as a parent? 3. In what other ways are parents involved in the school? 4. Are you generally satisfied with the level of parental involvement at the school? Why? Why not? 5. What do you expect your children to gain from attending this school? 6. How well is the school meeting your expectations? 7. What are your opinions about the literacy program at this school, if there is one? a. Probe: Positive aspects b. Challenges 8. What language do you speak at home? 9. What types of reading or storytelling activities do you do at home? 10. In what language do you carry out reading or storytelling activities at home? 11. Do you review your child’s homework? 12. Does your child read at home? 13. How often does your child read at home? 14. In what language does your child read at home? 15. Does your child have materials to draw or write at home? 16. Does your child draw or write at home? 17. How often does your child draw or write at home? 18. What recommendations do you have for improving the literacy program at this school? (skip if there is no literacy program) 19. How often do your children miss school due to illness? a. Probe: Types of illnesses, duration of missing school 20. Are your children less likely to experience health-related absences when receiving meals at school? 21. What types of health practices do your children learn about at school? 22. How do they apply their knowledge of improved health practices at home? 23. What are your opinions about the school feeding program at this school, if there is one? a. Probe: Positive aspects, Challenges (skip below questions if no program) 24. What are the benefits of your child or children receiving a meal at school? 25. What are your thoughts about the type of food that is provided for the meal? 26. How does the school meal affect the amount of food or number of times per day you give your school-going child or children at home? Abt Associates June 2022 ▌169 27. How does the school meal affect the types of food you give your school-going child or children at home? 28. What recommendations do you have for improving the school feeding program? 29. What are your opinions about the school garden, if there is one? a. Probe: Positive aspects b. Challenges 30. What recommendations do you have for improving the school garden? 31. Tell me about what happened during COVID-19? a. What kind of education did your children get during school closures due to COVID-19, if any? i. Literacy? ii. Health? b. Did your children still receive deworming? c. Did you children receive any take-home rations? d. If so, what kind of rations did they receive? e. How frequently were take-home rations given to your children between March 2020 and March 2021? 32. How is COVID-19 affecting school now? a. Shifted schedules to 3 days a week? b. Literacy a grade level behind? 33. Do you think any of the knowledge from the trainings provided by World Vision (if in Maputo) or PAI (if in Nampula) will continue to impact the school in the future? On which topics? (health, literacy)? 34. What would it take to sustain the knowledge gained? (More trainings? More materials?) Abt Associates June 2022 ▌170