February 18, 2018 McGovern Dole Mozambique Impact Evaluation and Literacy Assessment Baseline Report Prepared for U.S. Department of Agriculture DOI, Interior Business Center, AQD Division 2/Branch 2 381 Elden St Suite 4000 Herndon, VA 20170 Prepared by RTI International 3040 E. Cornwallis Road Research Triangle Park, NC 27709 GSA PSS Schedule No. GS00F354CA Task Order No. D16PD00635 _________________________________ RTI International is a registered trademark and a trade name of Research Triangle Institute. MCGOVERN-DOLE MOZAMBIQUE IMPACT EVALUATION AND LITERACY ASSESSMENT: BASELINE REPORT RTI International GSA PSS Schedule No. GS00F354CA Task Order No. D16PD00635 February 2018 The views, opinions, and/or findings contained in this report are those of the author(s) and should not be construed as an official Government position, policy, or decision, unless so designated by other documentation. iv USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report CONTENTS Section Page List of Acronyms x 1. Executive Summary 1-1 2. Background & Methodology 2-4 2.1 Background ......................................................................................... 2-4 2.2 Methodology ........................................................................................ 2-5 2.2.1 Theory of Change ......................................................................... 2-5 2.2.2 Impact Evaluation ........................................................................ 2-5 2.2.3 Research Questions ...................................................................... 2-7 2.2.4 Indicators ................................................................................... 2-7 2.2.5 Instruments ................................................................................ 2-9 2.2.6 Programming ............................................................................. 2-10 2.2.7 Field Work ................................................................................. 2-10 2.2.8 Sampling .................................................................................. 2-11 2.2.9 Data Management ...................................................................... 2-16 2.3 Analysis Methodology ......................................................................... 2-17 2.3.1 Basic Quantitative Analysis .......................................................... 2-17 2.3.2 Advanced Quantitative Analysis ................................................... 2-17 2.3.3 Qualitative Analysis .................................................................... 2-17 3. Analysis 3-1 3.1 Baseline Indicators ............................................................................... 3-1 3.1.1 Literacy ...................................................................................... 3-1 3.1.2 Student Attendance ...................................................................... 3-7 3.1.3 Secondary Indicators .................................................................... 3-8 3.2 Baseline Descriptive Statistics .............................................................. 3-10 3.2.1 Students ................................................................................... 3-10 3.2.2 Teachers ................................................................................... 3-20 3.2.3 Head Teachers ........................................................................... 3-33 3.2.4 Warehouse Managers and Cooks .................................................. 3-44 3.2.5 Parents ..................................................................................... 3-47 4. Summary 4-1 5. Limitations & Recommendations for Impact Evaluation 5-6 Limitations .......................................................................................... 5-6 Recommendations ................................................................................ 5-6 6. References 6-1 7. Annex A: Within-Province Literacy Analysis 1 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report v 8. Annex B. Precision Estimates 3 9. Annex C. Impact Evaluation recommendations (table) 5 vi USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report LIST OF TABLES AND FIGURES Figure 1. McGovern-Dole Program Coverage in Mozambique 2-4 Figure 2. Theory of Change 2-5 Table 1. Research Questions 2-7 Table 2. Literacy, Nutrition and Health, and School Management Indicators 2-8 Table 3. Instruments for Data Collection at Each School 2-9 Table 4. Description of Baseline EGRA Subtasks 2-9 Figure 3. Intervention and Comparison Districts 2-12 Table 5. Target Population across Intervention and Comparison Districts 2-12 Table 6. Summary of the Sampling Methodology and Target Sample Size 2-13 Table 7. Final Sample Counts 2-15 Table 8. Listening Comprehension Subtask Scores 3-2 Table 9. Letter Identification Subtask Scores 3-3 Table 10. Syllable Reading Subtask Scores 3-4 Table 11. Oral Reading Fluency Subtask Scores 3-5 Table 12. Reading Comprehension Subtask Scores 3-6 Table 13. Word Dictation Subtask Scores 3-6 Figure 4. Percent of Students Absent At least One Day Last Week, by Grade and Study Arm 3-7 Figure 5. Average Percent of Students Absent in Classroom 3-8 Figure 6. Percent of Students Achieving Nutrition Indicators, by Study Arm 3-8 Figure 7. Student Nutrition and Hygiene and Sanitation Knowledge Scores 3-9 Figure 8. Percent of Teachers Absent At least Once Last Week 3-9 Table 14. Student Responses on Household Demographic Characteristics, by Grade and Study Arm 3-10 Table 15. Student-reported Absenteeism, by Grade and Study Arm 3-11 Table 16. Teacher-reported Student Attendance, by Study Arm 3-12 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report vii Materials, by Grade and Study Arm 3-13 Arm 3-14 d, by Grade and Study Arm 3-15 3-16 Table 21. Student Responses on School Meals, Current Hunger, and Household Hunger, by Grade and Study Arm 3-18 Table 22. Student Nutrition Practices and Knowledge, by Grade and Study Arm 3- 19 Table 23. Student Hygiene and Sanitation Practices and Knowledge, by Grade and Study Arm 3-20 Table 24. Teacher Profiles, by Study Arm 3-21 3-22 Table 26. Teacher-reported Absenteeism, by Study Arm 3-23 Table 27. Teacher Access to Supplies and Learning Materials, by Study Arm 3-23 Table 28. Teacher Use of School Supplies and Learning Materials, by Study Arm 3- 24 3-25 Table 30. Teacher Perspectives on Supervision and Support, by Study Arm 3-26 Table 31. Teacher Knowledge and Skills, by Study Arm 3-27 Table 32. Teacher Perceptions of Student Attentiveness, by Study Arm 3-29 Table 33. Teacher-reported School Conditions, by Study Arm 3-31 Table 34. Teacher-reported Nutrition and Hygiene Teaching, by Study Arm 3-32 Table 35. Number of Teachers and Classes per Grade, by Study Arm 3-33 Table 36. -related Amenities and Activities at Schools, by Study Arm 3-34 Table 37. HT Training, by Study Arm 3-35 3-36 3-37 viii USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Arm 3-38 3-38 Study Arm 3-39 3-40 3-41 udy Arm 3-42 and Hygiene Lessons, by Study Arm 3-43 3-44 Table 47. School Feeding Infrastructure, by Intervention Arm 3-45 Table 48. Cook Training and Knowledge, by Intervention Arm 3-46 Table 49. Use of School Meals and Cook Working Days, by Intervention Arm 3-47 Table 50. Number of FGDs and Parents by District and Province Involved in Qualitative Data Collection 3-48 Table 51: Grade 2 and 4 EGRA Results for Maputo Province, by Study Arm 1 Table 52: Grade 2 and 4 EGRA Results for Nampula Province, by Study Arm 2 Table 53. Grade 2 Reading Fluency Means and Precision Estimates of the Mean ORF 3 Table 54. Grade 4 Reading Fluency Means and Precision Estimates of the Mean ORF 3 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report ix [This page intentionally left blank.] x USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report LIST OF ACRONYMS ADPP Ajuda de Desenvolvemento de Povo para Povo (Development Aid from People to People) ANOVA analysis of variance CI confidence interval clpm correct letters per minute csspm correct syllable sounds per minute cwpm correct words per minute DEFF Design Effect EGRA Early Grade Reading Assessment EMIS Education Management Information System FAS Foreign Agricultural Service FGD focus group discussion GSA U.S. General Services Administration HT head teacher ICC intra-class correlation IFP Instituto de Formação de Professores (Teacher Training Institute) ORF oral reading fluency PAI Planet Aid Incorporated PSS Professional Services Schedule QC quality control SD standard deviation TTC teacher training college UP Universidad Pedagógica de Moçambique (Mozambique Pedagogical University) USAID United States Agency for International Development USDA U.S. Department of Agriculture WHO World Health Organization wpm words per minute WV World Vision USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report xi [This page intentionally left blank.] _________________________________ RTI International is a registered trademark and a trade name of Research Triangle Institute. 1. EXECUTIVE SUMMARY Background: The McGovern-Dole Program in Mozambique In 2012, the U.S. Department of Agriculture (USDA) Foreign Agricultural Service (FAS) selected World Vision (WV) and Planet Aid Incorporated (PAI) to implement McGovern-Dole activities in Mozambique. The key objective of the McGovern-Dole program is to reduce hunger and improve literacy and primary education. The program provides school meals, as well as teacher training, health, sanitation, and hygiene interventions and other activities. Since 2012, WV has been implementing program activities in Nampula Province, and Planet Aid has been working in Maputo Province. In 2015, both partners received a second cycle of funding to continue the programs and enhance the literacy interventions. However, some schools in Maputo continued to receive only school meals without the literacy intervention. The McGovern-Dole program theory of change posits that improved literacy, health, and hygiene instruction paired with regular school meals will lead to improved literacy, health, and dietary practices of students via various program impact pathways, including increased student and teacher attendance, improved student attentiveness, and increased knowledge of health and hygiene. The intended beneficiaries of the McGovern-Dole program are primary school students in targeted districts in Maputo and Nampula Provinces, their families, and communities. Baseline Literacy Assessment and Impact Evaluation In 2016, USDA contracted RTI International to design an independent impact evaluation of the McGovern-Dole Mozambique program and conduct a baseline assessment. RTI has no role in the implementation of the McGovern-Dole program and no relationship with either implementing partner. The evaluation considers three groups of schools: 1) those receiving school feeding and literacy interventions, 2) those receiving only the school feeding intervention, and 3) comparison schools not receiving any McGovern-Dole interventions. The impact evaluation is designed to compare the following key indicators across the three groups (also referred to as study arms): reading ability, student attendance, student nutrition and hygiene knowledge and practices, and teacher attendance. The baseline survey was designed to support a quasi-experimental impact evaluation to be completed in 2021, which will compare changes in key indicators across the different study arms. The baseline report provides a snapshot of the situation in 2017 and does not constitute an evaluation of the program or its impact until they can be analyzed in conjunction with midline and endline data. The results are intended to support decision makers with information about the situation in Nampula and Maputo and share with implementers a wider vision of the overall status in these provinces. The school-based survey used to obtain the results was conducted in the third trimester of the school year to allow for adequate exposure of the students to standard education and school meals and, in the future, adequate exposure to the intervention. The instruments were developed to address the components of the theory of change, calculate the relevant indicators, address the research questions for the evaluation, and assess the similarities and differences between intervention and comparison schools at baseline. To assess literacy, 1-2 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report enumerators were trained to administer the Early Grade Reading Assessment (EGRA) tool developed by RTI,1 which is an individually administered, 15-minute oral assessment that measures the basic skills a child must have to eventually be able to read fluently and with comprehension. The student survey also included questions on student attendance, the school feeding program, and nutrition and hygiene knowledge and practices. In addition, teachers, head teachers (HTs), and school cooks were interviewed to obtain information on teacher attendance, school amenities, and the school feeding program, among other factors. The survey was paired with qualitative data collection, which provides supplemental insight from parents about the McGovern-Dole program. Baseline Findings Most of the baseline indicators ranged across study arms, and in some instances, comparison schools outperformed the intervention schools. This was expected given the non-random selection of intervention districts and the limited options for comparison districts within intervention provinces. Student Reading Ability Literacy ability is generally very low; at least 40% of Grade 2 students in McGovern-Dole schools received zero scores on all EGRA subtasks. Grade 4 students scored higher than Grade 2 students across all components, as expected. Students in School Feeding Only schools generally scored higher than those in the other two groups; however, it is important to note that these schools are in Maputo only. A higher percentage of students scored zero in the listening comprehension subtask in both intervention groups than in Comparison schools, despite the level of the text read aloud being representative of texts used for instruction. This is an important reading performance. In general, student data also show that at home, students speak languages other than Portuguese, which contributes an added challenge to literacy acquisition. Student Attendance At the time of the baseline survey, intervention schools had been providing school meals for nearly a full school year (and in most cases, four years.) The McGovern-Dole theory of change hypothesizes that providing school meals will increase student attendance, but no significant difference in attendance was detected between intervention and comparison schools at baseline. There was also no significant difference in absence between male and female students. The percentage of students reported absent at least one day in the past week was in the range of 25 30% across study arms. 1 https://globalreadingnetwork.net/resources/early-grade-reading-assessment-egra-toolkit-second-edition USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 1-3 We also compared the number of students present in a classroom on the day of the survey to the official class roster. School Feeding & Literacy schools had the highest proportion absent (28 29%), which differed statistically significantly from the Comparison schools2 (20 22%.) Among students who had missed school during the last week, the most commonly cited reason was student illness (55 60%), which helps to explain why we do not see higher attendance in McGovern-Dole schools despite the provision of meals. Student Nutrition Practices inadequate, with less than half of them reporting a minimum acceptable diet and approximately half achieving minimum meal frequency. Diet diversity scores were higher, with 74% of students in Literacy & School Feeding schools, 54% in School Feeding Only schools, and 69% in Comparison schools achieving minimum dietary diversity. Contrary to the expectation that school meals would add to the home diet, most students noted that they eat less at home on days when they receive school meals; this was supported by findings from the focus group discussions (FGDs) with parents. Student Nutrition and Health Knowledge hygiene/sanitation knowledge was moderate. Students in the Comparison schools had significantly higher knowledge scores than those in McGovern-Dole intervention schools. Teacher & Head Teacher Attendance Teacher absence was common. Across study arms, 21 28% of teachers were absent during the last week, and approximately 22 30% of head teachers were absent. This was reported in the survey with teachers and head teachers and brought up by parents in FGDs as an important issue. Recommendations for the Impact Evaluation The findings from the baseline study highlight the need to 1) continue tracking activities related to the program in both intervention and comparison schools within sampled districts; 2) invest additional effort in matching intervention and comparison schools, students, and teachers (e.g., propensity score matching) in future analyses; 3) conduct a meta-analysis to account for variance across implementers and provinces; and 4) use the proposed difference-in-differences methodology to account for the significant differences between the comparison and intervention arms at baseline and track the rate of progress of these schools over time; the intervention schools would be expected to have higher rates of improvement once the intervention activities take place. 2 A small fraction of Comparison schools was receiving some type of school meal program (not McGovern￾Dole), and approximately half of Comparison schools were found to have a school garden. 2-4 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 2. BACKGROUND & METHODOLOGY 2.1 Background In 2012, the U.S. Department of Agriculture (USDA)/Foreign Agricultural Service (FAS) selected World Vision (WV) and Planet Aid Incorporated (PAI) to implement McGovern-Dole school feeding and some literacy activities in Mozambique. In 2015, both organizations received an additional cycle of funding to continue their work in providing school meals; further the quality of literacy activities; teacher training; school gardens; water, sanitation, instruction. With the 2015 funding cycle, both implementing partners began rolling out enhanced literacy programs, including curriculum development and teacher training. The coverage zones for each organization are shown in Figure 1. Schools covered in 2015 were the same schools where the program has been active since 2012, except for a small number of new schools that opened because of enrollment overflow in pre-existing schools. Although the goals of the intervention are the same, the two implementers take slightly different approaches. What different intervention schools receive to achieve their targets also varies within each implementer. For example, for PAI in Maputo Province, not all McGovern-Dole schools receive literacy interventions: some receive school feeding only, while other (larger) schools receive both school feeding and literacy. In addition, some schools receive the literacy program in Portuguese only, whereas other schools receive both the mother tongue and Portuguese (bilingual) programs. The relatively new bilingual programs were incorporated by McGovern-Dole taking into consideration of evolving educational priorities in Mozambique. In contrast, WV delivers both school feeding and literacy across all targeted schools and implements the program in Portuguese only. (GSA) Professional Services Schedule (PSS) No. GS00F354CA, the USDA/FAS awarded to RTI -Dole Mozambique Impact Evaluation and Liter evaluation and conduct the baseline data collection for its McGovern-Dole Food for Education projects in Mozambique. RTI has no role in the implementation of the McGovern￾Dole program and no relationship with either implementing partner. The baseline assessment was conducted independently from the implementing partners. Figure 1. McGovern-Dole Program Coverage in Mozambique Nampula/World Vision Maputo/Planet Aid USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 2-5 2.2 Methodology The main objective of the impact evaluation design and the baseline data collection is to set the stage to answer th McGovern￾2.2.1 Theory of Change To guide the evaluation, RTI developed a theory of change that links the literacy, health, hygiene, nutrition, and dietary components of the McGovern-Dole program and is based on the McGovern-Dole learning agenda. The theory of change serves as a framework for the design of the impact evaluation, the indicators, and the instruments used to collect baseline data. At the highest level, the McGovern-Dole program posits that improved literacy instruction paired with regular school meals will lead to both improved literacy and improved health and dietary practices of students via various program impact pathways. The linkages between the key elements of the McGovern-Dole program and the other factors that may contribute to improved outcomes are shown in the theory of change diagram (Figure 2). Figure 2. Theory of Change 2.2.2 Impact Evaluation The impact evaluation uses a quasi-experimental design in which comparison schools serve as the counterfactual (i.e., our best estimate of what would have happened in the absence McGovern-Dole SO1 Improved Literacy Outcomes in School￾Aged Children McGovern-Dole 1.1 Improved Quality of Literacy Instruction McGovern-Dole 1.1.2 Better Access to School Supplies and Materials McGovern-Dole 1.1.4 Increased Skills and Knowledge of Teachers McGovern-Dole 1.1.5 Increased Skills and Knowledge of School Administrators McGovern-Dole 1.1.3 Improved Literacy Instructional Materials McGovern-Dole 1.1.1 More Consistent Teacher Attendance Overlap between WV and PAI Specific to WV LEGEND McGovern-Dole 1.2 Improved Attentiveness McGovern-Dole 1.2.1 Reduced Short￾term Hunger McGovern-Dole 1.3.1.1 Increased Access to Food/ Improved Dietary Intake McGovern-Dole 1.3 Improved Student Attendance McGovern-Dole 1.3.5 Increased Community Understanding of Benefits of Education McGovern-Dole 1.3.1 Increased Economic and Cultural Incentives McGovern-Dole 1.3.4 Increased Student Enrollment McGovern-Dole 1.3.3 Improved School Infrastructure McGovern-Dole 1.3.2 Reduced Health Related Absences McGovern-Dole SO2 Improved Health and Dietary Practices of Students McGovern-Dole 2.1 Improved Knowledge of Health and Hygiene Practices McGovern-Dole 2.3 Increased Knowledge of Nutrition McGovern-Dole 2.4 Increased Access to Clean Water and Sanitation Services McGovern-Dole 2.5 Increased Access to Preventative Health Interventions McGovern-Dole 2.6 Increased Access to Requisite Food Prep and Storage Tools and Equipment McGovern-Dole 2.2 Increased Knowledge of Safe Food Prep and Storage Practices McGovern-Dole 1.2.1.1 School Feeding Timing of the School Meal Availability of Bednets at Home Availability of Food at Home Access to Improved Water and Sanitation at Home Parental Knowledge and Use of Improved Health and Dietary Practices 2-6 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report of the McGovern-Dole program). An experimental design with randomized assignment of schools would have more accurately estimated the causal effect, but this was not possible because intervention districts and schools were pre-determined since phase one in 2012, long before the impact evaluation was planned. Because of the variations in interventions described above and the goal of measuring the impact of phase two of the interventions, the impact evaluation design was structured to compare outcomes across three groups of schools: -Dole interventions, including both the literacy and school feeding interventions; School Feeding Only: Schools receiving McGovern-Dole school feeding interventions without any McGovern-Dole literacy intervention (Maputo only); and Comparison: Schools not receiving any McGovern-Dole intervention. It is important to note the following: The aim is not to compare outcomes between the two implementing partners nor between Maputo and Nampula but rather to focus on the impact of the McGovern￾Dole program across both intervention provinces in Mozambique. At the impact evaluation design stage, no schools were identified to receive the literacy intervention only (i.e., without school feeding); thus, we are not pursuing a full factorial design. Schools in the School Feeding Only group are exclusively in Maputo Province, creating some level of imbalance that should be considered during analysis and when interpreting the results. The McGovern-Dole program in Mozambique targets Grades 1 3 for the literacy component and Grades 1 6 for the school feeding component. We assessed Grade 2 and 4 students for two main reasons. First, we expect to see more variability in Grades 2 and 4 reading ability, which would allow us to detect significant differences. Second, Grade 4 students will have more years of cumulative exposure to the intervention, which should yield a better understanding of changes over time associated with the intervention activities. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 2-7 2.2.3 Research Questions We have subdivided the overarching evaluation question into a set of primary and secondary questions (Table 1). Table 1. Research Questions Primary Research Questions Literacy How does reading ability among Grade 2 and 4 students who attend schools with the McGovern-Dole Literacy & School Feeding program compare with that of students who attend 1) schools with the McGovern-Dole School Feeding Only program and 2) schools without any McGovern-Dole program (Comparison)? Is there a larger skills from Grade 2 to Grade 4 in schools with the McGovern-Dole Literacy & School Feeding program than in 1) schools with the McGovern-Dole School Feeding Only program and 2) schools without any McGovern-Dole school feeding/literacy program (Comparison)? Student Attendance Is there higher student attendance at schools with the McGovern-Dole Literacy & School Feeding program than in 1) schools with the McGovern-Dole School Feeding Only program and 2) schools without the McGovern-Dole school feeding/literacy program (Comparison)? Secondary Research Questions Health & Nutrition Do a greater proportion of Grade 2 and 4 students attending schools with the McGovern-Dole Literacy & School Feeding program have knowledge of recommended health and nutrition practices than students at 1) schools with the McGovern-Dole School Feeding Only program or 2) schools without any McGovern-Dole school feeding/literacy program (Comparison)? Do a greater proportion of Grade 2 and 4 students attending schools with the McGovern-Dole Literacy & School Feeding program report recommended health and nutrition practices, including dietary diversity and meal frequency, than students in 1) schools with the McGovern-Dole School Feeding Only program or 2) schools without any McGovern-Dole school feeding/literacy program (Comparison)? Teacher Attendance Is there higher teacher attendance at McGovern-Dole Literacy & School Feeding schools than at 1) schools with the McGovern-Dole School Feeding Only program or 2) schools without any McGovern￾Dole school feeding/literacy program (Comparison)? Do McGovern-Dole Literacy & School Feeding schools have a higher percentage of teachers present on any given day compared to 1) schools with the McGovern-Dole School Feeding Only program or 2) schools without any McGovern-Dole school feeding/literacy program (Comparison)? 2.2.4 Indicators To answer the above research questions, a set of indicators linked to the theory of change was developed and used as the basis for questionnaire design, data management, and analysis. Table 2 highlights some of the main indicators. 2-8 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Table 2. Literacy, Nutrition and Health, and School Management Indicators Literacy Indicators Data Level Reported % of schools with access to teaching and learning materials School % of teachers who are present at the school Student performance on EGRA subtasks (Portuguese) % of students who report being absent or late the past week Student % of students who report being absent because of sickness % of teachers who report attending the teacher training designed to prepare them to use the teaching and learning materials supported by the reading program Teacher/ Classroom % of schools where teachers are using the teaching and learning materials needed to teach reading and promoted by the program/Ministry of Education % of teachers with access to materials for reading instruction promoted by the program (i.e., teacher guides, student books, and supplementary readers) % of teachers who report that students are attentive before/after a meal Nutrition and Health Indicators % of students fed a school meal School % of schools where meals were served on time % of schools where cooks have correct knowledge about safe food preparation and handling % of schools with improved sanitary facilities % of schools where students receive deworming medication % of students who consumed minimum meal frequency on the previous day Student % of students who consumed the minimum number of food groups on the previous day % of students receiving a minimum acceptable diet % of students with correct knowledge of nutrition practices taught by the program % of students with correct knowledge of hygiene and sanitation practices % of students reporting hand washing before meals % of trained teachers with copies of nutrition education and hygiene education materials Teacher/Classroom % of students who report that their school meal replaces a meal at home Household School Management Indicators % of schools that have enrollment and (updated) attendance registers for students, teachers, and HTs (i.e., attendance rate, dropout rate, transfer rate, and lateness rate by gender and grade) School % of schools that have and maintain an updated inventory of food/cooking supplies % of schools that have a place to properly store food/cooking supplies Note: EGRA = Early Grade Reading Assessment; HT = head teacher; USDA = U.S. Department of Agriculture. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 2-9 2.2.5 Instruments Based on these indicators, RTI developed several data collection instruments, which are listed in Table 3. Table 3. Instruments for Data Collection at Each School Instrument Source Data Level Notes: HT Questionnaire and School Inventory HT School Cook Questionnaire and Kitchen Inventory Cook School Schools with 1 or more cooks Teacher Questionnaire Teacher Classroom 1 Grade 2 & 1 Grade 4 Classroom Inventory Assessor w/ Teacher's help Classroom 1 Grade 2 & 1 Grade 4 Student EGRA Student Student 8 Grade 2 & 8 Grade 4 Student Questionnaire Student Student 8 Grade 2 & 8 Grade 4 Parent Focus Group Discussion Guide Parents Parents In selected Literacy & School Feeding and School Feeding Only schools Note: EGRA = Early Grade Reading Assessment; HT = head teacher. All instruments and instructional dialogues (for explaining instructions to the respondents and asking for consent) were developed in English and circulated for peer review internally, with USDA, and with literacy consultants in Mozambique (for the Early Grade Reading Assessment [EGRA] portion). The instruments were never shared with the implementing partners. Once these instruments were finalized, they were adapted and translated into Portuguese and reviewed for accuracy. The only exception was the EGRA, which is an individually administered, 15-minute oral assessment that measures the basic skills a child must have to eventually be able to read fluently and with comprehension the goal of reading (Tobia & Bonifaci, 2015). The EGRA instrument used at baseline consisted of six subtasks and was developed directly in Portuguese and administered to Grade 2 and Grade 4 students in Portuguese3 (Table 4). Table 4. Description of Baseline EGRA Subtasks # Subtask Students must: Reading Skill 1 Letter Name Knowledge (clpm) Provide the name of 100 upper- and lowercase letters presented in random order; timed at 1 minute. The ability to read the letters of the alphabet naturally and without hesitation. 2 Syllable Reading (csspm) Provide the sound represented by 50 upper and lowercase letters presented in random order; timed at 1 minute. The ability to recognize graphemic features of a letter and accurately map it to its corresponding sound. 3 Note that EGRAs in local languages was not developed at baseline since curriculum in local language was still under development. EGRAs in local languages are recommended as part of the efforts for midline and endline instrument updates and data collection. 2-10 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report # Subtask Students must: Reading Skill 3a ORF Read a narrative text of 57 words. Timed at 1 minute. The ability to read connected text with accuracy, little effort, and at a sufficient rate of speed. 3b Reading Comprehension Respond to 5 questions (4 literal and 1 inferential) about the entire text or parts they have read; 15 seconds to start to answer each question. The ability to make meaning from (understand) what they have read. 4 Listening Comprehension Listen to a connected text of approximately 30 words and respond correctly to 5 questions (4 literal and 1 inferential); 15 seconds to start to answer each question. The ability to make sense of oral language and vocabulary (considered a necessary skill for reading comprehension). 5 Dictation Write and correctly spell five words; 20 seconds to scribe each word. The ability to listen to individual words and spell (encode) them correctly. Note: clpm = correct letters per minute; csspm= correct syllable sounds per minute; ORF = oral reading fluency The EGRA adaptation and development process included a review of relevant curriculum materials. For this adaptation, RTI worked with a Mozambican literacy specialist, developed short reading passages using appropriate words that can be used across Grades 2 and 4, and developed appropriate reading comprehension questions to complement the short passages. The adaptation process covered language analysis, the selection of appropriate subtasks for the language and grade levels (Grades 2 and 4), the verification of instructions, and the translation of directions. 2.2.6 Programming All assessments and questionnaires4 mentio an open-source suite of software developed by RTI for education and health surveys. Data collectors could connect to the internet. RTI programmed and tested tablet-based scripts for the questionnaires before beginning data collector training. During and after data collector training, RTI updated the Tangerine instruments to accommodate any adjustments in terminology or survey question items resulting from issues identified in pilot testing. 2.2.7 Field Work Data collection tools and procedures were piloted and adjusted prior to the full-scale assessor training conducted in each province before data collection. Detailed reports of the piloting and adaptation process are available separately. The full-scale training for survey data collection included practice visits to two schools in each province. Data collection occurred in the third trimester of the academic year (September and October 2017) to allow the students to be exposed to a full school year. Quantitative assessment teams consisted of four members: one supervisor, one teacher/classroom assessor, and two student assessors. One assessment team visited one school per day. 4 Excluding the qualitative FGD guide, which cannot be administered via tablet. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 2-11 Qualitative data collection teams worked separately from the quantitative teams. Qualitative teams consisted of two members: a moderator and a note-taker. They contacted selected schools in advance (the school selection process is described in the Sampling section below) and asked the head teacher (HT) to help them invite parents to participate in the focus group discussion (FGD). At each of the selected schools, one FGD was held with parents of students. Lead observers were hired to periodically attend FGDs to provide technical oversight and ensure proper implementation of the data collection methods. During the FGDs, hand-written notes and audio recordings were used to record the data. 2.2.8 Sampling Province- and District-level Sampling Frame At the time of this baseline assessment, PAI was implementing the McGovern-Dole program in 273 schools across four districts in Maputo, and WV was implementing in 150 schools across two districts in Nampula. Although PAI in Maputo Provide covers schools with both Portuguese and local language literacy programs, and WV in Nampula Province covers Portuguese literacy programs only, the baseline literacy assessment was performed in Portuguese because the local language curriculum was still under development. At the time of the baseline, only Grade 1 students would have begun to receive the local language curriculum, so the Grade 2 and 4 students being assessed would not have been exposed. Selection of Comparison Districts We used the following criteria for selecting comparison districts: located within the same province as the intervention districts (Nampula and Maputo) does not have any other known literacy interventions taking place or planned for the foreseeable future (5 years) geographically close to districts with intervention schools has similar urban/rural makeup to the intervention districts has similar numbers of students and schools (thus, similar numbers of students in each school) has similar makeup of languages spoken at home Based on these criteria, schools in the remaining districts in Maputo, except for the City of Maputo, were automatically selected as comparison schools. The City of Maputo was not selected because of its all-urban make up. Despite this deliberate choice, Maputo comparison districts still tended to be more urban than the Maputo intervention districts. Maptuo comparison districts were also more likely to speak Portuguese at home than any of the intervention districts. The two Nampula comparison districts were selected by eliminating districts in Nampula that have or are planning to have some type of (non-USDA) literacy intervention or are primarily urban (City of Nampula) or coastal. This choice was made because such districts tend to have different demographic profiles than the intervention districts. 2-12 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report The selected comparison districts are as follows: Meconta and Mogincual in Nampula Province; and Boane, Marracuene, and Namaacha in Maputo Province (Figure 3). Target Population within Districts The population of interest within the targeted districts included the HTs, teachers, cooks, and Grade 2 and 4 students attending government schools in intervention and comparison districts. Table 5 lists the total number of schools and the expected numbers of Grade 2 and 4 students in each province and study arm. School lists were compiled from the list frames provided by the Ministry of Education, PAI, and WV. Table 5. Target Population across Intervention and Comparison Districts Province Group Districts & Primary Language* No. of Schools Students Sample Frame Grade 2 Grade 4 Maputo Literacy & School Feeding Magude Changana 139 11,475 8,544 School and enrollment figures provided by PAI/ADPP. Manhiça Changana Matutuíne Changana/Rhonga Moamba Changana/Portuguese School Feeding Only Magude Changana 134 4,745 3,323 School and Enrollment figures provided by PAI/ADPP. Manhiça Changana Matutuíne Changana/Rhonga Moamba Changana/Portuguese Comparison Boane Portuguese/Changana 133 19,382 12,722 2016 EMIS data Marracuene Portuguese/Changana Namaacha Changana/Portuguese Figure 3. Intervention and Comparison Districts USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 2-13 Nampula Literacy & School Feeding Muecate Nacarôa Macua 152 16,300 10,985 WV 2016 Status Report Comparison Meconta Mogincual Macua 150 15,832 11,123 2016 EMIS data Note: ADPP = Ajuda de Desenvolvemento de Povo para Povo (Development Aid from People to People); EMIS = Education Management Information System; PAI = Planet Aid Incorporated; WV = World Vision. Number of students is based on available data at the time of design in 2016. *Primary language based on student reports of language spoken at home. See Table 20 for further detail. Sampling Design We selected separate samples for the quantitative and qualitative components of the baseline study. The quantitative sampling design was a multi-stage stratified cluster probability based random design aiming to represent each of the targeted populations (schools, students, teachers, and cooks). However, for the qualitative component, the sampling was purposive, aimed at giving good representation of parents with different characteristics in a subset of the schools from the quantitative sample. The strategies for selecting both samples are described in below in more detail. Quantitative Survey Sampling Design For the quantitative sample, we used a three-stage stratified cluster probability-based design to select a representative random sample of schools, classrooms, teachers, head teachers, cooks, and students. Schools were stratified by province, district, and intervention arm and then sampled with equal allocation. The use of equal allocation rather than probability proportional to size was because enrollment information was missing for many schools in the frame, and even when this information was available, it was determined to be inaccurate based on comparison with other sources. Classrooms and students were sampled once the data collection team reached the school. One Grade 2 classroom and one Grade 4 classroom were randomly sampled with equal probability within the sampled and verified schools. Eight students within each sampled Grade 2 and eight students within each sampled Grade 4 classroom were then selected with equal probability. Table 6 provides the details of the final targeted sample. Table 6. Summary of the Sampling Methodology and Target Sample Size Stage # Item Sample (Sample Size) Stratification (Strata Description) Sample Allocation Probability of Selection 1 Schools (450) Province x Intervention (5) Equal (90 per stratum) Equal 2 Classrooms (900) Grade (2 & 4) Equal (one Grade 2 + one Grade 4 per school) Equal 3 Students (7,200) None Equal (8 students in selected Grade 2 classroom + 8 students in selected Grade 4 classroom) Equal 2-14 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Verification of Schools Once the sample was drawn, and prior to data collection, district officials were visited to verify that each sampled school met the following inclusion criteria. Intervention schools must be an active part of the McGovern-Dole School Feeding Only or Literacy & School Feeding intervention; be located in the specified district; have Grade 2 and Grade 4 enrollment; instruct (or intend to instruct) Grade 2 in either Changana, Ronga, or Portuguese (if in Maputo); and instruct (or intend to instruct) Grade 2 in Portuguese only (if in Nampula). Comparison schools must not be a part of any intervention and have no plan to be a part of an intervention for the foreseeable future (~5 years); be located in the specified district; and have Grade 2 and Grade 4 enrollment. Additional information collected/verified at the verification processes includes the number of classroom shifts in the school; the start and end times of shift(s); the total number of Grade 2 classes and Grade 4 classes; the total Grade 2 enrollment and Grade 4 enrollment; the location; and the HT name. Replacement Schools The verification process also helped the team compare the information provided in the list frames with what was on the ground. If a school did not have the required attributes for inclusion, it was replaced with the school that was most similar based on geographic location. A total of 32 schools were replaced. Seventeen comparison schools in Nampula were replaced because they were involved in a literacy intervention. Eight schools were replaced because they did not have Grade 4 enrollment. Another three schools had no Grade 4 enrollment but were not replaced because no additional replacement schools were available. The field team was instructed to proceed with the data collection in these schools. Additionally, seven schools were replaced because they were used in the pilot or practice sessions. Final Quantitative Sample Counts and Estimates A total of 448 schools were visited during September 4 October 6, 2017. The final sample included 429 HTs, 214 cooks (intervention schools only), 239 warehouse managers USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 2-15 (intervention schools only), 412 Grade 2 teachers, 394 Grade 4 teachers, 3,420 Grade 2 students, and 3,289 Grade 4 students (Table 7). Overall, the response rate was high and reflects a good representation of the targeted population with most of the response rates exceeding 85%, except for the population of cooks. Letters from the Ministry and districts helped with school collaboration and reducing non-response. Non-respondents were individuals who did not consent to be interviewed or were not present during data collection. As will be highlighted in the results sections, teachers and HTs report an average 25% rate of absenteeism, which is one of the main reasons for non-response. In addition, it is important to note that cooks are volunteer parents who do not necessarily go to schools daily and are not obligated to be in the school during the full school day, which resulted in lower response rates for this population specifically. However, compared to industry standards, the response rate of 79.3% of cooks is considered good for a population of volunteer workers. Table 7. Final Sample Counts Provinces Study Arm Schools HT Cooks Warehous e Managers Teachers Students Grade 2 Grade 4 Grade 2 Grade 4 Maputo Literacy & School Feeding 90 84 62 82 83 78 661 638 School Feeding Only 90 85 70 79 82 72 704 681 Comparison 89 86 n/a n/a 86 83 666 616 Nampula Literacy & School Feeding 89 85 82 78 79 80 699 678 Comparison 89 89 n/a n/a 82 81 690 676 Actual Total 448 429 214 239 412 394 3,420 3,289 Expected Total 450 450 270 270 450 450 3,600 3,600 Overall Response Rate 99.6% 95.3% 79.3% 88.5% 91.6% 87.6% 95.0% 91.4% Note: HT = head teacher Qualitative Sample For the qualitative sample, we stratified schools that had completed survey data collection before qualitative data collection started by number of Grade 2 classes (as a proxy for school size), HTs randomly selected schools to represent those categories across Maputo and Nampula Provinces. FGDs were conducted at 18 schools in Maputo and 12 schools in Nampula. In Maputo, FGDs were held in Magude (four FGDs), Manhiça (six FGDs), Matutuíne (three FGDs), and Moamba (five FGDs) districts. In Nampula, FGDs were held in Muecate (six FGDs) and Nacarôa (six FGDs). A larger number of schools was selected from Maputo to achieve 2-16 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report representation in both the School Feeding Only and the Literacy & School Feeding study arms. 2.2.9 Data Management Data Monitoring and Processing Data monitoring and quality control (QC) were conducted daily throughout data collection. Data were downloaded from the Tangerine site daily for each of the six instruments and checked by statisticians. Some checks were performed at the individual assessor level to check whether assessors were making common data collection errors. Regular communication between the RTI office and field teams ensured that any errors were addressed immediately. Preliminary analysis of variable distribution and logic QC were also conducted, such as checking that enrollment was larger than or equal to the number of students present and Weights Calculation Final sample weights were calculated for each stage of sampling (i.e., at the school, classroom, and student levels). The final weights consisted of the sampling weights adjusted for non-response. Sampling weights were calculated as the inverse of the probability of selection at each stage. School weights were calibrated to the known populations of schools by province and study arm. To account for appropriate inference to the target population and for the clustering effect in the sample, weights were calculated at each stage of the sampling. The complex sampling plan provides the stratification, clustering, sampling units, finite population correction, and final sampling weights for each stage. Unless explicitly mentioned otherwise, all estimates provided in the report were analyzed using the appropriate weights reflecting the complex sample design and adjusting for non-response. Sample Size Calculation The sample size calculation was derived using 95% confidence intervals (CIs) with a standard deviation (SD) of 14.5 and the average number of words read per minute in the EGRA 5 . Some of the other assumptions driving the sample size calculation included aiming for a balanced sample design while accounting for an intra-class correlation (ICC) coefficient of 0.15 and a design effect of 3.25. The sample size and the sampling methodology were developed together to obtain the most cost-effective and statistically rigorous sample possible. The sample size and sampling methodology were aimed to provide a 95% CI of ±2.5 words per minute (wpm) for students in each province and study arm6 . Annex B provides full mean and precision estimates. 5 This EGRA was conducted with Grade 2 and 3 students in Nampula and Zambezia provinces at the beginning of the 2013 school year. The assessment was conducted by International Business & Technical Consultants Inc. and funded by the United States Agency for International Development (USAID). 6 USAID typically uses sample sizes and methodologies that provide 95% CIs of ±3.5 wpm. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 2-17 2.3 Analysis Methodology The baseline analysis consists of calculating descriptive statistics and testing hypotheses to compare the core indicators across the Literacy & School Feeding, School Feeding Only, and Comparison schools. As a baseline, these results provide a snapshot of indicators at a moment in time and should not be interpreted as reflecting the impact of the program. Impact will be assessed using difference-in-differences analysis at midline and endline. 2.3.1 Basic Quantitative Analysis The quantitative analysis consisted of running both un-weighted and weighted frequencies and means across all variables to explore the data overall. The weighted frequencies and descriptive statistics were followed by crosstabulations by relevant groups (e.g., grade level). This report highlights the results of the weighted analysis. For core indicators, two types of testing were conducted: 1) For continuous variables: Analysis of variance (ANOVA) F-tests were conducted to investigate whether a significant difference exists between the two intervention groups and the comparison group, followed by multiple comparisons to detect the source of this significant difference. 2) For categorical variables: Chi-square tests were conducted to compare the two intervention groups and the comparison group across response categories, followed by Kruskal-Wallis tests to investigate two-by-two comparisons to detect the source of the significant difference. 2.3.2 Advanced Quantitative Analysis Schools across districts and provinces vary in their performance because of many exogenous factors, such as urban versus rural setting, the resources available to these schools, accessibility, and socio-economic level. When the comparison group outperformed the intervention groups across some indicators, it was not clear whether this difference was related to the purposeful selection of the intervention districts and schools because of their need or to other unknown reasons. For a more robust comparison across schools, RTI attempted to run propensity score matching at the school level to account for the variability among schools across the three study arms, provinces, and districts. A multinomial logistic regression was conducted, as were several logistic regression models. School enrollment was the only significant variable that could match the schools and was not linked to the main indicators of interest. With school enrollment, the model explained only 8% of the variability and was, therefore, not considered to be a robust model for matching schools. Although school-level information was not sufficient to achieve robust matching between schools across the study arms, we believe that matching at the student and teacher levels within schools should be conducted as part of the impact evaluation analysis effort. 2.3.3 Qualitative Analysis Qualitative research methods were used to provide greater contextual understanding of the quantitative baseline results and account for parents perspectives and experiences with the program. These methods consisted of conducting FGDs among parents with children enrolled in School Feeding Only program and Literacy & School Feeding schools. Our goal 2-18 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report the impact of the McGovern-Dole program on their children; therefore, we did not conduct parent FGDs in Comparison schools Recordings of the FGDs were transcribed and translated from the local languages into English for analysis. Once translated, the transcripts were coded according to 10 themes using the qualitative data analysis program Dedoose. Coded themes included parental ption of how gardens (when applicable). Reports compiling all FGD content relating to each of the themes were then generated. Researchers analyzed these reports to identify common topics corresponding to each of the themes in the original transcripts. _________________________________ RTI International is a registered trademark and a trade name of Research Triangle Institute. 3. ANALYSIS This part of the report is divided into two subsections. The first subsection focuses on the indicators linked to the primary research questions and an assessment of differences between the two intervention groups and the comparison group. The second section provides descriptive statistics for all other indicators, reflecting a wider snapshot of the situation at baseline, including qualitative analysis of the focus group data. 3.1 Baseline Indicators In this section, we present key indicators related to the primary and secondary research questions for the baseline and document statistical differences in the indicators across the study arms at baseline. The primary research questions focus on literacy ability and student attendance. The seco practices and teacher attendance. 3.1.1 Literacy The key indicator addressing the primary research questions on the literacy ability of students in Grades 2 and 4 is student performance on the EGRA. EGRA results are presented by subtask in Tables 8 13 in the sections below. The tables include mean scores for timed subtasks (i.e., letter identification, syllable reading, and oral reading fluency [ORF]) and mean percent scores for all non-timed subtasks (i.e., reading comprehension, listening comprehension, and dictation). Zero scores for each subtask are representative of students who could not, for example, identify a single letter or read a single word of connected text. When comparing across the study arms, it is important to consider that the School Feeding Only study arm exists exclusively in Maputo, whereas the Literacy & School Feeding and Comparison study arms represent averages from across Maputo and Nampula Provinces. For EGRA scores, we provide p-values to show the statistical significance of the difference between the score of each treatment group and that of the comparison group. Smaller p￾values (p<0.05) are indicative of strong statistical significance. At baseline, Grade 2 students in the Comparison group had fewer percent zero scores across all subtasks, indicating this group had stronger basic literacy skills compared to Grade 2 students in both intervention groups. Grade 4 students in the Comparison group had fewer percent zero scores than Grade 4 students in both intervention groups on the listening comprehension, ORF, and reading comprehension subtasks. In contrast, Grade 4 students in the School Feeding Only group had fewer percent zero scores on the letter identification, syllable reading, and word dictation subtasks than Grade 4 students in the Comparison and Literacy & School Feeding groups. The section below presents student performance on the EGRA by subtask, grade level, and study arm. Listening Comprehension Subtask: Although listening comprehension is not a reading ability assessment, it is a prerequisite that students be able to understand the target language of instruction. The percentage of 3-2 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report students with zero scores in this subtask indicate that these students were not able to correctly respond to a single listening comprehension question. In Grade 2, the percent of students with zero scores on listening comprehension varied: 33.8% (Comparison), 50.8% (School Feeding Only), and 62.7% (Literacy & School Feeding). This difference was statistically significant, with a p-value of less than 0.001 in both intervention groups. Grade 2 students in the Comparison schools performed better on the Listening comprehension subtask than Grade 2 students in the intervention schools. The higher percentage of zero scores among Grade 2 students in both intervention groups suggests that these students did not understand the meaning of the words read aloud in the passage. Table 8. Listening Comprehension Subtask Scores GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Percent Zero Scores 33.8 50.8 62.7 21.2 25.6 42.4 Mean Percent Scores 33.0 21.2 15.0 45.0 39.0 28.6 p-values <0.001* <0.001* 0.043* <0.001* The p-values shown represent the statistical significance of differences in mean percent scores relative to the Comparison group. In Grade 4, students in Comparison schools performed better than students in both intervention schools. This difference was statistically significant, with p-values of 0.043 and less than 0.001, respectively. Grade 4 students in the Literacy & School Feeding intervention showed the highest percentage of zero scores in the listening comprehension subtask compared to Grade 4 students in the other two groups. A higher percentage of Grade 2 and 4 students scored zero in the listening comprehension subtask in both intervention groups compared to Grade 2 and 4 students in Comparison schools. This finding suggests these students struggled to listen with understanding to a story read aloud in the target language of instruction, although the level of the text was representative of texts used for literacy instruction in Grades 2 and 4. This is an important finding given that researc performance (Scarborough, 2001). In general, student data also show that at home, students speak languages other than the target language of instruction. This finding may pose an added challenge to reading acquisition given that there is a mismatch between the language of instruction and the language spoken at home language (Heugh et al., 2007). Letter Identification Subtask: In the letter identification subtask, zero scores indicate the percentage of students not able to correctly name a single lower- or upper-case letter. Grade 2 students in the Literacy & School Feeding intervention had a lower percentage of students with zero scores (42.9%) USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-3 than Grade 2 students in the School Feeding Only group (49.4%). This difference was statistically significant in the Literacy & School Feeding group, with a p-value of 0.003. Mean scores on the letter identification task (Table 9) show that on average, Grade 2 students in the Comparison group and the School Feeding Only group could name approximately 11 correct letters per minute (clpm), whereas Grade 2 students in the Literacy & School Feeding group correctly name approximately 8 clpm. Given that letter knowledge is a predictor of reading acquisition (Hammill, 2004), the finding that 34 49% of Grade 2 students across study arms could not correctly name a single lower or upper-case letter helps explain the high percentage of zero scores on the syllable reading task in Grade 2 across all study arms. Table 9. Letter Identification Subtask Scores GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Percent Zero Scores 34.0 49.4 42.9 10.2 8.6 13.9 Mean Percent Scores 11.4 11.0 8.6 33.0 37.7 24.4 p-values 0.708 0.003* 0.008* <0.001* The p-values shown represent the statistical significance of differences in mean percent scores relative to the Comparison group. The percentage of Grade 4 students in the School Feeding Only group with zero scores (8.6%) was lower than the percentage of Grade 4 students in the Comparison group (10.2%) and the Literacy & School Feeding (13.9%) groups. The differences were statistically significant, with a p-value of 0.008 in the School Feeding Only group and a p-value of less than 0.001 in the School Feeding & Literacy group. Mean scores on the letter identification subtask (Table 9) show that on average, Grade 4 students in the School Feeding Only intervention could correctly name more letters per minute (approximately 38 clpm) than students in the Comparison group (approximately 33 clpm) and School Feeding & Literacy group (approximately 24 clpm). In general, Grade 4 students performed better than Grade 2 students on the letter identification subtask. Syllable Reading Subtask: In the syllable reading subtask, zero scores indicate the percentage of students not able to correctly read a single syllable. In Grade 2, 68 75% of students across study arms scored zero on the syllable reading subtask. Mean scores show on the syllable reading subtask (Table 10) show that on average, Grade 2 students in the Comparison group and the School Feeding Only group correctly read approximately 6 syllables per minute, whereas Grade 2 students in the School Feeding & Literacy group correctly read approximately 4 syllables per minute. 3-4 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report These results help explain low performance on the oral reading fluency subtask among Grade 2 students in both intervention groups. Research shows that when students struggle to read basic syllables, they often struggle with reading words made up of two or more syllables (Ficksman & Adelizzi, 2010). Table 10. Syllable Reading Subtask Scores GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Percent Zero Scores 68.2 69.7 74.7 29.8 22.1 41.0 Mean Scores 6.5 6.0 4.0 25.3 29.4 16.8 p-values 0.539 0.001* 0.019* <0.001* The p-values shown represent the statistical significance of differences in mean scores relative to the Comparison group. In the syllable reading subtask, Grade 4 students in the School Feeding group had fewer zero scores (22.1%) than Grade 4 students in the Comparison group (29.8%) and the Literacy & School Feeding group (41.0%). These findings were statistically significant, with p-values of 0.019 in the School Feeding Only group and less than 0.001 in the Literacy & School Feeding intervention. The percentage of zero scores across study arms (22 41%) indicate that Grade 4 students still struggle to read basic syllables, with students in the Literacy & School Feeding intervention performing the lowest on this foundational skill. Mean scores on the syllable subtask (Table 10) show that on average, Grade 4 students in the School Feeding Only Group could correctly read more syllables (approximately 29 syllables per minute) than Grade 4 students in the School Feeding & Literacy Group (approximately 16 syllables per minute) and the Comparison group (approximately 25 syllables per minute). In general, Grade 4 students performed better than Grade 2 students. Across study arms, Grade 2 and 4 student performance on the syllable reading subtask is in line with Grade 2 and 4 student performance on the letter identification subtask. These findings align with research showing that when primary grade students struggle to correctly identify letter names, they often have difficulty learning about letter sounds and letter sound relationships, all of which are necessary steps in learning to correctly read syllables and simple words (Wood & McLemore, 2001) in the target language of instruction. Oral Reading Fluency (ORF) Subtask: In the oral reading fluency (ORF) subtask, zero scores indicate that students were not able to correctly read a single word in the passage. In Grade 2, 56% of students in the School Feeding Only schools and 52% of students in the Literacy & School Feeding schools could not read a single word, compared to 38% of students in Comparison schools (Table 11). Grade 2 students in Comparison schools could read slightly better than students in the intervention groups. The difference in performance on the ORF subtask is statistically significant, with p- USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-5 values of 0.013 for School Feeding Only schools and less than 0.001 for the School Feeding & Literacy schools. Mean scores on the ORF subtask show that on average (Table 11), Grade 2 students in the Comparison schools could correctly read more words per minute (approximately 5 cwpm) than Grade 2 students in the School Feeding Only (approximately 4 cwpm) and School Feeding & Literacy (approximately 3 cwpm) schools. The average number of words read correctly per minute across study arms indicates that Grade 2 students struggled with reading connected text. Table 11. Oral Reading Fluency Subtask Scores GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Percent Zero Scores 38.5 56.5 52.1 23.0 24.1 33.1 Mean Scores 5.0 3.6 2.9 23.4 24.4 14.3 p-values 0.013* <0.001* 0.595 <0.001* The p-values shown represent the statistical significance of differences in mean scores relative to the Comparison group. In Grade 4, the percentage of students with zero scores on the ORF subtask was 23 33% across study arms. Grade 4 students in the Literacy & School Feeding schools had the highest percentage of zero scores compared to Grade 4 students in the other two groups. This finding was statistically significant for the Literacy & School Feeding schools, with a p￾value of less than 0.001. Mean scores on the ORF subtask show that on average (Table 11), Grade 4 students in the Comparison and School Feeding Only study arms correctly read approximately 23 24 wpm, respectively. In contrast, students in the School Feeding & Literacy schools correctly read approximately 14 wpm. The average number of cwpm across study arms suggests that Grade 4 students struggled with reading the entire passage. Abadzi (2012) states that while reading rates necessary for reading comprehension may vary across languages, in general, students need to correctly read 45 60 wpm to understand what they read. In general, Grade 4 students performed better than Grade 2 students on the ORF subtask. Reading Comprehension Subtask: In the reading comprehension subtask, zero scores indicate the percentage of students not able to correctly respond to a single reading comprehension question (Table 12). Across study arms, the difference in the percentage of Grade 2 students with zero scores in the reading comprehension subtask is statistically significant: 87.8% (Comparison), 93.1% (School Feeding Only), and 92.6% (Literacy & School Feeding), with p<.001. In general, this finding suggests that the majority of Grade 2 students in both intervention groups were not able to understand what they read. This aligns with their inability to read most of the words in the passage. 3-6 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Table 12. Reading Comprehension Subtask Scores GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Percent Zero Scores 87.8 93.1 92.6 52.2 57.6 70.9 Mean Percent Scores 5.1 2.0 2.2 26.8 23.2 14.2 p-values <0.001* <0.001* 0.189 <0.001* The p-values shown represent the statistical significance of differences in mean percent scores relative to the Comparison group. In Grade 4, 52.2 70.9% of students scored zero on the reading comprehension subtask. Grade 4 students in the Literacy & School Feeding schools (70.9%) had the highest percentage of zero scores in the reading comprehension subtask, compared to Grade 4 students in the Comparison schools and School Feeding Only schools. This finding was statistically significant in the Literacy & School Feeding schools, with a p-value of less than 0.001. When students read too slowly, as shown by the low range of ORF rates across study arms among Grade 4 students (14 24 cwpm), students are primarily focusing their time and attention on trying to decode the words, and in the process, they compromise their understanding of the meaning that the words convey (Samuels & Flor, 2007). Word Dictation Subtask: In the word dictation subtask, zero scores indicate that students were not able to correctly write a single word. In Grade 2, the percentage of students who scored zero was 74.1 81.5% across study arms (Table 13), indicating that students struggled to apply their knowledge of the relationship between letter names and sounds in Portuguese to write words found in grade-level text. Grade 2 students in the Literacy & School Feeding intervention had the highest percentage of zero scores compared to the other two groups. This finding was statistically significant, with a p-value of 0.002 in the Literacy & School Feeding intervention. The findings align with research that shows a strong correlation bet , given that across study arms, 68.2 74.7% of Grade 2 students could not read a single syllable (Table 10). Table 13. Word Dictation Subtask Scores GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Percent Zero Scores 74.7 74.1 81.5 38.1 25.2 52.1 Mean Percent Scores 13.4 13.0 8.8 41.7 55.1 29.8 p-values 0.828 0.002* <0.001* <0.001* The p-values shown represent the statistical significance of differences in mean percent scores relative to the Comparison group. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-7 In Grade 4, the percentage of students with zero scores was 25.2 52.1%, with students in the Literacy & School Feeding intervention showing a higher percentage of zero scores. This suggests that more Grade 4 students in the Literacy & School Feeding intervention struggle with writing basic words than in the other two groups. The difference on the word dictation subtask is also statistically significant in both intervention groups (p<0.001). In the word dictation subtask, Grade 4 students performed better than Grade 2 students. In general, this finding is in line with better Grade 4 student performance than Grade 2 student performance on subtasks that measure foundational skills, such as letter identification, which are needed to develop the ability to write basic words. 3.1.2 Student Attendance At baseline, no significant difference was detected in student attendance by study arm in either grade. The percentage of absent students was high across all groups with no significant differences across gender. Overall, 28% of Grade 2 students and 27% of Grade 4 students were absent during at least one day in the last week (Figure 4). Figure 4. Percent of Students Absent At least One Day Last Week, by Grade and Study Arm As an additional measure of student attendance, we compared class enrollment rosters with the number of students present in the classroom on the day of the survey (Figure 5). School Feeding & Literacy schools had higher absence rates than Comparison schools (p<0.001). School Feeding Only schools had lower absentee rates than Comparison schools, but the difference was only significant in Grade 2. Student illness was cited as the primary reason for absences across gender. This is discussed further in Section 3.2.1. 29.4 30.3 25.6 28.0 27.3 24.8 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Grade 2 Grade 4 3-8 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Figure 5. Average Percent of Students Absent in Classroom 3.1.3 Secondary Indicators At baseline, we found several differences in secondary indicators by study arm. In general, diversity, and minimum acceptable diet (World Health Organization [WHO], 2010; Figure 6). Minimum meal frequency is defined as three or more meals per day. Minimum dietary diversity is defined as consumption of four or more of seven food groups (grains, roots, and tubers; legumes and nuts; dairy products; flesh foods; eggs; vitamin A-rich fruits and vegetables; and other fruits and vegetables). Minimum acceptable diet comprises individuals who have both minimum meal frequency and minimum dietary diversity. Less than half of the students surveyed had a minimum acceptable diet. Fewer students achieved minimum meal frequency than minimum dietary diversity. The percentage of students with minimum dietary diversity and minimum acceptable diet ranged from lowest in School Feeding Only schools to highest in Literacy & School Feeding schools, with Comparison schools falling in between (both p<0.001). The percentage of students with minimum meal frequency was lowest in the Comparison arm and highest in the Literacy & School Feeding arm, with School Feeding Only arm falling in between (p<0.001). Figure 6. Percent of Students Achieving Nutrition Indicators, by Study Arm 20.7 16.8 28.7 22.3 19.7 29.5 Comparison School Feeding Only School Feeding & Literacy Comparison School Feeding Only School Feeding & Literacy 69.4 48.6 38.7 54.4 56.2 32.8 73.6 59.6 47.5 Minimum Dietary Diversity (MDD) Minimum Meal Frequency (MMF) Minimum Acceptable Diet (MDD + MMF) Comparison School Feeding Only Literacy & School Feeding Grade 2 Grade 4 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-9 We asked students eight nutrition and seven hygiene and sanitation questions based on the curriculum provided by the program implementers. The nutrition questions focused on food groups, nutrients, and consumption of a balanced diet. The hygiene and sanitation qu nutrition knowledge was very low and that their hygiene/sanitation knowledge was moderate (Figure 7 the Literacy & School Feeding schools had the lowest nutrition and hygiene/sanitation knowledge scores, whereas those in Comparison schools had the highest scores; School Feeding Only schools fell in between (both p<0.001). Figure 7. Student Nutrition and Hygiene and Sanitation Knowledge Scores Teacher absenteeism was common. Across the study arms, 23% of teachers were absent at least once during the last week, based on self-report data (Figure 8). We found no significant differences in the percentage of teachers absent by study arm. However, the mean number of teachers absent differed significantly by study arm (p<0.001), based on data reported by HTs. School Feeding Only schools had the fewest teachers absent, and Literacy & School Feeding schools had the most, with Comparison schools falling in between. Figure 8. Percent of Teachers Absent At least Once Last Week 3.2 1.0 3.0 0.7 2.7 0.5 Hygiene & Sanitation Knowledge Score (max 8) Nutrition Knowledge Score (max 7) Comparison School Feeding Only School Feeding & Literacy 20.7 27.8 24.3 Comparison School Feeding Only Literacy & School Feeding 3-10 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3.2 Baseline Descriptive Statistics In this section, we provide descriptive results for all survey variables included in the theory of change for the evaluation by type of respondent: students, teachers, HTs, and cooks. The qualitative findings from parents are also presented and triangulated with the survey results. 3.2.1 Students Student Demographic Characteristics Students provided informati characteristics. On average, students in Grade 2 were 8 years old, and students in Grade 4 were 10 11 years old (Table 14). Across study arms, approximately half of the students interviewed were female. A higher percentage of students in Comparison schools had piped water or a tap in their homes (approximately 32%) compared to those in School Feeding Only (approximately 18%) and Literacy & School Feeding (approximately 15%) schools. For students in School Feeding Only and Literacy & School Feeding schools, wells or boreholes were the most common household water source, followed by communal taps. Similar percentages of students in Comparison schools had access to water from wells or boreholes and piped water. moderate, ranging from three to four items out of nine. On average, students in the Literacy & School Feeding study arm had the lowest number of household assets. Table 14. Student Responses on Household Demographic Characteristics, by Grade and Study Arm GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding n=1,386 n=666 n=1,368 n=1,349 n=616 n=1,323 Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Student age 8.5 [±.2] 8.1 [±.2] 8.7 [±.2] 10.9 [±.2] 10.4 [±.2] 11.3 [±.2] % (n) % (n) % (n) % (n) % (n) % (n) Female students 50.3 (702) 50.0 (344) 49.5 (680) 51.2 (685) 50.4 (310) 50.3 (670) Household water source River, stream, or lake 12.7 (273) 15.5 (132) 20.0 (321) 12.7 (266) 19.3 (152) 17.5 (297) Well or borehole 29.9 (460) 40.3 (250) 41.0 (549) 28.6 (425) 35.8 (216) 43.6 (569) Communal tap 21.4 (311) 24.8 (158) 23.2 (297) 20.8 (287) 23.3 (146) 20.8 (260) Water pipe/tap in home 31.7 (268) 17.3 (73) 14.0 (130) 33.7 (300) 18.8 (64) 16.1 (145) Water truck or tank 2.1 (24) 1.9 (17) 1.1 (21) 2.4 (29) 2.7 (12) 0.7 (18) USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-11 GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Other 2.2 (22) 0.2 (1) 0.7 (6) 1.8 (19) 0.1 (2) 1.3 (9) Household asset index Radio 57.9 (748) 61.2 (398) 58.9 (780) 56.8 (731) 57 (369) 59.8 (800) Mobile phone 73.3 (928) 86.8 (575) 66.7 (930) 76.1 (962) 90.5 (544) 72.4 (960) Television 51.8 (511) 55.9 (304) 34.4 (415) 53.8 (542) 52.9 (290) 32.9 (410) Computer 20.7 (216) 24.6 (106) 12.5 (139) 18.7 (190) 12.7 (74) 10.2 (107) Refrigerator 41.7 (379) 35.6 (164) 22.0 (229) 39.1 (368) 31.5 (161) 21.4 (225) Bicycle 41.0 (593) 44.3 (304) 4.09 (648) 43.2 (615) 50.4 (315) 51.0 (710) Motorbike 18.0 (224) 16.2 (100) 19.1 (237) 17.8 (224) 13.9 (87) 20.5 (233) Car/truck 24.6 (248) 32.9 (185) 14.9 (184) 22.2 (227) 29.9 (157) 14.6 (177) Electricity 57.9 (590) 47.2 (251) 35.6 (415) 56.0 (599) 44.7 (224) 30.4 (353) Mean ± 95% CI Band Mean ± 95% CI Band Mean ± 95% CI Band Mean ± 95% CI Band Mean ± 95% CI Band Mean ± 95% CI Band Asset index (9 items) 3.8 [±.2] 4.0 [±.3] 3.1 [±.2] 3.8 [±.2] 3.8 [±.3] 3.1 [±.2] Note: CI = confidence interval. Student Absenteeism and Attendance Approximately 25 30% of students across the study arms were absent from school on any day last week (Table 15). Illness was the main reason for students being absent, with 56 66% of students who were absent saying they missed school because they were ill. Other top reasons for missing school were other work at home (8 18%) and unavailability of a school uniform (7 10%). Table 15. Student-reported Absenteeism, by Grade and Study Arm GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding n=1,394 n=666 n=1,368 n=1,357 n=616 n=1,323 % (n) % (n) % (n) % (n) % (n) % (n) Absent from school any day last week 29.3 (346) 30.3 (182) 25.6 (343) 28.0 (377) 27.3 (146) 24.8 (335) Reason for being absent Illness 59.2 (177) 66.4 (104) 55.8 (173) 58.3 (211) 57.5 (85) 57.5 (192) Had other work at home 7.6 (34) 13.9 (25) 11.7 (46) 11.2 (42) 11.6 (18) 17.8 (52) Had to care for family member 3.2 (10) 1.3 (4) 3.8 (11) 6.2 (17) 6.4 (4) 5.7 (19) No transportation 1.9 (7) 0.0 (0) 0.4 (4) 0.8 (3) 0.2 (1) 0.5 (1) 3-12 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Bad weather 2.5 (8) 1 (3) 0.6 (1) 0.8 (3) 1.5 (4) 0.3 (3) Had an emergency 2.9 (9) 2.5 (5) 1.8 (8) 1.9 (10) 1.1 (2) 0.9 (5) Getting to school was too dangerous 0.3 (1) 0 (0) 1.3 (2) .7 (2) 0.0 (0) 0.0 (1) Woke up late 1.3 (6) 0 (0) 2.6 (5) 1.3 (1) 0.9 (2) 0.9 (2) Could not find uniform or uniform was not ready 6.7 (28) 6.7 (4) 7.5 (25) 9.6 (35) 8.5 (9) 5.9 (20) Treated poorly by teachers 0.7 (2) 0.7 (1) 0.3 (3) 0.5 (2) 0 (0) 0.9 (4) Treated poorly by other students 1.4 (3) 0 (0) 2.0 (3) 1.2 (6) 0 (0) 0.2 (1) Other 12.4 (38) 7.5 (15) 12.0 (47) 7.5 (34) 12.3 (8) 9.5 (31) Note: CI = confidence interval. We also asked teachers about the total number of students formally enrolled in the class. In general, enrollment for boys and girls was lower in School Feeding Only schools compared to Literacy & School Feeding and Comparison schools (Table 16). Across the study arms, the mean number of students present was lower than the mean number enrolled, and the mean number of girls enrolled was lower than the mean number of boys enrolled. Enrollment numbers were compared to the number of students present on the day of observation to generate the attendance rates discussed in Section 3.1.2. Sixteen to nineteen percent of teachers across the study arms did not have a student attendance register available. Most of those with student attendance registers completed them daily. Table 16. Teacher-reported Student Attendance, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=331 n=163 n=312 Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Number of boys enrolled in the class 27.2 [±.8] 14.0 [±1.3] 23.9 [±.0] Number of boys present 18.3 [±1] 11.0 [±1.4] 14.9 [±1.3] Number of girls enrolled in the class 25.1 [±.8] 13.7 [±1.5] 22.3 [±.9] Number of girls present 17.7 [±1.2] 10.7 [±1.7] 14.3 [±1.2] Number of students absent on a typical day 7.3 [±.5] 3.3 [±.4] 7.8 [±.5] Number of students who are late on a typical day 7.3 [±.5] 4.6 [±.7] 6.2 [±.5] % (n) % (n) % (n) Status of class attendance register USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-13 Register was not available 19.1 (67) 16.2 (27) 15.6 (51) Attendance records completed daily 65.4 (213) 69.7 (112) 70.2 (214) Attendance records completed weekly 5.9 (21) 7.6 (12) 5.6 (20) Attendance records completed biweekly 0.5 (3) 0.0 (0) 1.3 (2) Attendance records completed monthly 7.4 (21) 1.1 (2) 3.2 (8) Other 1.8 (6) 5.4 (10) 4.0 (16) Note: CI = confidence interval. ing on Access to School Supplies and Learning Materials The percentage of Grade 2 students who had a supplementary reader available on the day of the interview was 50 60% across study arms (Table 17). The percentage of Grade 2 students who had the Portuguese textbook available on the day of the interview was 79 82% across study arms. The percentage of Grade 2 students who had the exercise notebook available on the day of the interview was 83 85% across study arms. When asked if they read books daily in the classroom or school library, across study arms, the rate of affirmative responses among Grade 2 students was 42 58% The percentage of Grade 4 students who had a supplementary reader available on the day of the interview was 48 53% across study arms. The percentage of Grade 4 students who had the Portuguese textbook available on the day of the interview was 71 82% across study arms. The percentage of Grade 4 students who had the exercise notebook available on the day of the interview was 92 94% across study arms. When asked if they read books daily in the classroom or school library, across study arms, the rate of affirmative responses among Grade 4 students was 52 62%. Table 17. Grade and Study Arm Grade 2 Grade 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding n=1,394 n=666 n=1,368 n=1,357 n=616 n=1,323 % (n) % (n) % (n) % (n) % (n) % (n) Student has supplementary reader available today 52.0 (655) 59.8 (405) 50.8 (690) 48.1 (603) 53.8 (341) 48.9 (658) Student has Portuguese textbook available today 82.1 (1027) 84.0 (542) 79.2 (1016) 74.2 (1011) 82.8 (509) 71.7 (975) Student has exercise notebook available today 84.4 (1169) 83.5 (538) 85.7 (1143) 94.9 (1265) 93.1 (581) 92.6 (1219) Student reads books daily in the classroom or school library 42.7 (524) 58.3 (357) 47.5 (587) 52.8 (681) 62.4 (386) 55.4 (689) 3-14 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report The percentage of Grade 2 students who reported attending an afterschool learning club was 16 18% across the study arms (Table 18). Reading clubs, homework clubs, and math clubs were the most common afterschool learning activities reported by students across the study arms. When asked if they had received a certificate or award in school to celebrate progress in reading, 8% of Grade 2 students responded affirmatively. The percentage of Grade 4 students who reported attending an afterschool learning club was 18 25% across the study arms. Reading clubs, homework clubs, and math clubs were the most common afterschool learning activities reported by students across the study arms. Among the Grade 4 students who reported attending an afterschool learning club, approximately 29 34% said they attended a reading club, 19 32% said they attended a homework club, and 17 32% said they attended a math club. When asked if they had received a certificate or award in school to celebrate progress in reading, 10 12% of Grade 4 students responded affirmatively. Table 18. Grade 2 Grade 4 Comparison School Feeding Only Literacy +& School Feeding Comparison School Feeding Only Literacy & School Feeding n=1,394 n=666 n=1,368 n=1,357 n=616 n=1,323 % (n) % (n) % (n) % (n) % (n) % (n) Attends afterschool learning club 17.2 (183) 16.2 (107) 18.6 (228) 20.3 (254) 18.7 (130) 25.2 (297) Type of afterschool learning club student attends Math club 19.0 (35) 21.5 (23) 18.4 (38) 17.5 (40) 32.1 (41) 19.5 (52) Science club 2.0 (2) 1.0 (1) 1.5 (2) 12.3 (21) 14.1 (16) 10.1 (21) Gardening club 0.9 (1) 1.3 (1) 0.0 (0) 1.3 (2) 0.4 (1) 0.4 (1) Reading club 23.3 (46) 35.7 (33) 35.9 (65) 29.4 (74) 33.7 (43) 34.1 (85) Homework club 35.4 (52) 15.8 (20) 9.7 (22) 32.7 (71) 27 (34) 19.1 (56) Other 1.6 (6) 10.3 (14) 13.4 (30) 5.1 (14) 13.3 (20) 13.7 (43) Student received a certificate or award in school to celebrate progress in reading 8.4 (89) 8.8 (57) 8.7 (105) 10.6 (115) 16.2 (104) 12 (158) The percentage of Grade 2 students who reported that the teacher uses Portuguese as the language of instruction was 83 88% across the study arms (Table 19). The percentage of students in each study arm who reported that the teacher uses another language of instruction (Macua or Changana) was 31 41%. The percentage of Grade 4 students who reported that the teacher uses Portuguese as the language of instruction was 91 96% across the study arms. The percentage of students in each study arm who reported that the teacher uses another language of instruction (Macua or Changana) was 27 39%. Given the geographical spread of local languages, Table 19 also USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-15 shows that 0% of students in the School Feeding Only study arm reported that the teacher uses Macua to teach reading since students in this study arm are located in a region of Mozambique where Portuguese and other mother tongues are spoken. When asked what the teacher does when the student is unable to answer a question, the most common responses among both Grade 2 and Grade 4 students across study arms were Table 19. onal Support Received, by Grade and Study Arm Grade 2 Grade 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding n=1,394 n=666 n=1,368 n=1,357 n=616 n=1,323 % (n) % (n) % (n) % (n) % (n) % (n) Student￾reported language teacher uses to teach reading in class Portuguese 88.7 (1229) 87.3 (561) 83.9 (1148) 96.2 (1297) 96.6 (591) 91.2 (1222) Rhonga 0.1 (6) 0.7 (4) 0.5 (25) 0.2 (9) 0.3 (2) 0.5 (17) Changana 5.9 (101) 37.6 (309) 16.2 (266) 3.0 (60) 32.5 (227) 13.2 (196) Macua 31.7 (515) 0.0 (0) 41.5 (497) 27.5 (439) 0.0 (0) 39.1 (495) Other 3.3 (27) 0.1 (2) 0.3 (3) 0.6 (9) 0.0 (0) 0.8 (6) According to student, what teacher does when child is unable to answer a question: Teacher rephrases/ explains question 14.7 (207) 13.8 (72) 11.4 (210) 26.2 (327) 19.1 (105) 19 (272) Teacher provides example 14.0 (139) 7.5 (45) 5.9 (100) 14.7 (153) 8.6 (61) 8.0 (134) Teacher encourages child to try again 6.6 (84) 2.7 (16) 5.9 (91) 8.5 (104) 6.1 (37) 7.0 (100) Teacher asks another student 6.9 (95) 6.1 (31) 6.6 (87) 8 (92) 4.3 (20) 8.9 (101) Teacher asks again 7.5 (87) 5.0 (33) 6.3 (94) 8.4 (101) 9.4 (54) 8 (105) Teacher corrects student 8.1 (107) 10.3 (70) 7.2 (92) 11 (134) 18.4 (110) 11 (162) Teacher assigns a student helper 5.1 (69) 3.2 (10) 4.2 (52) 4.8 (54) 1.8 (11) 4.9 (59) Teacher hits the student 30.0 (441) 40.5 (294) 35.1 (505) 23.8 (379) 37.6 (250) 34.0 (474) Other 4.6 (54) 6.9 (35) 5.5 (74) 5.3 (68) 5.6 (33) 8.8 (94) Home Literacy Practices study arms (Table 20). Slightly more than half of Grade 2 students reported that someone at home helps with homework when needed. The percentage of Grade 2 students who said they bring home reading books from the classroom was 64 72% across the study arms. Among the Grade 2 students who reported that they bring home reading books from the 3-16 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report classroom, approximately 92 96% reported that the books are in Portuguese. When asked if they bring home books from the school library or book bank, the rate of affirmative responses was 12 14%. Among the Grade 2 students who reported that they bring home books from the school library or book bank, roughly 80 98% reported that the books are in Portuguese. Grade 2 students in all study arms reported that a language other than Portuguese is the most widely used at home. When asked about how often they read at home, a higher percentage of Grade 2 students in the Comparison study arm (46%) and Literacy & School Feeding study arm (49%) responded per 49% across the study arms. comparison and both intervention study arms. Slightly more than half of Grade 4 students reported that someone at home helps with homework when needed. The percentage of students who said they bring home reading books from the classroom was approximately 68 76% across the study arms. Among the students who said they bring home reading books from the classroom, across the study arms, roughly 96 98% also reported that the books are in Portuguese. When asked if they bring home books from the school library or book bank, the rate of affirmative responses across the study arms was 10 18%. Among the students who reported that they bring home books from the school library or book bank, approximately 95 100% reported that the books are in Portuguese. Grade 4 students reported that a language other than Portuguese is the most widely used at home. Table 20. Grade 2 Grade 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding n= 1,394 n= 666 n=1,368 n=1,357 n=616 n=1,323 % (n) % (n) % (n) % (n) % (n) % (n) Someone at home helps with homework when needed 62.0 (741) 58.6 (328) 52.4 (683) 61.2 (747) 59.5 (335) 57.7 (756) Student brings home reading books from the classroom 64.9 (816) 65.2 (424) 72 (918) 68.4 (908) 74.8 (465) 76.8 (1000) Language of classroom books brought home Portuguese 96.4 (705) 96.6 (384) 92.5 (785) 98.1 (853) 98.7 (456) 96.2 (938) Rhonga 0.3 (3) 0.0 (0) 0.4 (6) 0.1 (3) 0.0 (0) 0.0 (0) Changana 0.6 (9) 3.1 (16) 2.1 (17) 0.1 (1) 1.2 (6) 0.0 (2) Other language 0.3 (3) 0.1 (1) 0.2 (2) 0.6 (4) 0.1 (1) 0.3 (2) Brings home books from the school library/book bank 12.3 (137) 14.8 (91) 12.5 (128) 10.5 (111) 18.0 (88) 13.6 (135) USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-17 Grade 2 Grade 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding Language of school library/book bank books brought home Portuguese 98.6 (129) 98.4 (85) 80.7 (104) 100 (110) 97.6 (84) 95.4 (126) Rhonga 0.8 (1) 0.0 (0) 0.3 (2) 0.0 (0) 0.0 (0) 0.4 (1) Changana 0.3 (1) 1.6 (3) 8.1 (7) 0.0 (0) 1.6 (2) 0.7 (3) Macua 0.3 (1) 0.0 (0) 9.2 (11) 0.0 (0) 0.0 (0) 3.5 (6) Other 0.0 (0) 0.0 (0) 1.7 (1) 0.0 (0) 0.7 (1) 0.0 (0) Language child speaks at home Portuguese 38.1 (357) 21.7 (81) 12.1 (146) 40.2 (365) 23.7 (83) 10.7 (123) Rhonga 0.7 (20) 1.2 (10) 0.9 (43) 0.9 (22) 0.6 (5) 1.1 (51) Changana 17.7 (321) 74.8 (552) 31.8 (477) 16.2 (304) 75.6 (522) 34.4 (471) Macua 43.1 (652) 0.0 (0) 54.8 (658) 42.0 (636) 0.0 (0) 53.1 (652) Other 0.5 (7) 2.2 (13) -.5 (18) 0.7 (14) 0.1 (1) 0.7 (21) Frequency student reads on his/her own at home Never 46.3 (662) 38.4 (273) 49.8 (675) 31.9 (436) 26 (152) 29.7 (405) Sometimes 43.7 (547) 50.5 (283) 40.4 (519) 53.5 (676) 54.5 (339) 53.9 (705) Every day 9.9 (95) 11.0 (67) 9.8 (85) 14.6 (193) 19.5 (105) 16.5 (165) Frequency someone tells the student a story at home Never 49.3 (683) 46.4 (339) 48.1 (634) 44.3 (594) 42.2 (278) 43.6 (538) Sometimes 41.9 (547) 44.4 (265) 46.3 (599) 50.4 (644) 51.7 (289) 49.2 (662) Every day 8.8 (79) 9.2 (40) 5.6 (64) 5.3 (61) 6.2 (37) 7.3 (83) When asked how often they read on their own at home, the percentage of Grade 4 students 54%. When asked how often someone at home tells them a story, the percentage of Grade 4 students who responded Nearly all students in schools with School Feeding Only and Literacy & School Feeding reported that their school serves school meals, whereas approximately 7% of students in Comparison schools said that their school serves school meals (Table 21). Among the students reporting school meals, approximately three quarters of those in schools with School Feeding Only and Literacy & School Feeding interventions said their school served a meal yesterday or on the previous day they were in school. Porridge was the most common meal in schools with School Feeding Only or Literacy & School Feeding, whereas Comparison schools with a feeding program mainly served food other than porridge. Of the students who had school meals served at their schools, more than half in some cases, more than two thirds said they ate fewer meals at home when they received a school meal. Approximately 80 90% of students said they felt less hungry on days when they received a school meal. We asked students if they felt hungry at the time of the interview, and affirmative responses ranged from 31% to 56% across study arms and grades. The percentage of students who 3-18 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report said their household had experienced hunger since the beginning of the school year was 8 21% across all study arms. Among those reporting household hunger, roughly 12 34% said it occurred every month. Table 21. Student Responses on School Meals, Current Hunger, and Household Hunger, by Grade and Study Arm GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding n=1,386 n=666 n=1,368 n=1,349 n=616 n=1,323 % (n) % (n) % (n) % (n) % (n) % (n) School serves school meals 6.9 (114) 98.5 (645) 96.7 (1317) 7.6 (111) 98.8 (605) 97.7 (1292) School meal served yesterdaya 58.9 (79) 79.7 (496) 74.6 (998) 66.9 (83) 74.8 (457) 71.0 (985) Food served at school meal yesterdaya Porridge 28.5 (19) 99.4 (496) 96.9 (980) 38.6 (17) 98.9 (443) 97.1 (941) Porridge & other foods 2.2 (2) 0.2 (1) 1.4 (7) 4.0 (4) 0.4 (2) 1.0 (6) Other foods only 69.3 (56) 0.4 (2) 1.7 (28) 57.4 (62) 0.7 (2) 1.9 (28) Eats fewer meals at home when receives a school meal 54.1 (60) 69.6 (438) 46.0 (677) 63.0 (68) 69.2 (415) 52.7 (688) Feels less hungry on days when receives a school meal 89.0 (104) 90.9 (600) 83.1 (1107) 89.4 (102) 91.1 (556) 84.9 (1096) Feels hungry nowb 55.9 (755) 40.3 (231) 51.9 (725) 46.1 (660) 30.8 (191) 39.9 (551) Household hunger 21 (273) 14.7 (94) 19 (243) 20.1 (243) 7.5 (48) 17.1 (216) Household hunger frequency Every month 28.7 (50) 15.3 (9) 16.5 (31) 26.7 (57) 12.2 (4) 33.5 (45) During the rainy season 20.3 (38) 12.2 (8) 15.4 (31) 13.2 (35) 15.2 (4) 14.4 (31) Other 50.9 (92) 72.6 (53) 68.1 (99) 60.0 (94) 72.6 (27) 52.0 (87) aThis is question was asked only if students reported that their school serves school meals. bWe used Bennett & Blissett (2014) to develop this question. Student Nutrition Knowledge and Practices -report, varied across study arms (Table 22). Higher percentages of Literacy & School Feeding (approximately 74%) and Comparison (approximately 69%) schools than School Feeding Only (approximately 54%) schools had achieved minimum dietary diversity on the previous day. The food groups consumed by the USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-19 fewest students were dairy products (18.5%) and eggs (23.5%), whereas grains, roots, and tubers were consumed by nearly all students (98.5%). Approximately 40 60% of students across the study arms and grades achieved minimum meal frequency on the previous day. Achievement of minimum acceptable diet was low: 31 50% across the study arms. On average, students had eaten four food groups and two to three meals on the previous day. ow. On average, students correctly answered one out of eight nutrition knowledge questions. Table 22. Student Nutrition Practices and Knowledge, by Grade and Study Arm GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding n=1,386 n=666 n=1,368 n=1,349 n=616 n=1,323 % (n) % (n) % (n) % (n) % (n) % (n) Minimum dietary diversitya 67.7 (905) 53.5 (356) 71.8 (899) 71.6 (930) 55.7 (348) 76.1 (937) Minimum meal frequencyb 42.8 (545) 55.3 (332) 57.4 (703) 56.9 (688) 57.5 (331) 62.4 (795) Minimum acceptable dietc 34.1 (419) 30.5 (186) 46 (516) 45.2 (537) 36.0 (211) 49.6 (596) Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Dietary diversity 4.2 [±.1] 3.7 [±.2] 4.2 [±.1] 4.3 [±.1] 3.8 [±.2] 4.4 [±.1] Meal frequency 2.4 [±.1] 2.6 [±.1] 2.7 [±.1] 2.6 [±.1] 2.7 [±.1] 2.8 [±.1] Nutrition knowledge score (8 items) 0.7 [±.1] 0.4 [±.1] 0.4 [±.1] 1.3 [±.1] 1.0 [±.2] 0.7 [±.1] Note: CI = confidence interval aMinimum dietary diversity is four or more of seven food groups (grains, roots, and tubers; legumes and nuts; dairy products; flesh foods; eggs; vitamin A-rich fruits and vegetables; and other fruits and vegetables). bMinimum meal frequency is three or more meals per day. cMinimum acceptable diet comprises individuals who have both minimum dietary diversity and minimum meal frequency. Student Hygiene and Sanitation Knowledge and Practices Among students in schools that provide school meals, less than half the students across all study arms said that they always wash their hands before eating their school meal (Table 23 to 3.5 items out of seven, on average. Grade 2 students had lower hygiene and sanitation knowledge than Grade 4 students. 3-20 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Table 23. Student Hygiene and Sanitation Practices and Knowledge, by Grade and Study Arm GRADE 2 GRADE 4 Comparison School Feeding Only Literacy & School Feeding Comparison School Feeding Only Literacy & School Feeding n=1,386 n=666 n=1,368 n=1,349 n=616 n=1,323 % (n) % (n) % (n) % (n) % (n) % (n) Frequency of washing hands with soap or ash before school meal Never 7.4 (7) 3.1 (28) 15.4 (158) 6.9 (5) 4.4 (27) 11.2 (126) Sometimes 44.6 (47) 54.8 (340) 49.1 (559) 45.8 (51) 47.0 (294) 45.4 (539) Always 48.1 (45) 42.1 (205) 35.5 (411) 47.3 (48) 48.6 (235) 43.4 (501) Mean ± 95% CI Band Mean ± 95% CI Band Mean ± 95% CI Band Mean ± 95% CI Band Mean ± 95% CI Band Mean ± 95% CI Band Hygiene and sanitation knowledge score (7 items) 2.9 [±.1] 2.6 [±.2] 2.5 [±.1] 3.5 [±.1] 3.5 [±.2] 3.1 [±.1] Note: CI = confidence interval. 3.2.2 Teachers Teacher Profile Sixty percent of teachers interviewed in the Comparison study arm were female (Table 24), whereas less than half of teachers in the School Feeding Only and the Literacy & School Feeding study arms were female. The percentage of teachers who reported that they taught reading in Portuguese was 98 99% across study arms. The percentage of teachers who said they had attended and finished studies at a teacher training college (TTC) was 95 96% across the study arms. The percentage of teachers who reported they had received preservice training on the teaching of reading and writing to students in the primary grades was 58 61% across the study arms. The percentage of teachers who reported attaining certification was 95 97% across groups. Less than 20% of teachers across the study arms attended an Ajuda de Desenvolvemento de Povo para Povo (Development Aid from People to People [ADPP])-supported TTC. The percentage of teachers who reported the certificate level as the highest level of academic education attained was 64 71% across the study arms. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-21 Table 24. Teacher Profiles, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=331 n=163 n=312 % (n) % (n) % (n) Female teacher 60.4 (153) 36.8 (53) 47.8 (124) Languages in which reading is taught Portuguese 99.7 (300) 98.2 (144) 98.8 (288) Rhonga 9.1 (19) 2.4 (4) 3.6 (16) Changana 15.6 (47) 60.4 (93) 29.1 (74) Macua 28.2 (112) 0.0 (0) 39.9 (122) Other 0.0 (0) 0.0 (0) 1.6 (7) Attendance of TTC Did not attend TTC 1.8 (7) 1.8 (3) 1.4 (3) Still attends TTC 1.9 (3) 3 (3) 3.0 (6) Attended and finished TTC 96.3 (292) 95.3 (141) 95.5 (283) TTC teacher attended IFP 77.5 (232) 87.9 (127) 78.5 (224) ADPP 13.3 (39) 9.7 (12) 16.3 (47) UP 9.2 (23) 2.4 (4) 5.3 (11) Attainment of certification 97.4 (291) 97 (142) 95.7 (282) Teacher is a currently trainees expected to graduate from a TTC 7.7 (20) 4.7 (8) 6.1 (17) TTC teacher is expected to graduate from IFP 21.5 (5) 62.5 (5) 76.6 (14) ADPP 2.5 (1) 0.0 (0) 0.0 (0) UP 76.1 (12) 37.5 (3) 23.4 (5) Highest level of academic education Form 6 (Basic) 18.8 (78) 23.1 (37) 25.0 (76) Certificate 64.4 (192) 71.6 (105) 69.3 (199) Diploma 1.4 (2) 0.0 (0) 2.1 (6) Advanced diploma 15.4 (30) 5.3 (5) 3.4 (10) Other 0.0 (0) 0.0 (0) 0.2 (1) During pre-service training, the teacher received specific training on how to teach reading and writing to students in the primary grades 61.4 (190) 60.9 (93) 58.9 (175) Note: IFP = Instituto de Formação de Professores (Teacher Training Institute); TTC = teacher training college; UP = Universidade Pedagógica de Moçambique Teacher Training The percentage of teachers who reported that they attended an in-service training on how to teach reading in the primary grades varied across the three study arms (Table 25). Fifty￾two percent of teachers in the Literacy & School Feeding arm, 38% of teachers in the School Feeding Only arm, and 26% of teachers in the Comparison arm reported that they attended an in-service training on how to teach reading in the primary grades. The percentages of teachers who attended a WV or PAI training were 70% in the Literacy & School Feeding arm, 3-22 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 63% in the School Feeding Only arm, and 10% in the Comparison arm. When asked if they received training on the fundamentals of reading instruction, the percentage of affirmative responses was 57 64% across study arms. The most common activities in which teachers participate during in-service training were analyzing the student exercise book, learning how to plan and prepare for a lesson, and learning how to assess students during the lesson. Table 25 Comparison School Feeding Only Literacy & School Feeding n=331 n=163 n=312 % (n) % (n) % (n) This year, the teacher attended any in-service training on how to teach reading in the primary grades 26.2 (87) 38.1 (54) 52.0 (157) Teacher attended WV/PAI training 10.9 (29) 63.1 (90) 70.3 (212) Teacher received training on the fundamentals of reading instruction 57.5 (174) 58.0 (91) 64.0 (190) Types of activities teacher participates in during in￾service training Analysis of curriculum and syllabus 35.1 (109) 47.3 (68) 25.9 (88) Analysis of teacher guide 22.9 (71) 29 (42) 19.5 (63) Practice teaching lessons from the teacher guide 23.4 (63) 22.5 (33) 19.5 (52) Analysis of the student exercise book 35.9 (114) 61.5 (89) 42.3 (131) How to plan and prepare for a lesson 64.6 (208) 81.1 (121) 67.1 (204) Improvisation and use of teaching and learning aids 35.2 (126) 55.6 (80) 42.0 (134) How to assess students during the lesson 41.7 (147) 65.1 (100) 54.8 (169) How to use assessments to adjust instruction 23.1 (80) 40.8 (63) 30.7 (100) How to give students feedback 22.6 (67) 42.6 (66) 28.8 (98) How to use supplementary readers/decodable texts 25.2 (87) 29.6 (48) 22.0 (72) How to apply new strategies for reading instruction 26.8 (80) 33.7 (52) 28.2 (88) Teacher Absenteeism Across study arms, 21 28% of teachers were absent from school during the last week, and 13 17% of teachers were late to school during the last week (Table 26). USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-23 Table 26. Teacher-reported Absenteeism, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=331 n=163 n=312 % (n) % (n) % (n) Teacher absent from school any day last week 20.7 (57) 27.8 (43) 24.3 (73) Teacher late to school any day last week 15.7 (46) 17.2 (26) 13.2 (46) Note: CI = confidence interval. Teacher Access to School Supplies and Learning Materials The percentage of teachers who reported that they have access to a teacher guide was 51 58% across study arms (Table 27). A higher percentage of teachers in the School Feeding Only study arm (72%) reported that students have exercise books that can be used during reading instruction than in the Comparison (58%) and Literacy & School Feeding (65%) study arms. Higher percentages of teachers in the School Feeding Only arm (95%) and the Literacy & School Feeding arm (95%) reported having access to school supplies than in the Comparison arm (87%). Among the teachers who reported having access to school supplies, most reported access to supplies such as pens/pencils, notebooks, and chalk. Teachers across the study arms reported low access to school supplies such as paint, markers, and chart paper. The percentage of teachers across the study arms who reported access to supplementary readers was 72 88%. Table 27. Teacher Access to Supplies and Learning Materials, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=331 n=163 n=312 % (n) % (n) % (n) Teacher has a teacher guide for reading instruction 51.8 (156) 51.5 (76) 58.2 (162) Students have exercise books that can be used during reading instruction 58.2 (164) 72.0 (107) 65.7 (192) Teacher has access to school supplies 87.6 (270) 95.2 (138) 95.7 (273) Supplies the teacher can access Pens/pencils 91.9 (243) 93.1 (129) 88.3 (251) Notebooks 88.7 (239) 88.7 (123) 83.8 (229) Chalk 83.8 (225) 75.6 (104) 84.3 (234) Cardboard 73.6 (196) 78.1 (106) 66.7 (188) Paint 7.1 (26) 31.9 (48) 13.9 (43) Markers 9.7 (31) 25.6 (38) 21.9 (66) Chart paper 27.1 (80) 45.6 (67) 40.2 (118) Other 12.5 (42) 10.0 (15) 16.5 (54) Supplementary readers available for use in this classroom 72.7 (51) 88.7 (50) 78.0 (105) 3-24 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Teacher Use of School Supplies and Learning Materials The percentage of teachers who said they used the teacher guide for reading instruction varied across the study arms (Table 28). Fifty-two percent of teachers in the Comparison arm, 48% of teachers in the School Feeding Only arm, and 60% of teachers in the Literacy & School Feeding arm reported that they used the teacher guide for reading instruction. The percentage of teachers who reported that students use the exercise books during reading instruction was 75 86% across the study arms. The percentage of teachers who reported using school supplies during reading instruction was 91 97% across the study arms. Of the teachers who said that they used school supplies during reading instruction, most reported using supplies such as pens/pencils, notebooks, chalk, and cardboard. Teachers across the study arms reported low use of school supplies such as paint, markers, and chart paper. Among the teachers who reported using school supplies during reading instruction, most indicated that they used supplementary readers once a week. Table 28. Teacher Use of School Supplies and Learning Materials, by Study Arm Comparison School Feeding Only Literacy & School Feeding Count n=331 n=163 n=312 Estimates % (n) % (n) % (n) Teacher uses the teacher guide for reading instruction 52.9 (163) 48.2 (74) 60.8 (168) Student use the exercise books during reading instruction 75.3 (218) 86.9 (126) 83.2 (239) Teacher uses school supplies during reading instruction 91.2 (283) 95.2 (138) 97.1 (280) Teacher uses the following school supplies Pens/pencils 69.9 (202) 68.1 (91) 74.2 (205) Notebooks 71.0 (202) 65.0 (87) 65.5 (185) Chalk 77.6 (219) 80.0 (111) 80.2 (229) Cardboard 79.9 (231) 83.1 (115) 70.7 (203) Paint 12.3 (35) 20.6 (31) 11.7 (36) Markers 10.1 (28) 15.0 (22) 17.0 (49) Chart paper 28.3 (78) 49.4 (72) 39.8 (113) Other 11.6 (40) 11.9 (19) 18.1 (61) Frequency students use supplementary readers in this classroom Never 2.2 (3) 4.8 (3) 6.6 (11) Once a week 72.7 (51) 88.7 (50) 78.0 (105) Every day 25.1 (21) 6.5 (4) 15.4 (26) Extracurricular Activities The percentage of teachers who reported that special events are held at the school to 49% across the study arms (Table 29). Sixty￾two percent of teachers in the Comparison arm, 79% of teachers in the School feeding Only arm, and 48% of teachers in the Literacy & School Feeding arm reported that students received a certificate or award in school to celebrate their progress in reading. Nineteen USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-25 percent of teachers in the Comparison arm, 19% of teachers in the School Feeding Only arm, and 30% of the teachers in the Literacy & School Feeding arm reported that special events those, lower percentages of teachers in Literacy & School Feeding (45%) and School Feeding Only (67%) groups reported that students in their class had received a certificate or award in school to celebrate improvement in attendance than in the Comparison arm (82%). The percentage of teachers who reported the existence of an afterschool club in the school varied across the three study arms. Seventeen percent of teachers in Comparison schools, 40% of teachers in School Feeding Only schools, and 60% of teachers in Literacy & School Feeding schools reported that an afterschool club exists in the school. Among the teachers who reported an afterschool literacy club in the school, most teachers across the study arms said that half the students in their class attend an afterschool literacy club. Table 29. on Extracurricular Activities, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=331 n=163 n=312 % (n) % (n) % (n) Special events held at this school to celebrate students' progress in reading 33.9 (95) 48.5 (71) 49.3 (131) Students in this class received a certificate or award in school to celebrate progress in reading 62.9 (65) 79.5 (56) 48.6 (71) Special events held at this school to celebrate students' school attendance 19.3 (59) 19.1 (28) 30.5 (82) Students in this class received a certificate or award in school to celebrate improvement in attendance 81.7 (43) 67.7 (19) 45.5 (40) Afterschool literacy club in this school 17.4 (37) 40.7 (60) 60.5 (167) Number of students who attend an afterschool literacy club in this class None 3.3 (1) 5.5 (2) 3.1 (3) Half 61.2 (7) 60.0 (27) 73.0 (98) More than half 29.2 (4) 20 (10) 12.6 (19) All 6.3 (2) 14.5 (8) 11.2 (16) Teacher Perspectives on Supervision and Support The percentage of teachers who reported that the HT or assistant HT checked lesson plans was 97 98% across the study arms (Table 30). Among those teachers, higher percentages of teachers in the School Feeding Only and Literacy & School Feeding arms responded that the HT checks lesson plans once every week, whereas a higher percentage of teachers in the Comparison arm responded that the HT checks lesson plans daily. When asked whom they 3-26 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report consult when help is needed, in each study arm, most teachers reported that they discuss issues casually with other teachers. When asked about the frequency with which classroom observations have been conducted by the assistant HT or HT since the beginning of the school year, in all study arms, most teachers responded that classroom observations occurred once per trimester. The percentage of teachers who reported that they received feedback from the HT or assistant HT was 92 96% across the study arms. In all study arms, higher percentages of teachers since the beginning of the school year. Table 30. Teacher Perspectives on Supervision and Support, by Study Arm Comparis on School Feeding Only Literacy & School Feeding n=331 n=163 n=312 % (n) % (n) % (n) Does the HT or assistant HT check lesson plans 98.8 (294) 98.8 (144) 97.2 (278) Frequency at which HT checks lesson plans Once every 2 3 months 3.8 (11) 6.3 (6) 10.5 (25) Once every month 10.3 (29) 11.3 (17) 11.6 (36) Once every 2 weeks 14.0 (42) 15.7 (23) 12.4 (34) Once every week 30.3 (82) 39.0 (56) 39.7 (107) Daily 41.6 (126) 27.7 (40) 25.8 (73) Person the teacher consults when help is needed Never need help 0.6 (2) 0.6 (1) 0.4 (2) There is no one to ask for help. 1.2 (6) 0.6 (1) 0.0 (0) Discuss casually with other teachers 78.0 (226) 78.7 (112) 72.8 (211) Discuss at meetings with other teachers 37.6 (108) 34.3 (52) 33.8 (96) Reading coach 1.1 (4) 1.8 (2) 1.2 (4) Assistant HT 31.6 (97) 30.2 (45) 37.5 (114) HT 8.8 (36) 26.6 (41) 13.3 (48) Since the beginning of the school year, frequency of classroom teaching observations by the assistant HT or HT Never 16.1 (37) 13.7 (15) 6.2 (21) Once per trimester 50.4 (163) 44 (64) 56 (154) Monthly 20.7 (51) 25.6 (41) 28 (78) Weekly 12.8 (46) 16.7 (26) 9.8 (35) Provision of feedback from HT or assistant HT 92.6 (236) 94.4 (123) 96.6 (254) Since the beginning of the school year, frequency of reading coach visits Never 82.9 (225) 72.0 (101) 61.1 (169) Once per trimester 11.3 (33) 14.0 (19) 15.9 (46) Monthly 5.1 (16) 7.0 (7) 12.6 (29) Weekly 0.8 (4) 7.0 (10) 10.4 (32) Note: HT = head teacher. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-27 Teacher Knowledge and Skills When asked if they could name the core reading skills, no teacher in any study arm reported being able to do so (Table 31). However, the percentage of teachers who named letter knowledge as a core reading skill was 58 66% across study arms. Less than half of HTs across the study arms named phonemic awareness, fluency, or comprehension as core reading skills. ading and writing progress, more than half of the teachers across the study arms reported using written tests and oral evaluations. When asked about how they check for student understanding during lessons, more than half of the teachers across the study arms reported asking questions of individual students. When asked about the techniques used to teach a new vocabulary word, teachers across the study arms gave varied responses. More teachers in the Comparison arm (54%) reported providing a verbal explanation, whereas a higher percentage of teachers of teachers in the School Feeding Only arm (49%) reported using real objects. Finally, a higher percentage of teachers in the Literacy & School Feeding arm (55%) reported using pictures. Table 31. Teacher Knowledge and Skills, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=331 n=163 n=312 % (n) % (n) % (n) Teacher can name the core reading skills 0.0 (0) 0.0 (0) 0.0 (0) The reading skills the teacher can name Letter knowledge 64.8 (195) 58.6 (89) 66.6 (195) Phonemic awareness 34.5 (98) 25.4 (38) 34.1 (112) Fluency 23.1 (65) 15.4 (23) 21.3 (72) Vocabulary 35.1 (114) 37.3 (57) 51.5 (146) Comprehension 43.1 (128) 37.3 (57) 35.3 (101) Grade level at which the teacher expects children to read text in the target language fluently Grade 1 7.1 (15) 5.9 (9) 4.5 (13) Grade 2 37.6 (102) 32.0 (45) 22.0 (57) Grade 3 25.1 (70) 27.2 (44) 42.4 (115) Grade 4 or higher 30.1 (111) 34.9 (49) 31.2 (94) Grade level at which the teacher expects students to write a coherent and comprehensible short story correctly Grade 1 4.3 (6) 3.6 (5) 2.5 (8) Grade 2 23.1 (62) 20.8 (31) 17.1 (45) Grade 3 28.4 (82) 29.2 (47) 35.2 (87) Grade 4 or higher 44.3 (144) 46.4 (63) 45.2 (142) Ways the teacher measures students' reading and writing progress Written tests 64.1 (198) 60.9 (93) 70.0 (199) Oral evaluations 66.7 (211) 66.9 (95) 62.1 (181) Observation 40.6 (122) 34.3 (51) 40.6 (126) Portfolios and other projects 4.1 (16) 5.9 (10) 4.4 (18) Homework 36.8 (111) 31.4 (45) 34.9 (95) 3-28 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Comparison School Feeding Only Literacy & School Feeding Classwork 31.1 (99) 30.2 (45) 32.4 (91) End-of-unit evaluation 6.0 (17) 5.3 (9) 5.5 (17) End-of-term evaluation 2.6 (11) 3.0 (4) 5.8 (16) Ways the teacher checks for student understanding during the lesson Ask questions of individual students 57.7 (187) 56.8 (81) 56 (174) Ask questions of whole class 46.4 (141) 40.8 (57) 38.9 (107) Ask questions of students in groups 25.3 (76) 25.4 (35) 25.6 (80) Give students a task and correct the written responses before the end of the lesson 32.6 (118) 43.8 (65) 36.3 (102) Give students a task and correct the written responses after the end of the lesson 21.2 (62) 35.5 (47) 25.6 (71) Other 7.7 (23) 7.1 (12) 8.8 (29) Ways the teacher uses the results of assessments in instruction Grade students 54.1 (171) 47.3 (74) 46.9 (135) subject matter 59.8 (178) 56.8 (81) 53.5 (159) Plan teaching and learning activities 23.3 (76) 27.2 (37) 29.1 (82) needs 21.3 (66) 36.7 (50) 21.4 (73) Arrange students in ability groups 4.7 (16) 3.6 (6) 7.0 (19) Other 12.0 (34) 3.6 (6) 5.1 (18) Techniques teacher uses to teach a new vocabulary word Pictures 40.6 (140) 43.2 (65) 55 (156) Realia 40.1 (131) 49.7 (72) 47.1 (141) Gestures 26.1 (77) 23.7 (38) 20.4 (75) Verbal explanation 54.9 (158) 49.1 (74) 51.9 (161) Other 15.7 (58) 13.6 (21) 12.9 (42) Frequency of teacher switching between official language of instruction and a vernacular language during the teaching and learning process Never 15.0 (29) 6.6 (8) 3.7 (8) Occasionally 58.3 (185) 63.5 (94) 67.8 (189) Often 21.1 (66) 22.2 (32) 18.8 (65) All of the time 5.6 (20) 7.8 (12) 9.7 (28) reading skills in the target language of instruction Weak 12.8 (47) 13.7 (18) 11.8 (39) Average 75.2 (223) 75.6 (112) 81.4 (231) Strong 8.9 (25) 9.5 (14) 6.8 (21) Not applicable does not teach in the target language 3.1 (5) 1.2 (2) 0.0 (0) writing skills in the target language of instruction: Weak 11.9 (39) 12.4 (20) 9.2 (30) Average 71 (224) 72.8 (106) 73.5 (215) Not applicable does not teach in the target language 13.9 (35) 13.6 (19) 17.1 (46) The percentage of teachers across the study arms who reported that they occasionally switch between the official language of instruction and a local language was 58 67%. Across USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-29 the study arms, 75 average. Similarly, average was 71- 73%. Interestingly, while students were generally not competent in basic performance is average. When asked at what grade level the teacher expects children to read text in the target language fluently, varied. A higher percentage of teachers in the Comparison arm (37%) reported that by Grade 2, students should be able to read text in the target language fluently, whereas relatively more teachers in the School Feeding Only arm (34%) reported that students should be capable of this skill by Grade 4. In contrast, a higher percentage of teachers in the Literacy & School Feeding arm (42%) reported that students should be able to read text in the target language fluently by Grade 3. The percentage of teachers across the study arms who reported that students should be able to write a coherent and comprehensible short story correctly by Grade 4 or higher was 44 46% across study arms. Across the study arms, w . In addition, we asked teachers in the School Feeding Only and Literacy & School Feeding The percentage of teachers who reported that more than half of students are generally attentive in class was 62 66% across the study arms (Table 32). Teachers most often determined that students are attentive in class when students are seated facing and making eye contact with the teacher. In School Feeding Only and Literacy & School Feeding schools, less than 40% of the teachers said that the school meal makes their students more attentive, approximately 40% said the meal does not change student attentiveness, and approximately 20% said it makes their students less attentive. Table 32. Teacher Perceptions of Student Attentiveness, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=331 n=163 n=312 % (n) % (n) % (n) Students are attentive in class 91.8 (282) 97.0 (143) 96.1 (278) Approximate number of students who are generally attentive in this class Less than half 7.7 (23) 4.3 (7) 9.1 (24) Half 20.4 (64) 5.5 (8) 12.0 (37) More than half 63.4 (170) 66.9 (93) 62.6 (173) All 8.5 (22) 23.3 (34) 16.3 (43) Ways teacher gauges if students are attentive in class Students are seated facing and making eye contact with the teacher. 55.8 (162) 59.8 (86) 47.9 (141) Students actively listen and act upon teacher directions. 38.1 (137) 41.4 (59) 37.4 (122) 3-30 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Comparison School Feeding Only Literacy & School Feeding Students are working independently, in pairs, or in small groups on the current learning activity. 14.8 (46) 14.8 (24) 13.1 (39) Students respond to questions posed by the teacher. 43.1 (139) 42.6 (67) 43.1 (132) Students ask the teacher or another student for help related to the current learning activity. 14.6 (49) 10.1 (15) 18.7 (53) Students ask the teacher questions related to the current learning activity. 16.8 (51) 13.6 (21) 13.0 (41) Students are reading. 2.6 (11) 8.3 (10) 6.3 (18) Students are writing. 4.7 (20) 4.1 (7) 4.2 (15) Students are listening to a story. 2.1 (8) 0.6 (1) 3.5 (11) Other 9.0 (27) 3.0 (5) 6.1 (19) Student attentiveness in relation to the school meal Students are more attentive after the school meal. - 37.2 (55) 36.6 (108) Students are less attentive after the school meal. - 21.3 (28) 22.7 (65) Students show the same attentiveness before and after lunch - 41.5 (59) 40.7 (104) School Conditions and relationships both between the school leadership and the community and among students. The most common issues identified with school conditions were the structure of the school buildings and lack of clean drinking water (Table 33). Issues with school structures affected approximately 60% of schools, across study arms. In terms of the lack of clean drinking water, School Feeding Only schools were most affected (41%), followed by Comparison schools (33%) and Literacy & School Feeding schools (26%). Other issues, such as issues related to latrines and relationships with parents, affected less than one third of the schools, with Comparison schools being the most affected, followed by School Feeding Only schools and Literacy & School Feeding schools. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-31 Table 33. Teacher-reported School Conditions, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=329 n=163 n=312 % (n) % (n) % (n) Issues with structure of school buildings 60.0 (196) 62.1 (96) 63.3 (177) Issues with the surrounding area 21.5 (73) 15.4 (25) 9.6 (32) Lack of clean drinking water 32.9 (122) 41.4 (67) 25.7 (93) Issues with latrines for boys 31.3 (105) 15.4 (25) 9.4 (30) Issues with latrines for girls 26.8 (90) 14.2 (22) 10.0 (33) Issues with latrines for teachers 29.4 (96) 19.5 (31) 12.5 (36) Poor relationships with parents/community 16.6 (56) 10.7 (18) 11.2 (34) Bullying at school 3.6 (10) 3.0 (4) 3.8 (10) Other 19.9 (76) 24.3 (36) 29.7 (94) Nutrition, Health, and Hygiene Teaching The percentage of teachers trained to teach about nutrition varied widely across the three study arms (Table 34). Seventy-two percent of teachers in School Feeding Only schools, 52% of teachers in Literacy & School Feeding schools, and 12% of teachers in Comparison schools said they were trained to teach nutrition. The percentage of teachers who had lesson plans or other materials for teaching nutrition lessons was 27 36% across the study arms. Of the teachers who said they had nutrition lesson plans, across the study arms, 13 29% were unable to show any such lesson plans. Among the teachers who said they had lesson plans, 70% in the Literacy & School Feeding and 80% in the School Feeding Only schools had other materials, such as picture cards or posters for teaching nutrition. A higher percentage of teachers in School Feeding Only schools taught nutrition at least once per week (70%) compared to Literacy & School Feeding schools (58%) and Comparison schools (49%). A knowledge of nutrition as average compared to Literacy & School Feeding and Comparison schools (approximately 51%). Overall, fewer teachers reported that they were trained to teach about hygiene and sanitation than about nutrition. A higher percentage of teachers in School Feeding Only schools (39%) said they were trained to teach about hygiene and sanitation than in Literacy & School Feeding (30%) or Comparison (23%) schools. The percentage of teachers who had lesson plans for teaching about hygiene and sanitation was low: 24 29% across the study arms. Of the teachers who said they had hygiene and sanitation lesson plans, across the study arms, 13 33% did not have lesson plans that they could show to the interviewer. Approximately three quarters of the teachers in the School Feeding Only and Literacy & School Feeding study arms had other materials, such as posters or picture cards, for teaching about hygiene and sanitation. In contrast, just 41% of teachers in Comparison 3-32 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report schools had these materials. Sixty-seven percent of teachers in the School Feeding Only group teach about hygiene and sanitation at least once per week compared to approximately 55% of teachers in the Literacy & School Feeding and Comparison study arms. A higher percentage of teachers in School Feeding Only schools (65%) rated their ene and sanitation as average compared to those in the Literacy & School Feeding and Comparison schools (roughly 56%). Table 34. Teacher-reported Nutrition and Hygiene Teaching, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=329 n=163 n=312 % (n) % (n) % (n) Trained on teaching about nutrition 12.4 (35) 72.2 (105) 52.1 (153) Has lesson plans or other materials for teaching nutrition lessons 26.6 (66) 32.1 (49) 36.4 (109) Able to show All nutrition lesson plans 34.8 (23) 28.6 (13) 24.0 (24) Some nutrition lesson plans 45.9 (27) 58.9 (31) 46.9 (51) No nutrition lesson plans 19.3 (16) 12.5 (6) 29.1 (35) Has other materials for teaching nutrition, such as picture cards or posters 35.8 (25) 80.4 (40) 69.9 (75) Frequency of nutrition lessons Daily 4.1 (8) 12.8 (19) 9.8 (21) A few times per week 32.3 (71) 27.4 (43) 25.1 (67) Once per week 12.3 (34) 29.3 (40) 22.9 (64) A few times per month 17.9 (46) 17.7 (21) 16.2 (44) Once per month 5.3 (12) 2.4 (4) 4.9 (14) Less than once per month 7.4 (22) 3.0 (5) 1.8 (8) Weak 38.3 (120) 18.4 (25) 41.3 (112) Average 52.2 (137) 72.8 (100) 51.2 (146) Strong 9.6 (23) 8.9 (13) 7.5 (22) Trained on teaching about hygiene and sanitation 23.1 (56) 38.5 (57) 29.9 (96) Has lesson plans or other materials for teaching hygiene and sanitation lessons 29.0 (72) 23.7 (35) 27.7 (88) Able to show All hygiene and sanitation lesson plans 30.2 (24) 32.5 (10) 22.9 (22) Some hygiene and sanitation lesson plans 37.0 (32) 55.0 (20) 56.2 (49) No hygiene and sanitation less plans 32.8 (19) 12.5 (5) 20.9 (19) USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-33 Comparison School Feeding Only Literacy & School Feeding Has other materials for teaching hygiene and sanitation, such as picture cards or posters 40.8 (33) 72.5 (24) 77.6 (66) Frequency of hygiene and sanitation lessons Daily 20.9 (49) 27.2 (36) 18.1 (49) A few times per week 19.2 (53) 19.5 (31) 18.9 (56) Once per week 15.4 (50) 20.7 (30) 17.4 (56) A few times per month 21.1 (66) 19.5 (30) 18.4 (50) Once per month 2.6 (9) 3.0 (5) 4.6 (14) Less than once per month 1.4 (5) 1.2 (2) 1.9 (8) Weak 22.0 (79) 15.0 (24) 28.6 (79) Average 56.6 (169) 65.0 (89) 56.2 (164) Strong 21.4 (43) 20.0 (26) 15.2 (37) 3.2.3 Head Teachers School Profile We asked our sample of Grade 2 and 4 teachers for information about their individual characteristics, and we also asked HTs to provide data on all teachers in their schools and school amenities. The average number of teachers per school varied across the study arms (Table 35). Table 35. Number of Teachers and Classes per Grade, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 Mean ± 95% CI Mean ± 95% CI Mean ± 95% CI Total teachers 12.7 [±1.1] 6.7 [±1.0] 10.1 [±.7] Male 6.1 [±.4] 4.4 [±.6] 5.8 [±.3] Female 6.7 [±.8] 2.4 [±.5] 5.1 [±.6] Number of teachers currently teaching Grade 2 2.3 [±.2] 1.2 [±.1] 1.9 [±.1] Number of teachers currently teaching Grade 4 1.8 [±.1] 1.1 [±.0] 1.5 [±.2] Number of Grade 2 classes (streams) 2.3 [±.2] 1.4 [±.2] 1.9 [±.1] Number of Grade 4 classes (streams) 1.7 [±.1] 1.3 [±.2] 1.4 [±.1] Note: CI = confidence interval. School Feeding Only schools had the fewest teachers (approximately 7), whereas Comparison schools had the most (approximately 13). The numbers of male and female 3-34 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report teachers were evenly split in Literacy & School Feeding and Comparison schools, but there were fewer female than male teachers in School Feeding Only schools. Schools had slightly more Grade 2 than Grade 4 teachers, but the number of teachers for each grade was one two across arms. The number of classes per grade (streams) mirrored the number of teachers. We asked HTs to report about health-related amenities and activities at their schools, including school gardens, deworming, latrines, and tippy taps. The percentage of schools with gardens varied across the study arms (Table 36). More Comparison schools had gardens (50%) than Literacy & School Feeding (41%) or School Feeding Only (29%) schools. Comparison and Literacy & School Feeding schools tended to have gardens during the rainy season only (91% and 65%, respectively), whereas School Feeding Only schools more often had year-round gardens (76%). The majority of HTs in schools with gardens said they use the produce for school meals only when garden products are available. Most schools coordinate with local health facilities to provide deworming medication to students. Higher percentages of School Feeding Only (89%) and Literacy & School Feeding (87%) schools ensured that students were dewormed than Comparison schools (75%). Similarly, more School Feeding Only (88%) and Literacy & School Feeding (87%) schools than Comparison schools (64%) facilitated the deworming of students during this school year. Higher percentages of School Feeding Only (82%) and Literacy & School Feeding (87%) schools had improved latrines than Comparison schools (39%). Among schools with improved latrines, the average numbers of latrines were 3.1 in School Feeding Only, 4.3 in Literacy & Scho observations, the percentage of schools with clean latrines was low overall but higher in the intervention study arms (44% in School Feeding Only and 42% in Literacy Plus School Feeding) than in the Comparison study arm (27%). Of the schools with improved latrines, the numbers of schools with tippy taps or hand-washing stations near the latrines were low (41% and 45% in intervention schools and 32% in Comparison schools), and few of these schools had ash or soap at all the tippy taps or hand-washing stations (9 28% across study arms). Table 36. Report of Health-related Amenities and Activities at Schools, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 % (n) % (n) % (n) School has a garden 50.0 (83) 29.4 (25) 41.4 (69) Period garden operates In rainy season only 91.4 (75) 24.0 (6) 65.4 (44) Year round 8.6 (8) 76.0 (19) 34.6 (25) Use of garden to Every day 0.0 (0) 0.0 (0) 1.4 (1) 3 4 times per week 2.3 (2) 4.0 (1) 1.4 (1) USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-35 supply food for school mealsa 1 2 times per week 0.0 (0) 8.0 (2) 5.7 (4) Never 22.1 (18) 8.0 (2) 13.1 (9) During the rainy season 20.9 (16) 8.0 (2) 16.2 (11) Only when school garden products are available 47.4 (37) 72.0 (18) 58.0 (39) Students are given deworming medication 74.7 (130) 89.4 (76) 87.0 (144) Students received deworming medication at least once during this school year 64.3 (113) 88.2 (75) 81.2 (137) School has improved latrines 38.9 (71) 82.4 (70) 86.5 (146) Cleanliness of latrines All latrines are dirty with feces on the slabs 17.1 (12) 11.4 (8) 13 (19) Some latrines are dirty 56.1 (40) 44.3 (31) 45.4 (66) All latrines are clean with no feces on the slabs 26.9 (19) 44.3 (31) 41.5 (61) School has tippy taps or hand-washing stations near improved latrinesb 32.4 (23) 41.3 (29) 45.2 (65) All tippy taps or hand-washing stations have ash or soapc 8.5 (2) 27.6 (8) 9.0 (6) Note: HT = head teacher. aUse of garden to supply food for school meals includes schools with a school garden. Some schools without school feeding but with gardens provide students with food from the garden when it is available. bOf the schools with improved latrines. cOf the schools with improved latrines and tippy taps. Head Teacher Training In the Literacy & School Feeding arm, approximately 50% of the HTs attended WV or PAI training (Table 37). Among the HTs who reported attending WV or PAI training, 47% reported attending the training during this academic year. In the School Feeding Only arm, none of the HTs reported attending WV or PAI training this academic year. HTs reported receiving training on the fundamentals of reading instruction (37 53%), school management (69 82%), and the importance of access to and use of school supplies and materials (25 38%) across the arms. Table 37. HT Training, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 % (n) % (n) % (n) % of HTs who attended WV or PAI training 1.1 (2) 0.0 (0) 50.9 (83) % of HTs who attended training this academic year 0.0 (0) 0.0 (0) 47.9 (78) % of HTs who received training on the fundamentals of reading instruction 37.3 (63) 34.9 (29) 53.4 (89) % of HTs who report receiving training on school management 68.1 (119) 82.1 (69) 69.3 (117) % of HTs who report receiving training about the importance of access to and use of school supplies and materials 26.9 (47) 24.7 (21) 38.7 (65) 3-36 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Note: HT = head teacher; PAI = Planet Aid Incorporated; WV = World Vision. Head Teach Knowledge and Skills The percentage of HTs who reported they could name the core reading skills was 76 93% across the study arms (Table 38). However, when asked to name the core reading skills, few HTs across the study arms could do so. HTs in the Literacy & School Feeding arm named more of the core reading skills than HTs in the School Feeding Only and Comparison arms. Table 38 Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 % (n) % (n) % (n) HTs able to name the core reading skills 93.7 (57) 76.9 (20) 90.7 (75) Core reading skills named by HTs Letter knowledge 5.9 (11) 4.7 (4) 21.4 (35) Phonemic awareness 10.2 (18) 8.2 (7) 30.9 (51) Fluency 5.9 (11) 9.4 (8) 21.8 (36) Vocabulary 15.1 (27) 17.6 (15) 30.9 (51) Comprehension 17.1 (30) 15.3 (13) 22.2 (37) Other 11 (19) 9.4 (8) 7.1 (12) Ways HTs identify academic progress Classroom observation 58.3 (102) 48.2 (41) 62.5 (105) Monitor student results on tests given by teachers 55.9 (98) 50.6 (43) 48.1 (82) Evaluate students orally myself 32.3 (55) 29.4 (25) 41.5 (69) homework 32.7 (56) 32.9 (28) 24.8 (42) Teachers provide progress reports 35.8 (64) 36.5 (31) 26.3 (45) End-of-term evaluations 37.9 (66) 28.2 (24) 32.6 (55) Feedback from parents 7.0 (12) 0.0 (0) 1.2 (2) Feedback from school counselors 5.3 (9) 3.5 (3) 0.0 (0) Feedback from school committees 1.1 (2) 1.2 (1) 0.6 (1) Other 14.8 (27) 16.5 (14) 12.4 (21) Note: HT = head teacher. Across the study arms, the most common approaches that HTs reported using to identify academic progress were conducting classroom observations and monitoring student results on tests given by teachers. Head Reporting on the Training of Teachers The percentage of HTs who reported that teachers in their schools attended a literacy training this academic year was 7 14% across the study arms (Table 34). Of the HTs who reported that teachers attended a literacy training this academic year, 100% of HTs in the USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-37 School Feeding Only schools and 79% of the HTs in the Literacy & School Feeding schools reported that teachers in their school attended WV or PAI training. Of the 13% of HTs in the Comparison arm who reported that teachers in the school attended the literacy training this academic year, approximately 9% also reported that teachers in the school had attended WV or PAI training. Overall, the mean number of Grade 2 teachers who attended literacy training this academic year in the Comparison arm was greater than the mean numbers of Grade 2 teachers in the School Feeding Only and Literacy & School Feeding arm who attended training. The mean number of Grade 4 teachers who attended literacy training this academic year in the Comparison arm was greater than the mean numbers of Grade 4 teachers in the School Feeding Only and the Literacy & School Feeding arms who did so. Table 39. Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 % (n) % (n) % (n) HTs who report that teachers in the school attended literacy training this academic year 13.3 (22) 7.2 (6) 14.1 (23) HTs who report the teachers in the school attended WV or PAI training (of the HTs who reported teachers attended literacy training) 9.1 (2) 100 (5) 79.2 (72) Mean [95% CI] Mean [95% CI] Mean [95% CI] Mean number of female teachers in the school who attended literacy training this academic year 1.3 [±1.6] 0.4 [±.5] 3.6 [±3.6] Mean number of male teachers in the school who attended literacy training this academic year 1.9 [±.8] 1.0 [±.6] 2.6 [±.6] Mean number of Grade 2 teachers who attended literacy training this academic year 2.5 [±1.6] 0.7 [±.4] 1.4 [±.7] Mean number of female Grade 2 teachers who attended literacy training this academic year 2.0 [±1.3] 0.0 [±0] 1.5 [±1.5] Mean number of male Grade 2 teachers who attended literacy training this academic year 0.7 [±.5] 0.7 [±.4] 0.8 [±.2] Mean number of Grade 4 teachers who attended literacy training this academic year 1.6 [±.9] 0.7 [±.4] 0.9 [±.4] Mean number of female Grade 4 teachers who attended literacy training this academic year 1.1 [±.8] 0.2 [±.3] 0.1 [±.2] Mean number of male Grade 4 teachers who attended literacy training this academic year 0.5 [±.3] 0.3 [±.4] 0.7 [±.1] Note: CI = confidence interval; HT = head teacher; PAI = Planet Aid Incorporated; WV = World Vision. Reporting on the number of Teacher Training Sessions When asked about the expected number of WV or PAI teacher training sessions, the mean numbers reported by HTs were higher in the School Feeding Only (2.9) and Literacy & School 3-38 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Feeding (6.5) groups than in the Comparison (0) group (Table 39). When asked about the number of WV teacher training sessions held to date, the mean number reported by HTs in the Literacy & School Feeding study was 2.6. When asked about the number of PAI teacher training sessions held to date, the mean number reported by HTs in the School Feeding Only arm was 3, whereas that in the Literacy & School Feeding arm was 2.4. Table 39 Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 Mean [95% CI band] Mean [95% CI band] Mean [95% CI band] Expected number of WV or PAI teacher training sessions 0.0 [±0] 2.9 [±.6] 6.5 [±.4] Number of WV teacher training sessions held to date 0.1 [±.1] 0.1 [±.1] 2.6 [±.2] Number of PAI teacher training sessions held to date 0.0 [±.0] 3.0 [±.2] 2.4 [±.3] Note: CI = confidence interval; HT = head teacher; PAI = Planet Aid Incorporated; WV = World Vision. Reporting on the Number of Reading Coaches Higher percentages of HTs in the School Feeding Only (47%) and Literacy & School Feeding (59%) arms reported that reading coaches train teachers at the school than in the Comparison arm (24%) (Table 40). Among the HTs who reported that reading coaches train teachers at the school, the mean number of trainings provided by reading coaches this academic year ranged from approximately one to three across the study arms. Table 40 Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 % (n) % (n) % (n) % of HTs who report that reading coaches train teachers at this school 24.6 (43) 47 (39) 59.7 (97) % of HTs who report that reading coaches supervise teachers at this school 48.5 (20) 59.5 (22) 82.0 (78) Mean [95% CI] Mean [95% CI] Mean [95% CI] Mean number of trainings provided by reading coaches this academic year 2.0 [±.4] 1.9 [±.3] 3.2 [±.2] Frequency at which reading coaches supervise teachers at this school Never 45.1 (18) 42.9 (15) 11.2 (10) Once a year 15.3 (6) 11.4 (4) 3.4 (3) Once per term 21.5 (8) 8.6 (3) 27.4 (24) Once a month 12.8 (5) 11.4 (4) 30.5 (27) More than once a month 5.2 (2) 25.7 (9) 27.5 (25) Note: CI = confidence interval; HT = head teacher. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-39 Higher percentages of HTs in the Literacy & School Feeding (82%) and School Feeding Only (59%) arms reported that reading coaches supervise teachers at the school than in the Comparison arm (48%). Among the HTs who reported that reading coaches supervise teachers at the school, higher percentages of HTs in the Comparison (45%) and School Feeding contrast, a higher percentage of HTs in the Literacy & School Feeding arm (30%) reported that reading coaches supervise teachers once a month. Reporting on Access to School Supplies and Learning Materials Higher percentages of HTs in the School Feeding (29%) and Literacy & School Feeding (52%) study arms reported that the school has a library/book bank than in the Comparison arm (17%) (Table 41). Among the HTs who reported that the school has a library/book bank, 100% also reported that the print material is available in Portuguese. In contrast, the percentage of HTs across the study arms who reported that the print material in the library/book bank is available in the mother tongue was 8 12%. Among the HTs in the Comparison arm, the most common types of print material reported as being available in the library/book bank were primers/student manuals and leveled books. Among the HTs in the School Feeding Only and Literacy & School Feeding arms, the most common types of print material reported as being available in the library/book bank were storybooks and primers/manuals. Table 41 Arm Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 % (n) % (n) % (n) HTs who report that the school has a library/book bank 17.1 (30) 29.4 (25) 52.2 (86) HTs who report specific types of print material available Flipbooks 19.1 (6) 36.0 (9) 18.6 (16) Storybooks 61.8 (19) 84 (21) 84.1 (72) Primers/student manuals 70.2 (21) 84 (21) 57.9 (50) Fables 39.6 (12) 40 (10) 50.3 (43) Leveled books 63.6 (19) 52 (13) 53.5 (46) Supplementary readers 50.2 (15) 40 (10) 40.8 (35) Guidelines for parents 6.2 (2) 0.0 (0) 1.2 (1) Reading posters 29.8 (9) 28 (7) 20.8 (18) Other 2.3 (4) 8.2 (7) 4.3 (7) HTs who report that the print material in the library/book bank is available in Portuguese 100.0 (30) 100.0 (24) 100.0 (86) HTs who report that the print material in the library/book bank is available in the mother tongue 9.8 (3) 8.3 (2) 12.7 (11) Note: HT = head teacher. 3-40 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Reporting on Extracurricular Activities A higher percentage of HTs in Literacy & School Feeding schools (33%) reported that the school has a literacy afterschool program than HTs in the Comparison (12%) and School Feeding Only (15%) schools (Table 42). Higher percentages of HTs in the School Feeding Only (72%) and Literacy & School Feeding (73%) arm reported that the school has other types of afterschool programs than HTs in the Comparison arm (14%). Among the HTs in the School Feeding Only arm who reported that the school has other types of afterschool programs, the most common types were the Math Learning Club and the School Gardening Club. Among the HTs in the Literacy & School Feeding arm who reported that the school has other types of afterschool programs, the most common types were the Help with HW club, Math Learning Club, and Science Club. Among the HTs in the Comparison arm who reported that the school has other types of afterschool programs, the most common types were the Help with HW Club and the Math Learning Club. Higher percentages of HTs in the School Feeding Only (55%) and Literacy & School Feeding (55%) arms reported that progress in reading than in the Comparison arm (38%). Higher percentages of HTs in the Literacy & School Feeding (48%) and School Feeding Only (35%) arms report that special events are held at the school to recognize effective reading instruction than in the Comparison arm (29%). Higher percentages of HTs in the Comparison (47%) and Literacy & School Feeding (55%) arms reported that special events are held at the school to recognize teacher attendance than in the School Feeding Only arm (27%). Table 42 Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 % (n % (n) % (n) HTs who report that the school has a literacy afterschool program 12.9 (22) 15.3 (13) 33.5 (55) HTs who report that their school has other types of afterschool programs 14.3 (25) 72.9 (62) 73.2 (124) Other types of afterschool programs reported by HTs Help with HW Club 5.7 (10) 8.2 (7) 23.3 (39) Math Learning Club 4.0 (7) 36.5 (31) 30.3 (52) Science Learning Club 2.4 (4) 10.6 (9) 23.3 (39) School Garden Learning Club 1.2 (2) 23.5 (20) 12.6 (22) Other 7.6 (14) 55.3 (47) 42.7 (73) HTs who report that special events are held at the school to 38.7 (68) 55.3 (47) 55.7 (93) HTs who report that special events are held at the school to recognize effective reading instruction 29.0 (50) 35.3 (30) 48.0 (79) HTs who report that special events are held at the school to recognize teacher attendance 47.1 (81) 27.1 (23) 55.9 (92) USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-41 Note: HT = head teacher. The percentage of HTs who reported satisfaction with parental involvement in their and in the school was less than half across all three study arms (Table 43). When asked about the ways that parents show evidence of support for the school, the most common answers given by HTs across the study arms were volunteering time and resources. The percentage of HTs who reported that parents volunteer in the school feeding program was higher in the School Feeding Only arm (45%) than in the Literacy & School Feeding (28%) and Comparison arms (4%). When asked about the frequency of parental involvement according to teachers, HTs in the Comparison (47%) and Literacy & School Feeding (43%) arms reported that parents were involved once a quarter. In contrast, HTs in the School Feeding Only arm (64%) reported that according to teachers, the frequency of parental involvement was more than once a month. Table 43 Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 % (n) % (n) % (n) % of HTs who report satisfaction with parental involvement in 31.3 (56) 19.0 (16) 36.7 (62) school 30.2 (53) 19.0 (16) 35.0 (59) Ways that parents show evidence of support for the school None 19.1 (33) 7.1 (6) 9.9 (17) Visiting the school 18.9 (34) 16.5 (14) 19.2 (32) Actively seeking information about school matters 26.2 (45) 23.5 (20) 29.2 (49) Following up on academic progress 24 (42) 27.1 (23) 26.6 (45) Volunteering time 37.7 (66) 47.1 (40) 40.5 (68) Volunteering in school feeding 4.5 (8) 45.9 (39) 27.8 (48) Volunteering resources 38.7 (66) 40.0 (34) 33.4 (56) Participating in school meetings/committees 32.0 (57) 32.9 (28) 36.2 (61) Other 8.8 (15) 7.1 (6) 6.7 (11) Frequency of parental involvement reported by teachers Once a year 18.0 (24) 0.0 (0) 9.2 (13) Once a quarter 47.7 (63) 26.0 (20) 43.3 (62) Once a month 15.2 (20) 9.1 (7) 12.7 (18) More than once a month 19.1 (25) 64.9 (50) 34.9 (51) Note: HT = head teacher. Teacher Absenteeism Across study arms, most schools (88 96%) had teacher attendance registers (Table 44). HTs reported that on average, one two teachers were absent yesterday or the last day school absence were 3-42 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report allocating the class to another teacher or combining students together into one class. HTs themselves were frequently absent (22 30%) or late to school (10 19%). Table 44 Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Number of teachers absent yesterday or last day school was in session 1.4 [±.2] 1.0 [±.3] 1.7 [±.2] % (n) % (n) % (n) Teacher attendance register available 93.5 (164) 88.2 (75) 96.0 (162) How class is handled if teacher is absent Let class proceed without teacher 2.7 (5) 1.2 (1) 0.6 (1) Allocate class to another teacher 55.0 (95) 62.4 (53) 71.8 (119) Join all students in one class 26.9 (46) 15.3 (13) 15.0 (25) Dismiss students for the day 0.5 (1) 1.2 (1) 1.2 (2) Distribute students among other classes 6.6 (12) 11.8 (10) 7.5 (13) Other 8.3 (15) 8.2 (7) 4.0 (7) HT absent from school any day last week 21.8 (38) 29.4 (25) 30.1 (51) HT late to school any day last week 9.6 (16) 10.6 (9) 18.8 (31) Note: CI = confidence interval; HT = head teacher. Nutrition and Hygiene Training and Teaching We asked HTs to report on teacher training on nutrition and hygiene/sanitation to collect information about all teachers in each school and triangulate the results with those hygiene/sanitation mirrored those of Grade 2 and 4 teachers (Table 45). In the School Feeding Only and Literacy & School Feeding schools, more teachers were trained on nutrition than hygiene/sanitation, whereas the reverse was true in Comparison schools. y of teaching nutrition and hygiene/sanitation did not align with those of teachers. HTs described less availability of teaching materials for nutrition and hygiene/sanitation than teachers. They also reported less frequent teaching of nutrition, although their estimates of the frequency of hygiene/sanitation teaching were relatively close to those of teachers in the School Feeding Only and Literacy & School Feeding arms. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-43 Table 45 Hygiene Lessons, by Study Arm Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 Mean ± 95% CI band Mean ± 95% CI band Mean ± 95% CI band Teachers trained on teaching nutrition 0.5 [±.4] 3.5 [±.4] 4.5 [±.6] Teachers trained on teaching hygiene and sanitation 1.1 [±.4] 1.5 [±.4] 2.3 [±.5] % (n) % (n) % (n) Teachers have materials for teaching nutrition 12.4 (22) 59.0 (49) 66.0 (109) Frequency of teaching nutrition lessons to each class Once per week 15.2 (22) 64.6 (51) 60.9 (91) Once every two weeks 6.1 (9) 7.6 (6) 8.5 (13) Once per month 7.7 (11) 12.7 (10) 14.3 (21) Once per term 38.6 (56) 8.9 (7) 6.1 (9) Never 32.4 (45) 6.3 (5) 10.2 (15) Teachers have materials for teaching hygiene and sanitation 22.9 (39) 25.0 (21) 55.7 (90) Frequency of teaching hygiene and sanitation lessons to each class Once per week 35.0 (53) 70.0 (56) 66.9 (104) Once every two weeks 8.5 (13) 6.2 (5) 7.8 (12) Once per month 6.9 (10) 5.0 (4) 7.3 (11) Once per term 26.3 (39) 6.2 (5) 7.2 (11) Never 23.4 (34) 12.5 (10) 10.8 (17) Note: CI = confidence interval; HT = head teacher. School Feeding Program Nearly all School Feeding Only schools and all Literacy & School Feeding schools had a school feeding program, whereas only a few Comparison schools did (Table 46). In School Feeding Only schools, the program was supported exclusively by PAI/ADPP; in Literacy & School Feeding schools, the program was supported nearly equally by PAI/ADPP and WV; in Comparison schools, the program was supported by other organizations. Approximately 92% of School Feeding Only and Literacy & School Feeding schools served school meals daily, whereas all Comparison schools with a feeding program served meals daily. The main issue with school feeding programs was insufficient number of pots, utensils, or dishes, as mentioned by 21 school feeding programs, lack of water, lack of cooks/unavailability of cooks, lack of remuneration for cooks, lack of soap, and lack of firewood were mentioned most frequently. 3-44 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Table 46 Comparison School Feeding Only Literacy & School Feeding n=175 n=85 n=169 % (n) % (n) % (n) School has a school feeding program 5.4 (10) 98.8 (84) 100.0 (168) Organization supporting school feedinga PAI/ADPP 0.0 (0) 100.0 (84) 46.4 (81) WV 0.0 (0) 0.0 (0) 50.7 (82) Community support 0.0 (0) 0.0 (0) 0.6 (1) Other 100.0 (10) 0.0 (0) 4.1 (7) Frequency of school mealsa Daily 100.0 (10) 92.8 (77) 92.2 (155) A few times per week 0.0 (0) 4.8 (4) 2.9 (5) Whenever food is available 0.0 (0) 2.4 (2) 4.9 (8) Issues with school feeding programa,b No issues 10.0 (1) 29.4 (25) 17.6 (30) Food is not always available. 10.0 (1) 2.4 (2) 4.1 (7) Kitchen/warehouse not constructed or in disrepair 30.0 (3) 1.2 (1) 4.2 (7) Not enough pots, utensils, or dishes 40.0 (4) 21.2 (18) 31.7 (53) Meals not prepared on time 0.0 (0) 7.1 (6) 6.0 (10) Other 70.0 (7) 51.8 (44) 68.7 (115) Note: ADPP = Ajuda de Desenvolvemento de Povo para Povo (Development Aid from People to People); HT = head teacher; PAI = Planet Aid Incorporated; WV = World Vision. aOf schools with a school feeding program. b 3.2.4 Warehouse Managers and Cooks We originally planned to collect information about school feeding infrastructure and training from cooks, but during the pilot phase, we found that warehouse managers handle many of the logistical issues related to the school feeding program at each school. Therefore, we divided the questions between the manager and the head cook. We also determined that warehouse managers and cooks were not always present to answer questions. School Feeding Infrastructure More schools in School Feeding Only schools (92%) compared to Literacy & School Feeding schools (79%) had a school feeding log book where they tracked the quantities of food received and used (Table 47). In approximately 90% of schools with a log book, the log was filled out and up to date. Nearly all schools in both intervention arms had a kitchen and storeroom for the school feeding program. Furthermore, in both intervention arms, the percentages of schools in which the kitchen and storeroom were in very good condition were low (23% in School Feeding Only and 28% in Literacy & School Feeding schools). USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-45 We found some gaps in the availability of materials for school feeding programs. Half of the Literacy & School Feeding schools and more than one third of School Feeding Only schools had ever had a gap in their supply of corn-soy blend. Most schools had cooking pots and utensils, but 14% of Literacy & School Feeding and 8% of School Feeding Only schools did not. More School Feeding Only schools had sufficient numbers of cooking pots and utensils than Literacy & School Feeding schools. Approximately 80% of the schools had bowls and spoons for students, but only 50 60% had them in sufficient quantities. Clean water was available at 68% of Literacy & School Feeding and 60% of School Feeding Only schools. Many schools did not have soap for cleaning cooking materials and bowls. School Feeding Only schools more often had soap available to clean pots and bowls (61%) than Literacy & School Feeding schools (36%). A higher percentage of School Feeding Only than Literacy & School Feeding schools had a place for cooks and students to wash their hands and soap for hand washing. The availability of soap was the largest gap, especially in Literacy & School Feeding schools. School meals are most often served between 9 and 10 a.m. in both intervention study arms, which puts the meal in the middle of the morning school session. Approximately half the cooks reported that school meals are always served on time. Table 47. School Feeding Infrastructure, by Intervention Arm School Feeding Only Literacy & School Feeding n=79 n=168 % (n) % (n) School has a school feeding log book. 92.4 (73) 78.5 (125) Log book is filled out and up to date. 89.0 (65) 89.0 (111) School has a kitchen and storeroom for school feeding program. 98.7 (78) 98.2 (156) Condition of the kitchen and storeroom Not adequate (holes in roof or walls, no door, and/or no lock) 16.7 (13) 19.8 (32) Moderate (complete building but needs repairs) 60.3 (47) 52 (83) Very good (complete building with no repairs needed) 23.1 (18) 28.1 (44) School has ever had a gap in the supply of corn-soy blend. 36.7 (29) 50.3 (80) Cooking pots and utensils are available. 92.4 (73) 86.1 (137) Number of pots and cooking utensils is sufficient. 75.9 (60) 56.9 (91) School has bowls and spoons for students. 78.6 (55) 80.1 (114) Number of bowls and spoons is sufficient. 61.4 (43) 51.8 (74) School has clean water available. 60.0 (42) 67.7 (97) Soap is available to clean bowls, spoons, pots, and cooking utensils. 61.4 (43) 35.5 (52) School has a place for cooks to wash their hands. 74.3 (52) 58.7 (84) Soap is available for cooks to wash their hands. 60.0 (42) 28.5 (42) School has a place for children to wash their hands before eating. 72.9 (51) 65.2 (93) Soap is available for children to wash their hands before eating. 45.7 (32) 19.0 (28) 3-46 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report School Feeding Only Literacy & School Feeding Time of day when meals are served clock hour 0.0 (0) 1.9 (3) 60.8 (48) 56.7 (90) 10.1 (8) 14.1 (22) 12.7 (10) 12.7 (20) 13.9 (11) 10.9 (18) After noon 2.5 (2) 3.7 (6) School meals are served on time Never 5.1 (4) 3.9 (6) Sometimes 46.8 (37) 46.9 (74) Always 48.1 (38) 49.2 (79) Cook Training and Knowledge Less than half of the school cooks interviewed had received training (Table 48). Among those trained, the timing of the last training varied between the two intervention study arms, with 45% of School Feeding Only cooks and 27% of Literacy & School Feeding cooks receiving training less than 6 months ago. Cooks had low to moderate knowledge of food safety (three four out of eight items correct) and hygiene (two three out of five items correct). Table 48. Cook Training and Knowledge, by Intervention Arm School feeding only Literacy + School feeding n=70 n=144 % (n) % (n) School cook training received 41.4 (29) 47.3 (67) Timing of last school cook training Less than 6 months ago 44.8 (13) 27.2 (19) 6- 11 months ago 17.2 (5) 17.9 (12) 12 or more months ago 37.9 (11) 54.8 (37) Mean ± 95% CI Band Mean ± 95% CI Band Knowledge of food safety (8 items) 3.4 [±.3] 3.8 [±.2] Knowledge of hygiene (5 items) 2.4 [±.2] 2.5 [±.2] Note: CI = confidence interval. Use of School Meals Nearly 90% of cooks in both intervention study arms said that they usually eat the school meal when they are at the school cooking (Table 49). Fifty-three percent of School Feeding USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-47 Only schools and 40% of Literacy & School Feeding schools did not usually have leftovers from school meals. When there were leftovers, cooks most often gave more porridge to students or ate it themselves. On average, interviewers found two three cooks present at each school, and cooks reported that they prepare three four school meals per week. Table 49. Use of School Meals and Cook Working Days, by Intervention Arm School Feeding Only Literacy & School Feeding n=70 n=144 % (n) % (n) Cooks usually eat the school meal 87.1 (61) 89.7 (128) Use of leftover food at end of school meala Usually there are no leftovers 52.9 (37) 39.5 (57) Give more to students 27.1 (19) 37.3 (53) Give food to siblings/other children at school who are not students 7.1 (5) 4.9 (7) Give it to teachers 7.1 (5) 7.8 (11) Cooks eat it 15.7 (11) 16.1 (23) Cooks take it home 4.3 (3) 4.7 (7) Other 7.1 (5) 14.8 (21) Mean ± 95% CI Band Mean ± 95% CI Band Number of cooks present on day of survey 2.3 [±.3] 2.6 [±.2] Number of days per week cooks prepare meals at school 3.6 [±.3] 3.7 [±.2] Note: CI = confidence interval. a 3.2.5 Parents Overall, a total of 30 FGDs with parents were conducted. Fourteen and 16 FGDs were completed in rural and urban schools, respectively. Twenty-three of the schools where FGDs were held received the Literacy & School Feeding intervention, whereas seven received School Feeding Only (Table 50). A total of 285 parents participated in the FGDs. We included mothers and fathers in the same groups because there were no sensitive questions, and we did not expect different old, with half between 29 and 43 years old. Fifty-seven percent of the participants were female. On average, the participants had 3.4 children in total, and 47%, 34%, and 19% had a child in Grade 2, Grade 4, and both grades, respectively. Fifty-six percent reported speaking only a local language at home, 30% reported speaking both a local language and Portuguese, and 14% reported speaking Portuguese only. 3-48 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Table 50. Number of FGDs and Parents by District and Province Involved in Qualitative Data Collection Literacy & School Feeding School Feeding Only # of FGDs Mean # of Parents % of Parents with Grade 2 Child % of Parents with Grade 4 Child # of FGDs Mean # of Parents % of Parents with Grade 2 Child % of Parents with Grade 4 Child Maputo Magude 1 7.0 57% 43% 3 11.3 71% 53% Manhiça 4 7.3 79% 59% 2 8.0 75% 44% Matutuíne* 3 7.7 - - Moamba* 3 6.7 33% 56% 2 11.5 61% 39% Nampula Muecate 6 10.5 75% 46% - - - - Nacarôa 6 11.8 51% 63% - - - - Total 23 9.3 63% 55% 7 10.4 68% 47% Note: FGD = focus group discussion. * Demographic information was missing for three FGDs in Matutuíne and two in Moamba. Average and percent values are based on non-missing data only. in order to be so Parent, Namina Rio School, Muecate, Nampula Parents value education for their children. They consistently stated that they expected their children to gain knowledge, including reading and writing; find employment after finishing school; support th hope he continues to study to acquire knowledge and one day becomes a doctor, a teacher uture employment opportunities but also teaches children good behavior and helps them be e in schools were mixed, with some parents being very pleased, while others expressed frustration that teachers were not more patient and persistent in helping each child succeed. Most parents seemed very satisfied with the level of education their child received at school, citing the fact that children are gaining skills in reading and writing as proof that the school is working to provide an education. However, some parents expressed deep concerns about and extreme disappointment in their school because children were progressing through grade levels without advancement in their reading and writing skills. For example, one parent stated that her son was advanced to Grade 5 without the ability to read or write. Three consistent problems regarding the quality of education were identified by parents: 1) teacher absenteeism, 2) inadequate education materials (particularly school books), and 3) the lack of a water source at the school. Without water, school meals cannot be prepared. Parents often stated that their children must wake up extremely early (e.g., 12:00 a.m.) to fetch water before school and that some schools require children bring water to attend USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-49 classes. Other requests made by parents related to adding grade levels in the schools to offer complete elementary and secondary education, incorporating English into the curriculum, providing security for the school to prevent thievery and vandalism, providing assistance with purchasing school uniforms, addressing the need for additional classrooms and latrines, and fulfilling the need for additional shelters (i.e., roofs) where children can read and study during the rainy season. he overall literacy education in schools were mixed. While many parents indicated that they observed their children successfully reading and writing at home, parents in two of the School Feeding Only schools stated that the literacy education was not working because the children are advancing grade levels despite lacking the ability to read and write. A few themselves were not literate. Parents pointed to poor teaching quality and teacher absenteeism as problems leading to poor literacy program performance. The main suggestion for improvement was to increase have are incomplete, and not everyone has them which makes it difficult to study because on a lack of trust in those running the program when supplies are not consistently available. Children were promised materials, including books, notebooks, and snacks, and when these materials did not arrive, children stopped attending school. However, other schools have reported receiving materials, including notebooks and pencils, and parents viewed these parents were pleased with the literacy program, they greatly valued the program and everal parents proposed increasing the number of teachers implementing the school literacy program. Parents reported that nearly all the reading and writing activities that children engage in at home revolve around homework assigned by school teachers. Parents often mentioned that teachers should continue to assign homework as a means of ensuring that their children continue to engage in reading and writing activities. According to the parents, parental involvement in such activities at home mainly consists of encouraging their children to complete their homework. Many parents mentioned that children ask them questions about homework and that they try to answer such questions as best they can. However, the parents indicated that they were not always able to fully assist their children. Lack of fluency in Portuguese and illiteracy were consistently identified as the main barriers that prevent parents from engaging in reading and writing activities with their children at home. Some parents mentioned that older siblings in higher grade levels often help their younger 3-50 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report and writing activities are conducted solely at home; however, sometimes, a lack of materials parents noted that children engage in reading and writing activities daily, either once or twice a day. Nearly all parents mentioned that school workbooks were the main source of reading reading material included non-school books purchased by parents, newspapers, television ads, phone text messages, and the bible. Some parents stated that their children have pencils and notebooks in which they can write and draw pictures; however, parents more frequently indicated a lack of writing materials as a problem. Thus, parents reported that many children use sticks and wire to write and draw pictures in the dirt. Parents identified the following as factors that motivate their children to engage in reading and writing activities at home: assigned homework, access to adequate materials, parental guidance or supervision, support from older siblings, and peer support. Barriers included a lack of materials, a lack of electricity to provide light and allow children to read and write in the evening, a lack of time because of the need to complete household chores, insufficient teachers to provide additional support for low-achieving students, and the desire for children to play rather than complete homework. Parents also noted that children need to indicated available to complete homework. he Language Used in School and at Home Parents reported that local languages (i.e., Changana and Macua) are the most common languages used in the home setting, whereas Portuguese is used in schools. Very few parents indicated that they had the ability to read and write in Portuguese. Furthermore, they reported that the inability to read and write in Portuguese makes helping their children with homework challenging. When parents reported that they did engage in reading and writing activities at home with their children, they mainly described doing so in the local language rather than Portuguese. Some parents suggested that the sole use of Portuguese at school when children use Changana at home might hinder their ability to learn at school. One parent stated, is important and suggested using both Portuguese and the local language in school. erceptions of School Meals -Parent, Maguaza School, Moamba District, Maputo USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 3-51 All parents expressed that they were pleased with the presence of the school feeding program. Indeed, the parents frequently mentioned that school meals help motivate their children to attend school more regularly and stay at school longer and prevent children from dropping out o children because it guarantees that at least they have something to eat, and it encourages result of the school meals. Others noted that their children appeared to be stronger and happier when a school feeding program was implemented. All parents requested that the -being and success in school. Parents identified the main problem with the school feeding program as the lack of a water source for cooking meals at the school. One parent noted that installing a water source near the school would also improve school attendance, the consistent provision of school meals, and the hygienic preparation of school meals. Many parents suggested providing more variety in the types of foods offered through the school feeding program. Suggested foods included beans, meat, rice, and pasta. One parent proposed adding peanuts to the school porridge to increase the calories consumed by the children. Several parents also recommended improving the flavor of the porridge by adding salt or sugar. Other suggestions for improving the school feeding program included increasing the number of cooks, ensuring that no gaps occur in the supply of school meals, and constructing more buildings with roofs where children can eat during the rainy season. Changes in Diets at Home due to School Meals Although a few parents stated that the amount of food consumed at home by their children did not change because of the school meals, most indicated that their school-going children the school meals, children no longer complain about being hungry when they arrive home tea, prior to leaving for school. However, with school meals, they no longer do so and instead eat only at school. Others stated that their children either skip dinner entirely or eat very little for dinner because when school meals are served, they are not hungry in the evening. Some parents said that their children never missed school because of illness, whereas others stated that fever, cough, nausea, malaria, and/or diarrhea had caused their children to miss some school. Parents explained that at school, children mainly learned about personal hygiene, including bathing, washing their face, and washing hands, and about washing dishes. Most parents reported that their children applied these messages in their behavior at home, although some parents indicated that their children did not bathe regularly. No parents discussed improved sanitation, such as increased or proper use of 3-52 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report latrines; instead, they only mentioned that their children learned to wash their hands after going to the bathroom. Nine of the schools where FGDs were held had school gardens. Parents noted that school gardens were beneficial because they allow students to have a variety of foods; however, many barriers prevent gardens from being successful. At most schools, parents are responsible for managing the school garden, and at some schools, children are also involved in the farming practices. Parents frequently indicated that a lack of seeds and other tools is the most serious challenge to having a successful school garden. _________________________________ RTI International is a registered trademark and a trade name of Research Triangle Institute. 4. SUMMARY In this section, we return to our theory of change and describe the baseline findings related to the elements of the theory. We also triangulate results across types of participants and data collection methods, wherever possible. At baseline, we found several differences between groups in the key indicators for the primary and secondary evaluation questions and in the factors that may contribute to those key indicators. Student literacy: Baseline EGRA scores varied across treatment arms. In general, performance on the EGRA was better among Grade 4 students than Grade 2 students, as expected. In almost all cases, and for both grades, as the difficulty of the subtasks increased, Students in McGovern-Dole intervention schools struggle more with listening comprehension than students in Comparison schools. Students in School Feeding & Literacy schools can read fewer words per minute than in School Feeding Only and Comparison schools. Oral reading fluency (ORF rates across all groups were lower than the minimum rate necessary for students to read the entire passage. Percent zero scores were generally higher for the syllable sound task than for the ORF task, indicating that more students could read at least one word of a short passage than were able to read a syllable. This pattern was similar across all arms and in both grades. This finding suggests that students have not fully acquired the decoding skills necessary to read fluently and with comprehension but may have recognized or memorized familiar words that they frequently encounter in grade-level text. that their children read and write well, whether others saw their children promoted to the next grade despite struggling with reading and writing. Student attentiveness: Most teachers said that they can tell whether their students are attentive when they face the teacher and make eye contact or when they respond to questions posed by the teacher. Less than 40% of the teachers said that the school meal makes their students more attentive, approximately 40% said the meal does not change student attentiveness, and roughly 20% said it makes their students less attentive. Access to and use of school supplies: S is instruction. However, data from parent FGDs indicated that children do not have consistent access to school materials. Only slightly more than half of teachers said they had access to a teacher guide, and their reported use of the teacher guide was generally aligned with their reported access to the teacher guide. Although more teachers in intervention schools than in comparison schools reported having access to basic school supplies, most teachers across groups reported low access to materials such as markers and chart paper. Literacy instructional materials: Students had lower access to supplementary readers than to Portuguese textbooks and exercise notebooks. This point was supported by the data from teachers, who reported using supplementary readers approximately once a week. More than 50% of schools in the Literacy & School Feeding intervention have a library/book bank 4-2 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report with print materials, such as storybooks and primers/manuals in Portuguese. Less than 50% of schools in the School Feeding Only intervention have a library/book bank. Most intervention schools have few print materials in the mother tongue available. This finding aligns with parent FGD data, which revealed that parents would like to see an increase in the quantity of reading materials available to their children. Extracurricular activities: Less than 50% of the HTs in the intervention and comparison schools reported the presence of an afterschool literacy program. Among the teachers who reported the existence of a literacy club at the school, across intervention and comparison schools, most indicated that half the students in their class attend an afterschool literacy club. Consistent with the teacher data, the HT data showed that more than 50% of the schools in the School Feeding Only Less than 50% of all sampled schools 1) hold special events at the school to celebrate or 2) hold special events at the school to recognize effective reading instruction. More than 50% of schools in the Literacy & School Feeding study arm hold special events at the school to recognize teacher attendance. In contrast, this trend is not observed on the same magnitude in the School Feeding Only study arm. Skills and knowledge of teachers: Most teachers reported attending and finishing studies at a TTC. Additionally, most reported that they had attained certification. Slightly more than 50% of teachers received specific training on how to teach reading and writing to students in the primary grades as part of their preservice training. More than half of the teachers earned a certificate as their highest level of education. According to HTs, more than 50% of teachers in the Literacy & School Feeding intervention receive training from reading coaches, whereas less than half of teachers in the School Feeding Only intervention receive training from reading coaches. More than 50% of the teachers in both interventions are supervised by reading coaches. However, these data are not consistent with teacher data indicating that more than 50% of teachers reported that reading coaches had not visited since the beginning support, most teachers in intervention schools reported that HTs or assistant HTs check lesson plans once a week and conduct classroom observations once per trimester. Nearly all the teachers in both comparison and intervention schools receive feedback from HTs or assistant HTs. When they need help, teachers in both comparison and intervention schools most often have casual discussions with other teachers. More than 50% of the teachers in comparison and intervention schools could not name all the core reading skills. More than 50% of the teachers in the comparison and intervention schools use written tests and oral and writing progress. To check student understanding during lessons, more than 50% of teachers in both comparison and intervention schools question students individually. Data on teacher knowledge and skills contradict on the instructional support received. For example, when asked what the teacher does when a student is unable to answer a question, the most common response from students in both comparison and intervention schools was that the teacher hit the student. USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 4-3 Skills and knowledge of Head Teachers: This year, 50% of the HTs in the Literacy & School Feeding group attended a WV or PAI training. Fifty-three percent of the HTs in the Literacy & School Feeding group received training on the fundamentals of reading instruction, compared to 35% in School Feeding Only schools. More than 65% of the HTs in both the comparison and intervention schools received training on school management. Less than 40% of the HTs in comparison and intervention schools received training on access to and the use of school supplies and materials. Few HTs in comparison and intervention schools could name the core reading skills. Classroom observations and monitoring of student results on tests given by teachers were the most common approaches used to identify academic progress by HTs in both comparison and intervention schools. : Sixty to 75% of Grade 2 and 4 students bring reading books home from the classroom, and nearly all the books brought home are in Portuguese. However, 50% of Grade 2 students in the Literacy & School Feeding intervention said that they never read on their own at home, and 50% of Grade 2 students in the School Feeding Only intervention said they sometimes read on their own at home. More than 50% of Grade 4 students across groups said they sometimes read on their own at home. Across the study arms, less than a quarter of Grade 2 and 4 students bring books home from the school library or book bank. Parental involvement: Less than 50% of the HTs across the study arms are satisfied with the level of parental participation in terms of and in the school. Among the HTs who reported that some parents do participate, across the study arms, the most common forms of participation consisted of volunteering time and resources to the school. More parents in the School Feeding Only arm said they volunteered their time in the school feeding program than parents in the Literacy & School Feeding arm. Slightly more than 50% of Grade 2 and 4 students across groups reported that someone at home helps with homework when needed. This result is in line with parent FGD data, which show that parental involvement at home occurs mainly in the form of encouragement to complete assigned homework. In the FGDs, some parents reported that older siblings help their younger siblings with homework and with practicing reading and writing. Across groups, more than half of students in Grade 2 and 4 speak a language other than Portuguese at home. This finding is consistent with parent FGD data showing that parents are not fully fluent in Portuguese. When asked if someone reads them stories, the most common response among Grade 2 students was that no one at home tells them stories. In contrast, among Grade 4 students, the most common response was that someone at home sometimes tells them stories. Access to food/dietary intake and short-term hunger: More than 50% of the students said they eat fewer meals at home on days when they receive a school meal, indicating that school meals replace home meals for many students. Most students reported that they feel less hungry on days when they receive a school meal. These points were supported by the qualitative data obtained from parents, many of whom reported that their children skip meals at home on school days, in part because they are less hungry when they have a school meal. 4-4 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Timing of the school meal: At most schools, meals are served at 9:30 a.m., which is in the middle of the morning session. This timing may have an impact on student attentiveness, if meals do indeed improve attentiveness, because the first half of the school day is over before the meal is served. School feeding: Nearly all of the School Feeding Only and Literacy & School Feeding schools serve school meals; however, it appears that meals are not served daily in all schools. Instead, on average, schools serve meals three to four times per week. More than half of the intervention schools do not serve school meals on time. Parents appreciate the school feeding program because it encourages their children to go to school. They recommended enhancing or changing the foods included in school meals. A small number of Comparison schools had formal school feeding programs, but half of the Comparison schools also offered some type of school food during the rainy season when food from their school gardens was available. The presence of school feeding activities in some of the Comparison schools should be considered in the final evaluation analysis. Access to requisite food preparation and storage tools and equipment: The conditions of the kitchen and storeroom were very good in only approximately one quarter of the intervention schools. Some schools had experienced a gap in the supply of corn-soy blend. Most schools had cooking pots and utensils available, but at least one quarter noted that the number of these utensils was not sufficient. Roughly 20% of schools lacked bowls and spoons for students. At least one third of the schools did not have clean water available. Soap for cooks and students to wash their hands was lacking, especially in Literacy & School Feeding schools. Knowledge of safe food preparation and storage practices: Less than 50% of the school cooks had received training. Their knowledge of food safety and hygiene was low to moderate. Knowledge of health and hygiene practices: Students had very low knowledge of nutrition and moderate knowledge of hygiene and sanitation. In intervention schools, one quarter to one half of the teachers were not trained on teaching about nutrition, and only approximately one third had nutrition lesson plans. Fewer teachers were trained to teach about hygiene and sanitation than about nutrition, and even fewer had lesson plans for hygiene and sanitation than for nutrition. Health and dietary practices of students: We found that many students across study arms had inadequate diets, with inadequate meal frequency being more common than low dietary diversity. Less than half of the students reported that they always wash their hands with soap or ash before eating the school meal. School infrastructure and access to water and sanitation services: Less than 50% of the intervention schools had a garden. Most intervention schools had improved latrines, but less than half of those with improved latrines had tippy taps or hand-washing stations. According to teachers, the most serious issues with the school environment were the dilapidated state of the school buildings and the lack of clean drinking water. Parents who participated in the FGDs also mentioned these as major problems. The lack of water impacts USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 4-5 the students in some intervention schools because they are required to get up early to fetch water to bring to school to make the porridge. Access to preventive health interventions: Most intervention schools offered deworming to students. Student attendance and health absences: Student attendance was difficult to capture accurately. We used student recall and compared class enrollment lists to the number of students present on the day of data collection. Across the study arms, more than one quarter of students were absent during the last week with no significant differences between the groups and no significant differences by gender. Students were most often absent because they were ill. Community understanding of the benefits of education: Parents who participated in the FGDs understood that education and reading are important because their children need literacy skills to be gainfully employed later in life and to be able to support themselves and their family members. Parents mentioned insufficient quantities of books and workbooks for students in some schools, which led students to drop out. Teacher attendance: Approximately one quarter of teachers and HTs were absent during the last week. 5-6 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 5. LIMITATIONS & RECOMMENDATIONS FOR IMPACT EVALUATION In this section, we describe the limitations of the study and offer recommendations relevant to future data collection efforts (midline and endline) and impact evaluation analysis. Limitations and corresponding recommendations from a methodological perspective are included as a table in Annex C. Limitations The School Feeding Only study arm is only available in Maputo. As a result, the design is unbalanced, which will impact the distribution of degrees of freedom and the analytical approach taken during the midterm analysis and final impact evaluation. Intervention districts were matched with comparison districts manually based on the limited information available on these districts and current information on future donor activities. At baseline, we found that some comparison schools served school meals. Most intervention schools have participated in school feeding activities since 2012, so the school-level Because this study was not a randomized controlled trial, intervention schools may be inherently different from comparison schools. The purposive selection of intervention schools for inclusion in the program could be related to their past poor performance, which could be one of the factors explaining why comparison schools seem to outperform intervention schools on some indicators. Recommendations Difference-in-difference analysis should be used at midline and endline to account for the lack of a true baseline and the inherent differences between the three groups at baseline. Other donor and governmental interventions in target districts should be tracked over the course of the program so they can be accounted for or considered in subsequent parts of the evaluation. Matching at the school, teacher, and student levels should be considered in future analyses to control the variance. No matching below the district level was attempted in the baseline assessment. Based on findings from the baseline quantitative analysis, we recommend adding additional qualitative data collection from teachers and cooks/warehouse managers to facilitate a more comprehensive mixed methods approach at midline and endline. This would produce more information to explain and support the quantitative analysis findings. For example, to reduce changes to the questionnaire and shed light on data regarding access to and use of the teacher guide, mother tongue instruction, and materials such as those from the school library or book bank, we recommend that USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report 5-7 qualitative in-depth interviews with teachers be conducted at midline. Furthermore, adding comparison schools to the qualitative interviews would help elucidate issues at these schools compared to intervention schools. Given that the literacy intervention will include mother tongue instruction and materials in some areas, we recommend designing an EGRA in the new target language of instruction, in addition to the Portuguese EGRA at midline. We also recommend keeping the question that asks about the percentage of teachers who report access to and use of mother tongue materials. Additionally, we recommend retaining the question on We recommend the inclusion of a meta-analysis of the impacts in Maputo and Nampula to overcome the confounding factors and variance issues resulting from the differences in program implementation strategies across implementers, baseline levels, and contextual factors in the two provinces. _________________________________ RTI International is a registered trademark and a trade name of Research Triangle Institute. 6. REFERENCES Abadzi, H. (2012). Developing cross-language metrics for reading fluency measurement: Some issues and options. Global Partnership for Education working paper series on learning. No. 6. Washington DC: World Bank. Bennett, C., & Blissett, J. (2014). Measuring hunger and satiety in primary school children. Validation of a new picture rating scale. Appetite, 78, 40 48. Hammill, D. D. (2004). What we know about correlates of reading. Exceptional Children, 70(4), 453 468. Heugh, K., Benson, C., Bogale, B., Yohannes, M. A. G. (2007). Final report study on medium of instruction in primary schools in Ethiopia. Commissioned by the Ministry of Education. Samuels, S. J., & Flor, R. F. (1997). The importance of automaticity for developing expertise in reading. Reading & Writing Quarterly, 13, 107 121 Scarborough, H. (2001). Connecting early language and literacy to later reading (dis)abilities: Evidence, theory and practice. In S. Neuman & D. Dickison (Eds.), Handbook of family literacy (pp. 97 110). New York: Guilford Press. Tobia, V., & Bonifacci, P. (2015). The simple view of reading in a transparent orthography: The stronger role of oral comprehension. Reading and Writing, 28(7), 939 957. Weiser, B., & Mathes, P. (2011). Using encoding instruction to improve the reading and spelling performances of elementary students at risk for literacy difficulties: A best-evidence synthesis. Review of Educational Research, 81(2), 170 200. WHO. (2010). Indicators for assessing infant and young child feeding practices Part 2 Measurement. Geneva, Switzerland: WHO. Wood, J., & McLemore, B. (2001). Critical components in early literacy Knowledge of the letters of the alphabet and phonics instruction. Florida Reading Quarterly, 38(2), 1 8. 6-2 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report A-1 7. ANNEX A. WITHIN-PROVINCE LITERACY ANALYSIS Table 51 presents the Early Grade Reading Assessment (EGRA) results for Maputo Province by study arm. In Maputo Province, intervention arms include Comparison schools, School Feeding Only schools, and Literacy & School Feeding schools. The table provides the percent zero scores for all subtasks, mean scores for timed subtasks, and percent mean scores for untimed subtasks. Table 51: Grade 2 and 4 EGRA Results for Maputo Province, by Study Arm Maputo Province Grade 2 Grade 4 Subtask Study Arm Mean Scores Percent Zero Scores Mean Scores Percent Zero Scores Letter Identification (clpm) Comparison 16.4 29.4% 45.5 3.5% School Feeding Only 11 49.4% 37.7 8.6% Literacy & School Feeding 11.7 42.3% 37 5.6% Syllable Reading (csspm) Comparison 9.9 57.3% 36.6 14% School Feeding Only 6 69.7% 29.4 22.1% Literacy & School Feeding 5.9 66.8% 28.2 21.2% ORF Comparison 7.6 26.9% 35.6 10.7% School Feeding Only 3.6 56.5% 24.4 24.1% Literacy & School Feeding 3.9 49.9% 24.5 20.7% Percent Mean Scores Percent Zero Scores Percent Mean Scores Percent Zero Scores Reading Comprehension Comparison 8.7% 80.3% 43.7% 31.7% School Feeding Only 2% 93.1% 23.2% 57.6% Literacy & School Feeding 3.7% 87.9% 26.7% 50% Listening Comprehension Comparison 49.2% 12.7% 63.9% 4.4% School Feeding Only 21.2% 50.8% 39% 25.6% Literacy & School Feeding 24.6% 44.9% 44.1% 20.3% Word Dictation Comparison 22.4% 61.4% 65.4% 19.1% School Feeding Only 13% 74.1% 55.1% 25.2% Literacy & School Feeding 15.6% 71.1% 55% 27.1% Note: clpm = correct letters per minute; csspm = correct syllable sounds per minute; EGRA = Early Grade Reading Assessment; ORF = oral reading fluency. Sample sizes for Grade 2: Comparison, n=704; School Feeding Only, n=666; Literacy & School Feeding, n=661 Sample sizes for Grade 4: Comparison, n=681; School Feeding Only, n=616; Literacy & School Feeding, n=638 Table 52 presents the EGRA results for students in Nampula Province in Grades 2 and 4 by study arm. The study arms in Nampula include Comparison schools and schools that will receive the Literacy & School Feeding intervention. The table provides the percent zero scores for all subtasks, mean scores for timed subtasks, and percent mean scores for untimed subtasks. A-2 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Table 52: Grade 2 and 4 EGRA Results for Nampula Province, by Study Arm Nampula Province Grade 2 Grade 4 Subtask Study Arm Mean Scores Percent Zero Scores Mean Scores Percent Zero Scores Letter Identification (clpm) Comparison 5.4 39.6% 18.6 17.8% Literacy & School Feeding 6.4 43.4% 14.8 20.2% Syllable Reading (csspm) Comparison 2.3 81.6% 12.4 47.7% Literacy & School Feeding 2.7 80.2% 8.1 56.3% ORF Comparison 1.8 52.6% 9.4 37.1% Literacy & School Feeding 2.2 53.5% 6.3 42.6% Percent Mean Scores Percent Zero Scores Percent Mean Scores Percent Zero Scores Reading Comprehension Comparison .7% 96.9% 7.6% 75.7% Literacy & School Feeding 1.2% 96% 4.5% 87.1% Listening Comprehension Comparison 13.1% 59.6% 23.3% 40.3% Literacy & School Feeding 8.3% 75.2% 16.6% 59.4% Word Dictation Comparison 2.4% 91% 14.7% 59.9% Literacy & School Feeding 4% 88.8% 10.4% 71.4% Nampula Province Grade 2 Grade 4 Subtask Study Arm Mean Scores Percent Zero Scores Mean Scores Percent Zero Scores Letter Identification (clpm) Comparison 5.4 39.6% 18.6 17.8% Literacy & School Feeding 6.4 43.4% 14.8 20.2% Syllable Reading (csspm) Comparison 2.3 81.6% 12.4 47.7% Literacy & School Feeding 2.7 80.2% 8.1 56.3% ORF Comparison 1.8 52.6% 9.4 37.1% Literacy & School Feeding 2.2 53.5% 6.3 42.6% Percent Mean Scores Percent Zero Scores Percent Mean Scores Percent Zero Scores Reading Comprehension Comparison .7% 96.9% 7.6% 75.7% Literacy & School Feeding 1.2% 96% 4.5% 87.1% Listening Comprehension Comparison 13.1% 59.6% 23.3% 40.3% Literacy & School Feeding 8.3% 75.2% 16.6% 59.4% Word Dictation Comparison 2.4% 91% 14.7% 59.9% Literacy & School Feeding 4% 88.8% 10.4% 71.4% Note: clpm = correct letters per minute; csspm = correct syllable sounds per minute; EGRA = Early Grade Reading Assessment; ORF = oral reading fluency. Sample sizes for Grade 2: Comparison, n=690; Literacy & School Feeding, n=707 Sample sizes for Grade 4: Comparison, n=676; Literacy & School Feeding, n=685 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report A-3 8. ANNEX B. PRECISION ESTIMATES Table 53 shows the weighted oral reading fluency (ORF) mean estimate and the confidence in the mean estimates by province and study arm. All mean estimates for Grade 2 fall well within the ±2.5 wpm 95% confidence interval (CI) band. All groups fall under ±1.0 wpm, except for the Maputo Comparison group, which has a band of ±1.3 wpm. Table 54 provides the same information for the Grade 4 students. The mean ORF for Grade 4 students in Nampula stayed within the expected 95% confidence bands: ±1.5 wpm for the intervention group and ±1.7 wpm for the Comparison group. However, the mean ORF for Grade 4 students in Maputo exceeded the expected precision. The precision values in the Comparison group and the School Feeding Only group were similar: ±2.8 and ±3.2 wpm, respectively. The Literacy & School Feeding group had a precision of ±4.0 wpm because of the combined high inter-class correlation (ICC) and standard deviation (SD), which were not expected. Overall, the estimates were sufficiently precise. Table 53. Grade 2 Reading Fluency Means and Precision Estimates of the Mean ORF Provinces Study Arm n Range (wpm) Mean (wpm) 95% CI Band (wpm) SD (wpm) ICC DEFF Maputo Literacy & School Feeding 661 [0,46] 3.9 ±0.9 7.6 0.134 2.8 School Feeding Only 666 [0,55] 3.6 ±0.7 8 -0.018 0.7 Comparison 703 [0,89] 7.6 ±1.3 13.4 0.183 3 Nampula Literacy & School Feeding 707 [0,69] 2.2 ±0.9 5.6 0.427 7.4 Comparison 690 [0,64] 1.8 ±0.7 6.2 0.108 2.7 Note: CI = confidence interval; DEFF = Design effect; ICC = inter-class correlation; SD = standard deviation; wpm = words per minute. Table 54. Grade 4 Reading Fluency Means and Precision Estimates of the Mean ORF Provinces Study Arm n Range (wpm) Mean (wpm) 95% CI Band (wpm) SD (wpm) ICC DEFF Maputo Literacy & School Feeding 638 [0,112] 24.5 ±4.0 22.1 0.252 4.5 School Feeding Only 616 [0,115] 24.4 ±3.2 24.1 -0.005 0.9 Comparison 680 [0,122] 35.6 ±2.8 26.9 0.06 2.3 Nampula Literacy & School Feeding 685 [0,92] 6.3 ±1.5 12.2 0.115 2.6 Comparison 676 [0,89] 9.4 ±1.7 14.7 0.109 2.5 Note: CI = confidence interval; DEFF = Design effect; ICC: inter-class correlation; SD = standard deviation; wpm = words per minute. A-4 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report A-5 9. ANNEX C. IMPACT EVALUATION RECOMMENDATIONS (TABLE) Source Limitation Consequences Recommendations Exogenous Factors & Program Design Different conditions, environment, and access to resources between Maputo and Nampula High variance expected which might mask some of the significant differences Meta-analysis for midterm and final impact evaluation Exogenous Factors & Implementer Strategy Different implementation activities by Planet Aid and World Vision Implementer and province variances are confounded resulting in increased variance which might mask some of the significance differences Meta-analysis for midterm and final impact evaluation analysis Implementer Strategy Program phased out for developing curriculum Different levels of exposures for students across different cohorts in midline and endline Special care in the analysis on the exposure of each cohort relative to implementer activities and inclusion of indicators for intensity of the treatment Implementer Strategy No schools receiving a intervention The quasi￾experimental design is not a full factorial design Special care in midterm and final impact evaluation analysis Implementer Strategy School Feeding Only study arm is only available in Maputo Unbalanced design, which will impact the distribution of degrees of freedom Special care to account for this difference during the midterm and final impact evaluation analysis Exogenous Factor & Implementer Strategy Intervention changes in response to endogenous and exogenous factors (e.g., schools with Literacy Only in Maputo because of other school feeding programs overlapping) Implications on sampling and data collection efforts Keep track of activities by implementers over time and update school lists and relevant activities to account for these changes in sampling and analysis A-6 USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report Exogenous Factor Limited information on current and future activities on literacy and nutrition at the district and individual school level Manual matching of intervention and comparison districts Need additional model based matching effort (e.g., propensity score matching) and continue collecting information about relevant school level activities Exogenous Factors Some comparison schools serve school meals or have overlapping activities Confounding factors due to contamination of the comparison schools Continue tracking other donor and governmental institution activities throughout the intervention period to account for them in the impact evaluation analysis Exogenous Factors Wide range of differences across schools within districts with limited information available during sampling Large variance which could impact the ability to detect significant differences Conduct matching at school, teacher, and student level prior to impact evaluation analysis Program Design Intervention schools participated in nutrition activities since 2012 Not a true baseline across all indicators Use of difference-in￾difference approach for midterm and final impact evaluation analysis relative to 2017 Program Design Not a randomized control trial and purposive selection of districts and school for inclusion in the program because of past poor performance Comparison schools might be better and different relative to intervention schools because of inherent selection bias Use propensity score matching and difference￾in-difference approach for midterm and final impact evaluation Baseline Results Unexpected trends in quantitative findings Need to explain reasons for the observed trends Use a mixed methods approach by adding more qualitative data collection from teachers, cooks, and warehouse managers in midline and endline data collection Include comparison schools in qualitative USDA McGovern Dole Mozambique Impact Evaluation and Literacy Assessment: Final Baseline Report A-7 data collection and analysis Avoid too many changes to quantitative instruments for consistency and comparability Program Design and Implementer Strategy Integration of mother tongue literacy program No measurements at baseline of mother tongue reading assessment Develop Early Grade Reading Assessment (EGRA) in mother tongue based on curriculum developed once available for midline and endline Assume zero scores at baseline