WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Mid-Term Evaluation of the Mozambique McGovern Dole International Food for Education and Child Nutrition Educating Children Together - Phase II (ECT2) FFE-656-2015/011-00 José Diogo – Advisory Services Manager - Africa, India & Middle East (AIM) Ernst & Young - Rua Belmiro Obadias Muianga, Nº 179, Maputo, Mozambique Office: +258-21-353-000 | Direct: +258-21-353-000 | jose.diogo@mz.ey.com Fax: +258-21-321-984 | Cell: +258-82-756-1210 Website: http://www.ey.com OCTOBER 2018 DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Acronyms ADP Area Development Program DME Design, Monitoring and Evaluation EGRA Early Grade Reading Assessment EPC Primary School Grade 1 to 7 DPEDH Provincial Directorate of Education and Human Development FCD Foundation for Community Development FFE Food for Education FGD Focus Group Discussion FLA Functional Literacy Assessment GDP Growth Domestic Product HH Household KII Key Informant Interview MASA Ministry of Agriculture and Food Security MERL Monitoring, Evaluation, Research and Learning MINEDH Ministry of Education and Human Development MISAU Ministry of Health MSC Most Significant Change SDAE District Services of Economic Activities SDPI District Services of Planning and Infra-structure SDEJT District Services of Education, Youth and Technology SPSS Statistical Package for Social Sciences USDA United States Department of Agriculture WVM World Vision Mozambique WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Contents 1 Introduction ................................................................................................................................................................. 1 1.1 Project Background................................................................................................................................................................................... 1 1.2 Project Objectives..................................................................................................................................................................................... 3 1.2.1 General Purpose .............................................................................................................................................................................................. 3 1.2.2 Specific Objectives........................................................................................................................................................................................... 3 1.3 Evaluation Questions................................................................................................................................................................................ 3 1.3.1 Relevance .................................................................................................................................................................................................. 3 1.3.2 Effectiveness ............................................................................................................................................................................................. 3 1.3.3 Efficiency.................................................................................................................................................................................................... 4 1.3.4 Sustainability ............................................................................................................................................................................................. 4 1.3.5 Impact......................................................................................................................................................................................................... 4 1.3.6 Project Design Improvement............................................................................................................................................................... 4 2 Methodology and Sample Approach......................................................................................................................... 5 2.1 Research Methods..................................................................................................................................................................................... 5 2.1.1 Qualitative Research .............................................................................................................................................................................. 5 2.1.2 Quantitative Research............................................................................................................................................................................ 6 2.2 Sample Approach....................................................................................................................................................................................... 6 2.2.1 Sample approach – Quantitative .................................................................................................................................................................. 6 2.2.2 Students – Sample Size ................................................................................................................................................................................... 6 2.2.3 Schools – Sample Size..................................................................................................................................................................................... 7 2.2.4 Sample Approach – Qualitative.................................................................................................................................................................... 7 2.3 Evaluation Constrains............................................................................................................................................................................... 8 3 Evaluation Findings ................................................................................................................................................... 10 3.1 SO1- Improved Literacy of School-Aged Children .........................................................................................................................10 3.1.1 MGD1- Improved Literacy of School-Aged Children ..................................................................................................................10 3.1.1.1 MGD 1.1- Improved Quality of Literacy Instruction.................................................................................................11 3.1.1.1.1 MGD 1.1.1- More Consistent Teacher Attendance ...............................................................................................12 3.1.1.1.2 MGD 1.1.2- Better Access to School Supplies & Materials...................................................................................13 3.1.1.1.3 MGD 1.1.5- Increased Skills and Knowledge of School Administrators............................................................15 3.1.1.2 MGD 1.2- Improved Attentiveness...............................................................................................................................16 3.1.1.2.1 MGD 1.2.1- Reduced Short-Term Hunger ...............................................................................................................17 3.1.1.2.2 MGD 1.2.1.1/1.3.1.1- Increased Access to Food (School Feeding).....................................................................18 3.1.1.3 MGD 1.3- Improved Student Attendance....................................................................................................................20 3.1.1.3.1 MGD 1.3.1- Increased Economic and Cultural Incentives (Or Decreased Disincentives)............................21 3.1.1.3.2 MGD 1.3.2- Reduced Health Related Absences......................................................................................................22 3.1.1.3.3 MGD 1.3.3- Improved School Infrastructure............................................................................................................23 3.1.1.3.4 MGD 1. 3.4- Increased Student Enrollment..............................................................................................................24 3.1.1.3.5 MGD 1.3.5 Increased Community Understanding of Benefits of Education.....................................................25 3.1.1.3.6 MGD 1.4.1- Increased Capacity of Government Institutions...............................................................................25 3.1.1.3.7 MGD 1.4.3- Increased Government Support...........................................................................................................26 3.1.1.3.8 MGD 1.4.4- Increased Engagement of Local Organizations and Community Groups...................................27 3.2 SO2- Increased Use of Health and Dietary Practices ....................................................................................................................27 3.2.1 MGD2- Increased Use of Health and Dietary Practices..............................................................................................................27 WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 3.2.1.1 MGD 2.1- Improved Knowledge of Health and Hygiene Practices.......................................................................27 3.2.1.2 MGD 2.2 Increased Knowledge of Safe food prep and Storage Practices...........................................................29 3.2.1.3 MGD 2.3- Increased Knowledge of Nutrition ............................................................................................................30 3.2.1.4 MGD 2.4 Increased Access to Clean Water and Sanitation Services...................................................................31 3.2.1.5 MGD 2.5- Increased Access to Preventive Health Interventions..........................................................................31 3.2.1.6 MGD 2.6- Increased Access to requisite Food prep and storage tools and equipment..................................32 3.3 Regression Analysis.................................................................................................................................................................................33 3.4 Sustainability..............................................................................................................................................................................................34 4 Conclusions and Recommendations ....................................................................................................................... 36 4.1 Conclusions...............................................................................................................................................................................................36 4.1.1 General Conclusions............................................................................................................................................................................36 4.1.2 Specific Conclusions.............................................................................................................................................................................36 4.2 Recommendations...................................................................................................................................................................................39 5 References.................................................................................................................................................................. 41 6 Annexes ...................................................................................................................................................................... 42 6.1 Evaluation Matrix.....................................................................................................................................................................................42 6.2 Data Entry and Data Analysis...............................................................................................................................................................42 6.3 Enumerators Selection and Training...................................................................................................................................................42 6.4 Additional Evaluation Questions..........................................................................................................................................................42 6.5 Quality Assurance ...................................................................................................................................................................................42 6.6 Sample Execution ....................................................................................................................................................................................42 Table of Figures Figure 1: Reading Camp Session .......................................................................................................................................................................................15 Figure 2: SG Meeting in Minheuene School, Muecate District..................................................................................................................................22 Figure 3: EPC de Halaca before World Vision Intervention......................................................................................................................................24 Figure 4: EPC de Halaca after World Vision Intervention..........................................................................................................................................24 Figure 5: Community meeting for Feedbacks providing .............................................................................................................................................27 Figure 6: Suggestion Boxes in use.....................................................................................................................................................................................27 Figure 7: Teachers and School Council Health Matters Training Session ..............................................................................................................28 Figure 8: Health Education Sessions.................................................................................................................................................................................28 Figure 9: Students washing their hands from tippy-taps before school lunch in Nacaroa..................................................................................30 Figure 10: New blocks of latrines built at Sapala ..........................................................................................................................................................31 Figure 11: Students taking medication during distribution of medication...............................................................................................................32 Figure 12: Health Examination Sessions..........................................................................................................................................................................32 Table of Charts Chart 1: Percentage of students who by the end of two grades of primary schooling demonstrate that they can read and understand the meaning of grade level text – Baseline treatment Group vs Control Group.................................................................................................10 Chart 2: Percentage of teachers in target schools who demonstrate improved literacy instruction as identified by supervisors, mentors or coaches..............................................................................................................................................................................................................................12 Chart 3: Percentage of teachers in target schools who attend and teach school at least 80% of scheduled days per school year........13 Chart 4: Percentage of classrooms with literacy instruction materials (text books, work books) sufficient for effective instruction...14 Chart 5: Percentage of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance ..............................................................................................................................................................................................................16 Chart 6: Percentage of students in target schools identified by their teachers as attentive during class/instruction .................................16 Chart 7: Percentage of students in target schools who indicate that they are not hungry during school day .............................................18 Chart 8: Beneficiaries satisfaction with the quality of the food.................................................................................................................................18 Chart 9: Percentage of students that attend school at least 80% of the school days..........................................................................................21 Chart 10: Percent of students (Boys) in target schools who miss more than 10 school days per year due to illness. ..............................22 Chart 11: Percent of students (Girls) in target schools who miss more than 10 school days per year due to illness...............................23 Chart 12: Value of public sector investments leveraged as a result of USDA assistance...................................................................................26 Chart 13: Percentage of children in target communities who can identify at least 3 important health/hygiene practices ........................28 Chart 14: Percent of target beneficiaries who use appropriate hand washing practices....................................................................................28 WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Table of Tables Table 1: Percentage of students who by the end of two grades of primary schooling demonstrate that they can read and understand the meaning of grade level text – Actual vs. Target.................................................................................................................................................11 Table 2: Percentage of teachers in target schools who demonstrate improved literacy instruction as identified by supervisors, mentors or coaches – Actual vs. Target......................................................................................................................................................................................12 Table 3: Skills and Knowledge of Teachers - Indicators Performance – Actual vs. Target............................................................................12 Table 4: Percentage of teachers in target schools who attend and teach school at least 80% of scheduled days per school year - Actual vs. Target ............................................................................................................................................................................................................................13 Table 5: Percentage of classrooms with literacy instruction materials (text books, work books) sufficient for effective instruction – Actual vs. Target...............................................................................................................................................................................................................14 Table 6: Number of classrooms with literacy instruction materials (text books, work books) sufficient for effective instruction – Actual vs. Target ............................................................................................................................................................................................................................14 Table 7: Percentage of students in target schools identified by their teachers as attentive during class/instruction – Actual vs. Target ...............................................................................................................................................................................................................................................17 Table 8: Percentage of students in target schools who indicate that they are not hungry during school day – Actual vs. Target......18 Table 9: Access to Food Indicator Results.................................................................................................................................................................18 Table 10: Percentage of students that attend school at least 80% of the school days – Actual vs. Target................................................21 Table 11: Percent of students in target schools who miss more than 10 school days/year due to illness – Actual vs. Target............23 Table 12: Number of Canteens built ...........................................................................................................................................................................24 Table 13: Number of Latrines Built..............................................................................................................................................................................24 Table 14: Number of Storerooms rehabilitated .......................................................................................................................................................24 Table 15: Number of students enrolled in school receiving USDA assistance .................................................................................................25 Table 16: Number of district and provincial MINED officials who know standard operating procedures and tools for management and oversight of school feeding programs and literacy boost – Actual vs. Target...................................................................................................26 Table 17: Percentage of Education coordination meetings where school feeding is part of the agenda....................................................26 Table 18: Percentage of children in target communities who can identify at least 3 important health/hygiene practices – Actual vs. Target ..................................................................................................................................................................................................................................28 Table 19: Percent of target beneficiaries who use appropriate hand washing practices - Actual vs. Target.............................................29 Table 20: Percentage of food preparers at target schools who can identify at least 3 key practices aimed at safe food preparation – Actual vs. Target...............................................................................................................................................................................................................29 Table 21: Number of individuals trained in child health and nutrition as result of USDA assistance .........................................................30 Table 22: Number of schools with appropriate food prep and storage equipment........................................................................................32 WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Executive Summary In the past few years, Mozambique has been making political efforts to minimize food insecurity among children. Indeed, the Five-Years Government Plan 2015 – 2019 (PQG) which aims to reduce stunting to 35%, and Multi-Sectorial Plan for the Reduction of Chronic Malnutrition 2010 – 2020, foresee strong recommendations to expand food security programs and initiatives to accelerate the reduction of stunting from 44% in 2010 to 20% in year 2020. As a consequence of the implementation of these strategies, the Government of Mozambique has been implementing school feeding programs under PRONAE (National School Feeding Program). This Program was approved in 2013 by the Mozambican Council of Ministers, which laid out the institutional foundation for school feeding within the educational system, and also reaffirming the potential of local food purchases in promoting food and nutrition security. World Vision, among its various Programs within Mozambique, is implementing a five-years Food for Education Project called Educating Children Together Phase 2 (ECT2) in coordination with the Mozambican Ministry of Education, specifically PRONAE. World Vision is a global Christian relief, development and advocacy organization founded in 1950. In Mozambique, World Vision started operations in 1983 and is currently one of the biggest humanitarian organizations, implementing Programs in health care, education, water and sanitation. The project expects to benefit approximately 59411 children in 150 schools in Muecate and Nacaroa districts in Nampula Province. The overall goals of the five year Educating Children Together (ECT2) McGovern Dole (MGD) project are to improve the literacy of primary school children and to improve health and dietary practices. To achieve this goal the project is focused on the following objectives: • Improve literacy of school-age children through improved quality of literacy education, improved attentiveness, and improved student attendance; • Increased use of health and dietary practices through improved knowledge of health and hygiene practices, increased knowledge of safe food preparation and storage practices, increased knowledge of nutrition, increased access to clean water and sanitation services, increased access to preventative health interventions and increased access to requisite food prep & storage tools and equipment. The project has been active since January 2016 and a mid-term evaluation was commissioned to assess progress of work with the main purpose of examining evidence of early changes in the target communities – both positive and negative – and compare them to the changes anticipated in the Project Framework. It will attempt to identify the factors in the program implementation or context that seem to promote or prevent those early changes that have been identified. The specific objectives are: • Assess implementation progress and constraints to determine the likelihood of achieving target results; • Assess relevance of interventions, provide a sign of the effectiveness of the interventions and identify, explain and learn from successful strategies as well as challenges; • Assess early signs of sustainability, review service and input delivery mechanisms and the quality of services (e.g. trainings, sensitization sessions, school activities, distribution of food WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation and non-food items, etc.) and highlight stakeholder’s views on or perception of project interventions, i.e. what is working and what adjustments need to be made; • Measure the extent of implementation of ECT Phase 1 evaluation findings and lessons learned from Phase 1; • Identify and document new or continuing lessons learned, challenges, good practices and recommendations. The evaluation results show that the implementation of ECT2 has been successful particularly when the baseline outcome indicators results are compared with the midterm outcome indicators results. Indeed, the project vision, strategies and standards are completely in line with Government of Mozambique National Policies and Strategies for Education and Food Security. The preliminary results of the project implementation bring positive lessons to PRONAE (School Feeding National Program) to support the actual school feeding Programs policy direction. In addition, a clear alignment of the project with the beneficiaries’ needs was perceived. Concretely, improving the quality of education, reducing food insecurity levels and improving the quality of children’s health is crucial and the priority is to improve the people’s wellbeing and therefore improve Mozambique Development Indicators and contribute to the achievement of the Sustainable Development Goals. Other important aspects in terms of the design are the way the activities are interconnected. This link ensures greater ownership from the school community members (Government, students, teachers, school administrators, school councils and the community in general). Thus, this link constitutes a very important way to ensure the sustainability of the project implementation actions. In general, the main challenges are Government related, which is to gather resources to keep the school feeding component after the project ends. Below is an illustrative comparison between the baseline and MTE and some highlighted results to support the general conclusion presented. Table: Summary of Outcome Indicators Performance at MTE MG D Result Indicator Gender/Type Unit Baseline MTE Final Targets Level of Achievem ent (%) Status SO1 - Improved Literacy of School-Aged Children 1 Improved Literacy of School Aged Children Percentage of students who by the end of two grades of primary schooling demonstrate that they can read and understand the meaning of grade level text (male/female) Male % 0.0 34% 45.0 76% In Progress Female % 0.0 31% 45.0 69% In Progress Total % 25.1 32.0 45.0 71% In Progress Number of individuals benefiting directly from USDA – funded interventions Male Nr. 34088.0 36544.0 36544.0 100% Achieved Female Nr. 29628.0 31900.0 31900.0 100% Achieved Continuing Nr 63716.0 68444.0 68444.0 100% Achieved 1.1 Improved Quality of Literacy Instruction Percentage of teachers in target schools who demonstrate improved literacy instruction as identified by supervisors, mentors or coaches % 74.8 100.0 100.0 100% Achieved 1.1.1 More Consistent Teacher Attendance Percentage of teachers in target schools who attend and teach school at least 80% of scheduled days per school year % 90.0 96.0 90.0 107% Achieved WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 1.1.2 Better Access to School Supplies and Materials Percentage of Teachers who received textbooks and other teaching and learning materials provided as a result of USDA assistance % 0.0 100 100.0 100% Achieved Number of textbooks and other teaching and learning materials provided as a result of USDA assistance Nr. N/A 1634.0 2400.0 68% In Progress Percentage of classrooms with literacy instruction materials (text books, work books) sufficient for effective instruction % 80.0 90.0 100.0 90% In Progress 1.1.3 Improved Literacy Instructional Materials Number of classrooms with literacy instructional materials (textbooks, workbooks,) sufficient for effective instruction Nr. 337.0 406 450.0 90% In Progress 1.1.4 Increased Skills and Knowledge of Teachers Number of teachers/educators/teaching assistants in targeted schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Nr. 0 566.0 566.0 100% Achieved Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance Nr. 0 708.0 708.0 100% Achieved 1.1.5 Increased Skills and Knowledge of School Administrators Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance Nr. 0 106.0 240.0 44% In Progress Number of school administrators and officials trained or certified as a result of USDA assistance Nr. 300.0 282.0 300.0 94% In Progress 1.2 Improved Attentiveness Percentage of students in target schools identified by their teachers as attentive during class/instruction % 98.0 90.0 98.0 92% In Progress 1.2.1 Reduced Short-Term Hunger Percentage of students in target schools who indicate that they are not hungry during school day % 52.6 100.0 100.0 100% Achieved 1.2.1. 1 Increased Access to Food (School Feeding) Number of individuals receiving take-home rations as a result of USDA assistant Nr. 0.0 728.0 728.0 100% Achieved Number of take-home rations provided as a result of USDA assistant Nr. 0.0 527,289. 00 1,134,000. 00 46% In Progress Number of school aged children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance Male Nr. 32775.0 35084.0 35084.0 100% Achieved Female Nr. 24726.0 30722.0 30722.0 100% Achieved Continuing Nr. 57501.0 65806.0 65806.0 100% Achieved Number of daily school meals (breakfast, snacks, lunch) provided to school aged children as a result of USDA assistance Nr. 12615812 .0 22,174,9 80.00 51,093,46 0.00 43% In Progress Number of social assistance beneficiaries participating in productive safety nets as result of USDA assistance Male Nr. N/A 36246.0 36246.0 100% Achieved Female Nr. N/A 31499.0 31499.0 100% Achieved Continuing Nr. N/A 67745.0 67745.0 100% Achieved 1.3 Improved Student Attendance Percentage of students that attend school at least 80% of the school days Male % 96.7 85.2 98.0 87% In Progress Female % 84.8 74.8 92.0 81% In Progress Continuing % 90.8 80.0 95.0 84% In Progress 1.3.1 Increased Economic and Cultural Incentives (Or Decreased Disincentives) Number of schools benefiting from saving groups social funds as result of USDA assistance Nr. N/A 146.0 120.0 122% Achieved 1.3.2 Reduced Health Related Absences Percent of students in target schools who miss more than 10 school days/year due to illness Male % 2.1 2.0 1.5 75% In Progress WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 1.3.3 Improved School Infrastructure Number of educational facilities (i.e., school buildings, classroom, and latrines) rehabilitated/const ructed as a result of USDA assistance Schools Nr. 11.0 17.0 15.0 113% Achieved Storerooms Nr. 150.0 48.0 90.0 53% In Progress Latrines Nr. 440.0 1430.0 1595.0 90% In Progress Canteens Nr. 150.0 123.0 150.0 82% In Progress Wells Nr. N/A 36.0 48.0 75% In Progress 1.3.4 Increased Student Enrollment Number of students enrolled in school receiving USDA assistance Male Nr. 32775.0 35084.0 35084.0 100% Achieved Female Nr. 24726.0 30722.0 30722.0 100% Achieved Continuing Nr. 57501.0 65806.0 65806.0 100% Achieved 1.4.1 Increased Capacity of Government Institutions Number of district and Provincial MINED officials who know standard operating procedures and tools for management and oversight of school feeding programs and Literacy Boost Nr. 0 48.0 48.0 100% Achieved 1.4.3 Increased Government Support Percentage of Education coordination meetings where school feeding is part of the agenda % 0 31.0 100.0 31% In Progress 1.4.4 Increased Engagement of Local Organizations and Community Groups Number of community groups that report having access to meaningful opportunities for feedback about project implementation Nr. 0.0 37.0 37.0 100% Achieved Number of Parent-Teacher Association (PTAs) or similar “school” governance structures supported as a result of USDA assistance Nr. 0.0 160.0 160.0 100% Achieved Number of public - private partnership formed as result of USDA assistance (nutrition) Nr. 0.0 80.0 80.0 100% Achieved SO2-Increased Use of Health & Dietary Practices 2 Increased Use of Health and Dietary Practices Percent of target beneficiaries who use appropriate hand washing practices Male % 0.0 94.0 95.0 0% In Progress Female % 0.0 95.0 95.0 0% In Progress Continuing % 0.0 94.0 95.0 99% In Progress Percent of school￾aged children receiving a minimum acceptable diet Male % 0.0 53.0 53.0 100% Achieved Female % 0.0 47.0 47.0 100% Achieved 2.1 Increased Knowledge of Health and Hygiene Practices Percentage of children in target communities who can identify at least 3 important health/hygiene practices % 61.7 86.0 90.0 96% In Progress 2.2 Increased Knowledge of Safe food prep and Storage Practices Percentage of food preparers at target schools who can identify at least 3 key practices aimed at safe food preparation % 0 100.0 100.0 100% Achieved 2.3 Increased Knowledge of Nutrition Number of individuals trained in child health and nutrition as result of USDA assistance Male Nr. N/A 482.0 818.0 59% In Progress Female Nr. N/A 1005.0 546.0 184% Achieved Continuing Nr. N/A 1487.0 1364.0 109% Achieved 2.4 Increased Access to Clean Water and Sanitation Services Number of schools using an improved water source Nr. 150.0 160.0 160.0 100% Achieved Number of schools with improved sanitary facilities Nr. 40.0 130.0 145.0 90% Achieved 2.5 Increased Access to Preventive Health Interventions Number of schools who receive at least 2 visits per year from health facility staff Nr. N/A 147.0 160.0 92% In Progress Number of students receiving de￾worming medication(s) Nr. N/A 55949.0 59411.0 94% In Progress WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 2.6 Increased Access to requisite Food prep and storage tools and equipment Number of schools with appropriate food prep and storage equipment Nr. 150.0 160.0 150.0 106% Achieved WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 1 Introduction 1.1 Project Background According to the Ministry of Education and Human Development (2017), gross primary school graduation rate lies at approximately 44%. This is a highly important statistic when considering the enrollment rate of 80%. The low completion rates are a consequence of dropout and repetition, and not having initial access or enrollment. Two of the main challenges to the retention of children in school are poverty and hunger. A quarter of Mozambique’s population is considered food insecure. Simply put, they do not know when their next meal will come or what it might include. For young children this can stunt their physical and intellectual development and limit their immediate ability to concentrate and perform what would otherwise be simple tasks. Therefore, and not surprisingly, in many poor households, hunger has been a major barrier to school participation and success. A hunger-stricken child is not only unable to enroll in school at the right age, but also cannot attend regularly even if enrolled. For those who are in class, studies have shown that hunger can often leave children distracted and less able to concentrate, which can have drastic, rippling effects on a child’s ability to succeed in the classroom, and later in the workplace. In addition, these children are also likely to quit school because they must address their own or their family’s immediate subsistence needs. Thus, low school enrollment, low class attendance, and limited participation and concentration, and high student drop-outs are recurring problems for children living in poor households, especially in areas of high food insecurity1. In the past few years, Mozambique has made political strides aimed at minimizing the critical situation mentioned above related to food insecurity particularly among school-aged children. Indeed, it has been implementing PRONAE (National School Feeding Program) through the Ministry of Education and Human Development in coordination with Ministry of Agriculture and Food Security and the Ministry of Health. This Program was approved in 2013 by the Mozambican Council of Ministers, which laid out the institutional foundation for school feeding within the educational system, and also reaffirming the potential of local food purchases in promoting food and nutrition security. World Vision, among its various Programs within Mozambique, is implementing a five-year school feeding program called Educating Children Together Phase 2 (ECT2) in coordination with the Mozambique Ministry of Education, specifically PRONAE. World Vision is an International Christian humanitarian aid, development, and advocacy organization founded in 1950. In Mozambique World Vision started operations in 1983 and is currently one of the biggest aid organizations implementing Programs in health care, education, water and sanitation. The overall goal of this five year Educating Children Together-Phase 2 (ECT2) McGovern Dole (MGD) program is to improve the literacy of primary school children and to improve health and dietary practices, through interventions aimed at improving the quality of literacy instruction, student attentiveness, and student/teacher attendance. The project expects to benefit approximately 59411 children in 150 schools in Muecate and Nacaroa districts in Nampula Province. To achieve 1 GITHUKU, Jackson, Effects of School Feeding Programme On Pupils Enrolment In Early Childhood Education In Karaba Zone, University of Nairobi, 2015, Nairobi. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation this goal, ECT2 focuses on the following objectives, all of which are fully integrated and interdependent: Increased use of health and dietary practices through improved knowledge of health and hygiene practices, increased knowledge of safe food prep and storage practices, increased knowledge of nutrition, increased access to clean water and sanitation services, increased access to preventative health interventions and increased access to requisite food prep & storage tools and equipment. Improve literacy of school-age children through improved quality of literacy education, improved attentiveness, and improved student attendance; 1 2 WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 1.2 Project Objectives 1.2.1 General Purpose The purpose of the mid-term evaluation is to examine evidence of early changes in the target communities – both positive and negative – and compare them to the changes anticipated in the Project Framework. It will attempt to identify the factors in the program implementation or context that appear to promote or obstruct those early changes that have been identified. 1.2.2 Specific Objectives The specific objectives are: • Assess implementation progress and constraints to determine the likelihood of achieving target results; • Assess relevance of interventions, provide a signal of the effectiveness of the interventions and identify, explain and learn from successful strategies as well as challenges; • Assess early signs of sustainability, review service and input delivery mechanisms and the quality of services (e.g. trainings, sensitization sessions, school activities, food and non-food items distributed, etc.) and highlight stakeholder’s views on or perception of project interventions, i.e. what is working and what adjustments need to be made; • Measure the extent of implementation of ECT Phase 1 evaluation findings and lessons derived from Phase 1 learning event; • Identify and document new or continuing lessons learned, challenges, good practices and recommendations. 1.3 Evaluation Questions 1.3.1 Relevance • To date, is the project addressing the most critical problems or constraints to improve educational outcomes? • Are the constraints faced by the target beneficiaries as outlined in the original design document remaining relevant? • What are the factors that have an impact on the achievement or non-achievement of the project results? 1.3.2 Effectiveness • Did increased access to school feeding lead to reductions in short-term hunger and improved attentiveness? • Did Literacy Boost approach contribute to increased literacy among school-aged children? • To what extent have teacher trainings led to increased quality of literacy instruction and increased literacy among school-aged children? WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 1.3.3 Efficiency • To what extent did the project resources lead to the achieved results? 1.3.4 Sustainability • What are the project’s prospects for sustainability and what are the efforts the project has taken to date to improve the sustainability of the project in the long-term? 1.3.5 Impact • To what extent do the indicators selected affect the well-being of beneficiaries? • To what extent have the outcomes contributed to capacity development of beneficiaries, government of Mozambique, and community-based organizations? 1.3.6 Project Design Improvement • How could the project be modified to improve its acceptability to targeted communities and the efficiency and effectiveness of achieving results? • In what ways were the foundational results well designed to contribute to the project outcome? WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 2 Methodology and Sample Approach 2.1 Research Methods 2.1.1 Qualitative Research The qualitative research aims to capture perceptions and understanding about the general nature of a question, opening space for its interpretation. Thus, qualitative research has allowed a deep understanding of the perceptions (positive and negative) about the factors that are affecting the project in positive and negative way, the project effectiveness, efficiency and sustainability. The qualitative approach was implemented through collection of desk review, semi-structured interviews and Focus Group Discussions. a. Desk Review The Desk review consists of the collection of elements (documentary evidence) that serve as the basis for the definition of critical analysis variables. Concretely, the desk review was the foundation for the Mid Term Review. It enabled an understanding of the ECT2 implementation context. The desk review was conducted through various documents such as the project appraisal report, the baseline report, program proposals, budgets, strategies, plans, reports, studies, assessments, monitoring forms, implementation guidelines and policies, training manuals, among others. In addition, to complement the desk review external sources were used such as statistics from the Mozambican Ministry of Education, list of key informants, contacts, PTA general plans, and food distribution reports. b. Semi Structured-Interviews The semi-structured interviews consisted of conducting meetings with key stakeholders and direct beneficiaries based on a script containing open questions that captured perceptions about the ECT2 implementation. Concretely the consultancy team held interviews with the project operational team, implementing partners, the Ministry of Education, Ministry of Health (provincial and district education services), Save the Children and USDA, school children and their families who have been receiving the project’s services, school councils, farmers associations, health committees and other community groups who have been participating in the ECT2 project. c. Focus Group Discussion Focus Group Discussions consisted of encouraging informal discussions by groups of about 7 participants, who are induced to explain the problem through their opinions, experiences, ideas, observations, and preferences. Focus group discussions provided basis for the responses given in the Semi Structured-Interviews (triangulation) and, on the other hand, provided insights about the indicative relevance of the project implementation which allowed an understanding of the level of alignment of the project appraisal with beneficiaries’ needs. Focus group research tools were developed in line with existing tools as well as the project documents. The focus groups were conducted with school councils, farmers associations, health committees and other community groups who have been participating in the ECT2 project. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 2.1.2 Quantitative Research Quantitative research, theoretically, is characterized using the quantification of the problems considering the magnitude or frequency of the variables in analysis. For this work, the quantitative research was used to capture measures of central tendency, namely averages, medians and mode as well as in the calculation of proportions. For the quantitative research a survey for students who have been benefitting from the program was applied. Further quantitative component included data from secondary sources namely: school and SDEJT records. a. Survey The survey is one of data collection tools used to gather information about student and household characteristics and perceptions. The survey questionnaire was characterized by closed-ended questions where the answers were pre-defined by the consultants. The preparation of the survey questionnaire was multidisciplinary and involved the project steering committee ensuring that the answers provided were as exhaustive as possible. b. Early Grade Reading Assessment (EGRA) The EGRA tool is an international tool developed and approved by UNESCO that aims to measure reading performance of students in grade 3 who have just completed their grade 2. The assessment requires about 15 minutes to administer per child. One key task requires that a child read aloud for 1 minute, and then answer questions based on that reading. The students were randomly selected for the purpose. In Mozambique the EGRA was adapted to Portuguese by USAID and approved by the Ministry of Education and Human Development. Initially, the EGRA tool was applied in the Literacy Boost project namely “Aprender a Ler” implemented by USAID but currently is being used for most of the NGOs in Mozambique that are implementing Literacy Boost (LB) projects. 2.2 Sample Approach 2.2.1 Sample approach – Quantitative The quantitative approach allowed for an understanding of what extent the expected outcomes have been achieved. The project has two main outcomes. The first one is to improve Literacy of School-Aged Children and the second one is to Increase the Use of Health and Dietary Practices. These outcomes, according to the MQL, are measured through the number of children from grade 3 demonstrating reading capacity (Education component) and the number of individuals benefiting directly from USDA (nutrition Component). Thus, for the first outcome indicator, literacy tests were conducted to understand how many children are demonstrating reading capacity. The indicator for outcome two was calculated based on food distribution records allowing to be collected the number of beneficiaries of the food Program. Based on this background, a sample approach used gave a statistically relevant sample size of students. 2.2.2 Students – Sample Size For the sample selection a simple random sampling method was used. This method was applied in the assumption that beneficiaries have similar characteristics. This method ensures that each WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation sample unit (student) has the same probability of being selected and this known probability is nonzero. In addition, the simple random sampling technique ensures that all students selected represent the population. It should also be noted that this method guarantees that the results obtained in the midterm review can be generalized for all student beneficiaries. The sample was calculated considering a confidence interval of 95% and an error margin of 5%, assuming a distribution response of 50%. Considering this premise, the total sampled students were 382. To obtain this number, a software was used, as shown in the figure below. Sample Size Calculator Software 2.2.3 Schools – Sample Size The number of schools to be visited was based upon a 0% condition criteria2. This condition considers that 10% is enough to have a representative perspective of the population. Thus, based on this assumption, 16 schools (from 160 schools) were considered representative to ensure valid results. The school selection was randomly based on the list obtained from District Services of Education Youth and Technology. Regarding school selection, to ensure district coverage and capture eventual differences at the analysis stage between schools near the village and schools far from village, the sampled schools were divided into three groups namely schools in the district village or near (within a radius of 15Km), schools in the middle way from district village (within a radius of 30km) and schools far from the village center. The assumption was that distances from the village/SDJET affect schools in terms of conditions and teacher performance. The issue is that access to schools farthest from the village is complicated due to a lack of road infrastructure, making the supervision and intervention of the SDEJT in these schools minimal. Both Nacaroa and Muecate have 3 administrative posts. To compare the results inside the district instead of having 8 schools per district we had 9 schools per district making 3 schools for each administrative post. So, the total of schools studied was 18 schools. 2.2.4 Sample Approach – Qualitative Best practices3, recommend at least three focus group discussion sessions per cluster. To build a better accuracy, 36 focus group discussions were conducted (18 FGD sessions with School council committees and 18 FGD sessions with farmers groups). For the in-depth 2 Gay, L.R. and Diehl, P.L. (1992) Research Methods for Business and Management. Mc. Millan Publishing Company, New York. 3 Burrows D & Kendall S (1997) Focus groups: What are they and how can they be used in nursing and health care research? Social Sciences in Health Namey, et al (2016). Evaluating Bang for the Buck - A Cost-Effectiveness Comparison Between Individual Interviews and Focus Groups Based on Thematic Saturation Levels WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation interviews it is recommended at least 5 interviews for each relevant cluster. We proposed 9 interviews per relevant cluster per district. Thus, we had 18 interviews in total with teachers, volunteers and school administrators. For the stakeholders, the sample approach was based on the relevance of the stakeholders. Indeed, were talked with all the relevant project stakeholders. The stakeholders considered relevant were Save the Children, Ministry of Education and Human Development, Deputy of Education and Human Development, Deputy of Health – NPL, Deputy of Public Works and Housing, District Services of Planning and Infrastructure, District Services of Health, Women and Social Affairs, District Services of Education and Technology, District Government and United States for Agriculture Development. It should be noted that were taken the saturation criteria into consideration. Control Zone The control zone theoretically is important to understand the situation where the project is not being implemented. Concretely the Control Zone is crucial to support the validation of the effectiveness of the World Vision intervention. For the control zone, similar to the ECT2 baseline study, was selected the district of Murrupula, In the control zone at total were interviewed 66 children. The sample of the control zone was reduced, when compared to the treatment zone, given the fact that control zone result be just to have an indicative picture of the zone without any literacy boost intervention. The interviews were conducted in 3 schools. The division was proportional. Each school represents each stratum. 2.3 Evaluation Constrains ► Absenteeism of Teachers and Students during the Field Work – In some schools particularly those far from the villages, some students or teachers were absent in the schools during the field work. This situation occurs particularly on Friday afternoons and Monday mornings. This happens because most teachers live far away and must leave school early to get transport to their home. On Mondays they arrive late because they leave village early morning to get to school but they usually arrive late. This is not a conclusive finding but was a fact that affected our field work strategy. To overcome this challenge these schools were replaced with the nearest school. In total 4 schools were replaced. ► Use of Paper Based Questionnaires – For the field work paper based questionnaires were used. This occurred given the tight time to Program the questionnaires into the tablets given that the inception report and research tools discussions took longer than forecast. The decision was made to ensure the deadlines were met. To ensure quality in the data collection process, the data entry was done during the field work period allowing for errors to be quickly fixed by going back to the selected school and re-conduct the interviews. In addition, it should be noted that a double data entry 4method was used to give more consistency to the data entered. ► Baseline Limitation on the Sampling Approach - The sample approach designed in the baseline has some limitations. The sample frame was not very clear. The calculations were not clear and were not presented. Thus, was complicated to understanding how the students and the schools were selected. In this MTE, the team 4 Double data entry is quality assurance method. It allow to identify inconsistencies in the data entry process. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation had to reconstruct the sample approach and there is some data that is difficult to compare with the baseline results. Nonetheless, with the baseline and some secondary results it was possible to make some comparisons and reach some indicative conclusions about the project’s progress. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 3 Evaluation Findings 3.1 SO1- Improved Literacy of School-Aged Children 3.1.1 MGD1- Improved Literacy of School-Aged Children The main purpose of the project is to improve the literacy skills of pupils in primary schools and thus contribute to improving the quality of education in general. As foreseen in the methodology, one of the main instruments for evaluating the effectiveness of the project was the EGRA Test, which is essentially aimed at capturing the number of student beneficiaries of projects that know / do not know how to read. The test was applied to a representative sample of the total beneficiary population of Muecate and Nacaroa students, considering a margin of error of 5%. The data was collected by teams of enumerators with previous experience in conducting education and child-sensitive research (work with children under 10 years old). Another important fact to be mentioned in this component is the classification criteria that was used. Four main categories were considered: non-reader (cannot read at least 6 words per minute), emergent reader (reads between 6 and 44 words per minute), established reader (reads between 45 and 80 words per minute and proficient reader (reads more than 80 words per minute). The results show that the ETC2 intervention has contributed to an improvement in the reading ability of the 3rd class students. Indeed, the study found that in Muecate and Nacaroa, about 32% of the pupils read between 6 and 44 words per minute. On average, they read about 21.6 words per minute. Within Sub-Saharan Africa, the common range for English Speakers, is to read between 10 to 20 words per minute which indicates that Mozambique is slightly above average. Reading between 6 and 44, it is considered that the student knows how to read, whilst presenting difficulties (Emergent Reader). Looking from a gender perspective the results do not show significant differences. It was found that 34% of boys read about 21.6 words per minute. It is 3% more that the proportion of interviewed girls. When the results from the Mid Term are compared to the baseline results in which only 25% of children could read between 6 and 44 words per minute as well as the control group in which 25% of children can read between 6 and 44 words per minute, we reinforce the positive effect generated by the ECT2 intervention because the midterm results are higher. Therefore, after 2 years of intervention in the pedagogical component as well as the school feeding component, we can capture the positive effects of the project implementation. The challenge is related to the achievement of the planned target for the project end date. So far, the indicator is at 71% of execution. Chart 1: Percentage of students who by the end of two grades of primary schooling demonstrate that they can read and understand the meaning of grade level text – Baseline treatment Group vs Control Group Baseline MTE Control Group Achieved 25.10% 32% 25% Target 45% 45% 45% 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% 45.00% 50.00% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Table 1: Percentage of students who by the end of two grades of primary schooling demonstrate that they can read and understand the meaning of grade level text – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Improved Literacy of School Aged Children Percentage of students who by the end of two grades of primary schooling demonstrate that they can read and understand the meaning of grade level text Total 32% 45% 71% In Progress Male 34% 45% 76% In Progress Female 31% 45% 69% In Progress As foreseen in the appraisal, the ECT2 intervention has today 68444 beneficiaries. The target was achieved in 100%. When compared with the baseline results, it is notable an increase of 7% in terms of the number of beneficiaries. Looking at gender perspective, the project is covering more boys than girls. In total, 36544 (53%) are boys and 31900 are girls. Indicator Gender Baseline MTR Final Targets Level of Achievement Status Number of individuals benefiting directly from USDA – funded interventions Male 34088.0 36544.0 36544.0 100% Achieved Female 29628.0 31900.0 31900.0 100% Achieved Continuing 63716.0 68444.0 68444.0 100% Achieved 3.1.1.1 MGD 1.1- Improved Quality of Literacy Instruction The Improved Quality of Literacy Instruction was analyzed considering the number of teachers who received training in teaching and learning skills namely the Literacy Boost method. As expected, in the schools covered by the project, all teachers (708) who teach between 1st and 3rd class, received training in Literacy Boost. The training was given by the coordinators of the ZIPs who, in turn, were trained by staff from the Institute for Teacher Training. Based on the interviews with the supervisors and coaches, it was possible to find that practically all trained teachers now demonstrate improved literacy instruction. It represents an improvement when compared to the baseline results (74,8%). To support this statement, in the schools visited, the implementation of the methodologies assimilated in these training sessions is visible. For instance, in all the schools visited, illustrations on the classroom walls were strongly visible. In addition, in many of the schools visited the students were organized in a "U-shaped" format allowing the teacher to better monitor the students' attentiveness. Another strong indication of the implementation of the methodologies assimilated with the training is the application of the dynamics/songs used to motivate the children. These dynamics make children happier and improve their level of attentiveness. Despite the teacher’s implementation of the contents assimilated in the trainings, a lack of a check list for the tools from the School Administrator to observe whether the methodologies were being properly applied, was observed. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Chart 2: Percentage of teachers in target schools who demonstrate improved literacy instruction as identified by supervisors, mentors or coaches Table 2: Percentage of teachers in target schools who demonstrate improved literacy instruction as identified by supervisors, mentors or coaches – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Improved Quality of Literacy Instruction Percentage of teachers in target schools who demonstrate improved literacy instruction as identified by supervisors, mentors or coaches 100% 100% 100% Achieved Table 3: Skills and Knowledge of Teachers - Indicators Performance – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Increased Skills and Knowledge of Teachers Number of teachers/educators/teaching assistants in targeted schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance 566.0 566.0 100% Achieved Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance 708.0 708.0 100% Achieved 3.1.1.1.1 MGD 1.1.1- More Consistent Teacher Attendance Teacher attendance is a very important assumption and determinant to improving the quality of education. The ECT2 project, recognizing this part, is providing a set of tools to reinforce this component. One of the methodologies introduced to improve teacher attendance is the teacher performance awards, which consists in providing award to the best performing teachers in the school. The assignment of awards includes different criteria such as teacher attendance and the students’ academic performance. The objective of this methodology is to increase the motivation of teachers while performing their duties. As foreseen, best teachers have been awarded once a year. This component was introduced in the year 2016 where the teachers were awarded bicycles. In 2017, the best teachers received mattresses, linen and kitchenware. Qualitative research shows that teachers are mostly satisfied with the 2017 awards. Another strategy used to improve teacher attendance is the involvement of School Councils in attendance monitoring. Within the scope of the members of the School Councils, they must visit the school with daily regularity to verify the teacher attendance. For this, the School Council members uses its attendance record as issued by the project. This method has also proved to be effective since it complements the tools used by school administrators, namely class books and sign in register. In addition, teachers are more accountable because in addition to responding to school administrators they must respond to the community / school councils. Baseline Mid Term Achieved 74.8% 100.0% Target 100.0% 100.0% 74.8% 100.0% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation As a result of these interventions, it was found that about 96% of teachers attend school at least 80% of scheduled days per school year. To reinforce this statement, with the student survey conducted, it was found that only 30% of students reported that their teachers did not attend classes more than once. It is also possible to observe improvements when comparing the data to the baseline results (90%). This therefore reinforces the positive signs of the interventions carried out. However, it has been found that most teachers have their main residence in the main population centers (villages). The implication of this issue is that on Fridays in some schools the teachers must leave slightly earlier than the planned teaching time to be able to catch transport. Another constraint is that, a significant number of teachers are studying in universities in Nampula and usually use weekends and part of Monday to attend their classes. This issue also affects the time that teachers stay in school. Chart 3: Percentage of teachers in target schools who attend and teach school at least 80% of scheduled days per school year Table 4: Percentage of teachers in target schools who attend and teach school at least 80% of scheduled days per school year - Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status More Consistent Teacher Attendance Percentage of teachers in target schools who attend and teach school at least 80% of scheduled days per school year 96% 90% 107% Achieved 3.1.1.1.2 MGD 1.1.2- Better Access to School Supplies & Materials It was found that about 90% of schools were equipped with school supplies. When comparing this indicator with the baseline results (80%) it is possible to observe an increase in the access to school supplies and materials. The material was distributed via the teachers within the scope of the training that they received. Specifically, teachers received boards, A4 sheets, markers, rulers, etc. This material served for the elaboration of word walls, the letters for alphabets, etc. The challenge in this issue is the amount of material available. According to the teachers, the material provided is not enough to meet their needs. For the case of word walls, the expectation of teachers is that the drawings be in accordance with the subject that the teacher is teaching. With the quantities currently received, there is a very significant deficit. Baseline Mid Term Achieved 90% 96% Target 90% 90% 90% 96% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Chart 4: Percentage of classrooms with literacy instruction materials (text books, work books) sufficient for effective instruction Table 5: Percentage of classrooms with literacy instruction materials (text books, work books) sufficient for effective instruction – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Better Access to School Supplies and Materials Percentage of classrooms with literacy instruction materials (text books, work books) sufficient for effective instruction 90% 100% 88% In Progress Table 6: Number of classrooms with literacy instruction materials (text books, work books) sufficient for effective instruction – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Better Access to School Supplies and Materials Number of classrooms with literacy instruction materials (text books, work books) sufficient for effective instruction 405 450.0 90% In Progress Establish libraries The distribution was contemplated within the context of equipping schools with reading books. These books are a mechanism to strengthening reading skills. As expected, all schools received the reading books. In the schools visited it was noticeable that the books are placed in the school administrator’s office. In some schools the books are placed on a table outside the school administrator’s office. It was found that teachers encourage students to take the books and then summarize the stories read in classrooms. This approach ensures that there is a significant adherence to the books. The books are mostly used by students from grades 3 to 7. Usually the students stay with the books between three to five days. After this period the student must return the book and if they want they can take it out again. One of the constraints reported by some teachers is that the books are in very small print which makes it difficult for the children to read from the initial classes. Reading Camps – Extra Curricular Activities A very innovative approach from this project is the implementation of the reading camp sessions. The reading camps are Literacy Boost sessions moderated by a facilitator. Indeed, the success of reading camps activities strictly depends on the facilitator's ability to interact with children. The promoters are community volunteers without any teaching training. To enable them to facilitate the sessions, training is mandatory on regular basis. In the last year, the project organized and conducted training for 142 (109 men and 33 women) reading promoters in March 2018 in Nacaroa District. The training aimed at strengthening the promoters’ capacities to lead Baseline Mid Term Achieved 80% 90% Target 100% 100% 80% 90% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation the reading camps, the daily routines for each camp and preparing lessons for the first two months of the current year. The promoters as part of facilitating the reading camp sessions are also responsible for mobilizing the community to support the reading camps. As a result of this mobilization, an impressive 12,662 children (6,157 boys and 6,505 girls) were enrolled in the reading camps for the current year with 208 Reading camps created in 150 communities conducting 3,289 reading camp sessions during the reporting period. According to the facilitators, the students have been improving their literacy skills, the sessions are very interactive, and the children feel comfortable to express their tasks and to discuss their homework with the facilitators and other school activities. In addition, the community has more belief in the school’s relevance. As a result, the parents send their kids to the reading camp sessions. Figure 1: Reading Camp Session 3.1.1.1.3 MGD 1.1.5- Increased Skills and Knowledge of School Administrators As part of the ECT2 detailed implementation plan, the School Administrators were trained. This activity arises from the fact that many of the School Administrators perform the role without prior training. In total, 296 school administrators were trained – in two-day training sessions. The training consisted of the improvement of management skills such as administrative and process management, management of financial resources of the school and the management of human resources. The training session took place at the district headquarters. The trainers were part of Institute for Teacher Training staff. At the end of this training exercise the expectation was that participants would get skills in problem solving. According to the visits made, some notable changes in the schools were noted. For instance, in a significant number of schools were built new classrooms using local materials, which will allow for a reduction of the number of students per classroom. This initiative was indicated by the school administrators, as a result of the incentive that the project gave in collaboration with the school council. Therefore, School Administrators realize today that they can, based on local resources, develop some solutions to improve the teaching and learning environment. In addition, Administrators are now observing more regularly and rigorously the teacher’s attendance in classrooms. According to them, with the training they now understand more clearly the correlation between student academic performance and teacher attendance. In addition to the issues mentioned, the school administrators are better skilled in terms of planning, school absenteeism monitoring (attendance and punctuality), supervision and use of school supplies and tools for decision making. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Chart 5: Percentage of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance 3.1.1.2 MGD 1.2- Improved Attentiveness The implementation of ECT2 predicts that one of the main results is to improve students' levels of attentiveness. To measure the level of attentiveness, the logical framework matrix provides the perception that teachers have of the proportion of students who are regularly attentive. The mid-term results show that, teachers report that 90% of students in their classes have been attentive to their classes. The results still on progress considering the final target however based on the trend the possibility of reach the target is good. The interviews with teachers show that this result may be associated with a school feeding component where, because students are not hungry, they are more willing to attend classes. On the other hand, the interactive methodologies used by the teachers, assimilated with the implementation of the ECT2 (teacher trainings), have also influenced the level of student attentiveness. Teachers capture the students' level of attention through group dynamics where the shyest students are considered inattentive. However, as a constraint, it was found that teachers do not record these observations. It means that teachers do not have a systematic tool to regularly and effectively capture the actual number of students who are attentive to their class. When compared the MTR with baseline results it is possible to observe a slight decrease. Nonetheless, the difference it is not critical considering that 90% of attentiveness is under acceptable parameters. Chart 6: Percentage of students in target schools identified by their teachers as attentive during class/instruction Baseline Mid Term Achieved 98.7% 99.3% 98.7% 99.3% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% Baseline Mid Term Achieved 98% 90% Target 98% 98% 98% 90% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Table 7: Percentage of students in target schools identified by their teachers as attentive during class/instruction – Actual vs. Target Result Indicator Baseline MTR Final Targets Level of Achievement Status Improved Attentiveness Percentage of students in target schools identified by their teachers as attentive during class/instruction 98.0 % 90% 98% 92% In Progress 3.1.1.2.1 MGD 1.2.1- Reduced Short-Term Hunger School feeding is one of the main components of the ECT2 project. The premise is that school feeding influences various components of student performance, such as the level of attentiveness, level of retention, dropout rates, pupil academic performance, etc. Thus, it is in this component that the project has been investing its greatest effort. For the implementation of this segment, the project foresees the provision/distribution of Corn Soya Blend (CSB) mix of maize, meal and soya flour, and vitamins. In addition to improving students' school performance, the distribution of these CSB is also intended to improve children's nutrition levels since chronic malnutrition is also a critical issue in the country, especially in the province of Nampula. The CSB have been distributed to all schools and expects to reach about 65,806 student beneficiaries. This number indicates that the targets foreseen by the project were 100% achieved. Of these, 35,084 are boys, equivalent to about 53% of the total student beneficiaries and 30,722 are girls, an equivalent of about 47% of the total student beneficiaries. In addition to the student beneficiaries, the project also includes other beneficiaries who are the volunteers and some teachers (supervisors of the school feeding). Thus, 1,939 beneficiaries were additionally included in the total. These receive the CSB at the end of the month and take them to their homes as compensation for the support that they give the school in the preparation, distribution, and control of food. Therefore, in total 67,745 people have benefited from the school feeding. The study found that about 99% of the students interviewed indicate that they are not hungry during school day and are satisfied with the food portion served. An increase was seen when compared to the baseline data (52.6%). This data therefore shows that practically none of the students receiving food under the project implementation report short term hunger. This is one of the major achievements of the project implementation. The study in addition explored perceptions regarding the taste of food. About 99% of the sample affirmed to like the food. According FGD discussions, the students that are not satisfied with the quality of food probably were influenced by the round of CSB that came with a different taste. However, the issue was resolved through sensitization of the cooks and students. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Chart 7: Percentage of students in target schools who indicate that they are not hungry during school day Chart 8: Beneficiaries satisfaction with the quality of the food Table 8: Percentage of students in target schools who indicate that they are not hungry during school day – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Reduced Short￾Term Hunger Percentage of students in target schools who indicate that they are not hungry during school day 99% 100% 100% Achieved 3.1.1.2.2 MGD 1.2.1.1/1.3.1.1- Increased Access to Food (School Feeding) The project core intervention is to provide meals to improve food security levels and improve children performance in schools. A total of 65806 school aged children receiving daily school meals (breakfast, snack or lunch) as a result of USDA assistance. The planned target was achieved. From the beginning of the project implementation to date 51,093,460.00 meals (breakfast, snacks or lunch) were provided. It represents 43% of the meals planned. To support the most vulnerable children the project planned the provision of take-home rations. To date, a total of 527,289 take home rations (46%) have been provided as against a total of 1,134,000 planned for about 728 beneficiaries. Apart from the school aged children, the project planned to provide meals to volunteers (cooks) and the teachers that support store rooms management. Thus, in total including this groups, it sums up to 67745 beneficiaries receiving social assistance where 54% are men. Table 9: Access to Food Indicator Results Baseline Mid Term Achieved 52.6% 100.0% Target 100.0% 100.0% 52.6% 100.0% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% Satisfied Non-Satisfied Achieved 99% 1% 99% 1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Result Indicators Unit Baseline MTE Final Targets Level of Achievemen t Status Increased Access to Food (School Feeding) Number of individuals receiving take￾home rations as a result of USDA assistant Nr. 0.0 728.0 728.0 100% Achieved Number of take-home rations provided as a result of USDA assistant Nr. 0.0 527,289. 00 1,134,000. 00 46% In Progress Number of school aged children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance Male Nr. 32775.0 35084.0 35084.0 100% Achieved Female Nr. 24726.0 30722.0 30722.0 100% Achieved Continuing Nr. 57501.0 65806.0 65806.0 100% Achieved Number of daily school meals (breakfast, snacks, lunch) provided to school aged children as a result of USDA assistance Nr. 12615812 .0 22,174,9 80.00 51,093,46 0.00 43% In Progress Number of social assistance beneficiaries participating in productive safety nets as result of USDA assistance Male Nr. N/A 36246.0 36246.0 100% Achieved Female Nr. N/A 31499.0 31499.0 100% Achieved Continuing Nr. N/A 67745.0 67745.0 100% Achieved Partnership with Farmers Groups The partnership with farmers’ groups is one of the strategies to minimize the project exit effect. Specifically, in the process of implementing the project, the creation of farmers groups to work with schools was planned. The objectives of these associations are to provide fresh food to schools namely, corn, peanuts, cassava, sweet-potato, beans, etc. In this process World Vision provides inputs and technical assistance to farmers' associations. As planned, farmers groups were created. In total, World Vision is working with about 42 associations in Muecate and 38 associations in Nacaroa. Each farmers group is composed of, on average, 25 members. The rationale is that each farmers group works with a school. However, there is an association that works with two schools because there are areas (where more schools are located) that do not have adequate agro-climatic conditions for agricultural production. It should be noted that not all farmers groups were created by the project in phase 2. Some farmers group already existed from phase 1. The direct result of the project was the legalization of the farmers group and support for them to obtain the DUAT (Right to Land Use). Therefore, all the farmers groups linked to the project have been legalized. In addition to the legalization, the ECT2 project has supported associations in terms of technical assistance. Specifically, in the dissemination of improved agricultural practices and technologies such as row planting, use of improved seeds, use of pesticides, and use of fertilizers etc. According to the report given by the farmers, this assistance has been providing added value to them since it has improved their production performance. In addition to the technical assistance, in the first year of the project implementation, the ECT2 supported farmers with inputs, namely improved seeds for horticultural crops, hoes, irrigators, etc. As initially foreseen by the project, associations have been cooperating with schools. Specifically, these have shared part of their production with schools. However, the data collection found a lack of an instrument to capture the quantities disbursed in relation to the quantities produced. This issue is relevant because the agreement was that 30% of the production from the associations should be donated to the school. Another constraint in this model is that historically, Muecate and Nacaroa are not districts with high potential for agriculture or with clear precedents for farming (Agricultural Census 2009–2010). This results in poor productivity. Considering the first and second constraints, the quantities available to schools remain WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation insignificant considering the number of pupils per school as well as the frequency with which the food should be distributed. For instance, in some cases the associations offer between one and five 50 kg bags of corn. These quantities can barely be used for one day's consumption. Despite the insignificance in terms of quantity, it was found that this strategy at least improves the linkages between the community and the school. On the other hand, it was noticed that the diet of the students and farmers group members is more diversified because the vegetable component was introduced. Another point is that some farmers besides consumption also do sell their production. As a result, these farmers improve their income levels alleviating their poverty levels. In addition, another interesting point is that the supply of vegetables in the markets of Nacaroa and Muecate has increased noticeably, according to the residents as a consequence of the ECT2 farmers production. 3.1.1.3 MGD 1.3- Improved Student Attendance The study shows that the student attendance issue has been stable over the last years. Today, about 80% (51,103) of the children who benefit directly from the project, attend school at least 80% of the school days. In terms of gender, about 80% of boys (27,252) as well as girls (23,852) beneficiaries of the project have participated in 80% of classes during the school season. These numbers do not differ much from the data collected during the baseline, where about 90% of the students participated in at least 80% of classes during the school year. According to the focus group discussions, the attendance reduction can be associated to the rise of living cost in the last two years and food scarcity. As strategy to overlap this situation, some parents have been forcing their children to support them on income generation activities (farm, sale of products, etc.) limiting them to go to school. When comparing the MTR results with the final target it is possible to observe that this target is on a positive way to be achieved. This result, in part, indicates the effectiveness of the school feeding program (first and second phase) where it is presumed that students attend school more to eat. But it is also the result of other interventions, such as the increased attendance of teachers in the classroom and the work that the school councils have done to monitor the attendance of students. In addition, lessons, particularly in the early classes, are more interactive and make students more motivated to attend school. The percentage of students in target schools who miss more than 10 school days per year due to illness is also low (about 2%). It can also be associated to the ECT2 Project given its strong influence on malnutrition levels reduction. Based on focus groups, which corroborates with theories on the subject, student attendance depends also on socio-cultural factors namely how parents and caregivers view education, the distance between home and school, number of members in the household, etc. In addition, attendance behavior may vary depending on the season of the year. For example, in lean season (September, October, November, January, February and March), sometimes students must support the parents in their search for food. At harvest season, students must support their parents with the harvest. Therefore, the exercise of control / management of student attendance must be conducted with regularity. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Chart 9: Percentage of students that attend school at least 80% of the school days Table 10: Percentage of students that attend school at least 80% of the school days – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Improved Student Attendance Percentage of students that attend school at least 80% of the school days 80% 95% 84% Achieved 3.1.1.3.1 MGD 1.3.1- Increased Economic and Cultural Incentives (Or Decreased Disincentives) Savings and Loans Groups The Savings and Loan Groups have proven to be a very powerful experience in the life of the community. To date the ECT2 has created about 163 Savings and Loans Groups. According to the interviews with the project managers, all the groups are working. The field team visited 18 groups and verified that the 18 groups are in operation. The groups meet once a week where members are required to make at least one transaction namely, savings or a loan. In general, all members have been active in the group. This means that the volume of savings and the volume of loans has tended to grow. It has been verified that the average daily saving is around 300 Meticais per group of 10 members and the average daily loan is about 200 Meticais for a group of 10 members. The loan interest rate is 10% of total amount. The loan is generally applied in activities that generate income, especially small businesses, such as selling cookies, selling soap, etc. Credit is also applied to lifetime events namely, deaths, births, marriages, etc. All groups have a treasurer who is responsible for keeping the cash in a safe. In none of the groups visited was any connection with a financial institution noted. At the end of the year the saved amount added to the interest is divided among the group members proportionally. The amount is generally used to purchase durable goods and to pay children’s school fees at the beginning of the year. Baseline Mid Term Achieved 90% 80% Target 95% 95% 90% 80% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Number of Groups Created and Working 163 Average daily saving 5 USD Average daily Loan 4 USD WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Another interesting fact about the savings groups is the existence of a social fund, which is usually allocated to school expenses. The social fund is the result of a contribution of 5 MT per member in each meeting, with the goal of providing some of the school needs. It is, in fact, an outlet for the purchase of dishwashing detergents for washing pans and other materials such as buckets, spoons, kitchen necessities. It was observed that all groups have been using the same approach which is to some extent an exit strategy to minimize the effect of project termination (sustainability). Figure 2: SG Meeting in Minheuene School, Muecate District 3.1.1.3.2 MGD 1.3.2- Reduced Health Related Absences The Percentage of students (Boys) in target schools who miss more than 10 school days per year due to illness have been reducing. Indeed, it was found that about 1.8% of the students miss more than 10 days per year. It, among various factors, is related to the school feeding initiative that contribute to better children nutrition levels. Once the nutrition levels improved reduces Poor immune function, which can cause the body to have trouble fighting off infections. Concretely nutrition prevents, of common diseases or symptoms such as dizziness, headache, bloated stomach, fatigue and low energy, Swollen and bleeding gums. Looking at gender perspective there were no major differences. Chart 10: Percent of students (Boys) in target schools who miss more than 10 school days per year due to illness. Baseline Mid Term Boys Achieved 2.1% 1.9% Target 1.5% 1.5% 2.1% 1.9% 0.0% 1.0% 2.0% 3.0% 4.0% 5.0% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Chart 11: Percent of students (Girls) in target schools who miss more than 10 school days per year due to illness. Table 11: Percent of students in target schools who miss more than 10 school days/year due to illness – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Reduces Health Related Absences Percent of students in target schools who miss more than 10 school days/year due to illness 2% 1,5% 75% In Progress 3.1.1.3.3 MGD 1.3.3- Improved School Infrastructure Construction/Rehabilitation: Classrooms To minimize one of the main challenges, not only for the project implementation but also at national level, which is the lack of classrooms, the ECT2 project has rehabilitated 17 schools (2 schools above the planned). This intervention, totalizes 72 classrooms. The school’s rehabilitation intended to reduce the number of students who study in precarious classrooms or under the trees. At the same time, this intervention contributes to lower student ratios per classroom. According to the project monitoring reports, the expected results were achieved as planned. For example, children above grade 5 from Halaca community in Nacaroa district used to have to travel 16 km to go to school, but now with the new classrooms built, Halaca EP1 has been upgraded to Halaca EPC. Another notable example is that of Namacozi School in Muecate district which has also been upgraded to EPC, and as a result the number of students enrolled in 2017 has increased from 59,341 to 63,911. This impacted on students that had had to walk 7 km to the nearest EPC school and contributed to reducing dropout rates. Baseline Mid Term Girls Achieved 2.3% 2.0% Target 1.5% 1.5% 2.3% 2.0% 0.0% 1.0% 2.0% 3.0% 4.0% 5.0% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Figure 3: EPC de Halaca before World Vision Intervention Figure 4: EPC de Halaca after World Vision Intervention Rehabilitation: Canteens The project anticipates the construction of canteens where students may seat and feed themselves. This activity was carried out as planned and 150 school canteens were built. The construction of school canteens was done using precarious material. The contribution of the ECT2 was in the acquisition of iron sheets for the roof. The schools visited reported that the canteens were used by students for the intended purpose. However, given the lack of quality of the equipment used in the construction, in the last rainy season, a significant part of the canteens were destroyed. Therefore, there is a need to review the construction carried out. A survey is currently being carried out to ascertain the concrete number of canteens affected and to design strategies to restore the destroyed canteens. Table 12: Number of Canteens built Indicator Untit Baseline MTE Final Targets Level of Achievement Status Number of Canteens built Nr. 150.0 123.0 150.0 82% In Progress Rehabilitation: Latrines A total of 1430 latrines have been built as a result of ECT1 and ECT2 intervention. Thus, this target is in progress however, the perspective of be achieved is positive considering the remaining time to close the project. As consequence of this intervention, the environmental sanitation levels in schools have improved greatly through community awareness work done by health volunteers. The old habits such as open defecation and the improper use of latrines have significantly reduced, as students have adopted proper use of latrines and are promoting similar behavior in their homes and communities. Table 13: Number of Latrines Built Indicator Untit Baseline MTE Final Targets Level of Achievement Status Number of Latrines built Nr. 440.0 1430.0 1595.0 90% In Progress Rehabilitation: Store Rooms and Stoves A total of 48 energy-saving stoves were rehabilitated out of 90 planned covering 53% of the target. However, a few stoves were destroyed due to the heavy rains in the last season. Therefore, a new proposal was submitted to the USDA to support reconstruction. Despite this constraint, the target schools affected keep providing meals for their children daily. Table 14: Number of Storerooms rehabilitated Indicator Untit Baseline MTE Final Targets Level of Achievement Status Number of Storerooms rehabilitated Nr. 0 48.0 90.0 53% In Progress 3.1.1.3.4 MGD 1. 3.4- Increased Student Enrollment Nampula province has come out on top of schools with highest number of school-aged children who do not go to school. This situation is mainly related to cultural issues. For example, girls culturally marry earlier because some parents think girls attending school adds no value to their lives (Ministry of Education and Human Development, 2017). According to the focus groups with the community, it was understood that this issue is now occurring with less frequency. The WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation community reports where the awareness work was carried out by World Vision on the importance of education, parents let their children attend the school at earlier ages. In the last semester, 27 community mobilization and awareness meetings were conducted and a total of 1,191 participants attended these meetings where issues such as the importance of education, community roles and responsibilities for student learning, and the relationship between food, health, nutrition and education were discussed. These meetings were aimed at encouraging parents, guardians and community members about their roles and importance of school and to seek their active participation in both school activities and the reading camps. According to ECT2 M&E reports, community mobilization meetings were reinforced by mobile radio broadcasting campaigns conducted by local radio stations, covering 53 communities and reaching an estimated 29,400 listeners in both districts. As result of World Vision intervention to promote enrollment, the foreseen targets were achieved in 100%. In the data collection period a total of 66750 children were enrolled. Looking at gender perspective, the targets were also achieved as planned. Table 15: Number of students enrolled in school receiving USDA assistance Result Indicator Result Unit Baseline MTE Final target Level of Achievement Status Increased Student Enrollment Number of students enrolled in school receiving USDA assistance Male Nr. 32775.0 36000.0 35084.0 101% Achieved Female Nr. 24726.0 30750.0 30722.0 100% Achieved Continuing Nr. 57501.0 66750.0 65806.0 101% Achieved 3.1.1.3.5 MGD 1.3.5 Increased Community Understanding of Benefits of Education The community understanding of education was measured through the parents' perception about education. This was captured in the focus group discussions. It was found that parents uniformly consider education as relevant to their children. Parents today, look at education as a strategy for poverty reduction. Indeed, to reinforce this finding, the number of children enrolled tends to grow. Currently, as mentioned before, there are 65806 children enrolled in Muecate and Nacaroa. It represents 15 % of increase when compared to the baseline year. Parents reported that in the community there are concrete examples of young people who today have better living conditions as a result of education and thus have been, encouraging their children to attend school. This awareness is also explained by ECT2 project education awareness campaigns. 3.1.1.3.6 MGD 1.4.1- Increased Capacity of Government Institutions The Government is one of the main stakeholders to ensure the project sustainability. Considering this perspective, within the scope of ECT2, actions are foreseen to generate greater ownership of the project by the Government. One of the key components considered in this line was the training of key education technicians, namely IFP (Institute of Teacher Training), DPEDH (Provincial Department of Education and Human Development) and SDEJT (District Services of Education) officers. Indeed, as planned, the training was carried out. In total, 48 Government officials were trained and given standard operating procedures and tools for management and oversight of school feeding programs and literacy boost. The training reviewed the general Literacy Boost approach; Literacy Boost-teachers training; 5 core reading skills (but main attention was given to knowledge of letters and phonemic awareness) and production and practice of the use of didactic material for these skills. Number of district and provincial MINED officials who know standard operating procedures and tools for management and oversight of school feeding programs and literacy boost 48 WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Table 16: Number of district and provincial MINED officials who know standard operating procedures and tools for management and oversight of school feeding programs and literacy boost – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Increased Capacity of Government Institutions Number of district and provincial MINED officials who know standard operating procedures and tools for management and oversight of school feeding programs and literacy boost 48 48 100% Achieved 3.1.1.3.7 MGD 1.4.3- Increased Government Support Given the relevance of the school feeding program and to ensure the sustainability of the feeding Programs interventions, World Vision has a role to influence the Government to have a more significant action in this field. This influence has been conducted through a presentation of the project results to the government. As a result, it is verified that the number of Education coordination meetings per year where school feeding is part of the agenda has increased. Currently at least 5 annual meetings are held where school feeding is part of the agenda. A total of 11 meeting where school feeding was discussed occurred since the beginning of the project. It corresponds to 31% of the foreseen target that is 35. Another indicator is the amount of public sector investment in school feeding that has increased. Currently the Government is spending 9.7 million USD annually to support school feeding in about 150 schools at national level. The amount is not significant, and the number of schools remains low but it indicates a progress when compared to the past years. Number of Education coordination meetings where school feeding is part of the agenda 11 Table 17: Percentage of Education coordination meetings where school feeding is part of the agenda Result Indicator Baseline MTE Final Targets Level of Achieveme nt Status Increased Government Support Percentage of Education coordination meetings where school feeding is part of the agenda 0 31.0 100.0 31% In Progress Chart 12: Value of public sector investments leveraged as a result of USDA assistance Source: MINEDH, 2018 2017 2018 Series1 8,000,000.00 USD 9,776,453.33 USD 0.00 USD 2,000,000.00 USD 4,000,000.00 USD 6,000,000.00 USD 8,000,000.00 USD 10,000,000.00 USD 12,000,000.00 USD Axis Title WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 3.1.1.3.8 MGD 1.4.4- Increased Engagement of Local Organizations and Community Groups To reinforce the ECT2 ownership, feedback sessions involving Government and community members were organized. The real purpose of this intervention is to align together the project expectations and make sure that the stakeholders have the same project perspective. For that purpose, the strategy was to create Humanitarian Accountability Committees. In total 37 Humanitarian Accountability Committees (HAP) were established and trained in each of the 20 Pedagogical Influence Zones (ZIPs) in Nacaroa district, and 17 ZIPs in the district of Muecate. Each committee is in one ZIP and involves 6 to 7 members from schools belonging to the ZIP. The committees have a management structure composed of a president, vice president and secretary, who are elected for a term of 1 year. These committees submit reports to the ECT2 informing them of community feedback on the project implementation. The community feedback is channeled through the 37 suggestion boxes that were established in the schools. Thus, based on the community feedback, sessions with the community are organized in order to discuss the suggestions ensuring thus the referred project implementation expectations are aligned. Figure 5: Community meeting for Feedbacks providing Figure 6: Suggestion Boxes in use As a challenge related to Government capacity, a need to strengthen school supervision was verified. On the one hand, from the perspective of there being tools that allow for this supervision, and, on the other hand, challenges related to a lack of resources for supervision. In relation to the first component, it should be noted that the ECT2 has updated its tools to become more adjusted to the intervention being carried out. These tools are not the domain of the Government staff, so there is a need to ensure their familiarization. For instance, the tools that are used by school councils to monitor teacher attendance. So, it makes sense to organize training sessions with the Government to discuss these tools and ways to supervise the implementation of this tools. 3.2 SO2- Increased Use of Health and Dietary Practices 3.2.1 MGD2- Increased Use of Health and Dietary Practices 3.2.1.1 MGD 2.1- Improved Knowledge of Health and Hygiene Practices Regarding the health component, the strategy used by the project has been to train key stakeholders in terms of health and hygiene, so they disseminate the knowledge acquired. Indeed, the ECT2 has trained to date, about 1487 individuals. These include volunteer teachers and members of school boards. The logic is that trainees organize sessions with the school community to share the acquired knowledge. The training covered children health related contents (immunization, de-worming, pregnancy, food and nutrition); drinking water storage and management (including water treatment); sanitation and hygiene (safe disposal of feces, hand WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation washing at critical times, personal hygiene); environmental hygiene (including rubbish pits, laundry areas, keeping animals away from food); and control of wastewater. As a result, during the project implementation, various health education sessions were conducted with focus on the students. The topics most covered and most involved were: hygiene, water treatment and conservation with 40 sessions, personal and collective hygiene with 39 sessions and transmission and prevention of malaria with 34 sessions. The visits made to the schools show that this health education exercise has been working. Teachers report that there has been a significant reduction in students who are not attending school as a result of hygiene disorders such as diarrhea. Furthermore, the number of pupils who are dirty in schools has been reduced. Indeed, as presented in the chart below, the percentage of children in target communities who can identify at least 3 important health/hygiene practices increased to 86% when compared to the baseline (62%). The hygiene practices considered in the analysis were taking a bath, using latrines and washing hands. The survey captured also that 94% (from 12 – 18 years) of the students know how to wash their hands. Figure 7: Teachers and School Council Health Matters Training Session Figure 8: Health Education Sessions Chart 13: Percentage of children in target communities who can identify at least 3 important health/hygiene practices Table 18: Percentage of children in target communities who can identify at least 3 important health/hygiene practices – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Increased Knowledge of Health and Hygiene Practices Percentage of children in target communities who can identify at least 3 important health/hygiene practices 86% 70% 123% Achieved Chart 14: Percent of target beneficiaries who use appropriate hand washing practices 3 hygiene practice 2 hygiene practice 1 hygiene practice Baseline 62% 30% 8% Mid Term 86% 10% 3% 62% 30% 8% 86% 10% 3% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation Table 19: Percent of target beneficiaries who use appropriate hand washing practices - Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Increased Use of Health and Dietary Practices Percent of target beneficiaries who use appropriate hand washing practices 94% 95% 99% In Progress Percent of target beneficiaries who use appropriate hand washing practices Male 94% 95% 99% In Progress Female 93% 95% 99% In Progress 3.2.1.2 MGD 2.2 Increased Knowledge of Safe food prep and Storage Practices It was observed that a significant portion of volunteers for food preparation were trained in the implementation of ECT1. However, there are volunteers who are new (a few) resulting from dropouts. The new volunteers received on the job training that was done by the teacher school feeding supervisor as well as by their senior colleagues. In fact, the monitoring reports as well as the visits, have demonstrated that volunteer cooks are preparing meals within the time allowed, kitchen volunteers are clean, arrive on time and abide by safe food preparation processes, including covering their heads, hand washing, and using cooking utensils appropriately. The consistency of CSB and its preparation is adequate and within the standards. The sanitation within the kitchen area is clean and satisfactory. Despite the positive aspects presented which is related to the performance of the volunteers, it has been understood that the volunteers feel that they should receive some compensation for the work done. Compensation would be symbolic to satisfy their basic needs such as purchasing hygiene detergents and eventually some school expenses. Currently, the beneficiaries' incentive has been the CSB take-home rations according to the working days as well as the health card that allows them to have medical examinations and free treatment in the health unit. Table 20: Percentage of food preparers at target schools who can identify at least 3 key practices aimed at safe food preparation – Actual vs. Target Result Indicator MTR Final Targets Level of Achievement Status Increased Knowledge of Safe food prep and Storage Practices Percentage of food preparers at target schools who can identify at least 3 key practices aimed at safe food preparation 100% 100% 100% Achieved Baseline - Not Available Mid Term Series1 0% 94% 0% 94% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 3.2.1.3 MGD 2.3- Increased Knowledge of Nutrition The training in nutrition was conducted as planned. The target were teachers and volunteers. A total of 1487 individual were trained from 2015 to date. These training sessions aimed to improve and increase teachers' and volunteer’s knowledge of good nutrition practices. The expectation was the teachers and volunteers at the end of the training, replicate the acquired knowledge in the form of education sessions in order to change students and community in general nutrition practices. All participants received didactic material as well as nutrition manuals. As planned, trained Teachers and board members disseminated age appropriate information to students. Grade 1 to 3 topics covered the importance of nutrition for growth, nutrition during the period of illness, the 4 food groups, and hygienic food preparation. Grade 4 to 7 topics covered balanced and adequate diet, nutritionally rich foods, locally available foods, the needs of pregnant women and women of childbearing age, exclusive breastfeeding. During the dissemination of information to students, teachers and school board members used serial albums, posters and manuals for demonstrations, illustration of the images, and actions to be taken to change behavior. Table 21: Number of individuals trained in child health and nutrition as result of USDA assistance Result Indicators Gender Baseline MTR Final Targets Level of Achievement Status Increased Knowledge of Nutrition Number of individuals trained in child health and nutrition as result of USDA assistance Male N/A 482.0 818.0 59% In Progress Female N/A 1005.0 546.0 184% Achieved Continuing N/A 1487.0 1364.0 109% Achieved Use of Tippy-Taps About 68500 individuals both from Nacaroa and Mueacate, from160 schools were trained in the usage and setting up of tippy-taps. Tippy Taps have appeared to be a powerful strategy for improving the hygiene levels of children and the school community. It is noted that all schools currently in Muecate as well as Nacaroa have tippy Taps and these are often used by the school community. The other positive effect is that parents also have introduced Tippy Taps into their homes which also improves the hygiene situation of all household members. The limitation found in some schools is the lack of water. Indeed, some schools must travel distances to get water for sanitary care. Figure 9: Students washing their hands from tippy-taps before school lunch in Nacaroa. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 3.2.1.4 MGD 2.4 Increased Access to Clean Water and Sanitation Services All schools covered by the project have water sources systems installed and therefore access to clean water. It was observed that the water sources are in good condition as a consequence of the water committee’s efforts. Indeed, besides the regular visits conducted by water committee members, a tax of 0.12 USD per month is applied to the beneficiaries’ households to ensure the maintenance of the water sources. According to the focus group discussions, this strategy has been effective and most of the households are collaborative. In spite of this model’s effectiveness, there are two main challenges faced. The first challenge is related to the lack of water (natural restriction). In these cases, the water comes with low pressure. The second challenge is related to number of beneficiaries that utilize the water source. In most of the cases the school must share the water source with the community. Thus, it affects the water source’s maintenance and consequentially the water source’s life time. As a result of these challenges, particularly the first one, some schools represented by the volunteers must travel long distances to find water particularly to cook the food for the children, which, in some cases, affects the feeding timings. Related to the Sanitation Services the results are also significantly positive. Indeed, 137 (85%) schools covered by the project have improved sanitation services and it was verified that the children use it. A total of 1,507 latrines were built. The built latrines are named Ventilated Improved Pit (VIP) and are equipped with ventilation pipes to get rid of flies and smells, and a concrete platform that is easy to keep clean. These improved latrines have proven so effective that local people have taken the initiative and built many more. The sanitation services are separate for students and teachers which is good because children are not exposed to possible adult infections. The sanitation units also obey gender division. According to the focus groups with the teachers, prior to the intervention, children were often exposed to diarrheal diseases hampering their ability to concentrate on their studies. In addition, the improved latrines are very important for girls. Reports from community indicate that when they reach adolescence, they had been forced to quit school given a lack of privacy when going to toilet. The challenge in the sanitation component is first related to a lack of water for the toilets due to the factors previously presented. The second challenge is the lack of someone strictly responsible to clean the sanitation units. The result is that the toilets are in most cases very dirty. To overcome this second challenge, the school councils have supported the schools but only once a week. In some schools the students clean the toilets, but the strategy is not completely effective due to the other activities that they have. Figure 10: New blocks of latrines built at Sapala 3.2.1.5 MGD 2.5- Increased Access to Preventive Health Interventions Number of Latrines Built 1,507 Number of Schools Covered 137 – 85% WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation As part of the ECT2 execution plan strong interventions were planned in terms of preventive health through the provision of vaccines, deworming as well as supplements. As a result of this intervention, around 55.000 students were dewormed in the last year of which 30,489 were boys and 24,570 were girls from 160 schools, 83 in Muecate and 77 in Nacaroa. The deworming was done by a team composed of Provincial Directorate of Health and project staff. These campaigns happen 2 times per year. Regarding vaccinations, in the last year about 9,200 children were vaccinated against tetanus where 4,939 (54%) were girls. It should be noted in addition that 9,200 children were vaccinated 30% were from Nacaroa and 70% from Muecate. This campaign also occurs 2 times a year at the same time as the deworming. Beside this vaccination and deworming supplements for girls with folic acid were also distributed. In total, 2056 supplements were distributed to the girls. The preventive health campaigns also included medical examinations. In total, about 13,000 children were examined, 7,000 boys and 6,000 girls respectively. The most common diseases found were scabies, eye problems and dental caries. Some students received prescriptions and others, depending on the disease complexity, were referred to the health units for further analysis and treatments. Number of Students Vaccinated 9200 Figure 11: Students taking medication during distribution of medication Figure 12: Health Examination Sessions 3.2.1.6 MGD 2.6- Increased Access to requisite Food prep and storage tools and equipment It was noted that 100% of targeted schools had been fully equipped with proper functioning cooking items. These comprised of a stove and a store room for the food. It was observed that materials are very well kept by the cookers. Also refer to the performance that teachers responsible for the store rooms as excellent. Considering the facts, the observed commitment it is expected that without project intervention the storage tool be well kept. The challenge probably would be ensuring that without project intervention the school replace the degraded equipment. Table 22: Number of schools with appropriate food prep and storage equipment Result Indicators Baseline MTR Final Targets Level of Achievement Progress Increased Access to requisite Food prep and storage tools and equipment Number of schools with appropriate food prep and storage equipment 150.0 160.0 160.0 106% Achieved WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 3.3 Regression Analysis The Regression Analysis was conducted to understand the ECT2 project effect on the academic performance. For this data was collected from grade 3 students. Specifically, marks in Portuguese were collected. Portuguese was the chosen subject since the ECT2’s intervention is more literacy oriented. So, the results constitute, in addition, a mechanism to also understand the effects of the Literacy Boost training. The data was gathered in the treatment and control area. The purpose was to understand whether the ECT2 beneficiary project affected positively the student academic performance in Portuguese. In total data was collected from 382 students in the treatment zone and 382 students in the control zone. This number comes from the sample that was calculated for the survey. For the regression analysis the Ordinary Least Squares Model (OLS) – Log Linear Model was used. Indeed, the equation below shows the interaction between grade 3 student final marks (Y) and belonging or not to ECT2 (X). The student’s final marks (Y) vary from 0 to 20. The Y variable was logarithmic to change the regression equation interpretation to perceptual. Indeed, equation informs to what percentage the fact of belonging to ECT2 influences the student’s final marks. 𝒍𝒍𝒍𝒍𝒍𝒍𝒍𝒍𝒊𝒊 = 𝜶𝜶 + 𝜷𝜷𝑿𝑿𝒊𝒊 + 𝝐𝝐𝒊𝒊 Regression Analysis Result The results show a positive correlation between being an ECT2 project beneficiary and grade 3 student performances in Portuguese. The results show that being an ECT2 beneficiary increases the final marks in Portuguese of students from grade 3 by 6.5%. Another important result, looking at R Square is that ECT2 project explains 5% of a student’s performance in Portuguese. These results are statistically valid considering the confidence interval of 95%. The results illustrate a positive effect of the ECT2. However, student performance in general is not only explained by ECT2 but also by other variables such as teacher academic performance, teacher absenteeism, school materials, student household composition, student extra school activities, etc. Regression Statistics Multiple R 0,227581516 R Square 0,051793347 Adjusted R Square 0,050548981 Standard Error 0,139734916 Observations 764 ANOVA df SS MS F Significance F Regression 1 0,812710409 0,81271 41,62228772 0,00000000019673 Residual 762 14,87869518 0,019526 Total 763 15,69140559 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Lower 95,0% Upper 95,0% Intercept 0,966688591 0,00714946 135,2114 0 0,952653614 0,980723568 0,952653614 0,980723568 X Variable 1 0,065230578 0,010110864 6,451534 0,00000000019673 0,045382123 0,085079033 0,045382123 0,085079033 WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 3.4 Sustainability The sustainability issue is always one of the main challenges for the implementation of development projects. The ECT2 project is a complex and complete project and its sustainability challenges are even greater. However, according to the visits made, there are very positive signs which indicate that in the way that the project is being implemented there is room for long-term effects and replicable results. Below issues that affect the ECT2 sustainability are presented. (+) Teacher Training- Teacher training through ZIP coordinators is an excellent initiative to easily replicate knowledge. The ZIP coordinators will be able to pass the knowledge on to incoming new teachers. On the other hand, the trainers of the Teacher Training Institutes also have knowledge in Literacy Boost that can be replicated to other districts. This model is very interesting and clearly guarantees that the results of this intervention will have continuous effects. (+) Reading Camps- The implementation of Reading Camps is an extraordinary Literacy Boost initiative. Reading camps sessions do not have any implementation cost requirements. The adherence of Reading camps is quite significant. Therefore, this initiative even without the intervention of the project is already appropriate for the beneficiaries/school community. In a scenario without ECT2, the implementation of Reading camps can easily be implemented and monitored by MINEDH to ensure that the contents are minimally in line with the students’ academic Program. The Reading camps initiative is already being replicated in other neighboring districts. It reinforces the relevance that the community sees in this initiative. (+) Councils Members- School Councils members are very clear about their role in the school. Indeed, on a regular basis, they have been controlling the attendance of students and teachers. This action, through field observations, will be perpetuated even without ECT2, which shows ownership from the school council members. Therefore, it is hoped that even without the project, School Council members will continue to collaborate with the school through initiatives of supporting school management. It should also be noted that members of school councils have supported the construction of attached classrooms which contributes to decreasing student ratio per class and therefore an improvement in teaching quality. (+) Reading Books Distribution- The distribution of school books is an initiative that will ensure improved reading skills for students that are currently enrolled as well as students to be enrolled in the future. It was understood based on the visits made that the teachers are encouraging the reading of the books offered. On the other hand, the books are well kept by the school administrators, which will allow longer life span. This intelligent initiative will work independently of whether the project is present or not. (+) Savings Groups- The savings groups have been an extraordinary initiative that contributes to the sustainability of the ECT2. Indeed, the saving group members make periodic contributions which is called social funds to support some school expenses such as buying soap to wash school utensils. This initiative generates greater independence of the school from government/project funds. In addition, savings groups also constitute a mechanism to increase family income because it has been found that most members apply the money to small businesses. The culture of saving and financial planning/organization is also a sustainable gain for the community as they can have better control of their expenses and, consequently, make better financial decisions. (+) Health and Hygiene Practices - The strategy of teacher training for health-related areas was crucial for health dissemination information given that they regularly give talks to students and the school community. This knowledge therefore stays with the school community and will WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation be transmitted to future students and will ensure that health hygiene practices are applied contributing to the improvement of health indicators (reduction of cases of diarrheal diseases, mortality, etc.). (+) Infrastructure Improvement - Construction is an excellent way to improve the quality of the teaching and learning environment by reducing the number of students per classroom. In addition, the construction of latrines will significantly reduce the presence of diarrheal diseases by improving hygiene conditions. For the maintenance of the infrastructures built, teachers were trained who are supporting in the control and use of the infrastructures. The School Council members were also trained to support the maintenance of the school infrastructure, which also contributes to the increase of its working life. (+/-) Farmers Group - With the implementation of the project, there were very positive results in terms of farmers' performance. Although there is no concrete evidence, there is an indication of an increase in the level of productivity of farmers as a result of the training initiatives in the use of improved technologies. Furthermore, farmers in the area of project implementation now produce and sell vegetables as a result of the intervention. Therefore, farmers today have alternatives for income generation activities that clearly contribute to poverty alleviation. However, on this issue there are natural challenges that are beyond the control of the ECT2 and which may obstruct the provision of food to school. The agriculture is still rudimentary, and dependent on the presence of rain. In addition, the soil fertility of Muecate and Nacaroa is not very good, which affects productivity levels. So, the fact is that farmers 'productivity remains low and unstable, limiting farmers' ability to feed the school. (+/-) School Feeding - The school feeding component is the main sustainability challenge of the project. It is not yet clear how the students will continue to receive food after the project ends. This challenge is, according to the interviews, due to a lack of resources to safeguard the implementation of the school feeding component. Despite this component, technicians could manage the logistics component. Indeed, the teachers trained in the food management component are fully aware of the procedures to be considered from the receipt of food to its distribution. The Government technicians have also received training related to commodity management, which gives them the ability to manage school food. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 4 Conclusions and Recommendations 4.1 Conclusions 4.1.1 General Conclusions The implementation of ECT2 has been successful. Indeed, the project vision, strategies and standards are completely in line with Government National Policies and Strategies for Education and food Security. The preliminary results of the project implementation bring great positive lessons to PRONAE (School Feeding National Program) to support the actual school feeding Programs policy direction. In addition, a clear alignment of the project with the beneficiary’s needs was perceived. Concretely, improving the quality of Education, reducing food insecurity levels and improving the quality of children health is crucial and a priority to improving the population’s wellbeing and consequentially improve Mozambique Development Indicators and contribute to the achievement of the Sustainable Development Goals. Another important aspect in terms of the design is the way how the activities are inter-connected. This interconnection ensures greater ownership from the school community members (Government, students, teachers, school administrators, school councils and the community in general). Thus, this interconnection constitutes a very important way to ensure the sustainability of the project implementation actions. In general, the main challenge, are Government- related, which is to gather the resources required to keep the school feeding component after the project end date. 4.1.2 Specific Conclusions Literacy Boost approach is a powerful strategy to improve reading skills of children in school age. Indeed, in Muecate and Nacaroa, about 32% of the pupils read between 6 and 44 words per minute. On average, they read about 21.6 words per minute. When the results from Mid Term are compared to the baseline results in which only 25% of children could read between 6 and 44 words per minute as well as the control zone in which 26% of children can read between 6 and 44 words per minute, it is possible to reinforce the positive effect generated by the ECT2 intervention because the midterm results are higher. There was a significant improvement in teacher skills. This improvement was a result of the teacher trainings in Literacy Boost. The application of content learned by teachers is evident in schools. All the schools visited have word walls, the students in the classrooms are sitting in a U-shape and the teachers use more interactive approaches that strengthen the students' attention in the classroom. The challenge is that the School Administrators do not have the tools to evaluate the effectiveness of the content that the teachers have assimilated in the trainings. The implication of this challenge may be the reduction in the technical quality of the implementation of the methodology. The teacher attendance has improved significantly as a result of the interventions that have been carried out within the scope of project implementation. At present, it is observed that about 96% of teachers attend and teach at school at least 80% of scheduled days per school year. This progress is a result of the reinforcement that the School Council has given to School Administrators in control of student attendance. In addition, the good performance of this indicator is associated with the prizes that have been given to the teachers with the best performance. Another factor that contributes to greater consistency in the attendance of the students is the fact that several teachers have their houses (which were in many cases built with the support of the community) close to the schools. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation School Administrators have better management skills that allow them to better manage school resources. As a result of the training, it was observed that the school administrators have better capacities for administrative organization, management of financial and human resources. It has been found that in many schools the administrators have been more rigorously controlling the attendance of students and teachers. It was also found that the school administrators have organized meetings on a regular basis to monitor the performance of their staff. Another important fact related to the management capacity is the volunteers' performance, which has been very good. The volunteers who cook, are health conscious and reading camp promoters are very committed to their activities. There are no children belonging to the target schools reporting hunger during the school time. All children in school on a regular basis receive Corn Soy Blend (CSB) enriched porridges mix of maize meal and soya flour and vitamins. In total 65,806 students receive CSB. Of these, 35,084 are boys, equivalent to about 53% of the total student beneficiaries and 30,722 are girls, equivalent to about 47% of the total beneficiary students. The result of this intervention is the improvement of student’s nutrition levels as well as the strengthening of their academic abilities. Another benefit of this intervention is the reduction of the financial pressure on parents (who in most of the cases are very poor) since the students eat at school. The level of attention of the students improved as consequence of school feeding and interactive dynamics implemented by the teachers. Indeed, on average, teachers report that 90% of students in their classes have been attentive to their classes. This result is associated to school feeding component since students are not hungry, they are willing to be more attentive. On the other hand, the interactive methodologies used by the teachers, assimilated with the implementation of the project, have also influenced the level of attention of the students. The challenge for this component is that teachers do not have a systematic tool to regularly and effectively capture the actual number of students who are attentive in class. The School Councils understand their role and are very important for School Development. In all the verified schools the School Councils clearly described their role in the school. This knowledge is a result of the trainings that the School Councils have had within the scope of ECT2 Implementation. The School Councils regularly monitor the attendance of students and teachers, sensitize students who are out of school to attend school, support the construction of classrooms, which contributes to a decrease in the pupil-per-classroom ratio, improving the quality of teaching and support management within the context of school budget planning. Parents and the community in general are increasingly aware of the relevance of children conventional education. All focus groups have pointed to this gain which is a result of the awareness campaigns that have been carried out by the School Council as well as by the ECT2. Community mobilization meetings were reinforced by mobile radio broadcasting campaigns conducted by local radio stations, covering 53 communities and reaching an estimated 29,400 listeners in both districts. In addition, because of the school feeding program, more parents are motivated to put their children in school. We have identified cases of parents who bring their children from other districts to enroll in schools that have school meals. Savings and Loan groups provide a sustainable mechanism for supporting schools and for alleviating family poverty. In total, today, through the project 113 Savings and Loans Groups have been created. These groups save on a regular basis and make contributions (social fund) to school to purchase kitchen material such as soap. In addition, savings groups also constitute a mechanism to increase family income because it has been found that most members apply the money to small businesses. With the earnings from the savings, families have purchased WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation durable goods for their homes like radios, bicycles, etc. and use the money to face lifetime events namely, births, deaths and marriages. The children clearly demonstrated that they know at least 3 health practices, that will contribute significantly to the reduction of health problems. The school community reports that there was a significant reduction of students who are not attending school as a result of diseases such as diarrhea. The number of students who appear dirty to schools has been reduced, students are washing their hands before receiving school meals. This result is a consequence of the various training sessions that covered topics such as child health (immunization, de-worming, pregnancy, food and nutrition); drinking water storage and management (including water treatment); sanitation and hygiene (safe disposal of feces, hand washing at critical times, personal hygiene); environmental hygiene (including rubbish pits, laundry areas, keeping animals away from food); and control of wastewater. Supporting farmers groups is a clear strategy for increasing agricultural productivity levels as well as an alternative for income generation and can in the long term be a sustainable exit for the feeding program in terms of fresh food supply. In all the associations visited, farmers are aware of improved agricultural technologies, namely fertilizer use, use of pesticides, use of improved seeds, planting in rows, etc. In addition, horticultural farmers have a more diversified food diet and sell the surplus which is an alternative for generating income. It is also noteworthy that some associations have contributed to school feeding. However, the challenge for this component is associated with the fact that Nacaroa and Muecate farmers remain young in terms of market-oriented agriculture production and still have low productivity rates. Indeed, they cannot meet the needs foreseen on the agreement with the school. It was also noted that there is no instrument to verify the effectiveness of the agreement so that it can be ascertained if the 30% is being offered in accordance with the agreement. There are now about 4,300 students that are now studying in better conditions as a result of new classrooms built. To minimize one of the main challenges, not only in project implementation but also at national level, the ECT2 project has built 72 classrooms in 24 schools. The classrooms built resulted in the upgrade of some schools from EP to EPC. This achievement contributed to shorten the distance between home and school for thousands of children. In addition, the construction of classrooms contributed to a reduction in the number of students per classroom. All schools currently have Clean Water and Sanitation Services which means reducing the risk of catching diarrheal diseases such as cholera. Apart from clean water source built/rehabilitated in all project schools, about 137 (85%) schools covered by the project have improved sanitation services and it was verified that the children use them. In total 1,507 latrines were built. All schools have infrastructure and are properly equipped to make food daily and on time. In addition, key members of the school feeding component have the skills and knowledge necessary for food preparation and food storage. This shows that schools are prepared for the provision of food in a recurring and sustainable manner. The challenge is simply linked to ensuring the availability of food on a permanent basis. The members of the Government as well as the other stakeholders of the project are fully aware of the importance of the project and have repeatedly and significantly collaborated with the implementation of the project. The political ownership of the project is notable, including the district administrator’s level. This is a key building block for a sustainable project exit. However, at Central level there is no indication of WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation funds to purchase the food after the project end date. This is a major risk to the project’s sustainability. 4.2 Recommendations ► Despite Literacy Boost’s positive results, it is crucial to keep improving the teacher’s performance through additional training. It was noted that the number of students that do not read remains high and the number of words read per minute could be greater. ► The Literacy Boost is new for most of the school administrators. Thus, it is fundamental to develop appropriate tools for them to improve the supervision of teachers that are implementing Literacy Boost methodologies. These tools must also be shared and implemented by the SDEJT technical supervisors. For instance, the School Administrators could regularly apply the EGRA test to the students in order to understand their reading performance. The information collected based on the tools to be developed will also help in additional analysis to understand the LB approach effectiveness. ► To keep the teachers motivated with the teacher rewarding approach, it is important to understand deeply what kind of awards to motivate them (through need assessment surveys) and try to meet this expectation. Also, it is important to ensure the selection criteria is very clear for all teachers. ► For the improvements of the school administrators in terms of management skills it would be important to have updated tools to measure the school’s administrator’s management performance. These tools could be used by the project and by the SDEJT technicians. These tools must contain additional components such as volunteer management, school council coordination effectiveness, financial organization and planning, etc. ► It is crucial to develop a clear and standardized tool for the teachers to measure student attentiveness in class. These tools must be recordable and should also be monitored by school administrators. ► It is important to select a school council member per school to be trained to train future school council members given that school council has just a 2-year mandate. With this mechanism the knowledge can be retained among the school council members. ► The savings and loan groups should be encouraged to promote the creation of new savings and loans groups in the community using the same model. This will be important to help the school with resources and will also constitute a mechanism for household poverty alleviation; ► It is important to find a mechanism to ensure that the latrines are cleaned and with the support given by the school councils, the schools must be mobilized to define strategies to clean the latrines. Considering the cleaning pattern verified (once a week), the infrastructure sustainability may be affected. In addition, it can lead to disease among children. ► The project should develop a tool to effectively monitor the production that farmer associations give to schools. In addition, there is a need to create more interaction and coordination between farmers and school administrators. It is fundamental to work with the school administrators to improve their commitment in this component. ► The project should keep encouraging the construction of classrooms using local materials (Earth/mud and wood). This intervention is very crucial since it reduces the students - classroom ratio which consequently contributes to the quality of education. WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation ► It is important to monitor the volunteer motivation and encourage financial or material contributions from the school community (School councils, parents, savings groups) to the volunteers to try to maintain their commitment; WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 5 References Burrows D & Kendall S (1997) Focus groups: What are they and how can they be used in nursing and health care research? Social Sciences in Health. Gay, L.R. and Diehl, P.L. (1992) Research Methods for Business and Management. Mc. Millan Publishing Company, New York. Gove, A. and Wetterberg Anna (2011), THE EARLY GRADE READING ASSESSMENT: Applications and Interventions to Improve Basic Literacy, Research Triangle Institute (RTI). Ross, N. Kenneth (2016), Sample design for educational survey research, UNESCO International Institute for Educational Planning. Republic of Mozambique (2015). Programa Quinquenal do Governo 2015-2019 Republic of Mozambique (2010). Plano de Acção Multissectorial para a Redução da Desnutrição Crónica em Moçambique, 2011- 2015 (2020) Ministry of Education (2012). Plano estratégico da Educação, 2012- 2016 (2019) Ministry of Education (2013). Programa Nacional de Alimentação escolar (PRONAE), 21 páginas. Maputo. World Vision Mozambique (2016). End of Program Evaluation Report; Educating Children Together Project WV Inc Mozambique USDA MGD FFE&CN–FFE-656-2015/011-00 Mid-Term Evaluation 6 Annexes 6.1 Evaluation Matrix Evaluation Matrix.docx 6.2 Data Entry and Data Analysis Data Entry and Data Analysis.docx 6.3 Enumerators Selection and Training Enumerarors Selection and Training 6.4 Additional Evaluation Questions Additional Evaluation Questions.docx 6.5 Quality Assurance Quality Control Process.docx 6.6 Sample Execution Target Group.docx