End of Programme Evaluation Report Educating Children Together Project 1 <> Agreement Number: FFE-656-2012/025-00 (and baseline for FFE-656-2015/011-00) Project Duration: 2012 - 2015 World Vision Mozambique 1 i. Table of Contents 1 Executive Summary ........................................................................................................................... 5 2 Evaluation Introduction/Background ............................................................................................. 7 3 Methodology ....................................................................................................................................... 8 3.1 Training and pre-testing............................................................................................................ 8 3.2 Data collection tools................................................................................................................. 9 3.3 Sampling........................................................................................................................................ 9 3.3.1 Selection of Schools and Surrounding Villages............................................................ 9 3.3.2 Population and Sampling for Qualitative Survey .......................................................10 3.4 Data analysis..............................................................................................................................12 4 Evaluation limitations......................................................................................................................13 5 Findings...............................................................................................................................................13 5.1 SO1: Improved Literacy of School-Age Children .............................................................13 5.1.1 (MDG 1.1) Improved Quality of Literacy Instruction..............................................15 5.1.2 (MDG 1.2) Improved Attentiveness............................................................................17 5.1.3 Improved Student Attendance......................................................................................19 5.2 SO2: Increased Use of Health and Dietary Practices ......................................................21 5.2.1 (MDG 2.1) Improved Knowledge of Health and Hygiene Practices ....................22 5.2.2 (MDG 2.2) Increased Knowledge of Safe Food Preparation and Storage Practices.............................................................................................................................................22 5.2.3 (MDG 2.3) Increased Access to Clean Water and Sanitation Services...............23 5.2.4 (MDG 2.4) Increased Access to Preventative Health Interventions....................23 5.2.5 (MDG 2.5) Increased Access to Requisite Food Preparation and Storage Tools and Equipment..................................................................................................................................24 5.3 Change (Impacts, Effectiveness)............................................................................................27 5.3.1 Infrastructure....................................................................................................................27 5.3.2 Provision of meals in schools........................................................................................27 5.3.3 Provision of school materials........................................................................................28 5.3.4 Community volunteers...................................................................................................28 5.4 What worked well?..................................................................................................................28 5.5 What did not work well (Challenges)?................................................................................30 5.5.1 Schools infrastructure: access to water and sanitation services...........................30 5.5.2 Food preparation and storage.......................................................................................30 5.5.3 Community participation and transparency...............................................................31 5.5.4 Water supply and sanitation..........................................................................................31 5.6 Project sustainability................................................................................................................32 5.6.1 Training ..............................................................................................................................32 5.6.2 Embedment of the project into local community.....................................................32 5.6.3 Keeping the children at schools....................................................................................32 6 Specific Conclusions and Recommendations.............................................................................33 6.1 Conclusions...............................................................................................................................33 6.2 Recommendations....................................................................................................................35 7 Most Significant Change Story ......................................................................................................37 8 Appendices........................................................................................................................................38 8.1 List of people interviewed, with affiliation and contact details......................................38 8.2 Evaluation TOR.........................................................................................................................38 8.3 Evaluation Tools.......................................................................................................................38 2 ii. Acknowledgements It has been a great pleasure for COSDER Consultants to work with World Vision Mozambique and we would like to express our gratitude for their engagement throughout the final evaluation process. Special thanks go to all community enumerators for sharing their knowledge and experience during the data collection phase. We extend our thanks to all respondents for setting aside their precious time to respond to our questions. The list of those who directly or indirectly contributed to the success of this study is too long to mention. Thank you all for being exceptionally supportive. iii. Affirmation The Final Evaluation has been conducted for Educating Children Together Project (ECT) sponsored by United States Department of Agriculture (USDA) McGovern Dole Food for Education and implemented by World Vision Mozambique led by Design Monitoring and Evaluation team through an External Consultant. World Vision Mozambique hired Consultoria em Serviços de Desenvolvimento (COSDER) represented by Celestino Salencia to carry out the training of data collection, field data collection and to write this evaluation report. Gratitude also goes to World Vision Nampula staff who supported the field work data collection for this evaluation. Disclaimer: This report was prepared by independent external consultants. Responsibility for the contents and presentation of findings and recommendations rests with the assessment team. Except as acknowledged by the references in this report to other authors and publications, the findings described herein consists of our own work. Primary data collected throughout the baseline process remain the property of the communities and families described in this document. Information and data must be used only with their consent. This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov 3 iv. Acronyms and Abbreviations ADP Area Development Programme COSDER Consultoria em Serviços de Desenvolvimento DME Design, Monitoring and Evaluation EGRA Early Grade Reading Assessment EPC Primary School Grade 1 to 7 FDC Foundation for Community Development FFE Food For Education FGD Focus Group Discussion FLAT Functional Literacy Assessment Tool GDP Growth Domestic Product HH Household KII Key Informant Interview MERLU Monitoring, Evaluation, Research and Learning Unit MSC Most Significant Change SDAE District Economic Activities Service SDPI District Planning and Infra-structure Services SPSS Statistical Package for Social Sciences USDA United States Department of Agriculture WVM World Vision Mozambique 4 v. Project Description Project Name Educating Children Together (ECT) Country/Province/ Specific Location Mozambique: Nampula Province Project Timing: Expected Start-up Date Expected Finish Date October, 2012 December, 2015 Evaluation Contact Person Tomas Doce Consultant Contact Person Celestino Salência 5 1 Executive Summary The following report consists of an evaluation of Educating Children Together Project (ECT) implemented in Mozambique‘s Nacarôa and Muecate districts of Nampula province. The three￾year project’s objectives were to improve the literacy of school‐age children and to increase the use of health and dietary practices, through interventions aimed at improving the quality of literacy instruction, student attentiveness, student enrolment and teacher attendance. For the first strategic objective (SO1), it was found that the quality of literacy instruction in the ECT Project in both implementing districts has improved significantly due to the implementation of the project. Compared to the baseline there has been an increase in children who can read and understand grade level text from 8.1% to 25.1% at Project endline. The Evaluation also reveals that the school feeding activities (provision of daily meal) improved the attentiveness (from 81.4% of inattentiveness in the baseline to 2.0% in this evaluation) as almost all the children no longer go to classes on empty stomachs. Further in this Evaluation it was found that school feeding has contributed to increased enrolment and retention rates considering the fact that as attentiveness increases, children tend to have an increased incentive to attend classes. Finally, the health of the children in the ECT project has also been boosted by good health and dietary practices. The good health and hygiene practices are closely linked with the health of the children and therefore the healthier the pupils, the more they are available to go to school and the better is the performance. The evaluation also found in its second strategic objective (SO2) that the ECT Project has successfully provided the necessary foundations for the literacy of School age children. The ECT project has successfully trained both the caretakers and the children with the necessary skills for health and hygiene practices. The communities have received and implemented the learnings about safe food preparation and storage, and improved stoves for CSB preparation. These learnings and methods were good for the ECT project because they equipped the project with the necessary skills to deliver the project goals. In addition, these lessons were also good for the household wellbeing because most of these health and hygiene techniques are applicable at household level. In terms of infrastructure, the project had a significant contribution improving the availability of good quality infrastructure instead of the existing precarious buildings and under the trees, 6 which therefore contributed to the wellbeing of children with construction of 39 classrooms. The project had a significant improvement on water supply and provision of sanitation facilities through installation of rain water harvesting systems in all schools as well as construction of latrines. However, the impact of the rainwater harvesting system might be limited because the rains are scarce and the capacity of the installed rainwater harvesting system is not enough to keep water for long periods of more than 8 months without rain. Also, the availability of latrines at both the household level and at school level has increased, although in limited number of schools are still considered an issue. Efforts in improving access to water were implemented in an integrated approach with the WV Mozwash Project which drilled boreholes and constructed latrines across Mozambique and included the ECT project area. 7 2 Evaluation Introduction/Background World Vision’s ECT project is an initiative funded by the United States Department of Agriculture’s (USDA) McGovern-Dole International Food for Education and Child Nutrition Program. The three-year project which started in October 1, 2012 overall objective is to, improve the literacy of school‐age children and to increase the use of health and dietary practices, through interventions aimed at improving the quality of literacy instruction, student attentiveness, student enrolment and teacher attendance in the districts of Muecate and Nacarôa in Nampula province. There are two strategic objectives under this project, namely:  MDG S01. Improved Quality of Literacy Instruction  MDG 1.1 Improved Quality of Literacy Instruction  MDG 1.2 Improved Attentiveness  MDG 1.3 Improved Student Attendance  MDG SO2 Increased use of health and dietary practices  MDG 2.1 Improved knowledge of health and hygiene practices.  MDG 2.2 Increased knowledge of safe food preparation and storage practices.  MDG 2.3 Increased access to clean water and sanitation services.  MDG 2.4 Increased access to preventive health interventions.  MDG 2.5 Increased access to requisite food preparation and storage tools and equipment. External Consultants contracted by World Vision conducted a baseline in April 2013 and a midterm review at the program midpoint from July through September 2014. As per contractual agreement with USDA, WV conducted this final evaluation from 19th October to 18th November 2015 to assess whether the project has achieved the expected results as outlined in the results framework. The final evaluation’s main objective was to assess areas of project design, implementation, management, lessons learned and replicability. This evaluation provides lessons learned and recommendations for USDA, program participants and other key stakeholders for future school feeding (food assistance) and literacy programs. The final evaluation used the same methodology that was used during the baseline study, i.e., quasi￾experimental design approach to assess the relevance, effectiveness, efficiency, impact and sustainability of the program. 8 The evaluation covered components of relevance of the ECT project, its effectiveness, efficiency, sustainability and impact. This report presents the main findings from inception of the project in October 2012 to December 2015. The report is comprised of seven components being (i) an executive summary with a full narrative of the project lifecycle, (ii) an introduction which presents the summary of the report and main components, (iii) the methodology used to conduct the evaluation, (iv) limitations - the issues that the study is not capable of addressing, (v) the main findings of the evaluation (vi) what these findings mean to the program and (vii) what can be learnt from the overall work conducted by the ECT Project. The Final evaluation aims at solving the following hypothesis “If ECT Project improves quality of literacy instruction and increases use of health and dietary practices then pupils will be more attentive in class and learn effectively. 3 Methodology The ECT project survey was conducted in Nacarôa, Muecate and Murrupula (Control) districts, Nampula province. The survey collected data for all indicators defined for the project implementation according to the M&E Plan. The data collection took place from 02 to 17 of November, 2015. The following components were covered as part of the Methodology. 3.1 Training and pre-testing Prior to the data collection process, twenty (20) local enumerators were trained. The training included sessions on research ethics, data confidentiality and survey implementation and consisted of equipping the enumerators with the abilities to conduct qualitative and quantitative interviews relevant to the evaluation. All enumerators attended a 5 day training in Nampula province and this was done from 26 -30 October and all the sessions were facilitated by the external consultants. In addition, 5 ECT M&E staff (1 Manager and 4 Officers) participated in the training and assisted with organizing logistics and monitoring all the training sessions. The ECT staff also assisted in general field coordination during data collection. On 29th October the surveys techniques were tested in an area called “EPC Mutepuehi” in Murrupula district, and after that, some feedback were included to refine the tools. By 30th October, the consultants tested the data collection instruments at the field and gathered 9 feedback from the enumerators, incorporating comments to the final version of the survey tools. 3.2 Data collection tools The Final Evaluation used a similar methodology that was used for the Baseline in 2013. Data collection tools were designed prior to enumerator training and were fully tested in the field as part of the enumerator training before finalizing them. The following comprised the data collection tools:  Functional Literacy Assessment Tool – FLAT  Early Grade Reading Assessment - EGRA (Literacy Assessment Tool)  Most Significant Changes (For Pupils and Guardian)  Key Informant  Focus Group Discussion Guides  Household Survey Questionnaire  Self-Administered Questionnaire (Teachers and Pupils) 3.3 Sampling 3.3.1 Selection of Schools and Surrounding Villages The sampling Design for this evaluation was similar to the one done in baseline. A stratified sampling method was used in which the population is divided into subpopulations (strata) and random samples are taken of each stratum. Each district will have 3 stratum composed of group one; Schools considered near District Centre, group two; Schools consider middle way from District Centre and group three; Schools considered far away from District Centres. It is believed that distances of schools from the district affect school and teacher performance since the level of support dwindles as you get further away from support services in the district. The focus was on primary schools from grade one to grade seven in the project area. To achieve this, in each district, six (6) schools were chosen using stratified sampling selecting methods totalizing 12 in project area. In Nacarôa district, the participating primary schools include Terrene Ponte, Inteta, Mucuthi, Nacuaia, Chicamalala and Muchelia whilst in Muecate, the schools that participated includes Imala, Napai, Terrene, Chipacane, Nchacha and Mocone. 10 The Control District had 3 schools selected namely Nihessue, Mulio and Kazuzo. These schools are in areas where are also WV programs running but without any School feeding projects going on and hence used as control areas. In each district the schools were randomly selected and categorized by location i.e. Near the District, Mid distance from the District and Far from the district as was done during Baseline evaluation. The selected schools per district: Table 1: Selected schools per district District Distance Schools Muecate Far 57 Km Terrene 37 Km Imala-Sede Middle 27 Km Chipacane 12 Km Mocone Near 15 Km Nchancha 12 Km Napai Nacarôa Far 28 Km Inteta 42 Km Muchelia Middle 16 Km Chicamalala 6 Km Terene - Ponte Near 0 - 5 Km Mucuthi 5 Km Nacuaia Control Area Murrupula Far 35 km Nihessue Middle 20 Km Mulio Near 17 Km Kazuzo 3.3.2 Population and Sampling for Qualitative Survey Qualitative data was collected using Key Informant Interviews (KII), Focus Group Discussions (FGD), stories of Most Significant Change and by direct observations: (i) Focus Group Discussions (FGDs) - This tool was conducted in the selected 12 schools with school councils, agricultural association Groups, cooks and health groups in the two districts comprising the project area (See table 2 below). The data was collected from community members based on their level of perception and participation of the project activities. The main focus was mainly on changes created by the project during 11 the three years of the implementation in ECT project areas (Muecate and Nacarôa districts). (ii) The Key Informant Interviews (KIIs) were held with key partners and stakeholders including government services and departments of Education, Health, Infrastructure, Agriculture and Economic Development. The KII were also conducted with school Directors & Pedagogical directors, World Vision ECT Management in Nampula, and with representatives from The Foundation for Community Development-FDC. The KII Guides were developed for collecting data from each of these key partners and stakeholders. The KII helped to understand the role of each partner and stakeholder in the ECT project objectives. The list of the KII is presented in annex 1 of this report. (iii) Literature review and observation were used to gather information that was used to provide context, validation and comparisons in this Report. The following documents Pictured: Participants during FGD in Nacarôa Pictured: Participants during FGD in Muecate Schools in Muecate FGD Agricultural associations Volunteers (Cooks) and Health School Councils Terrene 0 0 11 Imala-Sede 0 0 11 Chipacane 12 12 8 Mocone 0 9 0 Nchancha 0 10 0 Napai 0 12 7 Sub total 12 43 37 Schools in Nacarôa FGD Agricultural associations Volunteers (Cooks) and Health School Councils Inteta 9 12 9 Chicamalala 0 11 6 Mucheliua 12 8 9 Terene Ponte 0 6 6 Mucuthi 0 12 12 Nacuia 0 9 12 Sub total 21 58 54 Table 2: Number of participants in FGD per school in Muecate Table 3: Number of participants in FGD per school in Nacarôa 12 where used project reports, Baseline survey report 2013, USDA M&E Policy, Mid Term review report and other Project Documents, WV LEAP Evaluation Guidelines, the Project Indicator Performance Tracking Table (IPTT), the TORs of the Final evaluation, the Evaluation Plan for ECT project and other relevant documents to clearly understand the project and contribute well to this evaluation. (iv) In terms of quantitative data, self-administered questionnaires were administrated to 169 teachers, 284 students from aged 12-18 (grade 5- 7) in the 15 selected schools. On the other hand, Household survey (HH) data was collected using mobile phones (SMAP). A total of 2022 households were interviewed during the evaluation. The number of respondents in the same number of schools at Final Evaluation was increased to get more precision compared to the samples used at Baseline which had 84 Teachers, 150 students and a total of 600 Households, (v) Functional Literacy Assessment Tool (FLAT) - A sample of 216 students were taken in the project area aged 12 years old and above participated in the FLAT assessment as per questionnaire standard. The students were randomly selected, targeting 20 children from each school. The FLAT was used to find out the progress made towards achieving quality literacy in schools to enhance student capacity to read and write. (vi) Early Grade Reading Assessment (EGRA) - The EGRA tools aimed to measure reading performance of students in grade 3 who have just completed their grade 2 in a defined set of intervention and control schools. A total of 152 pupils were interviewed in the project area using this tool. The students were randomly selected targeting 20 pupils among grade 3 per each school. 3.4 Data analysis (i) Quantitative data from the mobile phones was analysed using appropriate statistical packages called Statistical Package for Social Sciences (SPSS), version 16 and Microsoft Excel, version 13. Both descriptive and inferential statistics were generated to support the report writing process. Assessments of relationships between two or more variables were conducted using cross tabulation and correlation techniques in order to further inform the process. 13 (ii) Qualitative data and gathered information from KII and FGD were analysed using content analysis. Coding and themes development were done by the Consultant Team (See Annex 2). 4 Evaluation limitations The timing of the Evaluation was limited considering that all data had to be collected two weeks before the schools closed. This made some adjustments to hire more enumerators and divide team so that all data is collected on schedule. Although this limitation was minimised through splitting the teams into two groups, it limited the direct supervision by the consultant in one area also considering that there was a rush during the data collection. However, this was reduced through asking the ECT M & E staff to assist in Field supervision at the leadership of the consultant. The control district had a USAID funded literacy project APAL (Aprender a Ler or Learning to Read). Even though this affected comparisons on Literacy component, the project is not doing school feeding, but they are doing extensive literacy activities and hence has likelihood to weaken the external validity of the literacy indicators when comparisons are made. In order to overcome this limitation, the comparisons were focused on the differences of the findings between the baseline and evaluation. 5 Findings This section builds on the preceding one and highlights what was found in the each key project indicator. It initially presents summaries of the information that was obtained and used to extract the indicators. The summaries are presented according to the sources from which they were obtained i.e. KII, FGD, Direct Observation and HHs survey. In addition the chart of indicators is also presented and explained. Since the methods of data collection were complementary, the findings are, in some instances, integrated and others separated to retain the meaning. The findings of this final evaluation are highlighted in subchapters according to the results framework and some key points presented in the Terms of References. 5.1 SO1: Improved Literacy of School-Age Children 14 Quality of literacy of school-aged children was measured during the ECT evaluation exercise through EGRA and FLAT. From EGRA analysis, the data shows that the number of children with improved ability of reading and comprehension has increased from 8.1% (in the Baseline) to 25.1% (in the final evaluation). This means that the proportion of pupils (grade two) in schools that demonstrates a level of reading and understands the grade level text has increased considerably after the project intervention. In terms of gender distribution, the results show an increase of 4.0% of girls and 17.0% of boys attaining reading and comprehension of school texts from 2013 to 2015. This is due to the fact that girls have more responsibility at home (e.g. cooking, cleaning, taking care of the siblings, fetching water, etc.) compared to the boys who have more time for playing and preparing the lessons at home as well as farming. The fact that the project target of 25.0% for this strategic objective was fully achieved demonstrates the project effectiveness and efficiency in addressing the issues of school literacy. The training sessions conducted by the ECT project have contributed to improve the ability of teachers and school directors to deliver reading and comprehension skills. However, the figures show that only 25% of school-aged students are currently equipped with basic literacy skills. More efforts are needed to improve the overall results as shown in Figure 1. Figure 1: Literacy tests results T test (P=0,006) (Significant Differences between Baseline and Final Evaluation) Using the same tool, the control district of Murrupula was found to have a highest growth from 5.5% to 59.0% of children with ability to read and comprehend while the ECT Project areas (Nacarôa and Muecate). The fact that the control district has higher results is because of another literacy program being run in that district by APAL. The 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% Baseline Final Target Percentage of Children able to read Level of Literacy of School Age Children 15 ECT Project needs to ask question what was the difference in the approach between APAL and ECT and why APAL was so much more successful. However, the scope work of the Evaluation did not allow for the consultant to do a detailed analysis of the operations of APAL in the control district. Table 3: FLAT Level of Reading District Frequency Percent ECT Project areas (Muecate & Nacarôa) Story and Comprehension 54 25.0% Letters 66 31.0% No reading 20 9.0% Sentences 42 19.0% Paragraph 34 16.0% Total 216 100.0% Control Murrupula Story and Comprehension 36 59.0% Letters 16 26.2% No reading 4 6.6% Sentences 2 3.3% Paragraph 3 4.9% Total 61 100.0% 5.1.1 (MDG 1.1) Improved Quality of Literacy Instruction During the final evaluation it was found that about 87.0% of the teachers demonstrate the threshold number of quality teaching, against 42.4% of teachers in target schools. The capacity was gained through formal training, from either World Vision ECT programme (85.8%) as well as by the Government (14.2%). As per the analysis from the Self-Administered questionnaire, the percentage of teachers in target schools that were trained in teaching techniques provided by the ECT in coordination with Fundação para Desenvolvimento Comunitário FDC was 74.6% in project area. The table below shows the percentage of teachers that have received some training from a National or an International Organization. Table 4: Percentage of Teachers that have received training during the project implementation District Frequency Percentage Nacarôa 38 76.0 Muecate 47 73.4 ECT Project Total 85 74.6 16 Control district Murupula 16 53.3  More Consistent Teacher Attendance The evaluation found that 90.0% of teachers in the project area are currently more present at schools compared to the baseline of 69.5% of teachers in target schools attend and teach school at least 90.0% of scheduled days per school year (Baseline). This means that that 90.0% of teachers are now present at least 90.0% of scheduled school days. According to the education officials, the increase in teacher’s level of attendance can be attributed to improved quality and quantity of infrastructures, more training opportunities as a result of the project, provision of school teaching material and incentives. This was also verified by school administrators during the interviews.  Better Access to School Supplies & Materials According to the schools administrators, the ECT project supplied school material to 150 schools as a result of project implementation. This represents 100.0% of target school that the project planned to supply with material. It is important to mention that the school directors claimed that the material received was not enough for the school needs. The school directors said that the materials received included wooden tools (Used for making local Materials), books, and drawing kits.  Improved Literacy Instructional Materials The final evaluation revealed that 74.83% of schools have now classrooms with literacy and instructional materials (stories books) compared to the baseline of 39.0%. This is based on Monitoring Data provided by the project which shows that general school supplies are available but Library and other story books are still missing. The ECT project distributed textbooks, reading material, working tools to schools in order to contribute to this improvement. However, when it comes to sufficiency of material to satisfy all the children, based on teachers’ self-administered questionnaires 33.6% of teachers reported having sufficient material in the sampled school.  Increased Skills and Knowledge of Teachers The final evaluation found that out of planned 1617 teachers to be trained (100%), the project trained 1124 (69.5%) in psycho pedagogical matters and the teachers were given a certificate 17 of participation after the accomplishment of the training. Findings from secondary data collected show that the project is consistent with the evaluation survey that found that 74.6% of teachers received training as a result of the project implementation. Data was not available for the control district but it was mentioned in key informant interviews the government staff that teachers there were probably trained through the USAID APAL project.  Increased Skills and Knowledge of School Administrators Along with school teachers, the administrators were also trained in their respective areas of operation to improve their effectiveness in this project and beyond. During the project lifetime, 300 school administrators were trained in school management. The fact that school administrators were trained in several skills including school management has contributed to teaching and learning skills. This is because the school management skills are designed to promote a good Teaching and learning in the schools which encourages teachers to also utilize teaching skills better in class. 5.1.2 (MDG 1.2) Improved Attentiveness In terms of improvement of inattentiveness, there was a substantial drop of students who are inattentive in the ECT implementing districts. As per the self-administered questionnaires analysis (qualitative) administrated to teachers during this evaluation, only 2.0% of the students were inattentive compared to 81.0% (19.0% Attentive) at baseline. See the graph below. Figure 2: Percentage of Pupil attentiveness In addition, because of project interventions 98.0% of students in the classrooms are identified as attentive, meaning that just 2% were considered inattentive during the final evaluation compared to the baseline of 81.4% of students identified as inattentive by their teachers. The 18 teachers said that these are pupils who are considered as being 50.0% or less inattentive in class. According to the ECT project objectives, this can be attributed to the provision of food so that the children can concentrate better in class. This was also verified by parents during the focus group discussions. In Murrupula the same indicator was found to be at 75.0% attentive. However, the evaluation scope of work didn’t allow to further investigate the reason that kept the levels of attentiveness high without any school feeding going on.  Reduced Short-Term Hunger It was found that 47.4% of students in target schools indicated that they are "hungry" or "very hungry" during the school day, compared to the baseline figure of 82.0% (Self-Administered Questionnaire). This is an improvement of almost 40.0% decrease in the percentage of school￾aged children who are no longer hungry. The results suggest that the provision of meals in schools has considerably improved the situation of short-term hunger in schools. Despite the fact that the figures have not hit the project goals of 25.0%, this result is very encouraging because of its impact in school attendance and attentiveness which are intrinsically affected by this parameter.  Increased Access to Food (School Feeding) During the project implementation, around 12,615,812 meals representing 58.11% of Daily number of meals were provided to pupils. The target number of meals planned by the ECT project was 21,711,950 thus only about 58% of the daily meals planned were provided. According to the project, the quantity of school meals to be provided was not calculated as per the planned quantity of food to be donated. The project disposed a total of 574.366 metric tons that got contaminated and had also a total of 4.69 metric tons of distribution loss. However, the variance was not because of the loss as the project usually get replacement for that. But, the variance was from the amendments done to the agreement with the Donor USDA. The ECT project has applied to reduce the tonnage of food received from 5200 to 2780 Metric tons. Of this a total of 2171 metric tons was distributed by the time of the Evaluation. The main reason for reducing was after critically analyzing the actual school attendance levels in the target schools which was low than initially anticipated. Unfortunately, the initial ECT target of meals was not revised to reflect this change and hence the variance. In addition, according to the school administrators, issues related to the delays in handling the CSB within the country such as lack of transport and poor road infrastructures linking the 19 intermediate warehouses and the beneficiary’s schools, has affected the timely delivery of CSB. During the Focus Group Discussion (FGD), the school councils mentioned: “Our school is a sub school of a big grouping of schools called ‘ZIP’. During the rainy season, the CSB was delivered in the ZIP headquarters where our school gets the CSB from, using the volunteers who take the CSB bags on foot, holding the bags on their heads 5.1.3 Improved Student Attendance The evaluation found that 90% of children (84.8% of girls and 96.7% of boys) regularly attend classes 80% of the school year calendar in the USDA-supported schools compared to 57.5% of students at baseline. The graph below, illustrates pupils attendance during the school calendar.  Increased Economic and Cultural Incentives (Or Decreased Disincentives) In terms of economic development of schools, discussions with WV project staff revealed that the extracurricular activities were supposed to be done with Gift In Kind (GIK mainly comprising sports equipment) but this did not work out because of the difficulties of bringing in goods duty free into Mozambique. However, the project supported schools with seeds and seedlings of vegetables such as tomatoes, lettuce, and cabbage. In each school, the delivery system was through associations of production or schools where water was available. In term of economic development 14.3% of the interviewed children in project site participated in the school garden activities. The target of 147 schools was reduced later to 36 since most schools didn’t have good conditions to start gardens especially water and good soils. The project staff managed to distribute seeds to all but just a few successes. (Self-Administered questionnaire).  Reduced Health-Related Absences As per data analysis of student self-administered questionnaire, only 7.1% against 18% (Baseline) of students in target schools missed more than 10 school days due to illness. On the contrary, Murrupula district did not registered any case of children that missed school for more than 10 school days due to illness. This decrease can be attributed to awareness program on health related issues, meals provided at schools, hygiene practices, health kit distributed by the ECT project during the implementation of the project. See table below for more details. 20 Table 5: Health-Related Absences Baseline Final Evaluation District Never Missed School Due to Illness Missed School Due to Illness for 10 days Missed School less than 5 days Never Missed School Due to Illness Missed School Due to Illness for 10 days Missed School less than 5 days Nacarôa 55.0% 16.0% 29.0% 38.0% 11.0% 48.0% Muecate 28.3% 20.0% 51.7% 34.5% 4.0% 58.0% Average ECTECT project 41.7% 18.0% 40.4% 36.0% 7.1% 53.8% Control district (Murrupula) 58.6% 2.1% 39.3% 95.0% 0.0% 5.0%  Improved School Infrastructure From the project target was 70 school infrastructures, the project built 63 school facilities that directly influence the teaching and learning environment namely 39 classrooms, 12 libraries, and 12 school cabinets. This represents 90 percent of school facilities. However, the project also engaged in construction of various facilities which supported the preparation of porridge namely 150 warehouses, 150 energy saving stoves, and 150 kitchens. Regarding the WASH in the schools, the project constructed 21 boreholes, and 270 latrines. In general, the project constructed/rehabilitated about 804 school facilities for learning, service, and water and sanitation. Rehabilitation ranges from cosmetic upgrades such as whitewashing walls, to structural improvements (replacing broken windows, fixing leaking roofs, rebuilding damaged walls or roofs, repairing latrines, and upgrading fixing school kitchens), and mending broken furniture. Latrines/toilets that are repaired must meet set local government standards and should also be counted. Toilets counted are only those that have hand washing facilities within or near the toilets.  Increased Student Enrollment The average percentage increase enrolment in the project area was 8.3% as opposed to 25% of the project target. This increase was higher among girls (14.9%) compared to boys (1.7%) because the project conducted enrolment campaigns, raising awareness in the communities. 21 Even though the change is small it can be seen that the increase is strongly connected with the fact that the “schools have porridge and the children do not need to spend more time at home waiting for a meal before class” said one Parent during the Focus Group discussions.  Increased Community Understanding of Benefits of Education About 96.4% of the parents interviewed understand that school is either very important or important for their children. This has almost doubled compared to the 59.5% of parents in target communities who say that it is important that their children attend school at inception of the project. This was verified by the parents during the Focus Discussion who mentioned that there a lot of awareness done by the project that could have contributed to these changes.  Increased Capacity of Government Institutions According to interview with project staff, the evaluation found that 1285 out of 24 initially planned of Government Officials and other school committee members were trained. This was supposed to be government officials but the project increased the PTAs members’ capacity as per the need and the recommendation from the Government. This training has been conducted by a Government college district government official. 5.2 SO2: Increased Use of Health and Dietary Practices Initially the project planned to train 61 government health workers. However, according to the project staff, this target was revised later in the life of the project and included even volunteers which raised the target to 2000. By the end of the project implementation, 2000 health workers including volunteers were trained for improved service delivery in all 150 schools. The train included topics around inclusive and priority and hygiene message, Malaria and HIV/AIDS prevention, child health (immunization, de-worming, pregnancy, food and nutrition); drinking water storage and management (including water treatment); sanitation and hygiene (safe disposal of feces, hand washing at critical times, personal hygiene); environmental hygiene (including rubbish pits, laundry areas, keeping animals away from food); and control and use of wastewater. According to the parents in Focus Group Discussion, this has improved health practices and environment which has directly linked to the children health that has, therefore, contributed to improved children attendance. 22 5.2.1 (MDG 2.1) Improved Knowledge of Health and Hygiene Practices The Household survey shows that 61.7% of communities (including children) can identify three important health/hygiene practices such as taking a bath, using latrines and washing hands compared to 45% of Communities at the baseline. The fact that the project has conducted health campaigns in the communities, has improved the knowledge of health and hygiene practices at the household level. The healthier the households, the more the availability of children to attend which ultimately contribute to improved children’s attendance. It should be noted that World Vision Mozambique also has been implementing Health projects in the program area in an integrated fashion with the ECT project. Table 6: Number of Children able to mention 3 types of hygiene practices Area Frequency Valid Percent ECTECT Project 1 hygiene practice 17 7.9% 2 hygiene practice 65 30.4% 3 hygiene practice 132 61.7% Total 214 100.0% Murupula 1 hygiene practice 0 0% 2 hygiene practice 5 8.3% 3 hygiene practice 55 91.7% Total 60 100 5.2.2 (MDG 2.2) Increased Knowledge of Safe Food Preparation and Storage Practices All schools by the end of the ECT Project had a safe and improved form of stove, with volunteers well prepared and showed capacity of passing a safe food preparation test. According to the project report, about 1017 food preparers were trained for safe food preparation and storage during the implementation stage of the ECT project. It is important to note that by the time of the baseline no meals at all were being prepared in schools. Despite assessing the food preparation tests, children were also asked in terms of the quality of porridge they took. The majority of children (57.8%) informed that they like the quality of porridge they take. 23 5.2.3 (MDG 2.3) Increased Access to Clean Water and Sanitation Services The percentage of target schools with year round access to clean and safe water source has increased from 3.5% at baseline to 100% during the final evaluation in the ECT project area. The ECT project improved the access to clean water in schools through provision of water collection tanks. The fact that children have access to water potentially contributes to keep the children at school and therefore increase the likelihood of attaining reading and writing skills according to parents interviewed in a focus group. Roughly 90% of children interviewed during the self-administered questionnaire have access to a latrine in their houses against 62.5% at the baseline. However, when it comes to access to latrines at school, the evaluation found that out of 150 schools, just 40 schools representing making a total of 90% had a functioning latrine in schools. Achievements here area also in collaboration with the WV MOZWASH project that is also supporting WASH activities in the area. 5.2.4 (MDG 2.4) Increased Access to Preventative Health Interventions The project didn’t provide the deworming tablets as planned as the government didn’t approve but the project just supported the vaccinations by giving fuel and transport logistics and manpower to undertake the activities. Household’s access to supplements such as Vitamin A in target communities increased from 66.5% in baseline to 83.0% in evaluation. This activity was done in coordination with district health authorities. Increased access to Vitamin A among school-aged children indicates an improved access to health services that has an impact on school attendance and quality of school literacy among children. The fact that the Government through the support of the ECT project has been working with school-aged students and parents to encourage the children to take Vitamin A, has brought a positive message about the benefits of health services in the households. Therefore, more parents and guardians have been sensitized to adhere the child health campaigns. Short and long term benefits such children health status, knowledge on dietary practice as well as improved children school attendance have been reflected among the children in terms pupils performance in schools according to FGD with parents. Table 7: Households Access to Vitamin A in target communities Project Area % Vitamin A Baseline 66.5 24 Final Evaluation 83 5.2.5 (MDG 2.5) Increased Access to Requisite Food Preparation and Storage Tools and Equipment The ECT project baseline figures indicate that only 6.1% targeted schools had food preparation and storage equipment in 2013. In the final evaluation, it was noted that 100% of targeted schools had been fully equipped with proper functioning cooking Stoves. This indicates that the project has exceeded the goal of 80.0%, which can be explained with the impressive ongoing changes in the school community in terms of food preparation, storage capacity and meals preparations. The ECT project has successfully achieved the capacity to provide meals in targeted schools on daily basis, which is critical to improved pupils’ attendance, attentiveness and the ultimate goal of quality of literacy.  Increased Access to Preventative Health Interventions According to the project report, the scope of work of the ECT project did not intend to provide soap for dishwashing and plates after having a meal. In order to guarantee hygiene and avoid diseases, the ECT project introduced a training on production of bar soap using ash and cooking oil. This process was done using local available resources, meaning it was a no-cost process. This process led the project on improvement of hygiene in the community through capacity building of the community. 25 Table 8: LOGRFRAME INDICATORS WITH TARGET, BASELINE AND FINAL EVALUATION INDICATORS Result Indicator Final Target Baseline Final Evaluation SO1 Improved Literacy of School-Age Children Proportion of students who, by the end of two grades of schooling, demonstrate that they can read and understand the meaning of grade level text 25.00 8.15 25,1% 1.1 Improved Quality of Literacy Instruction Percent of teachers (or classes) in target schools who demonstrate a threshold number of quality teaching techniques or tools 90.00 42.40 87.0% 1.1.1 More consistent Teacher Attendance Percent of teachers in target schools who attend and teach school at least 90% of scheduled days per school year 90.00 69.50 90.0% 1.1.2 Better Access to School Supplies and Materials Number of schools receiving school supplies and materials as a result of USDA 147 0.00 150 1.1.3 Improved Literacy of Instructional Materials Percent of classrooms in target schools with literacy instructional materials (textbooks, workbooks, ECT) sufficient for effective instruction 90.00 39.00 74.8% 1.1.4 Increased Skills and Knowledge of Teachers Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance 1617 0.00 1124.00 1.1.5 Increased Skills and Knowledge of School Administrators Number of school administrators and officials trained or certified as a result of USDA assistance 294 0.00 300.00 1.2 Improved Attentiveness Percent of students in classroom identified as inattentive by their teachers 25.00 81.40 2% 1.2.1 Reduced Short Term Hunger Percent of students in target schools who indicate that they are "hungry" or "very hungry" during the school day (collected through a student survey) 25.00 82.00 47.4% 1.2.1.1/ 1.3.1.1 Increased Access to Food (School Feeding) Number of daily school meals provided to school-age children as a result of USDA assistance 21,711,950.00 0.00 12,615,812.00 1.3 Improved Student Attendance Percent of girl students regularly (80%) attending USDA-supported schools 80.00 57.50 84.8% Improved Student Attendance Percent of boy students regularly (80%) attending USDA-supported schools 80.00 57.50 96.7% 1.3.1 Increased Economic and Cultural Incentives(Or Decreased Disincentives) Percent of students that participate in extra-curricular activities 94.50 90 96.6% 1.3.2 Reduced Health Related Absences Percent of students (Girls) in target schools who miss more than 10 school days/year due to illness. 10 18.00 2.3% 1.3.2 Reduced Health Related Absences Percent of students (Boys) in target schools who miss more than 10 school days/year due to illness. 10 18.00 2.1% 1.3.3 Improved School Infrastructure Number of educational facilities rehabilitated/constructed as a result of USDA assistance 70 0.00 63.0 1.3.4 Increased Student Enrollment Percent increase in girl students enrolled in school as a result of USDA assistance 25 0.00 14.9% 26 Result Indicator Final Target Baseline Final Evaluation 1.3.4 Percent increase in boy students enrolled in school as a result of USDA assistance 25 0.00 1.7% 1.3.5 Increased Community Understanding of Benefits of Education Percent of parents in target communities who say that it is important that their children attend school (collected through a survey) 80 59.5 96.4% 1.4.1 Increased Capacity of Government Institutions Number of government staff in relevant ministries/offices certified to monitor the safety of food in school feeding programs 24 0.00 1285.00 1.4.3 Increased Government Support Number of “community meetings” organized in target communities by government offices to receive and respond to community concerns related to local schools and education. 16 4.00 16.00 SO2 Increased Use of Health and Dietary Practices Number of individuals benefiting directly from USDA-funded interventions 61 0.00 2000.00 2.1 Improved Knowledge of Health and Hygiene Practices Percent of parents in target communities who can identify at least three important health/hygiene practices 50 45.00 61.7% 2.2 Increased Knowledge of Safe Food Prep and Storage Practices Percent of food preparers at schools who achieve a passing score on a test of safe food preparation and storage 75% 0.00 100%+(1017/572) 2.4 Increased Access to Clean Water and Sanitation Services Percent of target schools with year round access to clean and safe water source 100 96.5 100.0% 2.4 Percent of target schools that have well-functioning latrines 80 62.5 89.1% 2.5 Increased Access to Preventative Health Interventions Percent of target schools that have at least a one month supply of soap (hand and dish soap) 70 0.00 N/A 2.5 Percent of households in target communities that have received/are receiving vitamin A and iron supplements sufficient to produce health benefits 80 66.5 82.7% 2.5 Percent of households in target communities that have received de-worming tablets during the previous six months 80 61.9 82.7% 2.6 Increased Access to Requisite Food Prep and Storage Tools and Equipment Percent of target schools with improved food prep and storage equipment 80 6.10 100.0% 2.7.4 Increased Engagement of Local Organizations and Community Groups Number of community groups engaged in contributing to safe food prep and storage practices 735 0.00 735 ECT Evaluation Final Report 27 5.3 Change (Impacts, Effectiveness) 5.3.1 Infrastructure Qualitative data shows evidence of change brought about by the ECT project in communities. School council members were interviewed to share their views about project impact and effectiveness. In terms of school infrastructure, the project has helped the community to build classrooms, warehouses, kitchens and energy saving stoves, including improved toilets. The schools were also equipped with food stores and kitchens for food storage and preparation. The new infrastructure brought by the project have improved both the teaching and learning conditions in the schools, and has also improved community resilience in cases of extreme weather conditions (heavy rains, heat, strong wind with dust). According to the School council members the schools provide a good place for children to study and play even during the holidays. One of most important change was the provision of drinking water and latrine facilities. 5.3.2 Provision of meals in schools The ECT project is providing meals for every child in target schools. The school meals are contributing to address food security issues that are very common in Nacarôa and Muecate. The meals have, on one hand, helped improve school attendance as highlighted in the findings. It has played a critical role of bringing children to schools regardless the conditions of the children in terms of food security in their homes. The community and children are very grateful to be served meals at schools particularly because these districts have been severely affected by food insecurity. Before the project about 82% of children were hungry during the school day, but now 47.4% of children reported to have felt hungry during the school day. The figures Pictured: Warehouse and kitchen built by the project Pictured: Latrine built by the project ECT Evaluation Final Report 28 suggest that nearly half of the school children face hunger during the school day but it has improved more than 30% since the project inception. Interviews with Project staff and also observation revealed that meals are prepared in the middle of classes in some schools due to logistic challenges that are involved in preparing the food and making children eat at one time. Focus groups discussions with cooks also revealed that they usually don’t have enough time to cook the food and administer before class and end up giving the meal at break-time when all the children are seated. 5.3.3 Provision of school materials Addressing the issue of lack of school material (notebooks, pens, and pencils) has been mentioned as one of great benefit for children at schools. The fact that many parents were not able to acquire school material for their children adversely affected pupil’s performance and willingness to keep going to school. When it comes to the availability of school materials in a remote area such as Nacarôa, this has brought a great relief among parents who have to ensure that their children are well equipped for the school. Ultimately, this has given every child equal opportunity to attend school with proper material. 5.3.4 Community volunteers Cooks and other community volunteers were asked to share their experiences and work with the ECT project in the community. Community leadership and authority was involved since project inception and this has been greatly appreciated. In terms of change, the fact that the ECT project signed memorandum of understanding with local community members has in a way empowered the community to deal and partner with outsiders for a common goal. These experiences are reported to be of a great importance and impact for the community. 5.4 What worked well? ECT project has been implemented through a community participation-based approach taking the advantage of using already existing institutions such as local Government, (Community Based Organisations) OCB, community leaders, local NGO (FDC). This approach helped the effective embedment of the project within community systems. ECT Evaluation Final Report 29 The fact that the project provided meals at schools catalysed the willingness and likelihood of school attendance. The meals have been triggering school attendance and this approach worked very well due to the fact that food security is a concern in Nacarôa and Muecate. Now parents feel more comfortable to send their children to school because they know that they will get a free meal at school. It is believed that that the pupils tend to be much more punctual now than before the ECT project. The project built toilets in some schools and improved the access to clean water in schools through water collection tanks. The children having access to improved water and sanitation facilities have great potential to improve the hygiene conditions as well as to prevention of waterborne diseases such as diarrhoea. On the other hand, the fact that the children have access to water and toilets will potentially contribute to keep the girls at school and therefore increase the likelihood of attaining reading and writing skills. Another aspect that did work well was the regular training of school directors, teachers in early education matters and provision of school material. The findings show that the trainings have significantly contributed to the quality of teaching and learning in schools, while providing an incentive to the teachers who are meant to improve the reading and writing skills. Pictured: Volunteer cooks preparing the FUBA in Muecate. Pictured: Group photo with School administrators and their pupils in Muecate. ECT Evaluation Final Report 30 5.5 What did not work well (Challenges)? 5.5.1 Schools infrastructure: access to water and sanitation services While all ECT supported schools were well equipped with water harvesting systems such as water tanks, the lack of rainwater has negatively contributed to access to water and sanitation services. Unfortunately, the majority of schools still need their own water systems and not those shared with the general community so children and volunteers don’t experience challenges of getting clean water during their time at school. 5.5.2 Food preparation and storage Interviews with Project staff and also observation revealed that meals are prepared in the middle of classes in some schools due to logistic challenges that are involved in preparing the food and making children eat at one time. Focus groups discussions with cooks also revealed that they usually don’t have enough time to cook the food and administer before class and end up giving the meal at break-time when all the children are seated. In addition, Community volunteers (Cooks) say that cleaning detergents should be provided by the project. In addition to incentives or other type of compensation, they should have proper kitchen gear. For example, these volunteers should have a uniform and shoes and maybe a subsidy as they operate like full time workers and do not have time to perform other work for their subsistence. It was also noted that hygienic and protection materials (e.g. soap, gloves, boots, uniform, hats) should be included in the package of Cooks to ensure clean and healthy cooking environment. Pictured: One of the school that was built by the WV, Muecate Pictured: students showing school supplies that are producing with WV support, Nacarôa. ECT Evaluation Final Report 31 5.5.3 Community participation and transparency With regard to the collaboration between the ECT Project and local authorities, in the FGD with community leaders, it was found that some of leaders were not adequately prepared in the selection of volunteers for food preparation. It would have been important to set clear criteria for selection of volunteers in the community so that every person get the opportunity to participate and learn where and how they should be involved in the project. This can help communities to understand why a selected group of people participated in this project 5.5.4 Water supply and sanitation The project came to construct rainwater tank systems in all schools that implement the ECT project in both districts. It is therefore a great chance to address some of issues related to lack of access to safe water and sanitation such as: water for drinking at school, handwashing before eating the CSB, water for cooking the CSB, cleaning the dishes before and after the CSB meals, cleaning the kitchen and its utensils for the preparation of the meal and general hygiene of the volunteers. In addition to lack of sufficient water in schools, the project could have explored other alternative water sources such as collection of water from streams, small earth dams, increased capacity of water harvesting, etc, to ensure that the children get water during the dry season. Pictured: Place where the dishes are washed in schools after the children feed. Pictured: Children eating their food in Nacarôa during the data collection. ECT Evaluation Final Report 32 5.6 Project sustainability 5.6.1 Training ECT project objective is to address some of the common challenges of education in early stage in a developing world context. The project is creating capacity at local level focussing on education training such as: school management, planning lessons and teaching methods with school directors and teachers. On the other hand the project has contributed to improve the knowledge of hygiene and dietary practices through community awareness campaigns. Sanitation messages such as hand washing and usage of toilets have been contributing to a healthier and knowledgeable community. 5.6.2 Embedment of the project into local community Initially the project signed a Memorandum of Understanding (MoU) between WV, the Schools involved and local leaders (“Régulos” Regulators). The MoU intended to describe and clarify the roles and responsibilities of the key stakeholders in project at activity level. For example, the project had three key areas of training, namely pedagogical matters, school material crafting and food preparation & storage. The fact that the ECT project signed an MOU with these key partners emphasizes the long-lasting partnership in those communities. On the other hand, teachers training in different matters including food distribution/ management skills as well as school material crafting is a long term legacy for the schools, teachers and pupils. 5.6.3 Keeping the children at schools ECT Project has substantially increased parent’s willingness to send their children to school. (96.4% now compared to the 59% at the beginning of project). Before the ECT project most parents struggled to keep the children at schools, as they did not have sufficient capacity and resources (hunger, lack of school material, knowledge of importance of education, etc) that would ensure the children to stay at school. Equally it is expected that key messages on health and hygiene practices will be kept sustainably to ensure that the future generation will get benefits from this project. ECT Evaluation Final Report 33 6 Specific Conclusions and Recommendations 6.1 Conclusions Following are conclusions arranged according to the Project Objectives: SO1: Improved Literacy of School-Age Children (i) Quality of literacy of school-aged children was measured through EGRA and FLAT. The number of children with the improved ability of reading and comprehension has increased by 17% from the baseline. This means that the proportion of pupils (grade two) in schools that demonstrates a level of reading and understands the grade level text has increased considerably after the project intervention. (ii) Improved Quality of Literacy Instruction - The evaluation found that 87% of the teachers are very well trained with improved teaching technics and equipped with teaching tools against 42.4% (baseline) of teachers in target schools. This means that the teachers have greater impact in contributing to improved quality of literacy instruction. The project has delivered regular training of school directors, teachers in early education matters and assembling of school material and this has contributed to a good result toward to improve student attendance. The trainings has significantly contributed to the quality of teaching and learning in schools, while providing an incentive to the teachers who are meant to improve the reading and writing skills. (iii) Improved Attentiveness -There was a huge decline of students who are inattentive in the ECT districts from 81% at baseline to 2.0% in the evaluation. This can be attributed to the fact that the project provided meals at all schools in target districts and this has catalysed the willingness and likelihood of school attentiveness by the pupils and concentration to lessons. (iv) Improved Student Attendance – About 90% of children regularly attend the classes during 80% of the teaching time in the USDA-supported schools compared to 57.5% of students (baseline). SO2: Increased Use of Health and Dietary Practices (i) Improved Knowledge of Health and Hygiene Practices – About 61.7% of communities (including children) can identify three important health/hygiene practices compared to 45% of parents at the baseline. The healthier the households, the more available to ECT Evaluation Final Report 34 attend the school are the children that ultimately contribute to improved children’s attendance. (ii) Increased Knowledge of Safe Food Preparation and Storage Practices – 100% of schools by the end of the ECT Project had a safe and improved form of stove, with volunteers well prepared and showed capacity of passing on a safe food preparation test. It is important to note that by the time of the baseline no meals at all were being prepared in schools. The improved knowledge of safe food preparation and storage practices increases the quality of the food produced and ultimately reduces the illness of the children. (iii) Increased Access to Clean Water and Sanitation Services - The percentage of target schools with small water systems has increased from 3.5% (baseline) to 100% during the evaluation. The project has therefore improved the water and sanitation infrastructures in schools through the provision of water collection tanks and latrines. The fact that the children have access to these facilities contributes to keep the girls and boys at school and therefore increase the likelihood of attaining reading and writing skills. (iv) Increased Access to Preventative Health Interventions – Households access the supplement to Vitamin A in target communities increased from 66.5% in baseline to 83% in evaluation. The fact that the ECT project had worked in partnership with the health authorities to sensitize school-aged students and parents to encourage the children to take Vitamin A has brought a positive message about the benefit of health services in the households. Therefore, more parents and guardians have been sensitized to adhere the child health campaigns. (v) Increased Access to Requisite Food Preparation and Storage Tools and Equipment -It was noted that 100% of targeted schools had been fully equipped with proper functioning cooking items. These comprised of a stove and a store room for the food. This indicates that the project has super passed the goal of 80%, which can be explained with the impressive ongoing changes in the school community in terms of food preparation, storage capacity and meals preparations. ECT Evaluation Final Report 35 6.2 Recommendations With regard to quality of literacy the results suggest a promising improvement of the quality of literacy, taking into considerations the time of project implementation. There is need to for the ECT Project to learn on the approach being used by APAL that seen to have brought more success than the FFE approaches after looking at the findings. However, more interventions on skills and knowledge of teachers as well as more consistent teachers attendance is needed to ensure that what has been achieved so far is kept growing in the future. Despite some challenges such as number of pupils in each classroom, as well as teachers training and motivation, it is recommended that pupils should be continuously encouraged to read and improve writing skills at classroom level. In terms of access to clean water and sanitation services, the construction of water tanks in all schools created a significant impact among pupils while creating a great potential to address some of issues behind lack of safe water and sanitation services in schools. Nevertheless, the tanks are currently empty not responding to the school needs due lack of rainfall. Lack of access to water is intrinsically related to lack of good sanitation services. Lack of water in schools can dramatically affect the food preparation and hygiene. It is therefore recommended that future project should assess the potential of water sources in different school locations and the rainfall pattern in order to determine where water system including rainwater tanks should be installed. It is also important to ensure that the water tank design is realistically appropriate with the school water consumption so that the pupils can have the access to clean and sufficient water and sanitation services consistently. With regard to the collaboration between the ECT Project and leaders, the key informant interviews provided a feeling that some leaders were not properly involved in the selection of volunteers for food preparation. It would have been important to set clear criteria for selection of volunteers in the community so that everybody get the opportunity to learn where and how everybody give their contribution. This can help people to understand why everybody cannot fit in the project. Community volunteers (Cooks) claim for working material such as cleaning detergents were unavailable. In addition, these volunteers’ should have a uniform and protection shoes and do not have time to perform other. It was also noted that hygienic material should be included in the package of cooks to ensure clean and healthy cooking environment. It is good to notice ECT Evaluation Final Report 36 that children are adhering the schools due to ECT interventions. This demands more food and more volunteers should be recruited so that every child will have a chance to get a well￾prepared meal during the lunchtime. As stated earlier in this document meals were prepared in the middle of classes in some schools due to logistic challenges that are involved in preparing the food and making children eat at one time. Since this clearly shows that there is need to increase the number of cooks as stated above, there is also need to negotiate with the Ministry of Education on the start time of class so the cooks can ensure that food is prepared before classes start, midway and not after. This should be done after a head count is established so that all children are catered for. ECT Evaluation Final Report 37 7 Most Significant Change Story The followings are Most Significant Change Stories to help conclude the impact made by the ECT project during the three years of implementation. The stories communicate some key successes of the project in Nacarroa and Muecate Districts. Story title: Bright Future “I was often sad, hungry and our performance at school was bad and we could see that our teachers had to make extra-efforts to get our concentration. Our teachers would often rebuke us because we were often not attentive in class..., said a child with a smile face. The girl is 15 years old. She tells her story with a smile in her face looking over the leaves of the trees shaking on a sunny hot day. I don’t remember the month, but the school feeding started in 2013 at our school and the news came with our teacher that from that day on, we would start having a school feeding programme. She recalls that the situation before the programme was bad, “we were sad, hunger, and it needed a lot of courage and patience for a teacher to work with us. Our parents had to force us to go to school. We now have a meal during break time, we continue with classes without feeling lazy or too hungry to focus on what the teacher is saying. I think the programme is very helpful and some other children of my community who sometimes don’t have food in their houses will appreciate”. “The most significant change that was brought by the programme is making us concentrate in the class without feeling hungry or tired or weak. This change was significant because with the porridge, we stayed in class for all the period to listen to them. The teachers had books and the started to read them. I can read and understand school lessons easily now that before” said the girl. She recalls the first announcement of her teacher about the school feeding and she says. I think that with the arrival of the project, many children like me had a good learning environment to start improving to read and understand what the teacher is saying. There will be good education and all children like me will dream to be important service providers of our community in the future. Myself I want to be a teacher. Teaching when kids are happy makes this profession very pleasant. Giggles the girl as she finished talking to us. ECT Evaluation Final Report 38 8 Appendices 8.1 List of people interviewed, with affiliation and contact details 8.2 Evaluation TOR Moz FFE Final Eval ToR 7Aug15-Last.do 8.3 Evaluation Tools ECT_TOOLS.zip