PLANET AID/USDA Food for Education phase 2 Baseline study report Author: Simone Doctors with Rosa Tinga, ADPP Mozambique and NFER 19 July 2018 (based on data collected between May 2017- March 2018) Agreement Number: FFE-656-2015/009-00 2 Table of contents Table of contents ................................................................................................................................................ 2 Acknowledgments .............................................................................................................................................. 5 List of abbreviations and acronyms .................................................................................................................... 6 Executive summary ............................................................................................................................................ 7 Introduction ................................................................................................................................................... 7 Baseline findings ............................................................................................................................................. 9 1. A review of the findings of the Final Evaluation report of the FFK1 project ............................................. 9 2. Pupil-level data .....................................................................................................................................10 3.Project schools .......................................................................................................................................10 4.Teachers working in project schools.......................................................................................................11 5.Teacher training colleges (EPFs) .............................................................................................................11 Conclusion ....................................................................................................................................................12 Introduction .....................................................................................................................................................13 Background ..................................................................................................................................................13 1. School feeding, water supply development, school gardens, and related activities ...............................14 2. Early grade reading and writing intervention ........................................................................................14 3. Nutrition education programme ...........................................................................................................15 Overview of the FFK2 Baseline study ............................................................................................................15 Workshop to define M&E needs for FFK2 ................................................................................................. 15 Alignment with previous evaluations ........................................................................................................16 Alignment with future evaluations ............................................................................................................16 Methodology ................................................................................................................................................17 Sampling for pupil-level data collection ........................................................................................................18 Challenges, delays and lessons learned .........................................................................................................22 Baseline findings ...............................................................................................................................................23 1. A review of the findings of the Final Evaluation report of the FFK1 project ...............................................23 Human Resources, capacity, collaboration and ownership ........................................................................23 3 Transport ..................................................................................................................................................24 CSB and Logistics ......................................................................................................................................24 Nutrition training ......................................................................................................................................25 Water and sanitation ................................................................................................................................25 Literacy, school clubs and kits ...................................................................................................................26 School gardens .........................................................................................................................................26 Teacher training ........................................................................................................................................27 Budget and costs ......................................................................................................................................27 Pupil attendance .......................................................................................................................................28 Sustainability and relevance to local and national school feeding policy and programme environment ....28 2. Pupil-level data .........................................................................................................................................29 Literacy assessment (EGRA): presentation and results ..............................................................................30 Other pupil-level results............................................................................................................................34 3. Project schools ..........................................................................................................................................43 Buildings ...................................................................................................................................................45 Water and sanitation ................................................................................................................................46 School feeding ..........................................................................................................................................48 School gardens and Home Grown School Feeding Gardens .......................................................................50 Nutrition, hygiene and school health ........................................................................................................51 Extra-curricular clubs ................................................................................................................................52 4. Teachers working in project schools .........................................................................................................54 Questions relating to the project ..............................................................................................................55 Hunger, attentiveness and learning ..........................................................................................................56 Nutrition ...................................................................................................................................................57 Literacy .....................................................................................................................................................59 5.Teacher training colleges (EPFs) .................................................................................................................64 Profiles of EPF trainers ..............................................................................................................................64 Specialised subjects: trainer supply and demand .....................................................................................66 Conclusion ........................................................................................................................................................69 4 Annexes ............................................................................................................................................................71 Annex 1: Results Framework of the Food for Education Project in Mozambique, capturing the Theory of Change upon which the project is based .......................................................................................................71 Annex 2: McGovern-Dole Results Frameworks .............................................................................................72 Annex 3: Workshop to define Monitoring and Evaluation needs for the Food for Knowledge programme, phase 2: Recommendations for project management and implementers .....................................................73 Annex 4: Technical appendix 1 – Introduction and analysis of EGRA subscales .............................................76 Annex 5: Technical appendix 2 – Analysis of Pupil and Teacher Surveys ......................................................106 Annex 6 : Instrument for pupil data collection (Portuguese version)...........................................................132 Annex 7: Questionnaire for teacher survey.................................................................................................148 Annex 8: Questionnaire for school survey ...................................................................................................160 Annex 9: Questionnaires for EPF survey......................................................................................................162 9a Trainers..............................................................................................................................................162 9b Supply and demand of trainers by discipline ......................................................................................163 5 Acknowledgments A large number of individuals and institutions have contributed to this baseline study. Recognition and thanks are due to all the pupils, teachers, school leaders and directors of Escolas de Professores do Futuro who contributed their time and information and to the Food for Education project staff who gathered and entered the data. The authors are grateful to the many ADPP, Planet Aid and Food for Education project staff and volunteers who contributed in different ways; to representatives of the Mozambican Ministry of Education and Human Development, the Directorate of Education and Human Development of the Province of Maputo and the District Services of Education, Youth and Technology of the eight districts of Maputo province, who provided information for the sampling plan and facilitated field work; to representatives of the World Initiative for Soy in Human Health, representatives of Cambridge Education and representatives of the Instituto Nacional para Desenvolvimento da Educação, project leadership team, project Monitoring and Evaluation team and component managers, for their contributions to the data gathering instruments and training of data gatherers. The project management team were unstinting in their support, energy and willingness to troubleshoot and fill gaps. The expert contribution of the statistical team from the National Foundation for Education Research is acknowledged with gratitude, as is the data verification provided by Valerie Legg. 6 List of abbreviations and acronyms ADPP Ajuda de Desenvolvimento de Povo para Povo Development Aid from People to People ASA American Soybean Association BMI Body Mass Index CSB Corn Soy Blend DPEDH Direção Provincial de Educação e Desenvolvimento Humano Provincial Directorate of Education and Human Development EGRA Early Grade Reading Assessment EPC Escola Primária Completa Complete primary school, which includes Grades 1-7 EPF Escolas de Professores do Futuro (teacher training colleges run by ADPP) FFE Food for Education (project title) FFK Food for Knowledge (project title) GoM Government of Mozambique HGSFG Home Grown School Feeding Garden HIV&AIDS Human immunodeficiency virus infection/acquired immunodeficiency syndrome INDE Instituto Nacional para Desenvolvimento da Educação National Institute for Education Development ISET/OWU Instituto Superior de Educação e Tecnologia One World University (ISET/OWU) M&E Monitoring and Evaluation MGD McGovern Dole (indicator) MINEDH Ministry of Education and Human Development NFER National Foundation for Education Research PAI PLANET AID Inc. PRONAE Programa Nacional de Alimentação Escolar National School Feeding Programme RTI Research Triangle Institute SDEJT Serviço Distrital de Educação, Juventude e Tecnologia District Office for Education, Youth and Technology SDSMAS Serviço Distrital de Saúde Mulher e Acção Social District Service for Women Health and Social Action SFC School Feeding Committee TB Tuberculosis ToC Theory of Change USAID United States Agency for International Development USDA FAS United States Department of Agriculture - Foreign Agricultural Service WFP World Food Programme WHO World Health Organisation WISHH World Initiative for Soy in Human Health 7 Executive summary Introduction Phase 2 of the Food for Education project (2015-2020), supported and funded by the Foreign Agricultural Service (FAS) of the United States Department of Agriculture (USDA), follows the 2012-2016 project of the same name. A comprehensive school-feeding project, complemented by a major literacy component, it also includes child health, nutrition education, teacher training, water and sanitation components. The project is implemented in Mozambique by Planet Aid (PAI) and Ajuda de Desenvolvimento de Povo para o Povo (ADPP), Planet Aid’s local partner in Mozambique, in partnership with the World Initiative for Soy in Human Health programme (WISHH) of the American Soybean Association (ASA) and Cambridge Education (CE). This baseline evaluation, based on four new primary datasets and, where appropriate, information from the final evaluation of phase one of the project, provides a “photograph” of the project at the beginning of phase 2, against which achievements may be measured at mid-term and end point. In addition to offering essential accountability to its donors and other stakeholders, this report presents additional information, which project staff and managers considered useful, to make informed decisions about their operations. The project aims to improve literacy of school-age children in Mozambique and to increase the use of health and dietary practices, through interventions designed to improve the quality of literacy instruction. This significant addition to phase two of the project will train teachers in project schools, school managers and pre￾service teacher trainers in literacy and give learners improved access to school supplies and materials, in line with the project’s theory of change. Improving school infrastructure and increasing enrolment and attendance, as a result of the school feeding programme and other interventions, is expected to further contribute to improving child literacy. The project also aims to increase the use of health and dietary practices, through increased knowledge of nutrition, increased access to clean water and sanitation services and increased access to preventative health interventions, such as de-worming and TB screening. Increased engagement of local organisations and community groups and increased capacity of government institutions are essential to achieving both goals; school Feeding Committees are a focus of community support for school feeding and related activities. The project acts as one of the models for the national school feeding programme (PRONAE), being developed and implemented by the Government of Mozambique, the World Food Programme (WFP) and the Government of Brazil. The project consists of three major components: 1. School feeding, water supply development, school gardens, and related activities for 74,000 students in 270 schools in 4 districts of Maputo Province, Mozambique (Moamba, Magude, Manhiça and 8 Matutuine), through a daily meal, school gardens, a safe and adequate school water supply, and de￾worming treatment. After-school clubs and kits of educational and recreational materials are provided to promote an improved learning environment, enrolment and attendance of female and male pupils. 2. Early grade reading and writing intervention for 1st , 2 nd and 3rd grade children in 109 schools In partnership with Cambridge Education; 63 of these schools will receive materials in either Xirhonga or Xichangana to help them provide improved instruction in local languages. Teachers will receive training in the new methods, with reading coaches providing follow up support to the trained teachers. 3. A nutrition education programme, in partnership with Planet Aid’s implementing partner, WISHH, to produce nutrition and health educational materials and information and implement a comprehensive nutrition campaign in the 4 beneficiary districts. Primary school teachers are trained as Nutrition Educators/Trainers, then train other teachers. The objectives of the baseline survey, which was carried out as close as possible to the onset of the phase 2 project interventions, were:  Provide a “photograph” of the initial situation before the phase 2 activities, giving reference points against which to measure the subsequent achievements, and allowing targets to be set;  Update knowledge of factors which should to be considered in implementing phase two of the project ;  Promote ownership of and awareness of the project;  Collect specific information requested by managers and staff of project components, making M&E more relevant to them and promoting their active participation in M&E systems;  Give project management and staff detailed knowledge of the situation and needs of the project schools and teacher training colleges (EPFs), to allow interventions and approaches to be fine-tuned accordingly;  Measure the current level of literacy of pupils in the grades targeted by the literacy intervention, to inform the preparation of literacy materials and delivery of the literacy intervention. At the end of February 2017, a workshop, in which approximately 35 members of the project leadership and section representatives participated, was held to define M&E needs for FFK2; the aim was to make M&E more relevant to the project team, promote ownership of M&E systems and agree on the M&E needs of the second phase of the FFK programme (what information the project wished to collect and for what purposes). The workshop formed the basis for refining the Programme Monitoring Plan (PMP) and Evaluation Plan (EP) and for the Terms of Reference for the Baseline Study for FFK2. Since some phase 2 project activities are continuations of activities from FFK1, where appropriate, baseline information for indicators relating to these activities has been taken from the baseline, mid-term or final evaluations of FFK1. In other cases, where baseline information is required for new indicators or for indicators which previous evaluations were not able to adequately address, new information has been gathered. In particular, the baseline study is based on a new cohort of pupils to ensure that the mid-term and final 9 evaluations of the FFK2 project are able to collect comparable information from the same students at three distinct time points. The methods used to collect the information for the Baseline included: 1. A review of the findings and recommendations of the Final Evaluation report of FFK1; 2. Literacy testing of 6,291 learners (from project schools and control/counterfactual school) using a specially developed Early Grade Reading Assessment (EGRA); 3. Interviews with the same learners; 4. Weighing and measuring of the same learners to establish BMI z scores; 5. A survey of all 270 project schools; 6. A survey of a sample of 458 teachers from project schools; 7. Surveys of all 11 EPFs regarding infrastructure and equipment, profiles of teaching staff and staffing levels relative to teaching needs. This produced four primary data sets:  Pupil-level information (results of the EGRA, interview, weighing and measuring of pupils);  Teacher-level information (results of the teacher survey);  School-level information (results of the survey of project schools);  EPF-level information (from surveys completed by all EPF directors). The report documents and presents a number of challenges and delays to the collection and analysis of data for the FFK2 baseline study. The process has been valuable for staff learning and engagement but took project staff away from other project activities. Baseline findings 1. A review of the findings of the Final Evaluation report of the FFK1 project To avoid redundancy and wasting resources, information collected during the final evaluation of FFK1 has not been collected again for the FFK2 baseline; new information was collected only where the new framework or indicators meant this was necessary. The main findings and messages on the following themes are presented succinctly:  Human Resources, capacity, collaboration and ownership  Transport  CSB and Logistics  Nutrition training  Water and sanitation  Literacy, school clubs and kits  School gardens  Teacher training  Budget and costs  Pupil attendance  Sustainability and relevance to the local and national school feeding policy and programme environment 10 The analysis of the four new data sets collected for this baseline study is then presented. 2. Pupil-level data A cohort of grade 1 pupils (from project schools and comparison schools) took an Early Grade Reading Assessment (EGRA), underwent a short interview and were weighed and measured. A small number of pupils from grades 2 and 3 also took part. The EGRA was developed in three equivalent versions, in Portuguese, Xichangana and Xirhonga, by specialists in early grade reading in Mozambique and based on the early grades literacy curriculum. It consists of 11 sub tasks, designed to reflect the progressive stages of literacy acquisition, so that a pupil continues until they have reached the threshold at which they can no longer perform the tasks. Overall, the EGRA results reveal very low literacy outcomes for the vast majority of the early grade learners tested. Grade 1 learners were unable to proceed beyond the first four sub tasks, as was expected since, at the time of testing, they had only been attending school for three to four months. 85% of grade 2 pupils were unable to read simple words, whether in a list or within a short text; only 5% of grade 2 pupils were able to read two thirds of the list of words correctly, with 6% managing to read more than half of the text. Around 50% of grade 3 pupils were unable to read any of the simple words in a list or within a text, 24% were able to read two thirds of the list of words correctly and 27% to read more than half of the text. This suggests that the majority of early grades pupils are failing to learn to read at all, whereas those who are learning are performing far below the expected level. The same pupils took part in an interview. The following outcomes are reported:  Height  Weight  BMI z-scores  Attitude to learning and clubs  Hunger  Attentiveness  Health and hygiene practices 3.Project schools Project staff and management wished to have access to detailed school-level information regarding the infrastructure built or improved and maintained as part of the project. The project is also procuring a tailor￾made database to manage project data more effectively. All 270 schools participating in phase two of the project took part in the survey; 73,630 pupils attend these schools. Extensive, detailed information on the following project components is reported:  Buildings: latrines and hand washing facilities, storerooms and kitchens  Water and sanitation 11  School feeding  School gardens and larger Home Grown School Feeding Gardens  Nutrition, hygiene and school health  Extra-curricular clubs 4.Teachers working in project schools Two teachers from each project school were asked to respond to the teacher survey; these were teachers with special responsibilities in relation to the project. Of 458 teachers surveyed, 37% are female; 63% male. Over half are aged 18-29. Well over half have received one year’s teacher training after completing 10 grades of secondary education. They responded to questions on the following themes:  Questions relating to the project: school feeding, extra-curricular clubs, materials distributed  Hunger, attentiveness and learning amongst pupils  Nutrition  Literacy A high percentage of teachers (67%-81%) did not respond to the questions on literacy, so there is a regrettable gap in the data. However, taken together with the very low results of the EGRA, the results of the teacher questionnaire confirm a deficit in teacher’s knowledge and skills, particularly in the area of literacy, which is perhaps not surprising, given their age profile and level of training. However, the teachers’ responses to the nutrition questions suggest many teachers have learnt about basic nutrition and hygiene from the project interventions. Hopefully, the training in literacy will also lead to teachers becoming more competent and more confident in this important area. 5.Teacher training colleges (EPFs) Based on the EPFs’ identified needs for specific information, data collection instruments were developed to gather information concerning the overall profiles of EPF trainers and to compare the supply and demand for specialized trainers. Information presented about the profiles of trainers working in the 11 EPFs reveals that they are predominantly male, 90% hold a Licenciatura qualification from One World University/ISET, the Universidade Pedagógica or another university and, in terms of their administrative category, 87% are classified as teachers, rather than teacher trainers. A survey of the needs and availability of trainers in specialised subjects reveals that, although most positions are filled in subjects such as Psycho-pedagogy, Portuguese language teaching methods, Expression Techniques and Language, School management and Organisation, Social Sciences teaching methods and Bantu languages and bilingual education, large numbers of posts in certain other subjects are unfilled. There are unfilled posts 12 in Research methods/ICTs, Visual education teaching methods, Building, Maintenance and School Production, Crafts teaching methods, Music teaching methods and Moral and Civic Education. This situation is not limited to the EPFs and may reflect the international shortage of specialised teachers caused by labour market forces. Many countries struggle to attract specialised teachers and teacher educators, due to the more lucrative positions appropriately qualified candidates can access in other sectors. The EPFs will need to develop strategies to respond to this situation and attract or develop the specialised trainers they need. Conclusion The final report of the first phase of the project documented a considerable degree of success and the beginnings of meaningful change in the lives of its beneficiaries, as well as some outstanding challenges which formed the basis of a series of recommendations for phase two. The four new data sets presented in the current report were collected because project managers and staff wished to collect and maintain good quality, detailed information, in many cases about highly specific aspect of their work. This extensive data collection, which coincided with the development of the project’s dedicated data base, will allow improved information management and use, providing the necessary human resources and systems are available and used well. In particular, the project M&E team requires reinforcement and development. This ambitious baseline study has not been without costs: mobilising project staff for longer than was ideal. Delays, exacerbated by gaps in the M&E team, mean the finalised report has taken longer than expected. Hopefully, the investment of gathering and processing this information will bring rewards, in terms of its utility to the project and of project staff’s investment in and ownership of the M&E process. The project needs sufficient M&E staff with the right knowledge and skill sets, combining the intellectual skills and strategic vision to grasp and communicate the “big picture” with the rigour and attention to detail needed to develop and follow precise data protocols meticulously and support others to do likewise. The baseline study and report owe a great debt to the implication and conscientiousness of the project management team. The different component managers have also contributed much; their reward should be having access to relevant, useful information to inform their strategic and operational functions. Likewise, the “professionals” should benefit from their investment in the data gathering process, in terms of new skills acquired and the information to which they now have access. As a priority issue for phase two, the final evaluation of FFK1 stressed the need for urgent focus on sustainability. All stakeholders must urgently address will happen after the end of the project in 2020. A transition plan needs to be made and implemented, to ensure the endurance of what has been achieved so far. This must become the basis of a sustained and sustainable school feeding programme for as long as one is needed. 13 Introduction The 2015-2020 Food for Education project (phase 2 of the 2012-2016 project of the same name) is a comprehensive school-feeding project with child health, nutrition education, teacher training, water and sanitation components, complemented by a major literacy component. In advance of the commencement of project interventions, this report offers a thorough baseline evaluation of four primary datasets, augmented by a comprehensive analysis consisting of detailed information about the project’s processes and implementing environment, with extensive attention paid to lessons from phase one of the project. This is provided with the intention of informing all aspects of planning and implementation of the project’s second phase project and providing a “photograph” of the project at the beginning of phase 2, against which achievements may be measured at mid-term and end point. This report has the ambition to go further than facilitating accountability to its donors and other stakeholders, as vital as this is. By greatly exceeding the donor requirements and collecting and analysing a large amount of information identified as important by project staff and managers, with this baseline the project takes ownership of its Monitoring and Evaluation (M&E) processes and systems, ensuring staff and managers can access the information they consider necessary to make informed decisions about their operations. Background The 2015-2020 Food for Education (FFE2) project, phase two of the project of the same name, usually referred to as Food for Knowledge (FFK2), is supported and funded by the Foreign Agricultural Service (FAS) of the United States Department of Agriculture (USDA). The project is implemented in Mozambique by Planet Aid (PAI) and Ajuda de Desenvolvimento de Povo para o Povo (ADPP), Planet Aid’s local partner in Mozambique, in partnership with the World Initiative for Soy in Human Health programme (WISHH) of the American Soybean Association (ASA) and Cambridge Education (CE). Mozambique ranks 181/188 on the 2016 UNDP Human Development Index. The project’s overarching goals are to improve literacy of school-age children in Mozambique and to increase the use of health and dietary practices. To this end, the project takes a holistic, multi faceted approach to promote the health, wellbeing and education of school aged children, combining a straightforward school feeding programme with health, water, sanitation, literacy and nutrition components. The government of Mozambique has made considerable progress in improving access to education. However, as far as improving the quality of education is concerned, major challenges remain. The rapid expansion has placed intense pressure on school management, teaching personnel and the ability to deliver effective classroom instruction, resulting in a large number of overcrowded multi-shift schools, growing student/teacher ratios and plummeting reading and mathematics test scores. According to the project’s theory of change (see the Results Frameworks in Annexes 1 and 2), the quality of literacy instruction is improved by providing intensive, targeted literacy training to teachers in project schools. 14 Increasing the skills and knowledge of primary school teachers, through on the spot coaching of in-service teachers within primary schools and giving learners improved access to school supplies and materials is expected to contribute to improving child literacy. This dimension was a significant addition to Phase 2 of the project. The findings of the mid-term and final evaluations of Phase 1, revealed that, although attendance and retention rates and reported pupil concentration in the classroom had improved since the onset of school feeding, in the absence of improved teaching, educational outcomes were unchanged. Improving students’ attendance, through a combination of improved school infrastructure and increased enrolment, is expected to further contribute to improving child literacy. Increased access to food, as a result of the school feeding programme and other interventions to increase enrolment, should also lead to increased attendance. The project aims to increase the use of health and dietary practices, through a combination of increased knowledge of nutrition, increased access to clean water and sanitation services and increased access to preventative health interventions, such as de-worming and TB screening. The approach recognises the importance to the achievement of the project goals of both increased engagement of local organisations and community groups and increased capacity of government institutions. The project promotes community support for school feeding and related activities through School Feeding Committees. Finally, the project is intended to serve as one of the models for the PRONAE, the national school feeding programme being developed and implemented by the Government of Mozambique (GOM) with collaboration and assistance from the World Food Programme (WFP) and the Government of Brazil. The project consists of three major components: 1.School feeding, water supply development, school gardens, and related activities The project aims to improve the health and nutrition of 74,000 students in 2701 target schools in 4 districts of Maputo Province, Mozambique (Moamba, Magude, Manhiça and Matutuine), through a daily meal, the development of school gardens, the provision of a safe and adequate school water supply, and the implementation of a de-worming campaign. After-school clubs and kits of educational and recreational materials are provided in all schools benefiting from school feeding, in order to promote an improved learning environment, enrolment and attendance, with a focus on ensuring gender parity in these areas. 2. Early grade reading and writing intervention Children in the beneficiary districts speak one of several local Mozambican languages when they enter school, but they have traditionally been taught to read in Portuguese, the official national language. This creates an obstacle in their acquisition of reading and writing skills as well as in their wider education. Under Phase 2, thanks to an intervention delivered in partnership with Cambridge Education, 109 beneficiary schools are to receive materials to reinforce reading and writing skills for 1st 2nd and 3rd grade children; 63 of these schools will receive materials in either Xirhonga or Xichangana to help them provide improved instruction in local 1 The original proposal was for 264 schools. 15 languages. The teachers will receive training in how to use the new methods, with reading coaches providing follow up support to the trained teachers, in the form of on the spot class/lesson coaching. 3. Nutrition education programme The project also produces nutrition and health educational materials and information and, in partnership with Planet Aid’s implementing partner, WISHH, implements a comprehensive nutrition campaign in the 4 beneficiary districts of Maputo province. Within this component, primary school teachers currently teaching in primary schools are trained as Nutrition Educators/Trainers and, in turn, they each train other primary school teachers as Nutrition Educators/Trainers in the respective schools in which they work. Overview of the FFK2 Baseline study The baseline survey was carried out as close as possible to the onset of the phase 2 project interventions. The baseline study had the following objectives:  Provide a “photograph” of the initial situation before the phase 2 activities, giving reference points against which to measure the subsequent achievements, and allowing targets to be set;  Update knowledge of factors which should to be taken into account in implementing the second phase of the project ;  Promote ownership of and awareness of the project;  Collect specific information requested by managers and staff of the different project components, so as to make M&E more relevant to these and promote their active participation in M&E systems;  Give project management and staff detailed knowledge of the situation and needs of the project schools and teacher training colleges (EPFs), allow interventions and approaches to be fine-tuned accordingly;  Measure the current level of literacy of pupils in the grades targeted by the project literacy intervention, in order to better inform the preparation of literacy materials and the delivery of the literacy intervention. Workshop to define M&E needs for FFK2 At the end of February 2017, a four day workshop was held to define M&E needs for FFK2 at the ADPP headquarters in Machava, Mozambique. The workshop, facilitated by Simone Doctors, in which approximately 35 members of the project leadership and section representatives participated, was designed to make M&E more relevant to the project team, promote ownership of M&E systems and agree on the M&E needs of the second phase of the FFK programme. In particular, building on the findings of the final evaluation of FFK1, project managers and employees were encouraged to reflect on what information the project wished to collect and for what purposes. The workshop identified lessons learned during the first phase of the programme, opportunities and constraints with regard to information collection, and requirements for data management instruments and systems. It formed the basis for refinement of the Programme Monitoring Plan (PMP) and Evaluation Plan (EP) and for the Terms of Reference for the Baseline Study for FFK2. The 16 recommendations which arose from the workshop with a view to strengthening M&E processes and practices for the second phase of the FFK programme can be found in Annex 3. Alignment with previous evaluations During the first phase of the Food for Education project (FY 2012-2016), hereafter referred to as FFK1, following the convention in Mozambique, where the title “Food for Knowledge” is often used, the following evaluations were performed:  a baseline study was conducted in May 2013 by project staff with support from an external evaluator;  a mid-term evaluation was conducted by an external evaluation team at mid point;  a final evaluation was conducted by an external evaluation team at end point (between September￾December 2016). The baseline, mid-term and final reports of the first phase of the project are available on request. Given that some of the phase 2 project activities are continuations of activities from the FFK1 project, to avoid redundancy and ensure resources are used efficiently, where appropriate, baseline data for indicators relating to these activities have been taken from evaluations of FFK1. However, in order to build on lessons learned during the evaluations of FFK1, and begin FFK2 with an improved evaluation plan, the baseline survey addresses both new indicators for which baseline information is required and indicators which previous evaluations were not able to adequately address. In particular, the pupil-level data was collected using an improved sampling frame and more rigorously defined intervention and control groups. Although the current baseline studies a newly defined cohort of pupils, and a new sample of teachers, for the sake of coherence and continuity, where possible, it follows up on many of the research questions used during phase 1. Alignment with future evaluations Conducting a fresh baseline study based on a new cohort of pupils will ensure that the mid-term and final evaluations of the FFK2 project are able to collect comparable information from the same students at three distinct time points; this is important since the cohort of learners studied during phase 1 have now moved on from primary school and are no longer benefiting from the school feeding or otherwise in contact with the project. In late 2018, a mid-term evaluation of the second phase will be conducted by an external evaluator or evaluators in order to measure progress against the baseline outcomes and formulate recommendations for the remainder of the project. The indicators and methods used in the mid-term evaluation will be based on those of the present baseline study, to allow the project to be evaluated against these same measures. In particular, the evaluation will follow the progress of the cohorts of intervention and counterfactual pupils studied at baseline. 17 A final external evaluation of the second phase will be conducted at the end point of the project in 2020, in order to assess the project’s relevance, effectiveness, efficiency, impact and sustainability. The final evaluation will use the same indicators, measures and methods, in order to assess whether the project objectives have been achieved, what has changed in the lives of the project beneficiaries and whether recommendations made following the mid-term evaluation have been integrated and implemented. Methodology The baseline survey used a mixed method approach to gather and analyse a combination of quantitative and qualitative information. The methods used built on those used during the evaluations of the FFK1 project. As stated above, these methods will form the basis of the approach to be used for the mid-term and final evaluations of the FFk2 project, thus ensuring the information collected is comparable, consistent and able to provide the basis for evaluating change over the entire lifespan of the project. The methods used to collect the information for the Baseline included:  A review of the findings and recommendations of the Final Evaluation report of the FY 2012-2016 FFK project;  Literacy testing of 6,291 learners (from project schools and control/counterfactual school) using a specially developed Early Grade Reading Assessment (EGRA);  Interviews with the same 6,291 learners, using an instrument based on that used using the FFK1 project evaluations, but extended to capture additional information;  Weighing and measuring of the same 6,291 learners to establish BMI z scores;  A survey of all 270 project schools;  A survey of a sample of 458 teachers from project schools;  Surveys of all 11 EPFs regarding infrastructure and equipment, profiles of teaching staff and staffing levels relative to teaching needs. This produced four primary data sets, which will be referred to as follows:  Pupil-level information (results of the EGRA, interview, weighing and measuring of pupils);  Teacher-level information (results of the survey completed by a sample of teachers from project schools);  School-level information (results of the survey of project schools completed by FFK2 “professionals” (project officers), in collaboration with school directors);2  EPF-level information (from surveys completed by all EPF directors). 2 “Professionals” is the term used by the project to refer to the project officers who are responsible for supporting the project implementation in individual schools. 18 Sampling for pupil-level data collection The stratified sampling frame of schools to collect pupil-level data for a quasi experimental cohort study was drawn up by experts in planning statistical analysis for education studies from the UK-based National Foundation for Education Research (NFER), using school-level information provided by the Provincial Directorate of Education regarding all the schools in the eight districts of Maputo Province. Since almost all schools in the four project districts (Manhiça, Magude, Moamba and Matutuine) are benefitting from the project interventions, the control group schools were selected from the four non-project districts (Namaacha, Matola, Marracuene and Boane). The selection of the sample was influenced by several further considerations:  Schools in the project districts (Magude, Manhica, Matutuine, Moamba) not receiving the intervention were excluded  Schools in the comparison districts (Boane, Marracuene, Matola and Namaacha) known to be receiving a different feeding programme were excluded  Xichangana and Xirhonga were combined into one “local language” group. This resulted in five categories for schools in the sampling frame (schools where categorisation was uncertain were excluded): o FFK + literacy (local language) o FFK + literacy (Portuguese) o FFK + literacy (local language and Portuguese) o FFK only o Comparison  A matched sample was then created between those receiving the literacy intervention (all of these schools were sampled) and the FFK only/comparison schools. The sample was matched on urban/rural, total number of pupils and pass rate.  The sample comparison table 1 below shows the composition of the five different groups in terms of urban/rural, total number of pupils and pass rate. The table demonstrates that it was not possible to achieve perfect matching, since the characteristics of the schools in the different groups were so vastly different in the first place. This is due to the fact that the sample was drawn based on the schools actually benefiting from the project, rather than based on their characteristics (as would be the case in a randomised control trial, where schools would be chosen by evaluators to take part in a trial specifically on the basis of their characteristics). Furthermore, since the project districts were selected to benefit from the project on the basis of their relative poverty and food insecurity, the comparison districts are likely to have lower levels of socioeconomic deprivation than the intervention districts. 19 sample selected sample 1 group 1 FFE + literacy (local language) 2 FFE + literacy (Portuguese) 3 FFE + literacy (both Portuguese and local language) Subtotal 4 FFE only 5 comparison group Count Column N % Count Column N % Count Column N % Count Column N % Count Column N % Count Column N % Urban/rural 0 rural 29 100.0% 39 95.1% 26 89.7% 94 94.9% 80 100.0% 76 95.0% 1 urban 0 0.0% 2 4.9% 3 10.3% 5 5.1% 0 0.0% 4 5.0% School size Percentile Group of total pupils 1 small 18 62.1% 2 4.9% 3 10.3% 23 23.2% 44 55.0% 15 18.8% 2 medium 8 27.6% 26 63.4% 8 27.6% 42 42.4% 33 41.3% 19 23.8% 3 large 3 10.3% 13 31.7% 18 62.1% 34 34.3% 3 3.8% 46 57.5% Pass Rate C Percentile Group of Pass Rate 1 low 3 10.3% 13 31.7% 12 41.4% 28 28.3% 20 25.0% 33 41.3% 2 medium 7 24.1% 16 39.0% 12 41.4% 35 35.4% 28 35.0% 28 35.0% 3 high 19 65.5% 12 29.3% 5 17.2% 36 36.4% 32 40.0% 19 23.8% Table 1: sample comparison table In each school, sixteen grade 1 pupils, four grade 2 and four grade 3 were randomly selected. Where schools benefit from literacy interventions in both Portuguese and the local language, 16 grade 1 pupils were selected from the class learning in Portuguese and 16 grade 1 pupils from the class learning in the local language. In total, 6291 pupils from 254 schools were tested. The number of schools and pupils in each pupil group by language of testing is given in Table 2 and the number of pupils by gender in Table 3. As an individual school could be involved in testing in more than one language and with more than one grade, a school can appear more than once in Table 2. 20 Intervention group by language of testing Language of Testing Portuguese Xichangana Xirhonga Total Pupils Schools Pupils Schools Pupils Schools Pupils Schools Pupil Group Grade 1: FFK +literacy 1066 66 690 47 141 11 1897 124 Grade 1: FFK only 1050 78 25 2 3 2 1078 82 Grade 1: Comparison 1064 69 41 4 70 6 1175 79 Grade 2 1087 248 0 0 0 0 1087 248 Grade 3 1054 244 0 0 0 0 1054 244 Total 5321 705 756 53 214 19 6291 777 Table 2: Intervention group by language of testing Intervention group by gender Gender Total Boy Girl Intervention Group Grade 1: FFK + literacy 954 943 1897 50.3% 49.7% 100.0% Grade 1: FFK only 605 473 1078 56.1% 43.9% 100.0% Grade 1: Comparison 572 603 1175 48.7% 51.3% 100.0% Grade 2 571 516 1087 52.5% 47.5% 100.0% Grade 3 541 513 1054 51.3% 48.7% 100.0% Total 3243 3048 6291 51.5% 48.5% 100.0% Table 3: Intervention group by gender Management and quality assurance of the Baseline Evaluation Terms of Reference for the baseline study were developed by PAI and ADPP with support from a consultant. The study was planned and conducted by the FFK2 project team, under the supervision of the FFK2 M&E manager and with extensive support from two external consultants, one international and one local. The international consultant, Simone Doctors, assisted the project team with the evaluation design and planning, the development of evaluation instruments, training of data gatherers, collation and analysis of data and drafting of the baseline report. She liaised with the statistical team responsible for sampling and for statistical 21 analysis of two of the data sets. This support to the M&E team was designed to be collaborative and participatory, so as to develop capacity amongst the project M&E team 3 . The sampling framework was drawn up by a team of statisticians and experts in educational studies from NFER, who also performed the statistical analysis of the pupil-level and teacher-level data collected (reported in the technical appendices in annexes 4 and 5). The four data collection instruments (for pupil-level, teacher-level, school-level and EPF-level data respectively) were developed in an iterative process between the managers of the relevant project components, the M&E team and the international consultant. All data were collected by project staff, following training from the international consultant, Simone Doctors, the national consultant, Rosa Tinga, members of the project management team, members of the project M&E team and members of the literacy team (for administration of the EGRA). The pupil- and teacher-level data was collected by teams of project “professionals” (see footnote 2, above), supported by supervisors: the process was overseen and coordinated by the project management and M&E teams, with assistance from the national consultant and in regular dialogue and coordination at distance with the international consultant. Quality assurance protocols were in place to ensure the consistency and internal validity of the data collection process. The national consultant spent the first days of data collection in the field with the data gatherers and thereafter paid regular visits to the data gathering teams. She was on call to deal with any issues which arose in the field and performed random spot checks on 10% of the completed instruments. The school surveys were completed in the respective schools by the project “professionals”, who had been trained by the M&E team in the administration of the survey, in collaboration with the school director or pedagogical director. The EPF surveys were completed by the directors of the respective EPFs. Spreadsheets with data validation options to reduce the possibility of errors in data entry were prepared by the international consultant for each data set. Data entry was performed by the projects data entry clerks, under the supervision of the M&E manager. Each line was entered by one clerk then checked by a second clerk or by the M&E manager. The completed spreadsheets were transmitted to the international consultant for checking and cleaning of the data. Where necessary, issues encountered were sent back to the M&E manager for verification against the paper instruments. The pupil- and teacher-level data were analysed by statisticians from NFER. The school-level and EPF-level data were analysed by the international consultant; these analyses were systematically verified by a data analyst. 3 Unfortunately, the high rate of turnover in the project M&E team has to some extent limited the impact of this ambition to build capacity. 22 The report was drafted by the international consultant then sent to Planet Aid, ADPP and the different implementing partners for factual verification. A small number of minor changes were suggested and incorporated into the final draft. Challenges, delays and lessons learned The collection and subsequent analysis of data for the FFK2 baseline study was delayed on several occasions, due to a number of factors:  Three of the four data sets (pupil-level, school-level and teacher-level information) were collected by the FFK “professionals”, in parallel to their other responsibilities.  The instruments, in particular the EGRA, the teacher survey and the school survey took longer than expected to develop, due to the number of individuals and organisations involved in their elaboration (Cambridge Education, INDE, FFK literacy team, FFK M&E team and the consultants) working at distance through email. This was exacerbated by the limited availability of the M&E team at times, due to other responsibilities, and to the departure of a number of key M&E individuals during the process.  The sampling plan developed by NFER was delayed by challenges in getting the necessary, complete information from the DPEDH. Consequently, the pupil-level data collection started later than intended. It then took longer than planned to complete the data collection, for a variety of reasons, including the fact that the FFK “professionals” had other obligations during this time so were not fully available for this task.  Turnover and capacity issues within the M&E team led to challenges and limited availability of M&E staff. The lack of a M&E manager able to champion and coordinate the processes (ultimately attributable to labour market issues within Mozambique, meaning it has been challenging to recruit and retain a sufficiently knowledgeable and competent M&E manager) led at times to a lack of technical leadership on the ground. The international consultant oversaw the process at distance, assisted by the national consultant, the M&E team and, increasingly towards the end of the process, the project leadership team on the ground in Mozambique, to transmit and implement her technical instructions and requests.  The FFK “professionals” found the extent and demands of data collection quite challenging, due both to the exigencies of the process and to their need to reconcile this activity with their other responsibilities. A number of quality issues and inconsistencies were identified in the learner data collected by the “professionals” and in the data entered. Given the large volume of learner data collected, capacity issues within the M&E department and the fact that the international consultant was working with them at distance, these took some time to rectify.  Because of the initial delays, both the NFER team and the international consultant had to honour other existing professional commitments, reconciling their work on the FFK2 baseline with other assignments, thereby compounding the delays. 23 The decision to do the baseline “in house” using project staff had both advantages and drawbacks. Collecting the data and the other tasks associated with the baseline study has been a useful learning process for the “professionals” and other project staff, who are better informed about the project and the schools they work in as a consequence, in addition to having developed their research skills and practice of data collection practices. However, the volume of data which the project had decided to collect meant that the process took them away from their other responsibilities for longer than they had hoped. The limited capacity of the M&E team (both in terms of numbers and competence) led to the project management team having to step into the breach and perform many of the coordinating and quality control functions, adding considerably to their own workload. In summary, the process has been a useful one in terms of staff learning and engagement but this came at a certain cost, in terms of disruption to other project activities. Baseline findings 1. A review of the findings of the Final Evaluation report of the FFK1 project Since the second phase of the project follows on directly from the first, many of the findings of the final evaluation of phase one may usefully be considered to constitute a baseline for phase two. Indeed, in order to maximise efficient use of resources and the work already conducted, much of the qualitative and quantitative information collected during the final evaluation of FFK1 has not been repeated for the purposes of the FFK2 baseline (new information has been collected only where the new framework or indicators meant this was necessary). The findings of the third section of part 3 of the final evaluation report of FFK1, entitled “Strategic relevance regarding effectiveness, efficiency, impact and sustainability: ‘Were these the right things to do? What can be learnt from this experience?’” are briefly reviewed here and provide a useful background to the new information presented in later sections. For more detailed information, or the complete text, including recommendations, the original final evaluation report of phase 1 may be consulted (pp 63-85). Human Resources, capacity, collaboration and ownership The Final evaluation found continued commitment by project staff, volunteers and partners and significant improvements in individual and organisational capacity, with more skilled and more professional project staff, improved management and leadership systems and practices and a stronger sense of ownership of the project. Collaboration with the SDEJT officers, community members, project volunteers is effective; the MINEDH considers the project as an integral part of the national school feeding programme, PRONAE. Despite a concerted attempt to provide the “professionals” (project officers) with computers and transport, some outstanding needs for computer and communications equipment remained and not all “professionals” had managed to obtain a driving licence, the prerequisite to being able to use project vehicles and 24 motorcycles. A need for improved data management and security systems and practices was also identified. The FFK1 final evaluation report recommended the project should acquire a dedicated database to allow integration of data currently stored in discrete spreadsheets. The evaluation also recommended recruiting additional specialist Communications staff. The final evaluation of FFK1 revealed the need for additional Monitoring and Evaluation (M&E) capacity, and this was confirmed by the process of conducting the baseline of FFK2, where issues of both quality and quantity of M&E staff and the high turnover of those staff have continued to cause difficulties for the programme. The question of incentives for volunteers remained a delicate one at the end of FFK1. The project was taking the issue seriously and trialling a number of strategic responses to the question, including offering non￾monetary incentives, such as soap, to volunteers. The final evaluation report of FFK1 recommended placing volunteering on a more formal basis, with a contract between volunteer and project laying out clearly defined responsibilities and privileges on both sides. It also suggested that solutions ultimately lay in a deeper sense of ownership by the local communities. Although some teachers were still requesting incentives in return for their participation in the project, the delicate question of their needing to fill in two class registers daily appeared to have been resolved. The project, with the agreement of the DPEDH, has designed and introduced a single class register which doubles as the record of pupil presences needed by the project. Communication and coordination between the project and district, regional and national government was found to be excellent. Transport The final evaluation of FFK1 revealed that many of the earlier challenges around transport of goods (CSB, building and water and sanitation supplies) and persons (in particular allowing the “professionals” to move between the project offices and the schools where they work) were in the process of resolution, thanks to the acquisition of vehicles and motorbikes and to a scheme to support the “professionals” to obtain driving licences. The ongoing availability and condition of these vehicles and the ability of all the “professionals” to obtain their driving licences should continue to be monitored. CSB and Logistics Changes made to the logistics team, the use of a commodity tracking system and improved systems and communications within the project team had produced significant improvements at the time of the final evaluation of FFK1. However, schools continued to run out of CSB from time to time and the number of “non feeding days” was still above target; a tracking system was also needed to manage the distribution of non-food items, such as kitchen utensils and school kits. 25 Nutrition training The final evaluation of FFK2 found much better collaboration and information sharing between the WISHH team and the rest of the project team than at mid-term. The WISHH team members had also greatly improved their capacity and confidence as well as their record keeping and reporting. The WISHH and ADPP monitoring systems, record keeping and reporting were more aligned than previously: this harmonised approach must continue throughout the second phase of the project. WISHH has had to adapt the level of its materials and training, reviewing these for readability to make them accessible to Mozambican learners. The team has also learned how to develop materials jointly with INDE, rather than asking for retroactive approval of materials. Although during the second phase of the project the focus of the WISHH intervention shifts from the EPFs to the project schools, the investment already made in the EPFs must be sustained, so nutrition education and awareness continue to be an integrated part of the EPF approach, functioning and training. A further challenge for the second phase of the project is to continue to train all volunteer food preparers to an acceptable standard in nutrition and dietary practices, including good kitchen hygiene practices. Finally, the WISHH programme ends in 2019, earlier than the other components. An exit strategy, with a focus on sustainability needs to be developed and implemented. Responsibility for sustaining the work and achievements of the nutrition component needs to be progressively handed over to project staff, schools and communities while there is still time for the WISHH officers to adequately support this process. Water and sanitation Having access to safe water is critical to school feeding, as well as many other aspects of community life, and the project’s success in bringing water to some communities transformed life in both schools and wider communities during phase 1. However, the final evaluation of FFK1 found major barriers to the project aim of bringing safe water supplies to all schools, given the depth of the water table, the saline quality of much of the subterranean water and the distance of many communities from the existing municipal piped water network. Ensuring all schools have access to a safe water supply must be a major focus of phase 2. There were challenges in recruiting and maintaining suitable water technicians and in continuously training and supporting the local water committees. The final evaluation of phase 1 of the project recommended introducing a decentralised system and fund for small water system repairs and maintenance, administered at district level. A further recommendation concerned improving reporting systems, including clearer criteria for considering water installations as completed and therefore “countable” and more focus outcomes (functioning water systems) than activities. 26 Literacy, school clubs and kits The first phase of the project lacked substantive input to improve the capacity of teachers, so did not contribute significantly to improving learners’ literacy outcomes. However the project’s support to school clubs and provision of kits of educational materials appeared to make school more pleasant for pupils and to motivate and encourage teachers. Within the context of FFK2’s explicit focus on supporting teachers in teaching literacy, the role of school clubs must be clarified, so clubs do not simply repeat what happens in lessons. The project needs to rethink its support to extra-curricular clubs, to allow them to make more of an impact on learning. Club leaders should receive clear guidance and training in using play-based and other alternative activities to promote learning and the value and potential of clubs should be communicated to government, school leaders and teachers. School gardens At the time of the final evaluation, school gardens were seasonal, functioning during term times, where water was available, and not maintained during school holidays. They were considered primarily as a pedagogical activity to teach pupils about agriculture, natural sciences, nutrition and livelihoods. They were maintained by pupils and teachers and, in some cases, parents and the wider community. They were not making a significant contribution to school feeding, although some produce might be consumed on festival days. There was no satisfactory system to monitor the activity and production of the gardens. Many gardens were not fenced, so were vulnerable to animals eating the produce. Many school gardens were not in production, even during term time, due to drought and the absence of a water source. School production is closely linked with pupils learning about nutrition and is a key to the sustainability of the project, in addition to being a pillar of the government’s wider school feeding project. The final evaluation of FFK1 highlighted the importance of the new “mega gardens” being created as part of phase 2 contributing to sustainability. It also highlighted the desirability of pupils learning to produce and consume “real” food, rather than soya porridge which, although nutritious, is not part of the traditional Mozambican menu and lacks the diversity essential to learning good dietary habits. It is important to differentiate between school gardens as a pedagogical activity for pupils to learn about nutrition and agriculture and the need to cultivate produce on a large scale to contribute significantly to school feeding. While their participation in the former is valuable, pupils should not contribute to the latter during school hours, to the detriment of their other academic work and should not be expected to undertake non-age appropriate heavy labour. If school gardens are to contribute to sustainable school feeding, they need to be scaled up and more integrated in the school feeding processes and system. They will need to be maintained throughout the year by adults, whether on a paid or voluntary basis, monitored and protected. Water, seeds, inputs and equipment are all needed. 27 Teacher training The final evaluation of FFK1 found that the eleven ADPP-run teacher training colleges (EPFs) are making a valuable contribution to Mozambique’s teacher pool since, after graduating, EPF graduates teach in Mozambican schools. The EPFs produce teachers able to work and teach in remote rural primary schools, with minimal support, to be creative in the classroom and to use local materials to produce teaching and learning materials, as well as providing them with practical skills such as community development and mobilisation, building, small scale agriculture and husbandry. It is not known what they do in the longer term: systematic monitoring is needed to shed more light on the long-term professional orientation of EPF graduates. Since candidates for teacher training are recruited after secondary school grade 10, and given the low quality of teaching in many schools, many trainee teachers arrive at the EPF with very poor levels of written Portuguese and poor handwriting; all the EPFs have put in place a system of remedial Portuguese lessons to address these needs. In order to maintain the infrastructure and equipment of the 11 EPFs, thanks to USDA funding, a coordinated plan identifies priorities and systematically improves and carries out maintenance of the EPF buildings and acquires equipment and furniture for classrooms and dormitories. Likewise, USDA funding has allowed much￾needed vehicles to be repaired or acquired: at the final evaluation of FFK1, all but two of the EPFs had at least one car. Access to books is also a challenge. Since materials have not yet been developed for the one-year teacher training programme, primary school text books and curricular programmes are used to train teachers. All EPFs also have a computer network to provide students with access to study tasks and a digital library, although these were not functioning in all EPFs at the end of phase 1 of the project. Shortages of teacher trainers in certain subjects, including natural sciences, English and physical education reflect the labour market in Mozambique. MINEDH criteria to qualify for the grade of “instructor”, in order to work as a teacher trainer in Mozambique, include a Licenciatura qualification, as well as 5 years’ primary teaching experience. Budget and costs The final evaluation of FFK1 examined the functioning of financial procedures and accountability and investigated whether the project was seeking to provide value for money. It found that, given its large scale, and the rural areas in which it functions, the project’s financial systems and procedures were of good quality, sound financial systems are in place and project funds are used responsibly, with an appropriate focus on obtaining value for money. 28 Challenges to financial reporting remain: due to poor infrastructure, the availability of supporting documentation is limited, as proper receipts are not always available for goods and services. The value of the Mozambican metical and recent fluctuations in exchange rates have brought other challenges. Pupil attendance The information in this sub-section is taken from part 2 of the final evaluation of FFK, pp 50-51 (the other sub￾sections present information from part 3). Throughout phase 1 of the project, there were challenges in obtaining systematic, reliable pupil-level attendance data from schools, meaning it was impossible to verify objectively the frequent claims by education official, school leaders and other stakeholders that, since the onset of school feeding, attendance rates had improved. The attendance recording process required teachers to complete a separate daily attendance list of pupils receiving school feeding, in addition to the school register. Many teachers lacked rigour in completing these attendance lists, rendering the resulting data extremely unreliable: often the same numbers of pupils were indicated as present for the whole month, strongly suggesting that the teacher has completed the task from memory at the end of the month. There were also delays in entering and collating this paper-based data electronically, due to high turnover and absences within the M&E team. The project tracking system uses school level enrolment data provided at the beginning of the school year by the district education offices (SDEJT), which does not always reflect mid-trimester changes in pupil numbers. Because, during phase one, the daily attendance data with which the project was functioning was based on annual enrolment figures, it was not possible to use this to systematically verify changes in attendance rates or, indeed, health-related absences. As indicted above, a solution to this problem appears to have been found, with the introduction of a single class register which doubles as the record of pupil presences needed by the project. It will be important to monitor the use of this instrument closely. It is to be hoped that it will allow the project indicator of “improved school attendance” to be measured accurately for the second phase of the project. Sustainability and relevance to the local and national school feeding policy and programme environment The final evaluation of FFK1 found the project to be well integrated with the national PRONAE and recognised as an integral part of the national school feeding programme, with functional, cooperative relationships and collaboration with the MINEDH at all levels (district, provincial and national). At the time of final evaluation of FFK1, the future of school feeding in Mozambique appeared uncertain. The national school feeding programme, PRONAE, supported by both the WFP and FAO, is based on local sourcing and aims to meet two complementary needs: regularly procuring food for school feeding from local producers, thereby boosting and supporting Mozambican agriculture. The PRONAE pilot, which was being piloted in 12 29 schools in Gaza, Manhica, Tete and Nampula since 2013 and had been expanded to a further 12 schools, was to be evaluated in 2017. The pilot encountered some major challenges: small scale producers struggle to consistently produce food of the necessary quality and quantity, transporting food is problematic and procurement process overly bureaucratic. In the pilot PRONAE schools, teachers lack assistance and support to create and maintain school gardens and foster community participation, in contrast to the support, equipment, seeds and inputs provided by ADPP. Wider challenges come from the current context of Mozambique, which is experiencing financial, political and environmental crisis. In this context, school feeding is very much still needed, so the project’s sustainability is a priority issue. The project as it currently operates is far from ensuring a sustainable future independent of donor input. Given the unlikelihood of school gardens (even the “mega gardens” being introduced in phase 2) consistently producing sufficient of the staple basic foodstuffs (soya, sweet potatoes, manioc or another “basic” food, or indeed a combination of these) needed as the basis for school feeding, these will need to be procured after 2020, once CSB, the current staple, is no longer being provided. The final evaluation of FFK1 strongly recommended the different stakeholders (ADPP, PAI, USDA, GoM, including the Ministry of Agriculture, other international partners) should engage with this question as a matter of urgency, so that the remaining years of the project are used to plan the future after the contract with USDA comes to an end. The evaluators were concerned at the seeming lack of a plan for post-2020, with many stakeholders apparently assuming that funding for the project would continue, because it would be needed (and possibly because Mozambique has had some sort of donor funded school feeding programme since 1977), an assumption which is both dangerous in the current global context and incompatible with aspirations for Mozambican autonomy. The question of sustainability is therefore the major issue for the second phase of the programme and should be a preoccupation for all stakeholders. Having reviewed the main findings and messages of the final evaluation of FFK1, the information from analysis of the four new data sets collected for this baseline study is now presented. These are structured by data set, although where there is thematic overlap between the data sets, this is addressed. The information is structured following the order used in the data gathering instruments, for ease of reference. The instruments can be found in annexes 6, 7, 8 and 9. Since this is a baseline study the information, including relevant contextual information about the instruments and data gathering process, is presented and discussed in the same section, since discussion is limited to what is strictly necessary at this stage. 2. Pupil-level data A cohort of grade 1 pupils took an Early Grade Reading Assessment (EGRA) in order to establish baseline measures against which their subsequent progress could be measured at mid-term and end point. In order to 30 gather information to assist the literacy specialists engaged in developing the curriculum and materials for the ongoing literacy intervention, a small number of grade 2 and Grade 3 pupils were also tested. These grade 1, 2 and 3 pupils constitute the sample of 6,291 learners, composed of five pupil groups, presented in detail on pages 19-21 above, the characteristics of which are summarised in table 4 below. The same pupils, underwent a short interview and were weighed and measured. Grade 1 Grade 2 Grade 3 FFK + literacy FFK only comparison Portuguese Xichangana Xirhonga Table 4: characteristics of the five groups of pupils of the sample tested N.B. All grade 2 and all grade 3 pupils were analysed as two unified groups, not divided into intervention and control, as the objective was to obtain information about current levels to inform materials development. All grade 2 and grade 3 pupils were tested in Portuguese, rather than local languages. The grade 2 group comprised 73% pupils from project schools (FFK + literacy and FFK only combined) and 27% from comparison schools. The grade 2 group comprised 72% pupils from project schools (FFK + literacy and FFK only combined) and 28% from comparison schools. The findings of the literary assessment are presented first, followed by the other pupil level information. Literacy assessment (EGRA): presentation and results The EGRA was developed by a team of specialists from Cambridge Education, language specialists from INDE, the FFK literacy team and the FFK M&E team, working closely with the international consultant. Three equivalent versions were developed in Portuguese, Xichangana and Xirhonga. The EGRA consists of 11 sub tasks, designed to reflect the progressive stages of literacy acquisition, so that a pupil continues until they have reached the threshold at which they can no longer perform the tasks. It is therefore able to be used to evaluate pupils within and across several grades. All pupils took each of the tests but most Grade 1 pupils were not expected to answer anything beyond subtasks 4 and 5. The 11 sub tasks were analysed as 9 subscales (reading fluency and reading comprehension are considered as one sub scale; writing tasks were considered as one sub scale. Each of the three language versions of EGRA consisted of the following sub tasks (all of which are language specific, reflecting the phonological and orthographic rules of the target language): 1. Oral vocabulary (name the object depicted in stimuli pictures – 10 items) 2. Comprehension of an oral text (respond orally to comprehension questions after hearing a short text – 4 items) 3. Phonological awareness (indicate the picture of an object whose name begins with the same initial sound as that of a stimulus object – 10 items) 31 4. Concepts of print (perform tasks to demonstrate familiarity with how printed language functions – 10 items) 5. Letter sounds (produce the letter sounds when shown printed lower and upper case letters – 100 items) 6. Syllable recognition (read aloud syllables consisting of consonant and vowel combinations permitted in the target language – 50 items) 7. Reading simple words (read aloud a list of words progressively longer and less common - 30 items) 8. Reading fluency (read a short passage – number of words read correctly within 60 seconds –70 items) and Reading comprehension (respond to questions based on the passage just read – 4 items) 9. Writing (correctly write first name and family name – 2 items) and dictation (correctly write words dictated – 10 items). Filters were in place so that learners who had reached their level of competence within the hierarchy of sub tasks (which were ordered to reflect the progressive, cumulative stages of pre-reading then reading skills) did not proceed with tasks which were clearly beyond their level of competence. However, all pupils attempted the writing task. The results for each sub task are briefly presented below (for more details, please see Technical Appendix 1, prepared by the statistical team from NFER, page 79). Oral vocabulary All pupils performed this pre-reading task well. The group with the lowest score was the Grade 1 FFK only group tested in Portuguese, with a mean score of 8.19 out of 10. Grade 1 pupils tested in Xichangana and Xirhonga performed slightly better than Grade 1 pupils tested in Portuguese. For each group, mean scores of pupils tested in local languages are consistently slightly higher than the mean scores of those tested in Portuguese. 81% of Grade 1 pupils tested in Xichangana and 83% of pupils tested in Xirhonga scored 10 out of 10, compared with 61% of pupils tested in Portuguese. It is not appropriate to run significance tests on these scores as they refer to different tests. However, given the fact that the tests were carefully constructed to be comparable, it seems likely that, at this early age, the pupils learning in local languages have a better mastery of oral vocabulary than those learning in Portuguese. The complete results are presented in Technical Appendix 1). Oral comprehension The mean score for Grade 1 pupils (across all languages) was between 0.91/4 for the comparison group and 1.29/4 for the FFK + literacy group, suggesting this was not a task grade 1 pupils were able to perform comfortably overall, with many scoring 0. Grade 2 performed better (mean 1.88/4) and grade 3 even better (mean 2.81/4). Given that the task was to answer four simple questions about a short text read by the enumerator, these results suggest that, overall, early grade pupils are struggling to process oral language 32 effectively. This is particularly true for grade 1 pupils, approximately half of whom could not answer a single question correctly, regardless of the language in which they were tested. Phonological awareness The mean score for grade one pupils (across all languages together) was between 2.27/10 for the comparison group and 3.24/10 for the FEE only group, with a large number of Grade 1 pupils scoring 0/10 on this task. Grade 2 performed better (4.76/10) and grade 3 even better (6.47/10). These results suggest that, although, overall, pupils going through the school system do master some level of phonological awareness by grade 3, this pre-reading skill is relatively weak throughout the early grades. Grade one pupils struggled particularly with this task: approximately half could not answer a single question correctly, regardless of the language in which they were tested. Concepts of print This sub task, in which pupils are asked to perform a number of activities as evidence of their familiarity with printed materials, produced better results overall: the mean score for grade one pupils (across all languages together) was between 3.65/10 for the comparison group and 3.83 for the FFK + literacy group. Grade 2 pupils performed better (5.42/10 mean) and grade 3 pupils even better (7.35/10). Letter sound recognition Grade one pupils were not able to perform this task, while grade 2 pupils scored on average 5 points, and grade 3 pupils 18 points out of a possible 100 points. Experience of the Mozambican education system demonstrates that, when shown a written letter, pupils are not generally able to produce the corresponding letter sound but respond to the stimulus of a written letter by naming that letter. Teachers and teacher trainers also have difficulty performing the same task: teaching reading does not usually involve phonics or letter sound recognition. This observation was confirmed during the training of enumerators in this task: several additional training sessions were necessary in order for enumerators to be able to administer and grade the task comfortably. It is possible that the enumerators’ lack of familiarity with the task may have impacted on the pupils’ performance. However, this notwithstanding, it is clear that the task was one the vast majority of pupils tested were unable to perform. Although this was a timed exercise, the timing was not taken into account in the analysis due to inconsistencies in the data. Reading syllables The great majority of grade 1 pupils were not able to perform this task. Grade 2 pupils scored on average 3.35 and grade 3 pupils 13.57 out of a possible 50 points (7% and 26% respectively). Once again, although this was a timed exercise, the timing was not taken into account in the analysis due to inconsistencies in the data. 33 Reading words Once again, Grade 1 pupils were not generally able to perform this task. Grade 2 pupils scored on average 2.51 and grade 3 pupils 9.37 out of a possible 30 points (8% and 31% respectively). Once again, although this was a timed exercise, the timing was not taken into account in the analysis due to inconsistencies in the data. Reading fluency and text comprehension Once again, the great majority of grade 1 pupils were not able to perform this task. Grade 2 pupils managed to read on average 4.65 words and grade 3 pupils 17.15 words out of a possible 70 words (7% and 25% respectively). The distribution of scores was uneven: a large number of pupils did not read any words correctly (partly because this task was not attempted by those who had not managed to perform previous sub tasks, which acted as a filter) whereas 58 pupils successfully read all 70 words and there was an odd peak with 162 pupils scoring 52/70. In all, three pupils from grade 1, 67 pupils from grade 2 (6%) and 282 pupils from grade 3 (27%), successfully read half or more of the words in the text (between 36 and 70 words) within 60 seconds. Although these pupils may be considered able to read, reading half the short text within 60 seconds barely constitutes reading fluently, so in fact the number of genuinely fluent readers is even lower. In terms of comprehension (pupils were only asked comprehension questions corresponding to the point they had reached in reading the text) 88% of Grade 2 pupils scored 0/4; with 5% scoring either 1 or 2 points and 8% scoring either 3 or 4 out of the possible 4. 60% of Grade 3 pupils scored 0/4, with 13% scoring either 1 or 2 point and 27% scoring either 3 or 4 out of the possible 4. These results for grades 2 and 3 correlate with those for fluent reading: the very small number of grade 2 pupils and larger (but still low) number of grade 3 readers who are able to read with some fluency are able to understand what they are reading. In order to investigate the relationship between reading fluency and comprehension, further analysis of these sub tasks was undertaken (see technical appendix 1). 88% of pupils scored 0 on the fluency and comprehension tests: only 58 pupils scored full marks for fluency and only 9% scored more than 0 for comprehension. The mean fluency score increases with the number of correct comprehension questions; however this is to be expected as the more text the pupils was able to read, the more questions they were asked. Although pupils were only supposed to be asked comprehension questions referring to those sections of the text they had read, there were many inconsistencies in how this rule was applied. Given the low number of pupils who scored above 0 on these tests and the difficulties in administering them, it is not possible to identify a systematic relationship between fluency and comprehension. Writing first name correctly Overall, the percentage of grade one pupils (across all languages) able to write their first name correctly was between 6% for the comparison group and 13% for the FFK only group. 52% of grade 2 and 80% of grade 3 pupils were able to write their first name correctly. 34 Writing family name correctly Overall, the percentage of grade one pupils (across all languages) able to write their family name correctly was between 1% for the comparison group and 3% for the FEE only group. 22% of grade 2 and 55% of grade 3 pupils were able to write their family name correctly. Writing dictation Most grade 1 pupils were not able to perform this task, regardless of the language of testing. Grade 2 pupils managed to write on average 0.74 and grade 3 pupils 2.98 out of a total of 10 words correctly. These low mean scores hide a somewhat uneven distribution: despite the large number of pupils of all grades who scored 0/10 on this task, 7 grade 1 pupils, 24 (2%) Grade two pupils and 138 (13%) of grade 3 pupils scored 10/10 on this task. Overall, the EGRA results reveal very low literacy outcomes for the vast majority of the early grade learners tested. Practically no grade 1 learners were able to proceed beyond the first four sub tasks, although this was to be expected, given that, at the time of testing, they had only been attending school for three to four months. 85% of grade 2 pupils were unable to read simple words, whether in a list of words or within a short text, and only 5% of grade 2 pupils were able to read two thirds of the list of words correctly, with 6% managing to read more than half of the text. Grade 3 pupils demonstrated more variation: although around 50% were unable to read any of the simple words in a list or within a text, 24% were able to read two thirds of the list of words correctly and 27% to read more than half of the text. Given that the EGRA items were developed by specialists in early grade reading in Mozambique and based on the early grades literacy curriculum, this suggests that the majority of early grades pupils are failing to learn to read at all, whereas those who are learning are performing far below the expected level. Other pupil-level results The pupils surveyed were the same as those who performed the EGRA. As explained above, the sampling design reflects the way the FFK and literacy programmes are being rolled out. The following outcomes are reported for each group (Grade 1: FEE + Literacy; Grade 1: FFK only; Grade 1: Comparison; Grade 2; Grade 3) but combined across languages:  Height  Weight  BMI z-scores  Attitude to learning  Hunger  Attentiveness  Health and hygiene practices 35 Height, weight and BMI z-scores In order to establish a baseline to assess whether school feeding impacts on pupil’s physical development, pupils’ height and weight were recorded, so that BMI z-scores could be calculated. Mean Grade height and weight were calculated first within each school and then the overall means calculated for each group. Mean height for Grade 1 pupils (across all groups) was 1.15m; Grade 2: 1.21m; Grade 3: 1.26m. Mean weight for grade 1 pupils was between 19.68 kg - 19.90kg; Grade 2: 22.41Kg; Grade 3 25.32kg. More detailed information is presented in Technical Appendix 2 (page 101). BMI Z-scores were calculated following WHO guidelines. Mean and median BMI-Z scores are presented in the Technical Appendix 2. Calculating the scores proved problematic in a large number of cases, where accurate dates of birth had not been recorded: in a large percentage of these cases the date of birth was given as 1 January4 . Attitude to learning and clubs Pupils were asked a series of questions designed to probe their attitudes to learning and extra-curricular clubs (the results are presented in more detail in Technical Appendix 2). Membership of learning clubs is relatively low: amongst grade 1 pupils, 26% of FFK + literacy pupils, 24% of comparison group pupils and 18% of FFK only pupils said they belonged to at least one learning club (Grade 2: 26%; Grade 3: 31%). Reported participation (ranging from occasional to three times a week) in the clubs is also low: 14% of FFK + literacy group; 11% of FFK only group; 40% of comparison group; 20% of Grade 2; 22% of Grade 3. It is striking that, although they do not report higher membership than FFK pupils, comparison group pupils report significantly higher participation in clubs: this may be due to the fact that schools in the comparison districts are more active in promoting participation in school clubs. Of the minority of pupils who reported membership or attendance of clubs, approximately 50% of all pupils claimed to enjoy them “a lot”, with the exception of the Grade 1 comparison group pupils, 23% of whom reported enjoying club attendance “a lot” (see figure 1 below). These responses contrast significantly with those given by teachers from project schools and in the school survey. 92% of teachers reported that clubs had been created in their schools, with 65% reporting that “all” or “most” pupils in their classes participate regularly (see discussion on pages 56-57 below). In the school survey, 98% of schools report having one or more extracurricular clubs (see pages 53-55 below). 4 There are several possible explanations for this: early grades pupils may not know their date of birth; school records may in incomplete or inaccurate; the enumerators may lack the experience and expertise to be able to elicit this information. When the same cohort of pupils is studied at mid-term, this will provide an opportunity for accurate dates of birth to be collected (enumerators should be briefed on the necessity of collecting this information accurately and given particular training in how to do so) and for the missing baseline BMI-Z scores to be calculated; the mid-term scores can then be compared against these. 36 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Enjoyment of school clubs Don't enjoy Enjoy "a little" Enjoy"a lot" Figure 1: reported enjoyment of clubs by those pupils who report participating Of the minority of pupils who reported membership or attendance of clubs, the perceived benefit in terms of club attendance helping pupils with their school work is rather limited: although the responses vary between different groups, considerably more pupils reported no benefit from clubs than reported that clubs helped “a lot” or “a little” with their school work, with FFK only pupils significantly more likely to report that clubs do not help than the other groups (see figure 2 below). Again these findings contrast with the teachers’ reporting of the situation: 88% of teachers reported that the clubs had a positive effect on pupils’ school work (see page 56 below). 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% School clubs help with school work Don't help Help "a little" Help"a lot" Figure 2: “Clubs help with school work” responses from those pupils who report participating Likewise, amongst the pupils who reported membership or attendance of clubs, considerably more report no improvement in their school work since they began attending the clubs, than report either “a lot” or “a little” 37 improvement in their school work since that time, with FFK only pupils significantly more likely to report no improvement than the other groups (see figure 3 below). 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% School work has improved Has not improved Improved "a little" Improved"a lot" Figure 3: “School work has improved since attending clubs” responses from pupils who report participating These findings suggest there is a need to improve the functioning of school learning clubs if they are to play a useful role in supporting learning. The contrast between the pupils’ reporting about clubs and the reporting in both the teachers’ and the schools’ surveys reveal a very different perception of clubs by pupils than that of teachers and head teachers. Even allowing for the fact that, during a project evaluation, teachers and head teachers are likely to give a more positive account of something they see as an activity supported by the project, the discrepancy between this and the pupils’ accounts suggests that some of their optimism is currently misplaced. Hunger and Attentiveness In order to measure the impact of the intervention on short-term hunger and attention problems in the classroom, postulated by the theory of change to constitute important steps in the project results chain, pupils were asked about their experience of hunger and attention problems whilst at school. To allow some degree of triangulation, pupils were asked both whether they sometimes felt hungry at school and whether they felt hungry at the time of interview. Unfortunately, a considerable proportion of the comparison group pupils (57%) did not answer this question, for reasons which are unclear. Across the different groups, between 52% - 69% of all pupils who responded reported sometimes feeling hungry at school, whereas between 38% - 66% of all pupils who responded reported feeling hungry at the time of interview (see figures 4 and 5 below). 38 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% "Sometimes hungry at school" No response No Yes 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% "Hungry now" No response No Yes Figure 4: Pupils reporting sometimes being hungry at school Figure 5: Pupils reporting being hungry at time of interview In response to questions designed to verify that school feeding is reaching its target beneficiaries, 93% pupils from project schools affirmed they always received food at school, and 94-95% that all pupils received the food (see figures 6 and 7 below). However, when asked whether they had received food that day or the previous day, the responses were far lower, with 63% of Grade 1 FFK only pupils and 75% of FFK + literacy pupils claiming to have received food the day of the interview, and 75% of both groups claiming to have received food the previous day. Even though some learners may have been interviewed before the distribution of food the same day, according to these figures, 25% of learners did not receive food the previous day, which is far from the project’s target of all learners in project schools receiving food each school day.5 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% "Always receives food at school" N/A No response No Yes 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% "All pupils receive food" N/A No response No Yes Figure 6: Pupils reporting they always receive food at school Figure 7: Pupils reporting that all pupils receive the food 5 At the end of phase 1, the project was reaching 82% of its monthly distribution targets, according to the final evaluation report and to the project’s own figures. 39 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% "Received food today at school" N/A No response No Yes 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% "Received food yesterday at school" N/A No response No Yes Figure 8: Pupils reporting receiving food the day of the interview Figure 9: Pupils reporting receiving food the day prior to the interview When asked whether the food the received tasted good, almost 90% of pupils in project schools replied affirmatively (see figure 10 below). 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% "Food tastes good" N/A No response No Yes Figure 10: Pupils reporting the food they receive tastes good When asked whether they experienced attention problems during classes, responses between the different groups demonstrated a similar pattern, with 5% to 7% claiming they often experienced concentration problems, 23%-31% claiming they sometimes experienced attention problems sometimes and 46%-56% claiming they did not encounter attention problems in the classroom. Due to the relatively high level of “no responses” (12%-24%), the tables below show pupils’ answers to this question both with and without inclusion of the “no responses”6 . Teachers were asked about their experience of pupils experiencing attention problems in the classroom, and reported a very different picture. 86% of teachers reported that pupils 6 The issue of “no responses” is problematic throughout much of the data. In future evaluations, enumerators should be trained to use this option only rarely and where unavoidable. 40 “sometimes” had attention problems in class (2% “often”; 11% “never”) and 81% that pupils “sometimes” had difficult learning (1% often; 16% never) (see page 58 below). 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% "Attention problems at school" no response no sometimes often 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% "Attentiveness problems" (without non responses) no sometimes often Figure 11: Pupils reporting attention problems during classes Figure 12: Pupils reporting attention problems (without non responses) In the case of reporting of pupils’ attention problems, it is perhaps not surprising that teachers would have a different experience of these and report these more frequently than pupils themselves (teachers and former teachers reading this report are likely to feel sympathy for the Mozambican teachers in this regard!). Indeed, this discrepancy was observed in all the evaluations of phase 1. Health and hygiene practices Pupils were asked a series of questions aimed at establishing a baseline in order to evaluate the impact of project interventions during the second phase on their health and hygiene practices. The questions asked were based on those used at the baseline, mid-term and end point of phase one, to allow comparability throughout the two phases; however, in phase two, the responses are from a different cohort of pupils. It should be borne in mind that respondents in project schools (in particular grade 2 and 3 pupils, but also those in grade 1) are likely to already be demonstrating the impact of phase one project intervention (so this baseline is of the situation following phase one and prior to phase two but does not capture a pre-intervention “tabula rasa”). Pupils were asked what they did after using the toilet (enumerators were trained not to mention hand washing in their questioning, so as not to prompt this response). The grade 1 pupils of the two groups benefiting from the intervention mentioned hand washing in their responses significantly more frequently (75% and 80% respectively) than grade one pupils from the comparison group (63%) (see figure 13 below). 81% of grade 2 pupils and 88% of grade 3 pupils mentioned hand washing, which is consistent both with the fact that almost three quarters of the pupils in these groups are from intervention schools, with just over one quarter from comparison schools, and that the pupils are older and have had more time to assimilate health and hygiene messages and, hopefully, practices. 41 Pupils were asked what they did before eating (likewise, enumerators were trained not to mention hand washing in their questioning, so as not to prompt this response). The responses show a very similar pattern to those of the previous question, with the grade 1 pupils of the two groups benefiting from the intervention mentioned hand washing in their responses significantly more frequently (78% and 80% respectively) than grade one pupils from the comparison group (68%) and 82% of grade 2 pupils and 89% of grade 3 pupils mentioned hand washing before eating (see figure 14 below). 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Hand washing after using toilet No response No Yes 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Hand washing before eating No response No Yes Figure 13: Pupils reporting of hand washing after using the toilet Figure 14: Pupils reporting of hand washing before eating When asked why hand washing was important, pupils were less able to give an age-appropriate explanation, although the different groups’ responses were in similar proportions: 66% of the grade 1 pupils of the two groups benefiting from the intervention were able to give an adequate explanation , compared with 52% of grade one pupils from the comparison group. 71% of grade 2 pupils and 82% of grade 3 pupils were able to explain the importance of hand washing adequately (see figure 15 below). 42 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Explaining importance of hand washing No response No Yes Figure 15: pupils’ ability to explain the importance of hand washing In order to establish a baseline allowing future evaluations of the extent to which teacher training in nutrition is having an impact on their classroom practice, pupils were asked whether their teachers talk about nutrition during school. A similar pattern emerged across all the pupils groups, with between 5-12% of pupils claiming teachers did so often; 15-27% claiming they did so sometimes and 49-67% claiming their teachers did not talk about nutrition. Once again, there were between 9%-27% of non responses (see figure 16 below). This pattern notwithstanding, the comparison group pupils were significantly more likely to report that teachers did not speak about nutrition than the other groups. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Teacher talks about nutrition no response no sometimes often Figure 16: pupils reporting their teachers’ talking about nutrition in school One of the project activities is to provide logistical support to the local health services to provide de-worming tablets to all pupils once annually. Pupils were asked whether they had undergone de-worming during the current school year. Although the responses vary between the pupils, the grade 1 comparison group pupils 43 were significantly less likely to report having received de-worming treatment (64%), with reporting of de￾worming in the other groups varying between 71% to 88% (see figure 17 below). 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Deworming this year No response No Yes Figure 17: pupils reporting of receiving de-worming in the current year As the primary beneficiaries of many of the project’s interventions, it is important to hear and include pupils’ perceptions and experiences. Although pupils’ recall of what happened the previous day, for example, must be treated with some caution, information from pupils can provide very useful triangulation of more “official” information, for example about the frequency of school feeding. As noted above, in the case of reporting of pupils’ attention problems, it is perhaps to be expected that teachers would report these more frequently than pupils themselves. It is nevertheless valuable to have some insight into pupil’s experience of this and other questions of importance for the project. 3. Project schools During the workshop to define M&E needs for the FFK2 project, project staff and management expressed the desire to have access to detailed school-level information regarding the infrastructure built or improved and maintained as part of the project, with the possibility of updating this on an ongoing basis as part of the project monitoring system. This corresponded with a decision to procure a tailor-made database to manage project data more effectively. It was therefore decided that the baseline survey should collect school-level information, as a necessary step towards both these ambitions. A data collection instrument was elaborated by the M&E team, with extensive support from the consultant, based on the information needs expressed by the managers of the different project components. This goes significantly beyond the minimum M&E requirements of USDA for the project based on the MGD indicators and represents a real investment in and ownership of project M&E by staff and management alike. 44 The school surveys were completed in the respective schools by the project “professionals” in collaboration with the school director or pedagogical director. Delays in completing the instrument meant that the information was collected between 8-17 December 2017, almost one year after the beginning of phase 2 of the project7 . Phase one of the project had worked in 242 schools. During phase 2, this was extended to 270 schools. All 270 schools participating in phase two of the project took part in the survey. These schools have a wide variety of sizes and profiles. They range from schools with one or two teachers, to one school with 72 teachers, the mean number of teachers being 9.2. The schools have between 2 and 31 classes, the mean being 9 classes. 222 of the 270 schools are “complete” primary schools, with pupils from grade 1 to grade 7. Of the remaining schools, most have pupils in all of the first five grades. All have first and second grade pupils; all except four have third grade pupils; all except 12 have fourth grade pupils; all except 16 have 5th grade pupils. 100 (37%) of the schools have mixed classes, with pupils from two or more grades in the same class. 18 schools operate with one shift. 237 operate with two shifts and 15 with 3 shifts. The total number of pupils in a school ranges from 12 to 1,625. 19 schools have fewer than 100 pupils. 151 have between 100-200 pupils. 51 have 201-500 pupils. 32 have between 501 and 999 pupils. 16 schools have 1,000 or more pupils (see figure 18 below). The number of pupils declared in the survey total 73,630. 0 200 400 600 800 1000 1200 1400 1600 1800 Number of pupils Schools Numbers of pupils in project schools Figure 18: number of pupils in project schools 7 Where necessary, the situation of a particular component or school at the start of phase 2 can be established retroactively, using project records. The present report has not done this but it could be done in future. 45 Buildings Latrines All of the 270 schools except one declared there was one or more latrine in the school, with the number varying between 1 and 18, and a mean of 4.17 latrines per school. 92% of the total 1,123 latrines were declared to be functioning. 356 were declared to have been built by the project and 344 to have been rehabilitated by the project. Overall, 35% of functioning latrines were specifically for boys, 37% specifically for girls and 7% for both boys and girls, with 21% specifically for teachers. In 155 (57%) schools, there is a marked separation (by a picture or sign) between boys’ and girls’ latrines. 158 (59%) schools have no hand washing facilities attached to the latrines. The others have between one and six attached hand washing facilities. 207 (77%) schools have no hand washing facilities separate from the latrines, with other having between one and 6 separate hand washing facilities. Of the schools which have hand washing facilities, 92 (57%) have water for hand washing “always”, 28 (17%) have water “most of the time”, 20 (12%) schools have water for hand washing “sometimes”, 8 (5%) “rarely” and 14 (9%) “never”. Of the hand washing facilities enumerated, 196 (37%) are said to be in need of maintenance. The latrines are cleaned “daily” in 187 (70%) schools, “twice a week” in 6 (2%) schools, “weekly” in 66 (25%) schools, “from time to time” in 4 (1%) schools and at an “other” frequency, for example whenever necessary, in 5 (2%) schools. In 98% schools they are cleaned by the pupils (with help from others such as school council members, community volunteers, teachers or school maintenance staff in 5% of cases) and in 2% schools by school staff. Storerooms and kitchens All 270 schools declared they had a storeroom, of which 266 (99%) were built by the project. 220 (81%) are reported to be in good condition, with 50 (19%) in need of some maintenance. 267 schools declare they have a kitchen, of which all but one were built by the project. 226 (84%) are reported to be in good condition, with 44 (16%) in need of some maintenance. 264 schools report having a wood saving stove; these come in three different sizes: stoves with apertures for 25 litre and 50 litre pans; others with apertures for 50 litre and 100 litre pans; and a small number with two apertures for pans of 100 litres. Of the stoves with 25 litre and 50 litre pans, 218 (84%) are in good condition, with 43 (16%) needing some maintenance. Of the stoves with 50 litre and 100 litre pans, 97 (87%) are in good condition; 15 (13%) need some maintenance. Of the stoves with two 100 litre pans, ten (71%) are in good conditions, with four (29%) in need of some maintenance. One school has a stove with apertures of different dimensions, which is in good condition. 264 (98%) schools declare their kitchen are cleaned daily, with 5 (2%) cleaned weekly. In 259 (96%) schools, the kitchens are cleaned by the volunteer cooks; pupils clean the kitchen in 8 (3%) schools, the teacher who is storeroom supervisor does so in 2 schools and a member of the school council in one school. The storeroom is reported to be cleaned daily in 213 (79%) schools, weekly in 48 (18%), twice weekly in 3 (1%) and “from time 46 to time” in two (1%). The storeroom is cleaned by volunteer cooks in 208 (77%) schools, by pupils in 39 (14%) schools, by the teacher who is the storeroom supervisor in 17 (6%) schools and by a school council member in 3 (1%) schools, with parents and community members or other teachers performing this role occasionally. 105 schools (39%) report having a maintenance plan, with 135 respondents (50%) saying their school does not and 28 respondents (10%) saying they do not know. 81 schools say their maintenance plan is followed, with 12 saying it is followed “sometimes”. The survey focussed on a number of specific points on which the building team wished to have detailed information school-by-school. These are presented in table 5 below. Building and maintenance points Number of schools Yes Yes % No No % No response No response % The storeroom has improved ventilation 159 59% 110 41% 1 0% The interior of the storeroom is painted with lime 185 69% 85 31% 0 0% The interior walls are clean 242 90% 28 10% 0 0% The door opens and closes easily 207 77% 63 23% 0 0% The door shows signs of rust 106 39% 164 61% 0 0% The door has a door stopper outside 125 46% 145 54% 0 0% The kitchen is plastered 232 86% 38 14% 0 0% The storeroom is plastered 230 85% 40 15% 0 0% The walls of the kitchen are cracked 63 23% 207 77% 0 0% The walls of the storeroom are cracked 92 34% 178 66% 0 0% The ceiling of the kitchen and storeroom is well nailed down 192 71% 78 29% 0 0% The roof sheets show signs of rust 99 37% 171 63% 0 0% The roof has a fireguard 168 62% 102 38% 0 0% The storeroom floor is burned 201 74% 69 26% 0 0% The kitchen and storeroom show signs of abrasion around the block 95 35% 175 65% 0 0% The storeroom floor is cracked 73 27% 197 73% 0 0% The kitchen floor is cracked 118 44% 152 56% 0 0% The wood saving stove has two irons in each cavity 211 78% 59 22% 0 0% The wood saving stove is cracked 131 49% 138 51% 1 0% The base of the wood saving stove lacks cement 84 31% 185 69% 1 0% The wood saving stove requires plastering inside with clay 132 49% 136 50% 2 1% The storeroom / kitchen has a plaque identifying the project 201 74% 68 25% 1 0% Table 5: school-level information regarding particular building and maintenance points requested by the building team Water and sanitation 254 schools (94%) have some sort of system to supply water; 16 (6%) do not. The systems include: piped mains water, electric water pumps, manual water pumps and rainwater harvesting. 96 schools are connected to the piped mains water supply, whereas 168 are not (6 non responses). 13 of these were connected by the project, while 78 were rehabilitated by the project. 61 schools reported that their water system works “every day”; 16 that it functions “most of the time”, 18 that it functions “from time to time”; in other cases the piped water system “doesn’t work” or is defective. 27 schools have an electric water pump. 16 of these work with a bore hole created by the project. 7 work with a well created by the project. In 3 cases, the pump was installed by the project. In 6 the system was renovated 47 by the project. 11 of these work “100% of the time so we have water every day”, 4 work “most days”, 5 work “most days during the rainy season”, 2 work “from time to time during the rainy season,” 3 do not currently work or are faulty. 114 schools have a manual pump. 62 of these work with a bore hole created by the project. 34 work with a well created by the project. In 9 cases the pump was installed by the project. In 42 the system was renovated by the project. 69 of these work “100% of the time so we have water every day”, 11 work most days, 4 work “sometimes”, 7 work “most days during the rainy season”, 10 work “from time to time during the rainy season,” 11 do not currently work or are faulty. 110 schools have a rainwater harvesting system with gutters and water tanks. 32 of these were installed by the project and 76 were renovated by the project. 36 of these systems “function 100% during the rainy season”, 18 of these schools “have water most days”, while 34 of them “have water from time to time.” 22 of the systems do not work. 90 schools (33%) reported having been without water during the past two weeks; 180 schools (67%) had not. When asked if their water was of good quality, without colour or odour, 172 schools (64%) replied “always”, 38 (14%) replied “most of the time”, 53 schools (20%) replied “sometimes” and 3 (1%) replied “never”. 38 schools (14%) “always” have cloudy water, 30 (11%) have cloudy water “most of the time,” 147 schools (54%) “sometimes” and 31 (11%) “never”. 21 schools (8%) “always” have salty water, 24 (9%) have salty water “most of the time”, 118 schools (44%) “sometimes” have salty water, whereas the water in 95 schools (35%) is “never” salty. Other problems of water quality “sometimes” occur in 119 schools (44%), 14 schools (5%) “always” have other water quality problems, 17 schools (6%) do so “most of the time” and 110 (41%) “never” do8 . 121 schools (45%) consider their water source is sufficient for the needs of the school, whereas 144 (53%) consider it is not. 105 schools (39%) have enough water for consumption and for their school garden all year, whereas 165 (61%) declare they do not. 220 schools (81%) consider they have sufficient water to prepare the school lunch, whereas 50 (19%) do not. 144 schools (53%) have water supplies able to supply the school and the neighbouring communities, whereas 125 (46%) do not. 243 schools declare having received a 1,000 litre kitchen water tank from the project. In 161 cases, this is mounted on a base and protected by a porch. In 110 cases, the tank holds water. Where it does not, the reasons given include missing a tap, missing guttering, the tank not yet having been mounted or the tank being damaged. In 27 cases, the tank is connected to guttering for rainwater harvesting. In 227 cases it is not. 8 Where the responses do not total 100%, in this and subsequent sections, this is due to some schools not having answered certain questions. 48 141 schools declare having received a 5,000 litre water tank from the project. In 113 cases, this is mounted on a base and protected by a porch. In 101 cases, the tank holds water. Where it does not, the reasons given include missing a tap and/or guttering, guttering being too small, the tank not yet having been mounted or the tank being damaged. In 68 cases, the tank is connected to guttering for rainwater harvesting. In 126 cases it is not. 130 schools report having a routine water system maintenance and cleaning plan, whereas 94 do not. 157 schools report having a water committee in the local community; 95 do not and in 14 cases, the respondent did not know. Where one exists, the school representative which belongs to the committee may be the school director (in 23 cases), the pedagogical director (5 cases), the teacher who manages the school feeding (104 cases), all the school teachers (31 cases) or there may not be a representative from the school (23 cases). In 22 cases, the respondent did not know. The water system is looked after by the community surrounding the school, through the water committee (in 57 cases), the school council (in 120 cases), project staff who are not connected to the school (17 cases), pupils from the school (7 cases) or member of the school leadership team (34 cases), assisted, in some cases, by teachers from the school. The water system is cleaned daily (in 69 schools), twice a week (in 25 schools), weekly (in 76 schools), “from time to time” (in 90 schools), or less frequently. School feeding Gaps in school feeding, and the reasons for these, were recorded. 92 (34%) schools stated that in the past two months there had been one day or more when they had not been able to prepare the lunch. 84 schools had had some days without CSB during that time: of these, 29 were without for one or two days, 30 for 3 or 4 days, 19 for between 5-10 days and 5 between 13-22 days. The reasons for not preparing the lunch included lack of CSB (50 schools), lack of wood (12 schools), lack of water (25 schools) and cooks not turning up (23 schools). The food is prepared with varying frequency in different schools. 82 (30%) schools prepare the CSB once a day; 183 (68%) prepare it twice daily and 5 schools (2%) prepare it three times daily. The CSB which is to be cooked is weighed and taken out of the storeroom by the teacher who supervises the storeroom (in 256 schools, 95%), by the cooks (11 schools, 4%), or by another teacher from the school (3 schools, 1%). The stock sheet and distribution report are filled in by the storeroom supervisor (or, in 1% schools, the head teacher). These are checked by the school director (in 114 schools, 42%), the storeroom supervisor (in 151 schools, 56%) or a member of the schools council (in 4 schools, 1%). The rota of volunteer cooks is managed up in various ways: in 18 schools it is the same pair or group of people who prepare the food each day; in 99 schools the task rotates between a group of mothers chosen from within the community; in 104 it rotates by neighbourhood, with a different neighbourhood providing the volunteers each day; in 32 schools mothers from the school council take it in turns to cook; in 3 schools the cooking is performed by volunteers who turn up on the day. In 14 schools a different system is used, such as mothers of children in different classes taking turns. In 266 49 schools (99%), it is the storeroom supervisor who fills in the register of volunteers; in the other cases, the school director or a member of the school council plays this role. 261 schools (97%) report that volunteer cooks receive some type of incentive. These include receiving soap from the project (247 schools, 91%), receiving some financial payment (31 schools, 11% ), receiving capulanas from the project (153 school, 57%), receiving tee shirts (57 schools, 21% ), taking part in training in agriculture, health, hygiene and nutrition (8 schools, 3%) and taking part in all the project’s commemorative activities (6 schools, 2%). Where financial incentives are given, these are paid by the school management (8 schools, 3%), the school feeding project (23 schools, 9%), members of the community though the school council (34 schools, 13%), parents and guardians (9 schools, 3%)9 . It is the cooks who wash the dishes in 262 (97%) schools, assisted by pupils in 19 (7%) of these schools; pupils perform this task in seven (3%) schools. 266 (99%) schools claim the dishes are “always” washed prior to school feeding sessions, with 4 (1%) schools stating this happens “most of the time”. 265 (98%) schools claim the dishes are “always” washed after school feeding sessions, with 4 schools (1%) stating they are washed “most of the time” and one school “sometimes”. 225 (83%) schools have a draining table for this purpose. 266 (99%) schools claim to have received detergent from the project during the previous 6 months. 179 schools (66%) claim they received utensils from the project during the same period. For the detail of these utensils, see table 6 below. Utensils received Number of schools Percentage of schools Pans 161 60% Bowls 169 63% Spoons 169 63% Glasses 135 50% Basins 148 55% Buckets 153 57% Jugs 145 54% Scales 148 55% Table 6: Schools reporting of utensils received from the project in the previous 6 months 211 schools (78%) reported that their existing utensils were not sufficient to prepare and distribute the food, compared with 59 (22%) who reported they had sufficient utensils. Details of the utensils which were reported to be lacking appear in table 7 below. Utensils lacking Number of schools Percentage of schools Pans 183 68% Bowls 191 71% Spoons 206 76% Glasses 187 69% Basins 188 70% 9 74 schools gave responses to this question, significantly more than the 31 which claimed that volunteer cooks received financial incentives. 50 Buckets 196 73% Jugs 183 68% Scales 177 66% Table 7: Schools reporting of utensils they are lacking Finally, schools were asked some questions about basic hygiene behaviour: 100% of schools reported that the cooks “always” wash their hands before preparing the food. 264 schools (98%) report that the pupils “always” wash their hands before eating; 3 schools report pupils wash their hands before eating “most of the time”; 2 reported “some of the time”; one “rarely”. 237 (88%) of schools report their pupils “always” wash their hands after eating, with 15 (6%) reporting pupils wash hands after eating “most of the time”; 10 (4%) “some of the time”; 4 (1%) “rarely” and 2 “never”. School gardens and Home Grown School Feeding Gardens 101 schools (37%) report having school gardens, with 55 (20%) reporting they have a “machamba” (larger plot for growing produce, known as a Home Grown School Feeding Garden or HGSFG). 85 schools (31%) report growing legumes, 111 (41%) growing vegetables, 78 (29%) tubers and root crops, 10 (4%) oil seeds, 67 (25%) cereals and 26 (10%) fruit trees. The frequency with which the school gardens and HGSFGs produce food varies, with 43 schools (16%) reporting they grow food once a year, 19 schools (7%) twice a year, 28 schools (10%) all year around and 2 schools (1%) three times a year. 63 schools (23%) report only growing food during the rainy season. The productive areas of the gardens or HGSFGs varies, with three schools (1%) having between 3 and 5 hectares, 9 schools (3%) having between 1 and 2 hectares, 54 schools (20%) having between 0.25 and one hectare and 77 (29%) having less than 0.25 hectares. Schools were asked about the quantity of produce they had grown in the past two years (2016 and 2017). The responses are summarised in table 8 below10.This shows that a small number of schools are beginning to produce quantities of food which can make a real contribution to school feeding. Schools report growing Schools report growing (%) Total quality (kg) Mean quantity (kg) Number of school producing 1-50 kg Number of school producing 51-100 kg Number of school producing 101-500 kg Legumes 85 31% 4,350 kg 52kg 62 11 11 Vegetables 111 41% 7,620 kg 70kg 62 31 16 Tubers and roots 78 29% 5,768 kg 75kg 56 8 13 Oil seeds 10 4% 565 kg 47kg 11 0 1 Cereals 67 25% 4,422 kg 70kg 39 16 8 Fruit trees 26 10% 3,085 kg 114kg 11 7 9 Table 8: school reporting of produce grown in the past two years 10 In some cases, the total number of schools which reported growing a particular type of crop does not exactly tally with the numbers of schools which reported producing a certain amount by weight. However, the discrepancy is small. 51 13 schools reported that the food they produced was consumed in the school on festival days. 1 school reported that food produced was sold in the community. 136 schools reported that the food grown contributed to the school feeding. 72 schools reported having evidence of this consumption, in the form of a document provided by the SDEJT (24 schools), a register of school production (27 schools), a school garden register (11 schools), the CSB report (4 schools) or other types of document (7 schools); 85 schools reported having no evidence. 98 schools report receiving seeds from the project; 66 schools reported receiving gardening tools, such as hoes, rakes and machetes. In 52 schools, the garden or HGSFG is irrigated by rainwater, in 64 by water from a water fountain, in 28 by river water, in 10 by other water sources, such as a manual pump or tap water. The water source is available all year around in 82 schools and is available only seasonally in 76. When the usual water source is not available, 31 schools continue to cultivate; 53 do not. 74 gardens or HGSFGs are enclosed, whereas 86 are not. 9 schools have already obtained a DUAT, the administrative document authorising them to exploit the land. The community contribution to school gardens varies: in 60 schools community members help several times a year; in 36 schools, community members help regularly, for example, weekly; in 57 schools, the community is not involved. Nutrition, hygiene and school health 248 schools (92%) have at least one teacher who has been trained in nutrition education. 179 of these were trained in 2016; 69 in 2017; 5 did not remember when they were trained. Of these teachers, 228 were trained by the project nutrition training team; 10 by a health trainer supplied by the district health office; 14 did not know who trained them. The teachers who had been trained in nutrition education reported having performed the following dissemination activities: repeating their training for other teachers (238 teachers, 96%); organising a meeting to include nutrition content in lesson plans (214 teachers, 86%); including nutrition content in lessons for pupils (229 teachers, 92%); introducing nutrition content into graduation ceremonies (220 teachers, 89%); organising school cleaning and health days (212 teachers, 86%); meeting with parents (1 teacher). In 240 schools (89%), the volunteer cooks received training in hygiene and nutrition from project staff during the previous year. 219 schools (81%) reported receiving nutrition posters from the project; 17 schools (6%) reported receiving pamphlets. 239 schools (89%) reported these materials were being used: by teachers using them in their lessons (in 228 schools, 84%), by being displayed on the wall of the kitchen, storeroom or teachers’ room (in 212 schools, 79%); by being distributed to pupils (in 194 schools, 72%) or distributed to the community (in 145 schools, 54%). 52 Extra-curricular clubs 264 (98%) schools report having one or more extracurricular clubs. 77 schools (29%) report having one or two clubs; 147 (54%) report having 3 or 4 clubs; 34 schools (13%) report having 5 or 6 clubs, with 6 schools not specifying the number. 261 (97%) schools report having reading clubs. In 256 cases (95%), these have fixed hours for club activities11 . They are organised by grade in 168 schools (62%) and by class in 88 (33%); in 7 cases (3%), “no organisation” is reported. The main activities are remedial lessons (in 165 schools, 61%), working in groups outside of the class (in 116 schools, 46%) and doing homework to understand the subject better (1 school). The number of pupils in reading clubs per school ranges from 5 to 203, with the mean number being 29. They function every day in 4 schools (1%), twice a week in 168 schools (62%), once a week in 82 schools (30%); in 6 schools (2%) there is no fixed frequency. The reading clubs are led by teachers in 256 schools (95%), by pupils in 4 schools (1%) and by a community member in one. 228 schools (84%) report having mathematics clubs. In 224 cases (83%), these have fixed hours for club activities. They are organised by grade in 141 schools (52%) and by class in 87 (32%); in 2 cases, “no organisation” is reported. The main activities are remedial lessons (in 162 schools, 60%) and working in groups outside of the class (in 71 schools, 26%). The number of pupils in mathematics clubs per school ranges from 4 to 203, with the mean number being 26. They function every day in 5 schools (2%), twice a week in 141 schools (52%), once a week in 84 schools (31%); in 2 schools there is no fixed frequency. The mathematics clubs are led by teachers in 225 schools (83%) and by pupils in 5 schools (2%). 136 schools (50%) report having clubs for “recreational arts and dance”. In 129 cases (48%), these have fixed hours for club activities. They are organised by grade in 79 schools (29%) and by class in 44 (16%); in 13 cases (5%), “no organisation” is reported. The main activities are singing and dancing, including traditional dances, and making crafts. The number of pupils in these clubs per school ranges from 5 to 203, with the mean number being 23. They function every day in 1 school, twice a week in 76 schools (28%), once a week in 51 schools (19%); in 6 schools (2%) there is no fixed frequency. These clubs are led by teachers in 129 schools (48%), by pupils in 6 schools (2%) and by a project volunteer in one. 152 schools (56%) report having sports clubs. All except two of these have fixed hours for club activities. They are organised by grade in 67 schools (25%) and by class in 75 (28%); in 13 cases (5%), “no organisation” is reported. The main activities are football, games and exercises, team games and athletics. The number of pupils in the sports clubs per school ranges from 4 to 203, with the mean number being 28. They function every day in 13 schools (5%), twice a week in 85 schools (31%), once a week in 46 schools (17%); in 8 schools (3%) there is no fixed frequency. The sports clubs are led by teachers in 155 schools (57%). 11 For the sake of consistency, percentages given in this and the following sections are given as percentages of the total number of schools, not to the total declaring this type of club. 53 70 schools (26%) report having school garden clubs. In 67 cases (25%), these have fixed hours for club activities. They are organised by grade in 33 schools (12%) and by class in 34 (13%); in 2 cases, “no organisation” is reported. The main activities are learning and practicing food production, planting, weeding and watering. The number of pupils in the school garden clubs per school ranges from 8 to 883, with the mean number being 56. They function every day in 27 schools (10%), twice a week in 23 schools (9%), once a week in 12 schools (4%); in 6 schools (2%) there is no fixed frequency. The school garden clubs are led by teachers in 65 schools (24%) and by pupils in one school. 19 schools (7%) report having other types of club. These include themes such as the environment, citizenship, health and hygiene, social sciences, drawing and painting, school cleaning and building. Of these, 11 (4%) have fixed hours for club activities. They report being organised by grade in 7 schools (3%) and by class in 10 schools (4%). The number of pupils in these clubs per school ranges from 8 to 203, with the mean number being 32. They function every day in one school, twice a week in 9 schools (3%), once a week in 6 schools (2%); in 2 schools there is no fixed frequency. These clubs are led by teachers in 14 schools (5%), by pupils in one school and by community volunteers in one school. 168 schools (62%) declared they had received teaching and learning materials from the project in 2016; 68 schools (25%) had received materials in 2017; 10 (4%) reported receiving materials in both 2016 and 2017. The types of materials schools reported receiving and their perception of the usefulness of these is summarised in table 9 below12 . Type of materials Schools which report receiving materials Schools which report receiving materials % Schools which report materials to be the most useful Schools which report materials to be the most useful % Pencil sharpeners 246 91% 256 95% Erasers 246 91% 259 96% Bostick 221 82% 234 87% Calligraphy exercise books 207 77% 249 92% Small lined exercise books 212 79% 252 93% Ballpoint pens 235 87% 258 96% Poster paint 210 78% 243 90% Crossword puzzles 193 71% 242 90% Charcoal 226 84% 261 97% Colour pencils 235 87% 257 95% Children’s story books 209 77% 246 91% Large paper 220 81% 254 94% Handwriting alphabet charts 209 77% 257 95% Multiplication tables charts 179 66% 252 93% Syllable charts (grade 1) 215 80% 255 94% Syllable charts (grade 2) 208 77% 250 93% 12 Although the question asked was “of the materials received, which were considered to be most useful”, the numbers of schools reporting the different materials as perceived useful is consistently higher than the number of schools who reported receiving the items. This suggests that the second question may have been misunderstood and answered in absolute terms, rather than in relation to the materials received. 54 Blackboards for pupils 177 66% 247 91% A4 black covered exercise book 208 77% 256 95% Football 224 83% 263 97% Paper glue 207 77% 258 96% Mathematics sets 204 76% 253 94% Reams of A4 paper 198 73% 260 96% White chalk 187 69% 255 94% Table 9: school reporting of materials received and their usefulness 263 schools (97%) report seeing “some differences” in the children who participate in the clubs. 250 (93%) report that they are more active in regular lessons. 252 schools (93%) report pupils have improved learning outcomes. 235 (87%) report improvements in the pupils’ communication and expression. 183 schools (78%) report having a teacher who has been trained in the area of extra- curricular clubs. In terms of challenges encountered in implementing clubs, 197 schools (73%) report low levels of pupil participation, 186 (69%) report difficulties finding time for club activities, 148 (55%) report a lack of interest amongst teachers in implementing clubs, 154 (57%) report lacking information about methodologies or activities which can be used in the clubs, 130 (48%) report lacking the materials needed. As noted in section 1 above (see pages 35-37), pupils’ reporting of their membership and participation in clubs gave a less vibrant picture than the one suggested by the numerical information reported by schools. The reporting of clubs’ existence by schools does not necessarily mean that these are particularly active or dynamic or that pupils’ attendance is regular. In particular, there may be a significant difference between membership and regular attendance (and between attendance and productive activity). Indeed, the number of schools reporting challenges, such as low pupils participation and lack of interest by teachers, in the previous paragraph suggests that many of these clubs may “exist” as lists of pupils on paper, rather than being particularly active or useful in practice. The final evaluation of FFK2, which went beyond the quantitative data and looked at the sector in more detail, including school visits, observations and interviews, found that the component of extra-curricular clubs was in need of substantial reinforcement and reinvigoration if it was to play a significant role in improving learning. It also suggested that the reporting criteria needed reform, so that presenting a paper containing a list of pupils with a school stamp would not be sufficient to allow a club to be “counted” for reporting purposes, and more evidence of actual activities should be required. 4. Teachers working in project schools Two teachers from each project school were asked to respond to the teacher survey, except where a school has only one teacher. 458 teachers responded13. The enumerators administering the survey were asked to identify teachers in the school who had special responsibilities in relation to the project: where possible, they were asked to survey the teacher who is the storeroom supervisor and the teacher who is the nutrition focal point or who is involved in teaching literacy. 13 This is an 85% response rate, given that, of the 270 schools which completed the school survey, only one reported having only one teacher. 55 Of the 458 teachers surveyed, 37% are female; 63% male. Well over half are aged 18-29. Well over half are in the administrative category N4, which means they have received one year’s teacher training after completing 10 grades of secondary education. More information about the characteristics of the teachers surveyed is provided in tables 10 below. Teacher characteristics Count Column N % Sex Female 169 36.9 Male 289 63.1 Age 18-29 245 53.5 30-39 169 36.9 40-49 33 7.2 50-59 9 2.0 Left blank 2 0.4 Administrative grade N1 10 2.2 N2 3 0.7 N3 206 45.0 N4 239 52.2 Table 10: Teacher characteristics 9% of the teacher respondents report being members of the School Feeding Committee; 42% are storeroom supervisors; 25% report having been trained in nutrition and health; 7% report being club supervisors. For more details about teachers’ responsibilities see Technical Appendix 2. Questions relating to the project The overwhelming majority of teachers reported that the organisation of food distribution was either “excellent” (39%) or “good” (59%), although 53% reported that aspects of the distribution could be improved. In relation to learning clubs, 92% of teachers reported that clubs had been created in their schools, with 65% reporting that “all” or “most” pupils in their classes participate regularly (see figure 19 below). As already noted, this finding contrasts with the pupils’ reporting of their club attendance. 85% of teachers reported knowing what activities are offered by the clubs. 88% reported that the clubs had a positive effect on pupils’ school work, again in contrast with the pupils’ own reporting. 93% reported their school having received materials for use in the clubs; about half of those who answered the question claimed to have been consulted about the choice of materials they received. 54% claimed to have used the materials either that day or the day 56 before; 99% considered the materials to be either “excellent” (42%) or “good” (57%). 62% of teachers claimed they now had access to sufficient teaching and learning materials. 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% All pupils in my class participate regularly Most pupils in my class participate regularly Some pupils in my class participate regularly Participation is irregular I do not know Left blank Teachers' reporting of pupils' attendance of clubs Figure 19: Teachers reporting of their pupils’ participation in clubs 83% of teachers reported having received training as part of the project. Of these, 34% evaluated the training to be “excellent” and 64% “good”. Hunger, attentiveness and learning Teachers were asked about pupils’ hunger, attentiveness in class and learning. 69% reported that pupils sometimes” appeared hungry at school (7% “often”; 23% “never”) (see figure 20 below). 0% 10% 20% 30% 40% 50% 60% 70% 80% often sometimes never no response Pupils appear to be hungry at school Figure 20: teachers reporting of pupils appearing hungry at school 57 86% reported that pupils “sometimes” had attention problems in class (2% “often”; 11% “never”) and 81% that pupils “sometimes” had difficulty learning (1% often; 16% never) (see figures 21 and 22 below). 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% often sometimes never no response Pupils have attention problems in the classroom 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% often sometimes never no response Pupils have problems learning Figure 21: teachers reporting of pupils attention problems at school Figure 22: teachers reporting of pupils having problems learning 91% of teachers reported observing changes in their pupils’ behaviour and 89% reported differences in their pupils’ hygiene practices since the beginning of the school feeding programme. As reported in the section on pupil-level information above, the teachers’ responses report these phenomena significantly more frequently than the pupils’. Nutrition 81% of teachers reported having received nutrition training as part of the project. 6% of teachers surveyed were trained in 2013; 4% in 2014, 16% in 2015, 25% in 2016 and 31% in 2017. The training is delivered as a series of modules. 76% teachers reported receiving training in Hygiene; 72% in Food groups; 69% in Hygiene and food storage; 72% in Handwashing; 70% in Balanced diet. Teachers were asked with whom they had shared the information. The most frequently cited group with whom learning had been shared was pupils (76%) (more than one response could be given: see figure 23 below for the details of other groups). 58 0% 10% 20% 30% 40% 50% 60% 70% 80% Pupils Your family Parents and guardians School council Community No response With whom teachers shared nutrition information Figure 23: “With whom did you share nutrition training information?”: teachers’ responses 97% of those who replied to the question considered that the training had brought about changes in their own lives or those of their pupils. The survey included questions designed to evaluate teachers’ knowledge of nutrition and hygiene. Overall, when asked to provide examples of different food group (basic foodstuffs, protein-rich food, food rich in vitamins/minerals, energy-rich foods), teachers were able to provide correct answers in between 61% and 85% of cases (see figure 24 below). 66% of teachers were able to explain correctly the three stages of hand washing (see figure 25 below). 0% 10% 20% 30% 40% 50% 60% 70% 0 stages 1 stage 2 stages 3 stages Teachers able to explain 3 stages of hand washing correctly Figure 24: Teachers’ ability to provide examples of different food groups Figure 25: Teachers’ ability to explain hand washing stages 0% 20% 40% 60% 80% 100% Teachers able to provide examples of different food groups no response no yes 59 Literacy This section is designed to provide a baseline to allow the evaluation of teachers’ knowledge of literacy teaching. There were a high percentage (67%-81%) of non responses to the questions in the section on literacy. In many cases, teachers who reported not receiving the literacy training did not answer the questions about literacy: it is possible they did not feel capable of answering these questions or that they thought the questions were not aimed at them. However, 37 teachers who had not received the training responded to at least some of the questions about literacy, whereas 5 teachers who reported receiving the training did not respond to the questions. The literacy section was the final section of the questionnaire. It is possible that many teachers found the survey too long and abandoned it when they reached this section, which may have appeared more challenging than the others. Whatever the reason for the high rate of non responses, it is not possible to draw any firm conclusions regarding those teachers who did not respond to the questions in this section14 . 28% of teachers surveyed claimed to be taking part in the literacy training provided by the project (20% claimed not to be taking part; 52% did not respond to this question). In order to test their knowledge of basic concepts of literacy, teachers were asked to define five commonly￾used terms using multiple choice questions. 65% of respondents did not reply to these questions. Overall, teachers gave the correct definition of each of the five questions as follows: “phonological awareness” 8%, “phonetics” 9%, “fluency” 30%, “vocabulary” 28% and “comprehension”31%15 . In order to test teachers’ knowledge and understanding of the three phases of the lesson cycle, they were asked to define each stage using multiple choice questions. 69%-70% of respondents did not reply to these questions. 16% chose the correct definition of the phase “I do”; 22% chose the correct definition of “we do”; 23% chose the correct definition of the phase “you do”. 22% of teachers reported their language of instruction to be Portuguese, compared with 8% for Xichangana and 2% for Xirhonga (67% did not respond to this question). 30% of teachers claim to read for lesson preparation “every day”. 24% claim to read for pleasure “every day” (68% non responses in each case) (see figures 26 and 27 below). The language of reading for pleasure is given as Portuguese by 26% of teachers, Xichangana by 4% and Xirhonga by 1% (68% non responses). 14 After the data had been collected, entered and collated, this significant gap in the data became clear. It was decided not to attempt to fill the gap by asking teachers a second time to attempt to respond to these questions, in order not to delay the baseline process still further. This unfortunate gap in the data raises the issue of enumerator training and the ability of enumerators to follow the instructions they have received and transmit instructions correctly to respondents. 15 These and all percentages reported are of the total number of teachers surveyed, not of those who responded to the question. 60 0% 10% 20% 30% 40% 50% 60% 70% 80% Every day About 3 times a week About once a week About once a month Never No response Frequency of reading for lesson preparation 0% 10% 20% 30% 40% 50% 60% 70% 80% Every day About 3 times a week About once a week About once a month Never No response Frequency of reading for pleasure Figure 26: Frequency of teachers’ reading for lesson preparation Figure 27: Frequency of teachers’ reading for pleasure When asked to assess their ability to read in Portuguese, 17% declare they can read fluently and understand everything, whereas 15% declare they can read fluently with some difficulties (68% non responses). When asked to assess their ability to read in Xichangana or Xirhonga, 6% declare they can read fluently and understand everything, whereas 16% declare they can read fluently with some difficulties (69% non responses). For more detail of the responses, see figures 28 and 29 below. 0% 10% 20% 30% 40% 50% 60% 70% 80% Read fluently and understand everything Read fluently with some difficulties Read and have difficulty understanding Read rarely/never Left blank (No response) Reading ability in Portuguese 0% 10% 20% 30% 40% 50% 60% 70% 80% Read fluently and understand everything Read fluently with some difficulties Read and have difficulty understanding Read rarely/never Left blank (No response) Reading ability in Xichangana/Xirhonga Figure 28: Teachers’ reading ability in Portuguese Figure 29: Teachers’ reading ability in XIchangana/Xirhonga Teachers were asked about the training in literacy. 16% of teachers stated they had received training in the five components; all but one of these had found it useful (73% did not respond). 21% stated they had received materials to improve their teaching of reading and writing; all but one of these had found the materials useful (77% did not respond). 15% had received a guide to help them with lesson planning; all but two had found the guide useful. 61 When asked about their use of certain materials distributed by the project (a guide of methodological suggestions; letter, syllable and word cards; teacher read alouds; reading books for pupils), between 24% and 30% of teachers stated they used these every day or about 3 times a week (68%-69% did not respond). For more detail, see figure 30 below. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Use of literacy materials provided by the project non response never about once a month about once a week about 3 times a week every day Figure 30: teachers’ reporting of use of literacy materials provided by the project When asked how much time their pupils have for individual reading during the school week, 12% of teachers affirmed that their pupils read for five minutes a day, the most frequent response (compared to 8% affirming 15 minutes per day; 2% affirming 5 minutes per week; 9% affirming 15 minutes per week). 69% did not respond to this question. For more detail, see figure 31 below. 0% 10% 20% 30% 40% 50% 60% 70% 80% 5 minutes per day 15 minutes per day 5 minutes per week 15 minutes per week no response Time pupils spend reading individually Figure 31: teachers’ reporting of time pupils have for individual reading 62 Teachers were asked how they evaluate pupils’ progress in reading and how they record this, through a series of multiple choice options. 69% did not respond. 14% stated they evaluated pupils’ reading by asking them to read one by one during the class; 16% by listening to pupils reading individually to help them and monitor their reading (see figure 32 below). 18% reported keeping a record book in which they record the stages pupils achieve; 12% correcting pupils’ exercise books during lessons (see figure 33 below). 0% 10% 20% 30% 40% 50% 60% 70% 80% Test at end of each trimester Listen to pupils read individually Pupils read one by one in class Only evaluate those who can read No response How teachers evaluate pupils' reading progress 0% 10% 20% 30% 40% 50% 60% 70% 80% Record the stages achieved in record book Give marks at end of trimester in record book Correct exercise books during classes Complete register at the end of year No response How teachers record pupils' reading progress Figure 32: teachers’ reporting how reading progress is evaluated Figure 33: teachers’ reporting how reading progress is recorded Finally, teachers were presented with a series of binary options, designed to test their familiarity with modern classroom practice and approaches to reading. These were adapted from the Learning Gains Study used as part of the EGRA developed in Malawi by USAID Malawi and RTI, 2016. Between 69% and 72% of teachers did not respond to this question. Table 11 below summarises teachers’ responses. The options which are compatible with modern classroom practice and approaches (so are considered as “correct” for the present purposes) are shaded. This shows that, of the teachers who responded, the majority chose the modern approaches in five of the seven cases. However they are fairly evenly split on the questions of whether or not grade one pupils are too young to be responsible for handing out books and whether young pupils learn to read best in front of the blackboard or when holding a book and at their own pace, revealing some very “traditional” attitudes, probably due to a lack of exposure to modern pedagogical theory and practice. When questions are framed in this way (with the focus on attitudes rather than actual practice), there is plainly a risk that teacher choose what they perceive to be the desired response, without any guaranty that this corresponds to their actual practice. 63 Answer A A % Answer B B % No response % It is important to allow pupils to take their books home 23% Pupils should not take school books home 7% 70% Pupils cannot practice reading at home because the majority of parents cannot read 2% Pupils should practice reading at home, even if their parents cannot read 29% 70% Grade one pupils are too young to be responsible for handing out books 12% Grade one pupils can learn to hand out books 14% 74% Pupils should be seated in rows facing the front so they can take part in the class 7% The way pupils are seated should be adapted to suit different activities 22% 71% Preparing classes in advance is necessary for good classroom management 27% Preparing classes in advance does not help much with classroom management 2% 71% Young pupils learn to read best in front of the blackboard 12% Young pupils learn to read best when they are holding a book and can learn at their own pace 17% 71% Independent practice is not useful for young pupils because they make mistakes 3% Being able to make mistakes during independent practice is an important part of learning 25% 72% Table 11: familiarity with modern classroom practice and approaches to reading: teachers’ responses The number of non responses to the teacher questionnaire has resulted in a regrettable gap in the data, which limits the usefulness of the information collected. It is in the section on literacy that this gap is greatest. It is possible that some teachers found the questions overwhelming and so far outside their sphere of knowledge and current practice that they did not feel comfortable attempting them, for fear of revealing their lack of knowledge in this domain. This would be compatible with findings during the baseline study of FFK1 that teachers’ reporting of their own levels of competence and professional knowledge were disproportionately high, compared with objective evidence of teaching quality and teachers’ qualifications and training. Even if one disregards the large number of non responses, taken together with the very low results of the EGRA reported above, the results of the teacher questionnaire confirm a deficit in teacher’s knowledge and skills, particularly in the area of literacy. Given their age profile and, in particular, their level of training (52% of teachers surveyed are in the administrative category N4, having left secondary school after grade 10 and received one year’s teacher training), this is perhaps not surprising. On the other hand, the teachers’ responses to the questions about nutrition suggest that many teachers have been able to learn about basic nutrition and hygiene from the project interventions (although one cannot rule out the possibility that they already possessed this knowledge). It is therefore to be hoped that the training in literacy will lead to teachers becoming more competent and more confident in this important area. 64 5.Teacher training colleges (EPFs) The workshop to define M&E needs for the FFK2 project concluded that M&E of EPF activities should be more integrated in the overall M&E processes of the project, with the EPF calendar aligned with the project planning framework and instruments and monitoring processes developed in partnership between the EPFs and the project M&E team. It was agreed that the EPFs should continue to gather data on the added value provided by the EPF remedial Portuguese programme, using a standardised diagnostic instrument in all EPFs. This had not been collected in 2016 or 2017 but the pre-test was performed at the beginning of the 2018 academic year, so data demonstrating added value will be available once the post-test has been performed at the end of the academic year 2018 (in time for the mid-term evaluation of FFK2). During the workshop, the EPFs were encouraged to identify the types of information which would be useful for their planning and operations and to regularly gather this. Based on the EPFs’ identified needs for specific information, a number of instruments were subsequently developed by the EPFs, with support from the international consultant. Two of these: one to gather information on the overall profiles of EPF trainers and another to compare the supply and demand for specialized trainers were used to collect information early in 201816, when all the EPF directors were required to supply the relevant information for the establishment. The results are presented below. Profiles of EPF trainers The EPF management team wished to have more information about the profiles of trainers working in the different establishments. The instrument used to collect this information was designed to address their specific information needs and priorities. The eleven EPFs have between 11 and 24 trainers, with the mean being 18. The ratio of female to male trainers varies: some EPFs have only one female trainer; overall, less than one quarter of the trainers are female (see figure X below). Maputo and Macuse have proportionately the most female trainers, with 14 males to 9 females in both establishments (see figure 34 below). 16 The funding of the EPFs by FFK2 is slightly out of sync with the rest of the project: the current phase of the project began funding the EPFs in 2016. This information was collected considerably after the onset of the second phase of project funding, so does not strictly speaking give baseline information. The fact that the EPFs are in the process of developing monitoring systems and procedures which correspond to their own needs, thanks to the project, is nevertheless extremely positive. 65 0 5 10 15 20 25 30 Number of female trainers Number of male trainers Figure 34: numbers of trainers per EPF (male and female) The vast majority of trainers (90%) hold a Licenciatura as their highest qualification. Only 3% hold a Master’s degree. 8% of trainers have a lower qualification than the Licenciatura (3% of the total number of trainers have a Bacharelato, 4% have the “Medium” level and 1% the “Basic” level)17. 44% of the trainers have a qualification in Education, Science of Education or School Management; 33% in the areas of the arts, languages or humanities and 23% in the areas of science, social sciences, mathematics or physical education. The trainers are graduates of a number of different universities and teacher education instructions: 27% are graduates of One World University/ISET; 44% were trained at the Pedagogical University; 23% at another university (usually either Universidade Eduardo Mondlane or the Catholic University); the remainder were trained at other institutions, including the state-run teacher training colleges or the EPFs. 99% of the trainers have received psycho-pedagogical training. Regarding their experience of teaching and teacher training, 30% of the trainers have had two or fewer years experience teaching at primary level; 15% have had between 2 years 1 month and 5 years; 25% between 5 years 1 month and 10 years and 30% have had more than 10 years 1 month (see figure 35). 13% have had two or fewer years experience training student teachers; 30% between 2 years 1 month and 5 years; 35% between 5 years 1 month and 10 years and 22% more than 10 years 1 month (see figure 36). 17 The “Licenciatura” is the equivalent of a European or US Bachelor’s Honours degree; a “Bacharelato” is approximately equivalent to an Associate Degree, following two or three years of higher education, and has been phased out in recent years, in favour of a three-year “Licenciatura” in alignment with other countries; “Medium level” is the teacher training qualification earned after graduating from 12th grade of secondary school; “Basic level” is the teacher training qualification earned after graduating from 10th grade of secondary school. In principle, in order to train teachers in Mozambique, trainers are now generally required to possess at minimum the level of “Licenciatura”, although this requirement has not always been systematically applied. 66 0 10 20 30 40 50 60 70 0-2 years 2 years 1 month - 5 years 5 years 1 month - 10 years 10 years 1 month + Experience of primary teaching Number of trainers 0 10 20 30 40 50 60 70 0-2 years 2 years 1 month - 5 years 5 years 1 month - 10 years 10 years 1 month + Experience of teacher training Number of trainers Figure 35: EPF trainers’ experience of primary teaching Figure 36: EPF trainers’ experience of teacher education The administrative categories of the EPF trainers vary, with the majority (87%) classified as “teachers” (docente), rather than teacher trainers. By far the most frequent category (69%) is DN1, the highest category of “docente”. Other trainers are classified at lower DN categories: DN2 6%; DN3 11%; DN4 1%. Other trainers are classified appropriately as teacher trainers (Instrutor Técnico Pedagógico): 13% are in the category ITP-N1; 0.5% in ITP-N2 (see figure 37 below). It is striking that, although the ITP category was introduced in 2007, only 14% of EPF trainers have been reclassified according to the ITP career path. Figure 37: administrative categories of EPF trainers Specialised subjects: trainer supply and demand The EPF management team were keen to have detailed information about trainer needs by speciality. Analysis of the overall relationship between the needs for specialised teacher trainers and the numbers of trainers employed per subject, throughout the eleven EPFs, reveal that, although all or almost all positions are 0 20 40 60 80 100 120 140 160 DN1 DN2 DN3 DN4 ITP-N1 ITP-N2 Administrative category Number of trainers 67 filled in subjects, such as Psycho-pedagogy, Portuguese language teaching methods, Expression Techniques and Language, School management and Organisation, Social Sciences teaching methods and Bantu languages and bilingual education, there are considerable numbers of vacant posts in certain other subjects. There are a large number of unfilled posts in Research methods/ICTs (8 posts vacant), Visual education teaching methods (8 posts vacant), Building, Maintenance and School Production (7 posts vacant), Crafts teaching methods (10 posts vacant), Music teaching methods (6 posts vacant) and Moral and Civic Education (8 posts vacant) (see figure 38 below). The data available does not allow any sort of explanation as to why this gap between supply and demand exists in these subject areas, but this situation is not limited to the EPFs: experience from other countries and sectors suggest it may reflect the international shortage of specialised teachers caused by labour market forces. The teaching profession struggles to attract specialised teachers and teacher educators in many countries, due to the more lucrative positions appropriately qualified candidates can access in other sectors. The EPFs will need to develop strategies to respond to this situation and attract the specialised trainers they need. 0 5 10 15 20 25 Social sciences teaching meth Bantu languages/ bilingal ed Psycho-pedagogy Research methods /ICTs Visual education teaching Building, Maintenance/SP Portuguese Lang teaching meth Natual Sciences teaching meth Crafts teaching meth Music teaching meth Mathematics teaching meth Moral and Civic Education Expression and language Physical Education School Management English Specialised trainers /Target Demand (unmet) Supply Figure 38: Relationship between specialised teacher trainer needs and posts filled Finally, 63% of the EPF trainers overall have at least five years experience of primary teaching. Figure 39 below shows that there is considerable variation between the EPFs in this regard: whereas in EPF Maputo 19 out of 68 the 20 trainers have at least 5 years’ primary teaching experience, in EPF Macuse, this is the case of only 2 of the 19 trainers18 . 0 5 10 15 20 25 30 Trainers WITHOUT 5 years experience in primary teaching Trainers WITH 5 years experience in primary teaching Figure 39: EPF trainers with and without 5 years experience in primary teaching It is hoped that having this information and the ability to keep it updated will assist the EPFs in planning their infrastructure and equipment renewal and maintenance and their strategic Human Resources management and development, including recruitment or professional development of trainers to meet their needs. 18 This is significant in terms of conformity with the Ministry of Education and Human Development’s requirement for teacher trainers in government teacher training institutions, although there has in the past been some uncertainty whether this applies to the EPF trainers. The requirement is based on a regulation regarding the required profile of teacher trainers, introduced in 2007 by the then Ministry of Education and Culture. Article 53 of this regulation states that the following people can apply for posts of instructor in one of the state-run teacher training colleges: a) Instructors and teachers who have a higher education qualification in the disciplinary areas required to teach the curriculum and who have a minimum experience of 5 years of basic level primary teaching; b) Professionals with a “medium” level qualification, who have undergone specific training and have the recognised professional competence and relevant professional experience, of 7 years minimum. 69 Conclusion This is in some ways an unusual baseline report, in that it concerns the second phase of a project which has already operated with a considerable degree of success during the approximately four years of its first phase and begun to produce meaningful change in the lives of its beneficiaries. The final report of the first phase of the project documented these successes and changes, as well as some outstanding challenges. These formed the basis of a series of recommendations for phase two. The current report began by outlining some of the main aspects of the project at the end of phase one, before presenting four sets of detailed new information, gathered from pupils, from teachers, from schools and from ADPP’s teacher training colleges. These four data sets were collected as a result of the desire of project managers and staff to collect and maintain good quality, detailed information, in many cases about highly specific aspects of their work, which will enable them to do their jobs better and, ultimately, produce better outcomes for their beneficiaries. The timing of this extensive data collection, which coincided with the development of the project’s dedicated data base, will allow much more efficient and effective information management and use, providing the necessary human resources and systems are available and used well. In particular, the project M&E team will need to be strengthened numerically and in terms of quality. This has been an ambitious baseline study and the scale of its ambition has not been without costs: in particular it has mobilised project staff for longer than was ideal. Due to a series of accumulated delays, exacerbated by gaps in the M&E team, the report has taken longer to finalise than was hoped. However, the investment of gathering and processing this information is expected to reap dividends, in terms of its utility to the project and of investment in and ownership of the M&E process by project staff. In the context of such a large scale and complex project, having access to the right information in real time and understanding about what is happening in each component and in each school is a key to making informed decisions and choices. The process of preparing this baseline study has highlighted the need for sufficient M&E staff and for the right knowledge and skill sets within the M&E team. There is a need for individuals who have both the intellectual skills and strategic vision to grasp and communicate the “big picture” whilst demonstrating the rigour and attention to detail necessary to develop and follow precise data protocols meticulously and to ensure others understand the need to do likewise. It is thanks to the implication and conscientiousness of the project management team, who stepped into the breach on a number of key occasions, that it has been possible to successfully finish the baseline study and produce this report. The project is fortunate to have a team of competent, committed leaders, who are prepared to go the extra mile when needs arise. The engagement of the different component managers has also been extremely valuable: hopefully they will be rewarded by having access to relevant, useful information to inform their strategic and operational functions. Likewise, the investment of the “professionals” in the data gathering process, at a time when they had other priorities, will hopefully bring them valuable dividends in terms of both the new skills acquired and the information to which they now have access. 70 The final evaluation of stage one made the case that sustainability should be the priority issue for phase two. All stakeholders: the GoM, in particular the Ministries of Education and Human Development, of Finance and of Agriculture, the PAI, ADPP, USDA and other international partners, including the WFP, must as a matter of urgency address the question of what will happen after the project come to an end in 2020. Phase two is an opportunity for a transition plan to be made and implemented, so that everything which has been achieved so far does not cease after the donor’s generous investment comes to an end but becomes the basis of a sustained and sustainable school feeding programme for as long as this is needed. 71 Annexes Annex 1: Results Framework of the Food for Education Project in Mozambique, capturing the Theory of Change upon which the project is based 72 Annex 2:McGovern-Dole Results Frameworks 73 Annex 3: Workshop to define Monitoring and Evaluation needs for the Food for Knowledge programme, phase 2: Recommendations for project management and implementers Workshop to define Monitoring and Evaluation needs for the Food for Knowledge programme, phase 2: Machava, Mozambique, 27 February - 2 March 2017: Recommendations for project management and implementers 9 March 2017 The workshop, facilitated by Simone Doctors, in which approximately 35 members of the project leadership and section representatives participated, was designed to identify the Monitoring and Evaluation (M&E) needs of the second phase of the FFK programme, implemented from [2015-2020] by Planet Aid and ADPP Mozambique, including what information the project wishes to collect and for what purposes. It identified lessons learned during the first phase of the programme, opportunities and constraints with regard to information collection and requirements for data management instruments and systems. It formed the basis for refinement of the Programme Monitoring Plan (PMP) and Evaluation Plan (EP) and for Terms of Reference for the Baseline Study for FFK2. The recommendations below arose from the workshop and are formulated by the facilitator as an additional output of the workshop, with a view to strengthening M&E processes and practices for the second phase of the FFK programme. Recommendations Monitoring instruments and reporting/recording processes  Each sector/component should review the instruments used to report and record the completion of an activity and improve these to make them fit for current purpose (the current instruments have evolved organically over several years, along with the activities; now it is necessary to take stock and analyse where changes are needed).  The chain of reporting/recording of activity completion should be reviewed to ensure clarity and coherence between the processes and instruments at each stage (the instruments should be harmonised so that same information is captured/collated/aggregated in the expressed in the same way, making reporting up the chain a smooth process). Sectors which currently send narrative reports, rather than raw data, to the M&E department should also send the data, via an appropriate instrument/process.  Establish fixed dates each month to send in reports and ensure sending reports on time is part of the organisational culture; provide professional development support training for colleagues who are not currently submitting reports on time. When reports are submitted, there should be acknowledgement and follow up, to demonstrate that the information submitted is being used/considered. Demonstrate to the project team the advantages of collecting standardised information, which can be used to benefit the programme.  For each sector, establish clear criteria to define when an activity is “countable” (for example whether a garden is “active” or what constitutes an installed water system) to avoid double counting or under￾reporting.  Clarify criteria to calculate numbers of beneficiaries of specific interventions, which need to be reflected in the accounting tools e.g. an unambiguous formula to show the number of pupils to benefit from one latrine, when several latrines will be built at different times in the same school.  Avoid repeatedly modifying tools and systems, given that all change is disruptive; in particular, when new colleagues or teams arrive, discourage modifying tools/systems until the new colleagues/teams have had a chance to observe the previous or existing ones at work.  As part of the Baseline Study for FFK2, conduct a survey of all project schools, including gathering information about existing infrastructure, clubs, school gardens, water source and management, the school feeding process etc. The instrument should be developed with input from the different components regarding what information they would find useful. 74  Staff at all levels and within all components should receive basic training in the principles and practices of M&E; the arrival of a new M&E team is a good opportunity for such training. Data management  Procure a database able to integrate information from the different sectors and components. Ensure: o The system provided can be updated to reflect changes and additional data management needs as these arise; o Information from existing spreadsheets can be easily imported into the new system o Adequate staff training and support are provided; o System maintenance and fine-tuning are provided.  Decisions to be made include: o Will only school-level information be collected or will pupil-level, teacher-level and volunteer￾level information be collected and used? Factors to consider include: the trade off between a) the burden on “professionals” and schools of collecting pupil-level information; number of data entry employees needed to process pupil-level information and b) the advantages of being able to analyse interaction between pupil-level information, such as educational outcomes, in the form of test results and other types of information, such as frequency of school feeding, exposure to literacy intervention, etc. o Will all project sectors be integrated in a single database? Will data from the literacy/reading interventions, the nutrition component managed by WISHH and the EPFs be managed in a single project database, or will data from these sectors be managed separately? If so, what provision will be made for management and analysis of data relating to the EPFs? How will questions of access and user/administrator rights be defined? o Will the database function online or offline and what are the implications of each of these options, given Internet coverage and reliability in Mozambique? New M&E team  Given the gap in M&E over the past months, efforts to induct the much-needed new M&E manager and specialist, along with the data entry clerk(s), and promote their integration with the rest of the project team should be a priority. Given this need, the services of the external evaluator of FFK1 may be used to support the imminent Baseline study of FFK2, so as to ensure the new M&E team have the opportunity to get up to speed on other M&E tasks and activities. Escolas de Profesores do Futuro Teacher Training Colleges (EPFs)  There should be more involvement of national headquarters in M&E of EPF activities: o the EPF calendar must become part of the project planning framework; o the EPFs should develop instruments and monitoring processes in partnership with the project M&E team.  Gather data on the added value provided by the EPF remedial Portuguese programme using a standardised diagnostic instrument in all EPFs: o Appoint a working group of Portuguese language trainers (preferably including one representative from each EPF) to adapt the “list of contents to test” developed in 2014-15 and, based on this list, to adapt the instrument itself, so that it reflects more closely the focus of the remedial Portuguese teaching and is viewed more favourably by the EPF trainers. o The instrument should be updated annually, within the agreed criteria defined in the “list of contents to test”. Each year, all students – from the one-year and the three-year course – should be pre- and post-tested to generate annual data on the added value provided by the Portuguese teaching within the EPFs. If necessary, separate instruments may be developed for students from the one-year and the three-year courses.  Gather data to show the evolution of teaching skills and methods amongst EPF student teachers: o Identify a working group of trainers specialised in classroom pedagogy (preferably including one representative from each EPF) to improve the teaching practice observation tool developed in 75 2014-15, so that it reflects more closely the focus of the pedagogical training and is viewed more favourably by the EPF trainers. In particular, it may be necessary to include a score. o Ensure the same instrument is used to assess teaching practice by all EPFs, to allow the collection of data showing the evolution of students throughout their training. o Develop a system to record the scores obtained by student teachers during teaching practice, to show the evolution of students’ teaching throughout their training. o Monitor the career evolution of EPF graduates: Collect contact information: cell phone number, an alternative number (for example of a family member) and email address for all EPF students before they graduate. o Contact them twice yearly to ask where they are working (name of school, district and province) and what job they are doing. This task can be decentralised to the individual EPFs, but the information should be centralised. o Develop a database (or a spreadsheet in the short term) of EPF graduates, which is updated twice yearly.  As part of the Baseline Study for FFK2, conduct a survey of all 11 EPFs, including information about current infrastructure, transport and other needs, trainers, etc. The instrument should be developed with input from the different EPFs and the national headquarters regarding what information they would find useful.  Demonstrate to the EPF leadership and trainers the advantages of collecting standardised information, which can be used to benefit the running and development of the EPFs.  Develop norms or standards to describe the responsibilities, desired attributes and training practices of an EPF trainer, to be used to support professional development and appraisals. N.B. any data collected for this purpose should not be considered as part of the professional or career advancement of the trainers, which is the responsibility of the Ministry of Education and Human Development (MINEDH). Literacy programme  Continue to develop instruments to monitor classroom teaching, bearing in mind the burden of capturing the information generated; a trade off is necessary between the ambitions to benefit from extensive information and the costs of capturing and processing this. o Instruments should be designed to achieve their specific objectives ; o Where possible, avoid binary evaluation instruments (x is achieved; x is not achieved) which do not allow room to record progression; o Within classroom observation instruments, avoid questions which combine assessment of teacher behaviour and pupil behaviour/responses within the same question.  In preparation for the FFK2 Baseline study: o Develop Terms of Reference (ToR) for the development of an early Grade reading Assessment (EGRA) able to assess the pre-reading and reading abilities of pupils in Grades 1,2 and 3; o Identify consultants to develop the EGRA content; o Plan and prepare training of the reading coaches and “professionals” who will conduct the data gathering in schools. WISHH/nutrition training  Continue to promote the integration of the WISHH team within the wider project team, with joint work plans and planning meetings.  Review the process of reporting/recording WISHH activities and data to the project M&E team and management team, to ensure the same narrative reports and raw data are available to the M&E department and to WISHH and there is coherence in the numbers recorded in their respective systems.  Develop and perform a survey to capture the impact of the nutrition training (N.B. this may be incorporated into the Baseline Survey of schools for FFK2). 76 Annex 4: Technical appendix 1 – Introduction and analysis of EGRA subscales Introduction NFER was given the task of assisting the external evaluator with the design of the study to establish a baseline for future impact evaluations; to draw the sample of schools and to carry out the analysis of baseline data. The work will has the following elements:  Advice on impact evaluation design  Advice on school background data required for sampling  Production of school and pupil sampling plan  Drawing of the school sample using matching  Checking baseline dataset for outliers and cleaning errors  Descriptive analysis of baseline pupil data (see 1.1 below)  Reliability of the 9 EGRA subscales  Production of BMI z-scores  Descriptive analysis of teacher survey The sampling design has reflected the way the FFK and literacy programmes are being rolled out. This, in turn, governs the planned baseline descriptive analysis of pupil survey data. Table 1 describes the planned cross-tabulations, depending on number of schools (and pupils) sampled. In particular, the number of Ronga schools may be too small for useful analysis. Where a literacy intervention is running in both Portuguese and a local language within the same school, this school will appear twice in the EGRA analysis (once in each language) but only once in the analysis of other variables. The following outcomes will be reported for each split but combined across languages:  Height  Weight  Hunger  Attentiveness  Attitude to learning  Health and hygiene practices  BMI z-scores The 9 EGRA subscales will be reported separately for each language. The statistic reported will depend on the measure. As sampling was done at the school-level, we calculated the mean (or proportion) within each school first and then aggregated these for corresponding population estimates. Note therefore that standard deviations are also calculated at the school level. As we were particularly concerned with the performance of the EGRA scales, we 77 followed up each table with histograms plotted from the pupil-level data. In each case, we plotted one for all the children split by intervention group and another for just Grade 1. Table 1. Planned cross-tabulations Grade 1 Grade 2 Grade 3 FFK + literacy FFK only comparison Portuguese Xichangana Xirhonga In total 254 schools took part in the project. The number of schools and pupils in each intervention group by language of testing is given in Table 2 and the number of pupils by gender in Table 3. As an individual school could be involved in testing in more than one language and with more than one grade, a school can appear more than once in Table 2. All pupils took each of the tests but most Grade 1 pupils would not be expected to answer anything beyond subtasks 4 and 5. Table 2: Intervention group by language of testing Language of Testing Portuguese Xichangana Xirhonga Total Pupils Schools Pupils Schools Pupils Schools Pupils Schools Intervention Group Grade 1: FFK + literacy 1066 66 690 47 141 11 1897 124 Grade 1: FFK only 1050 78 25 2 3 2 1078 82 Grade 1: Comparison 1064 69 41 4 70 6 1175 79 Grade 2 1087 248 0 0 0 0 1087 248 Grade 3 1054 244 0 0 0 0 1054 244 Total 5321 705 756 53 214 19 6291 Table 3: Intervention group by gender Gender Total Boy Girl Intervention Group Grade 1: FFK + literacy 954 943 1897 50.3% 49.7% 100.0% Grade 1: FFK only 605 473 1078 56.1% 43.9% 100.0% Grade 1: Comparison 572 603 1175 48.7% 51.3% 100.0% Grade 2 571 516 1087 52.5% 47.5% 100.0% Grade 3 541 513 1054 78 51.3% 48.7% 100.0% Total 3243 3048 6291 51.5% 48.5% 100.0% Language of Testing (Grade 1 pupils) Grade 1 pupils were tested in their language of instruction (Portuguese, Xichangana or Xirhonga)19 . All Grade 1 results are disaggregated according to language of testing. The number and proportion of grade 1 pupils by language and intervention group is illustrated in Table 4 and Figures 1 and 2 below. The majority of pupils were tested in Portuguese (3180), followed by Xichangana (756) and Xirhonga (214). It can be seen in the tables and graphs that 90.6% of the comparison group were tested in Portuguese, compared to 56.2% of the FFK + literacy group and that 3.5% of the comparison group were tested in Xichangana, compared to 36.4% of the FFK + literacy group. Table 4: Language of Testing of Grade 1 pupils by Intervention Group Intervention Group Grade 1: FFK + literacy Grade 1: FFK only Grade 1: Comparison Total Language of Testing Portuguese N 1066 1050 1064 3180 % 56.2% 97.4% 90.6% 76.6% Xichangana N 690 25 41 756 % 36.4% 2.3% 3.5% 18.2% Xirhonga N 141 3 70 214 % 7.4% 0.3% 6.0% 5.2% Total N 1897 1078 1175 4150 % 100.0% 100.0% 100.0% 100.0% 19The enumerators were instructed to provide the stimuli in the language of instruction but to use the language best understood by the child to explain the task and give instructions. 79 Figure 1: Language of testing of Grade 1 pupils by intervention group (count) Figure 2: Language of testing of Grade 1 pupils by intervention group (percentage) Outliers The data required a substantial amount of cleaning for example, to correct the consistency between school code and school name, to put dates of birth into a consistent format, remove duplicate pupils and then allocate a new unique identifier to all pupils. It was then possible to consider whether any 80 pupils should be removed from some analyses because their data was out of range or were considered to be outliers. A few pupils were removed for the analysis of the EGRA subscales. One pupil was removed from the analysis of subtask 6 as they had a score of 55 when the maximum possible score was 50. Similarly 14 pupils scored more than 30 for subtask 7 and one pupil scored 10 for the understanding element of subtask 8 when the maximum was 4; these pupils were treated as missing for the relevant analyses. Subscale 1: Oral Vocabulary The enumerator read 10 words from a list and pupils were asked to identify the corresponding image. The score represents the total number of correct answers, no distinction is made between no answer and a wrong answer. The results are presented in Table 5. Table 5: Oral Vocabulary score Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean SD Valid N Mean SD Valid N Mean SD Valid N Mean SD Valid N Grade 1: FFK + literacy 8.67 1.48 66 9.54 0.54 47 9.68 0.35 11 9.09 1.22 124 Grade 1: FFK only 8.19 1.36 78 9.30 0.99 2 9.25 1.06 2 8.24 1.36 82 Grade 1: Comparison 9.47 0.64 69 9.52 0.59 4 9.82 0.18 6 9.50 0.62 79 Grade 2 9.31 1.11 248 0 0 9.31 1.11 248 Grade 3 9.74 0.59 244 0 0 9.74 0.59 244 The following graphs show the percentage of grade 1 pupils achieving different scores in the test by intervention group and language of testing. For example, in the Portuguese test, 61% of pupils scored 10 compared to 81% of pupils in the Xichangana test and 83% in the Xirhonga test20 . 20 It is not appropriate to run significance tests on these scores as they refer to different tests. However, given the fact that the tests were carefully constructed to be comparable, it seems clear that the pupils 81 Figure 3: Oral Vocabulary – all pupils Figure 4: Oral Vocabulary – Grade 1 pupils by intervention group 82 Figure 5: Oral Vocabulary – Grade 1 pupils by language of testing Subscale 2: Oral Comprehension The enumerator read a short story and the pupils were asked to answer 4 simple questions. The score represents the total number of correct answers. No distinction is made between no answer and a wrong answer. The results are presented in Table 6. Table 6: Oral Comprehension Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean SD Valid N Mean SD Valid N Mean SD Valid N Mean SD Valid N Grade 1: FFK + literacy 1.26 1.02 66 1.46 1.10 47 0.75 0.59 11 1.29 1.04 124 Grade 1: FFK only 1.26 1.04 78 1.07 1.51 2 2.25 0.35 2 1.28 1.04 82 Grade 1: Comparison 0.93 0.79 69 0.46 0.58 4 0.96 0.68 6 0.91 0.77 79 Grade 2 1.88 1.10 248 0 0 1.88 1.10 248 Grade 3 2.81 0.85 244 0 0 2.81 0.85 244 83 Figure 6: Oral Comprehension – all pupils Figure 7: Oral Comprehension – Grade 1 pupils by intervention group 84 Figure 8: Oral Comprehension – Grade 1 pupils by language of testing Subscale 3: Phonological Awareness The enumerator read 10 words from a list and for each word, the pupil was asked to identify a picture from a choice of three whose name starts with same sound. A point was given for each sound matched correctly, no distinction was made between no answer and a wrong answer. The results are presented in Table 7. Table 7: Phonological Awareness score Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean SD Valid N Mean SD Valid N Mean SD Valid N Mean SD Valid N Grade 1: FFK + literacy 2.78 2.12 66 3.41 2.74 47 4.88 3.62 11 3.20 2.57 124 Grade 1: FFK only 3.20 2.25 78 3.73 5.28 2 4.25 1.06 2 3.24 2.28 82 Grade 1: Comparison 2.34 2.72 69 2.98 4.70 4 0.97 0.61 6 2.27 2.74 79 Grade 2 4.76 2.87 248 0 0 4.76 2.87 248 Grade 3 6.47 2.59 244 0 0 6.47 2.59 244 85 Figure 9: Phonological Awareness – all pupils Figure 10: Phonological Awareness – Grade 1 pupils by intervention group 86 Figure 11: Phonological Awareness – Grade 1 pupils by language of testing Subscale 4: Concepts of Print The pupil was given a story book and asked to perform a series of tasks that provide evidence that he/she is familiar with printed materials and reading them. A point was given for each of the 10 tasks performed correctly. No distinction was made between no answer and a wrong answer. The results are presented in Table 8. Table 8: Concepts of Print score Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean SD Valid N Mean SD Valid N Mean SD Valid N Mean SD Valid N Grade 1: FFK + literacy 3.85 2.11 66 3.84 2.25 47 3.67 2.36 11 3.83 2.17 124 Grade 1: FFK only 3.69 2.21 78 5.47 0.19 2 4.25 2.47 2 3.75 2.19 82 Grade 1: Comparison 3.64 1.63 69 3.71 2.02 4 3.77 0.61 6 3.65 1.58 79 Grade 2 5.42 2.21 248 0 0 5.42 2.21 248 Grade 3 7.35 1.91 244 0 0 7.35 1.91 244 87 Figure 12: Concepts of print – all pupils Figure 13: Concepts of print – Grade 1 pupils by intervention group 88 Figure 14: Concepts of print – Grade 1 pupils by language of testing Subscale 5: Letter Sound Recognition The pupils were asked to voice the sounds corresponding to letters (either upper case or lower case) displayed in a 10x10 grid. They were given one minute to complete the task. Scoring was out of 100 with a point given for each pair of letter-sounds matched correctly. The results are presented in Table 9. If the pupil couldn’t match any of the letters in the first line of the grid to its corresponding sound they were given a score of 0. If a pupil completed the task in less than 60 seconds the enumerator was asked to record the time taken. This information was recorded but for several reasons it looks as if the enumerators may have misunderstood the instructions. A time was recorded for 443 pupils in total, 116 were recorded as taking 60 seconds, one was 90 seconds and 81 were given a time even though they scored 0 for the number of correct sounds. The data for time to complete the exercise has not been analysed further. 89 Table 9: Knowledge of Letter Sounds score Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean SD Valid N Mean SD Valid N Mean SD Valid N Mean SD Valid N Grade 1: FFK + literacy 0.12 0.46 66 0.61 2.35 47 1.11 3.24 11 0.39 1.77 124 Grade 1: FFK only 0.07 0.28 78 2.03 2.88 2 0.00 0.00 2 0.12 0.52 82 Grade 1: Comparison 0.28 0.91 69 0.20 0.40 4 0.00 0.00 6 0.25 0.85 79 Grade 2 4.88 12.91 248 0 0 4.88 12.91 248 Grade 3 17.76 21.51 244 0 0 17.76 21.51 244 Figure 15: Letter Sound Recognition – all pupils 90 Figure 16: Letter Sound Recognition – Grade 1 pupils by intervention group Figure 17: Letter Sound Recognition – Grade 1 pupils by language of testing Subscale 6: Reading Syllables The pupils were asked to read syllables displayed in a 5x10 grid. They were given one minute to complete the task. Scoring was out of 50 with a point for each syllable read correctly. The results are presented in Table 10. If the task was completed in less than 60 seconds the enumerator was asked 91 to record the time taken. There were similar problems with this data to those seen for subscale 5 so the data was not used in further analysis. Table 10: Reading Syllables score Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean SD Valid N Mean SD Valid N Mean SD Valid N Mean SD Valid N Grade 1: FFK + literacy 0.07 0.35 66 0.07 0.49 47 0.06 0.20 11 0.07 0.40 124 Grade 1: FFK only 0.00 0.00 78 0.47 0.66 2 0.00 0.00 2 0.01 0.10 82 Grade 1: Comparison 0.07 0.40 69 0.00 0.00 4 0.00 0.00 6 0.06 0.38 79 Grade 2 3.35 7.36 248 0 0 3.35 7.36 248 Grade 3 13.57 13.23 244 0 0 13.57 13.23 244 Figure 18: Reading syllables – all pupils 92 Figure 19: Reading syllables – Grade 1 pupils by intervention group Figure 20: Reading syllables – Grade 1 pupils by language of testing Subscale 7: Reading Words The pupils were asked to read words from a list of 30. They are given 60 seconds to complete the task. Scoring was out of 30 with a point for each word read correctly. The results are presented in Table 11. Again the time to complete the exercise if less than a minute was not analysed. 93 Table 11: Reading Words score Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean SD Valid N Mean SD Valid N Mean SD Valid N Mean SD Valid N Grade 1: FFK + literacy 0.03 0.19 66 0.04 0.25 47 0.02 0.05 11 0.03 0.21 124 Grade 1: FFK only 0.00 0.00 78 0.03 0.05 2 0.00 0.00 2 0.00 0.01 82 Grade 1: Comparison 0.03 0.16 69 0.00 0.00 4 0.00 0.00 6 0.03 0.15 79 Grade 2 2.51 4.96 247 0 0 2.51 4.96 247 Grade 3 9.37 8.31 244 0 0 9.37 8.31 244 Figure 21: Reading words – all pupils 94 Figure 22: Reading words – Grade 1 pupils by intervention group Figure 23: Reading words – Grade 1 pupils by language of testing 95 Subscale 8: Reading Fluency and Text Comprehension Pupils were asked to read a 70 word text and were given a minute to complete the task. They were given a point for each word read correctly. The results are presented in Table 12. After reading the text pupils were asked up to four comprehension questions, how many they were asked was dependent upon how far they got through the text. They were asked none if they couldn’t read the first line of text. The results for reading fluency are presented in Table 13. 58 pupils successfully read all 70 words and there was an odd peak with 162 pupils scoring 52. As the numbers are so small for the text comprehension exercise the number of pupils attaining each score is presented in Table 13 rather than mean values. Table 12: Reading Fluency score Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean SD Valid N Mean SD Valid N Mean SD Valid N Mean SD Valid N Grade 1: FFK + literacy 0.04 0.34 66 0.05 0.31 47 0.00 0.00 11 0.04 0.31 124 Grade 1: FFK only 0.00 0.00 78 0.00 0.00 2 0.00 0.00 2 0.00 0.00 82 Grade 1: Comparison 0.07 0.51 69 0.00 0.00 4 0.00 0.00 6 0.07 0.47 79 Grade 2 4.65 10.46 248 0 0 4.65 10.46 248 Grade 3 17.15 17.21 244 0 0 17.15 17.21 244 96 Figure 24: Reading fluency – all pupils Figure 25: Reading fluency – Grade 1 pupils by intervention group 97 Figure 26: Reading fluency – Grade 1 pupils by language of testing Table 13: Number of pupils achieving scores for subtask 8: Text Comprehension Number of correct answers 0 1 2 3 4 Total Intervention Group Grade 1: FFK + literacy 1876 2 10 2 1 1891 Grade 1: FFK only 1078 0 0 0 0 1078 Grade 1: Comparison 1155 7 6 5 1 1174 Grade 2 954 22 27 36 48 1087 Grade 3 636 57 80 119 160 1052 Total 5699 88 123 162 210 6282 98 Figure 27: Text comprehension – all pupils Figure 28: Text comprehension – Grade 1 pupils by intervention group 99 Figure 29: Text comprehension – Grade 1 pupils by language of testing Further Analysis of Subscale 8 Pupils were asked between 0 and 4 comprehension questions depending on how far they were able to read through the text. The rules are documented in the instructions for evaluators and are slightly different for each language. Table 14 below shows the relationship between fluency and comprehension scores; it can be seen that there may be inconsistencies in the way the rules were applied. Table 14: Reading fluency: words from text read correctly in one minute by Understanding of text read Reading fluency: words from text read correctl y in one minute Count Understanding of text read 0 1 2 3 4 Total 0 5529 9 24 18 17 5597 1 11 0 1 0 2 14 2 29 1 6 0 0 36 3 17 1 2 6 1 27 4 16 0 1 0 2 19 5 0 1 1 2 0 4 6 6 0 0 0 0 6 7 6 0 0 1 1 8 8 7 4 2 0 3 16 9 5 5 1 0 0 11 10 17 10 3 3 2 35 11 2 2 0 0 0 4 12 1 1 1 2 0 5 100 13 1 0 0 0 1 2 14 3 1 0 0 0 4 15 0 2 0 0 1 3 16 2 3 0 0 0 5 17 6 4 1 0 1 12 18 5 8 1 0 1 15 19 7 11 6 2 3 29 20 0 5 0 1 1 7 21 1 1 5 0 0 7 22 0 0 2 0 1 3 23 0 4 6 2 0 12 24 0 0 1 0 2 3 25 0 0 2 0 1 3 26 0 0 1 0 0 1 27 2 0 0 0 0 2 28 0 0 0 0 1 1 30 0 1 3 2 2 8 31 1 1 2 1 1 6 32 1 2 7 2 2 14 33 1 2 0 0 0 3 34 1 1 0 0 1 3 35 0 0 1 0 1 2 36 1 0 4 0 0 5 37 1 1 5 1 1 9 38 0 1 0 3 1 5 39 0 3 11 0 4 18 40 0 0 3 0 0 3 41 0 0 1 1 1 3 42 2 0 0 1 3 6 43 0 1 0 1 1 3 44 0 0 4 12 0 16 45 0 0 1 4 0 5 46 1 0 0 0 2 3 47 0 0 0 1 0 1 48 0 0 0 0 1 1 49 0 0 0 0 2 2 50 0 0 2 3 1 6 51 0 0 0 1 1 2 52 10 2 4 61 85 162 53 1 0 2 2 0 5 55 0 0 0 2 0 2 56 0 0 0 1 0 1 57 1 0 0 0 0 1 101 59 0 0 2 4 1 7 60 0 0 0 1 0 1 63 0 0 1 1 0 2 64 0 0 0 0 1 1 65 0 0 0 2 0 2 66 0 0 0 8 8 16 67 0 0 0 0 2 2 68 0 0 0 2 1 3 69 0 0 0 1 0 1 70 2 0 3 7 46 58 Total 5696 88 123 162 210 6279 5529 (87.9%) of pupils scored 0 on the fluency and comprehension tests. Just 58 pupils scored full marks (70) on the fluency test and only 583 (9.3%) pupils of the 6291 total were given a comprehension score greater than 0. Consideration was given to excluding pupils with strange results in order to analyse the relationship between fluency and comprehension. General rules were applied using the fluency score as a proxy for how far the pupils read through the text. This would remove 467 pupils including all Grade 1 pupils leaving just 116 grade 2 and grade 3 pupils with a fluency and comprehension score. All the pupils have been kept in the analysis as it is impossible to establish accurate rules for excluding pupils using the fluency scores. In theory pupils could get a long way through the text but only read a few words correctly so, although the numbers look a bit strange, they could all be correct. The mean fluency scores are given in Table 15 below for each number of comprehension questions answered correctly. The mean fluency score increases with the number of correct comprehension questions but this is not surprising as more were asked if the pupil was able to read more text. Table 15: Reading fluency: words from text read correctly in one minute N Mean Std. Deviation Understanding of text read (number of questions answered correctly) 0 5696 0.4 3.4 1 88 17.2 11.6 2 123 24.3 19.2 3 162 41.6 21.6 4 210 46.8 21.8 Total 6279 3.7 12.9 12 pupils had missing values for one or both of these items With such small numbers involved and difficulties with the test it is impossible to draw any useful conclusions about the relationship between fluency and comprehension. 102 Subscale 9: Writing First Name Correctly Table 16: Writing first name correctly (%) Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean (%) SD Valid N Mean (%) SD Valid N Mean (%) SD Valid N Mean (%) SD Valid N Grade 1: FFK + literacy 9.5 14.5 66 4.8 9.0 47 1.4 3.1 11 7.0 12.3 124 Grade 1: FFK only 13.9 19.1 78 0.0 0.0 2 0.0 0.0 2 13.2 18.8 82 Grade 1: Comparison 6.6 11.9 69 0.0 0.0 4 1.9 4.5 6 5.9 11.3 79 Grade 2 51.5 34.5 248 0 0 51.5 34.5 248 Grade 3 80.2 27.5 244 0 0 80.2 27.5 244 Subscale 9: Writing Family Name Correctly Table 17: Writing family name correctly (%) Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean (%) SD Valid N Mean (%) SD Valid N Mean (%) SD Valid N Mean (%) SD Valid N Grade 1: FFK + literacy 2.6 7.2 66 0.8 2.1 47 0.8 2.5 11 1.8 5.5 124 Grade 1: FFK only 2.8 5.1 78 0.0 0.0 2 0.0 0.0 2 2.7 5.0 82 Grade 1: Comparison 1.5 4.6 69 0.0 0.0 4 1.0 2.6 6 1.4 4.4 79 Grade 2 21.9 27.4 248 0 0 21.9 27.4 248 Grade 3 55.3 34.3 244 0 0 55.3 34.3 244 Subscale 9: Writing Dictation Ten words were dictated to the pupils, they scored one point for each word written correctly. The average scores are presented in Table 18. The numbers are small for all groups but particularly Grade 1 so the actual number of pupils attaining each score is presented in Table 19. 103 Table 18: Writing Dictation score Language of Testing Portuguese Xichangana Xirhonga Total Intervention Group Mean SD Valid N Mean SD Valid N Mean SD Valid N Mean SD Valid N Grade 1: FFK + literacy 0.05 0.21 66 0.04 0.15 47 0.04 0.09 11 0.04 0.18 124 Grade 1: FFK only 0.00 0.01 78 0.00 0.00 2 0.00 0.00 2 0.00 0.01 82 Grade 1: Comparison 0.02 0.16 69 0.00 0.00 4 0.00 0.00 6 0.02 0.15 79 Grade 2 0.74 1.53 248 0 0 0.74 1.53 248 Grade 3 2.98 2.79 244 0 0 2.98 2.79 244 Table 19: Writing Dictation score (number of pupils) Intervention Group Total Writing: dictation score Grade 1: FFK + literacy Grade 1: FFK only Grade 1: Comparison Grade 2 Grade 3 0 1877 1077 1172 935 558 5619 1 2 1 0 21 47 71 2 1 0 1 15 40 57 3 0 0 1 14 31 46 4 1 0 0 18 33 52 5 0 0 0 13 32 45 6 0 0 0 11 35 46 7 0 0 0 12 40 52 8 0 0 0 12 43 55 9 1 0 1 12 56 70 10 7 0 0 24 138 169 1889 1078 1175 1087 1053 6282 104 Figure 30: Writing dictation – all pupils Figure 31: Writing dictation – Grade 1 pupils by intervention group 105 Figure 32: Writing dictation – Grade 1 pupils by language of testing 106 Annex 5: Technical appendix 2 – Analysis of Pupil and Teacher Surveys Height, Weight and BMI The enumerators measured the pupil’s height and weight. They were asked to follow clear instructions and to record weight to the nearest 0.1kg and height to the nearest cm. The thresholds for outliers were set at less than 10kg and more than 60kg for weight. For height the thresholds were less than 0.98m (WHO 99th percentile for 5 year old girls) and more than 1.929m (WHO 99th percentile for a 17 year old boy). No outliers were identified in the weight data and just 11 were identified as too short and excluded from the height and BMI statistics. Mean Grade height and weight were calculated first within each school and then the overall means calculated for each intervention group (Tables 1 and 2). Table 1: Mean Grade height in metres Intervention Group Mean Standard Deviation Minimum Maximum Valid N Missing Grade 1: FFK + literacy 1.15 0.02 1.08 1.22 95 0 Grade 1: FFK only 1.14 0.03 1.07 1.21 80 0 Grade 1: Comparison 1.15 0.04 1.04 1.27 75 0 Grade 2 1.21 0.05 1.04 1.37 248 0 Grade 3 1.26 0.06 1.02 1.41 244 0 Total 1.21 0.07 1.02 1.41 742 0 107 Table 2: Mean Grade weight in kilograms Intervention Group Mean Standard Deviation Minimum Maximum Valid N Missing Grade 1: FFK + literacy 19.90 1.48 16.55 26.31 95 0 Grade 1: FFK only 19.68 1.35 16.95 24.80 80 0 Grade 1: Comparison 19.85 1.82 12.73 25.45 75 0 Grade 2 22.41 2.63 12.63 32.77 248 0 Grade 3 25.32 3.22 13.05 33.00 244 0 Total 22.49 3.40 12.63 33.00 742 0 108 BMI Z-scores were calculated following WHO guidelines. An additional requirement for the BMI calculations was an accurate date of birth. Pupils were required to be at least 61 months (5 years old) and less than 204 months (17 years old) at the time of measurement. 1302 children had been given 1 st January as their birthday was unknown; these were treated as missing. 24 pupils had birthdays after the interview date or on the day of interview, these were classified as ‘too young’ and also excluded from the analysis. Pupils who had been classified as too short were also excluded from the BMI calculations. This left 4957 pupils from the total of 6291 for whom BMI z scores could be calculated. Of these 270 pupils had BMI z scores which the WHO consider to be outliers, 183 had z scores less than -3 and 87 had scores greater than 3. These 270 pupils were kept in the dataset. Individual pupil z scores were aggregated and grade means calculated within each school and then the overall means calculated for each intervention group (Table 3). Some of the minimum and maximum values for the grade means are still outside the +/- 3 range, this is because some of the grades had very small numbers of pupils and these pupils had extreme BMI values. These are probably having a disproportionate influence on the mean and we have therefore also included the median. The distribution of z scores for the three grades is illustrated by the histograms. 109 Table 3: Mean grade BMI z-score Intervention Group Mean Median Minimum Maximum Valid N Missing 1 Grade 1: FFK + literacy -0.42 -0.41 -2.65 2.47 89 6 2 Grade 1: FFK only -0.31 -0.29 -1.93 0.95 78 2 3 Grade 1: Comparison -0.37 -0.26 -7.19 3.17 67 8 4 Grade 2 -0.34 -0.32 -3.78 3.18 221 27 5 Grade 3 -0.42 -0.43 -8.04 2.86 220 24 Total -0.37 -0.35 -8.04 3.18 675 67 110 111 Pupil Survey 6291 pupils completed the survey. Table 4: Attitude to Learning and Clubs Intervention Group Grade 1: FFK + literacy Grade 1: FFK only Grade 1: Comparison Grade 2 Grade 3 Count Column N % Count Column N % Count Column N % Count Column N % Count Column N % B.1. a) Member of club(s) none 896 47.2 780 72.4 608 51.7 566 52.1 540 51.2 one 355 18.7 155 14.4 191 16.3 172 15.8 171 16.2 two 116 6.1 32 3.0 54 4.6 85 7.8 130 12.3 three or more 29 1.5 11 1.0 34 2.9 28 2.6 23 2.2 no response 462 24.4 70 6.5 287 24.4 194 17.8 175 16.6 N/A 39 2.1 30 2.8 1 0.1 42 3.9 15 1.4 B.1. b) Club attendance 3 or more times per week 0 0.0 0 0.0 75 6.4 16 1.5 20 1.9 3 times per week 0 0.0 0 0.0 1 0.1 0 0.0 0 0.0 twice per week 0 0.0 0 0.0 296 25.2 57 5.2 62 5.9 once per week 256 13.5 119 11.0 54 4.6 137 12.6 138 13.1 from time to time 0 0.0 0 0.0 48 4.1 12 1.1 12 1.1 no response 191 10.1 447 41.5 553 47.1 308 28.3 324 30.7 N/A 1450 76.4 512 47.5 148 12.6 557 51.2 498 47.2 B.1. c) Enjoyment of clubs a lot 308 16.2 113 10.5 119 10.1 163 15.0 199 18.9 a little 202 10.6 75 7.0 161 13.7 127 11.7 133 12.6 no 151 8.0 24 2.2 227 19.3 60 5.5 77 7.3 no response 418 22.0 111 10.3 465 39.6 252 23.2 249 23.6 N/A 818 43.1 755 70.0 203 17.3 485 44.6 396 37.6 B.1. d) Club helps with school work a lot 289 15.2 137 12.7 123 10.5 169 15.5 205 19.4 a little 207 10.9 68 6.3 151 12.9 122 11.2 125 11.9 no 417 22.0 583 54.1 280 23.8 241 22.2 235 22.3 no response 419 22.1 84 7.8 469 39.9 247 22.7 240 22.8 N/A 565 29.8 206 19.1 152 12.9 308 28.3 249 23.6 112 B.1. e) School work has improved a lot 284 15.0 126 11.7 119 10.1 157 14.4 188 17.8 a little 202 10.6 76 7.1 159 13.5 125 11.5 126 12.0 no 418 22.0 586 54.4 275 23.4 241 22.2 244 23.1 no response 423 22.3 84 7.8 474 40.3 255 23.5 242 23.0 N/A 570 30.0 205 19.0 148 12.6 309 28.4 254 24.1 Table 5: Hunger Intervention Group Grade 1: FFK + literacy Grade 1: FFK only Grade 1: Comparison Grade 2 Grade 3 Count Column N % Count Column N % Count Column N % Count Column N % Count Column N % B. 2. a) Sometimes hungry at school yes 1116 58.8 595 55.2 346 29.4 462 42.5 488 46.3 no 714 37.6 454 42.1 158 13.4 424 39.0 368 34.9 no response 67 3.5 29 2.7 671 57.1 201 18.5 197 18.7 N/A 0 0.0 0 0.0 0 0.0 0 0.0 1 0.1 B. 2. b) Hungry "now" yes 810 42.7 404 37.5 334 28.4 362 33.3 361 34.3 no 1033 54.5 647 60.0 175 14.9 532 48.9 499 47.3 no response 54 2.8 27 2.5 666 56.7 193 17.8 194 18.4 N/A 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 B.2. c) Always receives food at school yes 1763 92.9 1005 93.2 2 0.2 705 64.9 676 64.1 no 95 5.0 48 4.5 0 0.0 65 6.0 61 5.8 no response 38 2.0 25 2.3 0 0.0 18 1.7 13 1.2 N/A 1 0.1 0 0.0 1173 99.8 299 27.5 304 28.8 B.2. d) All pupils receive food yes 1790 94.4 1028 95.4 1 0.1 723 66.5 691 65.6 no 63 3.3 32 3.0 0 0.0 47 4.3 46 4.4 no response 38 2.0 18 1.7 0 0.0 20 1.8 13 1.2 N/A 6 0.3 0 0.0 1174 99.9 297 27.3 304 28.8 113 B.2. e) Received food today at school yes 1419 74.8 678 62.9 0 0.0 532 48.9 488 46.3 no 436 23.0 381 35.3 1 0.1 241 22.2 249 23.6 no response 38 2.0 19 1.8 0 0.0 17 1.6 13 1.2 N/A 4 0.2 0 0.0 1174 99.9 297 27.3 304 28.8 B.2. f) Received food yesterday at school yes 1419 74.8 811 75.2 0 0.0 577 53.1 551 52.3 no 436 23.0 247 22.9 1 0.1 193 17.8 181 17.2 no response 37 2.0 20 1.9 1 0.1 20 1.8 18 1.7 N/A 5 0.3 0 0.0 1173 99.8 297 27.3 304 28.8 B.2. g) Food tastes good yes 1636 86.2 954 88.5 0 0.0 664 61.1 627 59.5 no 35 1.8 28 2.6 0 0.0 21 1.9 24 2.3 no response 49 2.6 23 2.1 1 0.1 20 1.8 22 2.1 N/A 177 9.3 73 6.8 1174 99.9 382 35.1 381 36.1 B. 3. a) Number of meals per day at home none 1 0.1 0 0.0 1 0.1 0 0.0 2 0.2 one 199 11.0 50 4.9 122 10.9 88 8.9 63 6.3 two 1220 67.7 802 78.9 771 68.7 679 68.4 672 67.1 three 359 19.9 149 14.7 227 20.2 209 21.1 259 25.9 four 22 1.2 13 1.3 1 0.1 15 1.5 4 0.4 five 0 0.0 0 0.0 0 0.0 0 0.0 1 0.1 no response 1 0.1 3 0.3 1 0.1 1 0.1 0 0.0 B.4. a) Number of meals yesterday at home none 3 0.2 3 0.3 2 0.2 3 0.3 3 0.3 one 188 10.9 55 5.4 125 11.1 94 9.5 71 7.2 two 1127 65.5 781 76.6 764 68.0 660 66.5 653 65.8 three 329 19.1 169 16.6 230 20.5 206 20.8 253 25.5 four 23 1.3 6 0.6 2 0.2 9 0.9 2 0.2 five 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 no response 50 2.9 5 0.5 1 0.1 20 2.0 10 1.0 114 Table 6: Attentiveness Intervention Group Grade 1: FFK + literacy Grade 1: FFK only Grade 1: Comparison Grade 2 Grade 3 Count Column N % Count Column N % Count Column N % Count Column N % Count Column N % B.5. Attention problems often 90 4.7 59 5.5 56 4.8 76 7.0 78 7.4 sometimes 481 25.4 250 23.2 323 27.5 305 28.1 329 31.2 no 877 46.2 583 54.1 652 55.5 549 50.5 519 49.2 no response 449 23.7 186 17.3 144 12.3 157 14.4 128 12.1 Table 7: Health and Hygiene Practices Intervention Group Grade 1: FFK + literacy Grade 1: FFK only Grade 1: Comparison Grade 2 Grade 3 Count Column N % Count Column N % Count Column N % Count Column N % Count Column N % B.6. a) Wash hands after toilet yes 1426 75.2 858 79.6 742 63.1 880 81.0 922 87.5 no 295 15.6 147 13.6 379 32.3 132 12.1 87 8.3 no response 176 9.3 73 6.8 54 4.6 75 6.9 45 4.3 B.6. b) Wash hands before eating yes 1481 78.1 856 79.4 794 67.6 896 82.4 935 88.7 no 263 13.9 158 14.7 327 27.8 123 11.3 77 7.3 no response 153 8.1 64 5.9 54 4.6 68 6.3 42 4.0 B. 7. Explains importance of hand washing yes 1256 66.2 709 65.8 615 52.3 768 70.7 860 81.6 no 422 22.2 280 26.0 493 42.0 227 20.9 132 12.5 no response 219 11.5 89 8.3 67 5.7 92 8.5 62 5.9 115 B.8. Teacher talks about nutrition often 145 7.6 66 6.1 63 5.4 95 8.7 128 12.1 sometimes 383 20.2 161 14.9 212 18.0 227 20.9 286 27.1 no 940 49.6 628 58.3 791 67.3 598 55.0 520 49.3 no response 429 22.6 223 20.7 109 9.3 167 15.4 120 11.4 B. 10. Deworming this year yes 1353 71.3 948 87.9 746 63.5 799 73.5 784 74.4 no 271 14.3 49 4.5 338 28.8 130 12.0 137 13.0 no response 273 14.4 81 7.5 91 7.7 158 14.5 133 12.6 Teacher Survey 458 teachers responded to the teacher survey. Table 8:Teacher characteristics Count Column N % Sex Female 169 36.9 Male 289 63.1 Age 18-29 245 53.5 30-39 169 36.9 40-49 33 7.2 50-59 9 2.0 Left blank 2 0.4 Administrative grade N1 10 2.2 N2 3 0.7 N3 206 45.0 N4 239 52.2 116 Table 9: Time in teaching Count Column N % Time in this school less than 1 year 62 13.5 1-3 years 172 37.6 4-6 years 127 27.7 more than 6 years 96 21.0 Left blank 1 0.2 Time in this grade less than 1 year 122 26.6 1-3 years 240 52.4 4-6 years 53 11.6 more than 6 years 42 9.2 Left blank 1 0.2 Table 10: Grades taught Count Column N % Grade taught Grade 1 173 37.8 Grade 2 68 14.8 Grade 3 67 14.6 Left blank 38 8.3 other grade 112 24.5 Other grade(s) Grade 2 2 0.4 Grade 2 and 3 5 1.1 Grade 2 and 4 3 0.7 Grade 3 5 1.1 Grade 3 and 4 1 0.2 Grade 3 and 5 2 0.4 Grade 4 44 9.6 Grade 4 and 5 11 2.4 Grade 4, 6 and 7 1 0.2 Grade 5 61 13.3 Grade 5 and 6 1 0.2 Grade 5, 6 and 7 2 0.4 Grade 6 13 2.8 117 Grade 6 and 7 64 14.0 Grade 7 14 3.1 Left blank 4 0.9 N.A 225 49.1 Table 11: Teacher responsibilities Count Column N % Specific responsibility Member of School Feeding Committee 43 9.4 Storeroom supervisor 192 41.9 Teacher trained in nutrition and health 113 24.7 Other responsibility 57 12.4 Left blank 53 11.6 Other responsibility building 1 0.2 club supervisor 31 6.8 club supervisor Escoles 2 0.4 cook 1 0.2 culture 2 0.4 health and hygiene 1 0.2 just teacher 1 0.2 Left blank 41 9.0 Literacy 14 3.1 Literacy teacher 1 0.2 member of school feeding committee 1 0.2 N.A 329 71.8 none 1 0.2 Nutrition 1 0.2 nutrition and health 5 1.1 Other responsibility 1 0.2 pedagogical director 2 0.4 President of ONP 1 0.2 sanitation 1 0.2 school gardens 11 2.4 school head 1 0.2 school hygiene and health 1 0.2 storeroom supervisor 3 0.7 teacher in bilingual grade 1 0.2 trained in nutrition 4 0.9 118 Table 12: Questions about the project Section A, Question 2, 4, 6 and 8 Count Column N % A1 organization of food distribution Excellent 175 38.5 Good 268 59.0 Neither good nor bad 10 2.2 Bad 1 0.2 2. Aspects which could be improved Yes 244 53.3 No 212 46.3 Left blank 2 0.4 4.Clubs created in schools Yes 420 91.7 No 33 7.2 Left blank 5 1.1 5. Pupils' attendance of clubs All pupils in my grade participate regularly 117 25.5 Most pupils in my grade participate regularly 180 39.3 Some pupils in my grade participate regularly 96 21.0 Participation is irregular 21 4.6 I do not know 9 2.0 Left blank 35 7.6 6. Know which activities offered Yes 390 85.2 No 23 5.0 Left blank 45 9.8 8. Positive effects of clubs? Yes 404 88.2 No 11 2.4 Left blank 43 9.4 119 Table 13: Questions about the project Section A, Question 10, 11, 12, 13, 17 and 19 Count Column N % 10. Received materials for clubs Yes 420 93.3 No 30 6.7 Left blank 0 0.0 11.Consulted about choice of materials Yes 212 46.3 No 208 45.4 Left blank 38 8.3 13. Used materials today or yesterday Yes 246 53.7 No 157 34.3 Left blank 55 12.0 15. Materials are: Excellent 145 42.3 Good 194 56.6 Neither good nor bad 4 1.2 Bad 0 0.0 16.Access to sufficient materials Yes 283 61.8 No 165 36.0 Left blank 10 2.2 17. Training by school feeding project? Yes 378 82.5 No 65 14.2 Left blank 15 3.3 19.Training was: Excellent 132 34.2 Good 248 64.2 Neither good nor bad 6 1.6 Bad 0 0.0 120 Table 14: Questions about the pupils Section B, Question 1, 3 ,4, 6 Count Column N % B.1.Pupils appear hungry Often 33 7.2 Sometimes 316 69.0 Never 103 22.5 Left blank 6 1.3 3.Attention problems in graderoom Often 11 2.4 Sometimes 392 85.6 Never 51 11.1 Left blank 4 0.9 4.Problems learning Often 6 1.3 Sometimes 370 80.8 Never 72 15.7 Left blank 10 2.2 6.Changes in pupil behaviour yes 415 90.6 no 38 8.3 Left blank 5 1.1 8.Diferences in pupils' hygiene practices yes 409 89.3 no 36 7.9 Left blank 13 2.8 10. Other differences since introduction of project yes 338 73.8 no 93 20.3 Left blank 27 5.9 121 Table 15: Questions about nutrition Section C, Question 1 and 2 Count Column N % Nutrition training yes 373 81.4 no 80 17.5 Left blank 5 1.1 Which year 2013 25 5.5 2014 20 4.4 2015 73 15.9 2016 115 25.1 2017 144 31.4 Left blank 81 17.7 Table 16: Questions about nutrition Section C, Question 3 Which aspects of nutrition have you learnt about? (You can select more than one option) Count % 3.Aspects learned A Hygiene 347 76 3.Aspects learned B Food groups 330 72 3.Aspects learned D Handwashing 330 72 3.Aspects learned E Balanced diet 320 70 3.Aspects learned C Hygiene and food storage 314 69 No response 78 17 Total = 458 100 More than one answer could be given so percentages may sum to more than 100. A total of 380 respondents answered at least one item in this question. Table 17: Questions about nutrition Section C, Question 4 With whom have you shared this information? (You can select more than one option) N % 4.Information shared with A Pupils 350 76 4.Information shared with C Your family 213 47 4.Information shared with B Parents and guardians 199 43 122 4. Information shared with D School council 171 37 4.Information shared with E Community 154 34 No response 90 20 Total = 458 100 More than one answer could be given so percentages may sum to more than 100. A total of 368 respondents answered at least one item in this question. Table 18: Questions about nutrition Section C, Question 4f Count Column N % Information shared with Others (please specify) Acquaintances 1 0.2 church 1 0.2 Community and others 1 0.2 community and pupils 8 1.7 everyone 3 0.7 everyone who asks 2 0.4 family 2 0.4 Friends 5 1.1 Friends and neighbours 1 0.2 Friends, colleagues and families outside the area 1 0.2 Left blank 379 82.8 neighbours 1 0.2 Other acquaintances 1 0.2 parent and pupils 1 0.2 Pupils 2 0.4 Pupils and carers 9 2.0 Pupils and work colleagues 1 0.2 Pupils in community, carers and family 1 0.2 pupils, community, family 1 0.2 Teachers 10 2.2 Teachers and community 2 0.4 Teachers from school 2 0.4 Teachers and community in general 2 0.4 Teachers, pupils, parents and guardians and school council 1 0.2 Work colleagues 7 1.5 Work colleagues, pupils and family 1 0.2 123 Table 19: Questions about nutrition Section C, Question 5 Count Column N % Nutrition training brought changes yes 356 77.7 no 10 2.2 Left blank 92 20.1 Table 20: Questions about nutrition Section C, Question 7 Count Column N % Example 1 basic foodstuff yes 388 84.7 no 4 0.9 Left blank 66 14.4 Example 2 basic foodstuff yes 374 81.7 no 10 2.2 Left blank 74 16.2 Example 1 protein-rich food yes 377 82.3 no 14 3.1 Left blank 67 14.6 Example 2 protein-rich food yes 353 77.1 no 18 3.9 Left blank 87 19.0 Example 1 food rich in vitamins/minerals yes 377 82.3 no 10 2.2 Left blank 71 15.5 Example 2 food rich in vitamins/minerals yes 365 79.7 no 12 2.6 Left blank 81 17.7 Example 1 energy-rich foods yes 289 63.1 124 no 82 17.9 Left blank 87 19.0 Example 2 energy-rich foods yes 278 60.7 no 63 13.8 Left blank 117 25.5 Table 21: Questions about nutrition Section C, Question 8 Count Column N % Correct 3 stages of hand washing cited 0 57 13.9 1 46 11.2 2 37 9.0 3 269 65.8 Table 22: Questions relating to literacy Section D, Question 1 and 2 Count Column N % D1. Participates in literacy training yes 130 28.4 no 92 20.1 Left blank 236 51.5 2.i)Phonological awareness - definition A 0 0.0 B 6 1.3 C 100 21.8 D 5 1.1 E 10 2.2 F 36 7.9 Left blank 301 65.7 ii)Phonetics - definition A 0 0.0 B 0 0.0 C 42 9.2 125 D 1 0.2 E 9 2.0 F 108 23.6 Left blank 298 65.1 iii) Fluency - definition A 4 0.9 B 1 0.2 C 5 1.1 D 139 30.3 E 6 1.3 F 5 1.1 Left blank 298 65.1 iv) Vocabulary - definition A 10 2.2 B 130 28.4 C 1 0.2 D 6 1.3 E 6 1.3 F 4 0.9 Left blank 301 65.7 v) Comprehension - definition A 142 31.0 B 5 1.1 C 1 0.2 D 1 0.2 E 8 1.7 F 0 0.0 Left blank 301 65.7 Table 23: Questions relating to literacy Section D, Question 3, 4 and 5 Count Column N % 3. I do Give pupils the opportunity to practice saying/reading new words 28 6.1 Keep pupils quiet while teacher shows new activity 43 9.4 Left blank 311 67.9 Pupils practice new words or activities while teacher shows 1 0.2 Show a new skill to pupils so they know what there is to learn and what they should do 74 16.1 126 4. We do Give pupils opportunity to listen and then to copy 6 1.3 Give pupils the opportunity to practice a new skill 100 21.8 Give pupils the opportunity to work in pairs or groups 37 8.1 Keep pupils occupied 4 0.9 Left blank 311 67.9 5. You do Left blank 313 68.3 Move around the grade to keep order 7 1.5 Move around the grade to observe and help 104 22.7 Prepare the next activity 3 0.7 Read aloud 31 6.8 Table 24: Questions relating to literacy Section D, Question 6 and 7 Count Column N % 6.Language of instruction Portuguese 101 22.1 Xichangana 37 8.1 Xirhonga 11 2.4 English 0 0.0 English and Zulu 0 0.0 Other 2 0.4 Left blank 307 67.0 6. "Other" language of instruction Portuguese 1 0.2 Xichangana 1 0.2 Xirhonga 3 0.7 English 3 0.7 English and Zulu 1 0.2 Other 0 0.0 Left blank 449 98.0 7.Frequency of reading for preparation Everyday 135 29.5 About 3 times a week 0 0.0 About once a week 12 2.6 127 About once a month 2 0.4 Never 0 0.0 Left blank 309 67.5 Table 25: Questions relating to literacy Section D, Question 8 and 9 Count Column N % 8.Frequency of reading for pleasure Everyday 109 23.8 About 3 times a week 0 0.0 About once a week 31 6.8 About once a month 6 1.3 Never 1 0.2 Left blank 311 67.9 9. Language of reading for pleasure Portuguese 121 26.4 Xichangana 17 3.7 Xirhonga 5 1.1 English 0 0.0 English and Zulu 0 0.0 Other 4 0.9 Left blank 311 67.9 9. "Other" language of reading for pleasure Portuguese 0 0.0 Xichangana 1 0.2 Xirhonga 1 0.2 English 7 1.5 English and Zulu 0 0.0 Other 0 0.0 Left blank 449 98.0 128 Table 26: Questions relating to literacy Section D, Question 10 and 11 Count Column N % 10.Reading ability in Portuguese Read fluently and understand everything 79 17.2 Read fluently with some difficulties 68 14.8 Read and have difficulty understanding 0 0.0 Read rarely/never 2 0.4 Left blank 309 67.5 11. Reading ability in Xichangana/Xirhonga Read fluently and understand everything 29 6.3 Read fluently with some difficulties 74 16.2 Read and have difficulty understanding 23 5.0 Read rarely/never 15 3.3 Left blank 317 69.2 Table 27: Questions relating to literacy Section D, Question 12 Count Column N % 12. a) Training about 5 components yes 73 15.9 no 52 11.4 Left blank 333 72.7 b) Was the training useful? yes 72 15.7 no 18 3.9 Left blank 368 80.3 c) Materials to improve reading and writing? yes 95 20.7 no 18 3.9 Left blank 345 75.3 d) Were the materials useful? yes 94 20.5 no 11 2.4 Left blank 353 77.1 e) Guide to help plan grades yes 68 14.8 129 no 44 9.6 Left blank 346 75.5 f) Was the guide useful? yes 66 14.4 no 20 4.4 Left blank 372 81.2 Table 28: Questions relating to literacy Section D, Question 13, 14, 15 and 16 Count Column N % 13. Uses the guide Everyday 110 24.0 About 3 times a week 0 0.0 About once a week 17 3.7 About once a month 1 0.2 Never 17 3.7 Left blank 313 68.3 14.Uses the cards Everyday 119 26.0 About 3 times a week 0 0.0 About once a week 18 3.9 About once a month 2 0.4 Never 6 1.3 Left blank 313 68.3 15. Uses teacher materials Everyday 1 0.2 About 3 times a week 121 26.4 About once a week 15 3.3 About once a month 3 0.7 Never 3 0.7 Left blank 315 68.8 16. Uses pupil books Everyday 138 30.1 About 3 times a week 0 0.0 About once a week 7 1.5 About once a month 0 0.0 Never 2 0.4 Left blank 311 67.9 130 Table 29: Questions relating to literacy Section D, Question 17, 18 and 19 Count Column N % 17. Time to read stories 15 minutes per day 38 8.3 5 minutes per day 54 11.8 15 minutes per week 42 9.2 5 minutes per week 7 1.5 Left blank 317 69.2 18. How assess pupil reading progress Ask pupils to read one by one 63 13.8 I assess pupils with a test 9 2.0 I hear pupils read individually 71 15.5 Muitos alunos nao sabem ler... 1 0.2 Left blank 314 68.6 19. How record pupil reading progress I correct pupils' exercise books during grade 53 11.6 I have a register of names and I record it 91 19.9 Left blank 314 68.6 Table 30: Questions relating to literacy Section D, Question 20 Count Column N % 20.i Pupils should A) take their books home B) not take their books home Response A 107 23.4 Response B 31 6.8 Left blank 320 69.9 20.ii A) It is not possible to practice reading at home B) Pupils should practice reading at home even if parents are illiterate Response A 8 1.7 Response B 132 28.8 Left blank 318 69.4 20.iii A) Year 1 pupils are too young to hand out books B) Year 1 pupils can learn to hand out books Response A 55 12.0 Response B 64 14.0 131 Left blank 339 74.0 20.iv A) Pupils should sit in rows facing the teacher B) It is necessary to change the way pupils sit according to activities Response A 32 7.0 Response B 99 21.6 Left blank 327 71.4 20.v A) Grade preparation is essential to ensure good management of grade B) Preparation does not make management easier Response A 123 26.9 Response B 9 2.0 Left blank 326 71.2 20.vi A) It is better for young pupils to learn how to read standing in front of the blackboard B) It is better for young pupils to learn how to read handling their own books since they can learn at their own pace Response A 55 12.0 Response B 76 16.6 Left blank 327 71.4 20.vii A) Independent practice is not useful B) Independent practice is important part of learning process Response A 12 2.6 Response B 116 25.3 Left blank 330 72.1 132 Annex 6 : Instrument for pupil data collection (Portuguese version) Diagnóstico de Capacidades de Leitura e Escrita Português Nome do inquiridor: Data da realização do inquérito: ____/____/20___ Hora do inquérito: Nome do Distrito: Nome da escola: Código da escola: __ __ __ __ __ __ Nome do aluno: Código do aluno: __ __ __ __ __ __/__ __/__ __ __ Data de nascimento do aluno: ____/_____/____ Grade do aluno: Idade do aluno: Sexo do aluno: Língua que o aluno fala em casa: Sumário da notação Etapa Nota Descrição Etapa Nota Descrição 1. Vocabulário Oral Número de respostas correctas (10) Secção 7: Leitura de palavras Número de palavras lidas CORRECTAMENTE em 60 segundos (30) 2: Compreensão oral de texto Número de respostas correctas (4) Tempo (se menos de 60 segundos) 3: Consciência Fonológica Número de respostas correctas (10) 8: Leitura fluente e compreensão de textos Número total de palavras lidas CORRECTAMENTE em 60 segundos (70) 4: Conceitos sobre materiais impressos Número de respostas correctas (10) Número total de respostas correctas às perguntas de compreensão (4) 5: Conhecimento sobre os sons das letras Número de sons de letras correctamente lidas em 60 segundos (100) 9: Escrita (nome) Escreve o nome CORRECTAMENTE(sim/não) Tempo a completar o exercício (se em menos 60 segundos) Escreve o apelido CORRECTAMENTE(sim/não) 6: Leitura de sílabas Número de sílabas lidas CORRECTAMENTE em 60 segundos (50 ) 9: Escrita (ditado) Numero de palavras escritas CORRECTAMENTE (10) Tempo (se menos de 60 segundos) 133 Maputo, Maio de 2017 Instruções Gerais: É importante estabelecer uma relação relaxada e de confiança com o aluno a ser avaliado, por exemplo, através de conversas sobre tópicos de interesse do aluno. O aluno deve entender esta avaliação como uma actividade que não conta para a decisão no final do trimestre ou do ano (tipo AC, AF, etc). Deve manter-se em estado de confiança, livre e dar as suas respostas como lhe convier. Deve encorajar o alunoa dar o máximo de si, sem pressionar. Informe ao aluno que o resultado desta actividade não será do conhecimento de ninguém: (ninguém, mesmo o teu professor não vai conhecer o resultado). É importante que o(a) inquiridor (a) leia SOMENTE as secções presentes nas caixas, em voz alta, calmamente e de forma clara para que o aluno possa compreender os exercícios. Após o término da avaliação, agradeça o aluno pelo seu tempo e esforço. Consentimento Verbal Olá! O meu nome é ____________________. Eu trabalho para a ADPP-Moçambique. Nós estamos a tentar perceber como éque os alunos aprendem a ler e escrever. Tu foste escolhido, aleatoriamente.  Nós gostaríamos de poder contar com a tua ajuda para isso. Mas, não precisas participar nesta pesquisa, caso não querias. A tua participação é voluntária.  Nós vamos fazer uma actividade de leitura. Eu vou pedir-te para fazeres várias actividades, por exemplo, leres em voz alta algumas letras, silabas, palavras e uma pequena história.  Usando este relógio, eu vou ver quanto tempo tu levas a ler.  Isto NÃO é uma ficha de avaliação e não vai influenciar as tuas notas da escola.  Eu vou fazer-te algumas perguntas com a língua falada em tua casa.  Para avaliarmos o que tu disseres/escreveres, vamos usar códigos. Assim, ninguém saberá que as respostas foram tuas.  Mais uma vez, gostaria de te lembrar que não precisas participar nestaactividade, se não quiseres. Uma vez que começarmos anossaactividade, não há problema se preferires não responder a alguma pergunta. Podemos começar? 134 Secção 1. Vocabulário Oral Você vai pedir ao aluno para mostrar a imagem que corresponde à palavra que vai ouvir. O inquiridor marca a palavra correspondente a imagem identificada como correta ou incorreta ou não respondida com um “X”, na coluna certa do cartão de palavras. Se o(a) aluno(a) hesitar durante a identificação da imagem, por mais de cinco segundos, ou faz um esforço para identificá-la por cinco segundos, o inquiridor deve pedir ao (a) aluno(a) para imagem seguinte, conforme o cartão de palavras. NOTA: Auto-correção - se o aluno deu uma resposta errada, mas corrigiu-a mais tarde (auto-correcção), circule o (X) na coluna marcada como resposta incorrecta e marque o “X” na coluna correcta. Eu vou dizer algumas palavras em Português e tu vais me mostrar as figuras/imagens correspondentes. Por exemplo, "lapis". Outro exemplo, "cadeira "... Bravo! Vamos lá! CARTÃO DE PALAVRAS Corpus Correcto Incorrecto Não respondeu 1 Afiador 2 Gato 3 Cadeira 4 Mão 5 Peixe 6 Lápis 7 Sapato 8 Enxada 9 Mesa 10 Galinha TOTAL Número total de respostas correctas: / 10 135 Secção 2: Compreensão oral Peça a concentração do aluno para a audição da estória que vai ler para ele. Diga: Aqui está um conto que quero ler para ti. Hei-de ler, pausadamente,duas vezes, e peço que tu prestes atenção, porque, depois, vais responder a algumas perguntas. Estás atento? Podemos começar, por favor? O Piloto Era uma vez um cão, que se chamava Piloto. O Piloto era muito amigo do João. Todos os dias de manhã cedo, o Piloto seguia o João à escola. O Piloto ficava sempre no portão da escola porque o professor não o deixava entrar na sala de aulas. Depois de teres ouvido o texto, responde às perguntas que se seguem: 1. De que fala o texto? (Resposta: O texto fala do cão/Piloto/ cão chamado Piloto/João.) Se o aluno disser uma das alternativas, considere a resposta correcta. Correcto Incorrecto Não respondeu 2. Quem são as personagens do texto? (Resposta: As personagens do texto são o Piloto e o João.) Se o aluno disser uma das personagens, considere a resposta correcta. Correcto Incorrecto Não respondeu 3. De manhã cedo, para onde ia o Piloto? (Resposta: De manhã cedo o Piloto acompanhava o amigo à escola.) Correcto Incorrecto Não respondeu 4. Por que o Piloto não entrava na sala de aulas? (Resposta: O Piloto não entrava na sala de aulas porque o professor não o deixava entrar na sala de aulas.) Correcto Incorrecto Não respondeu A. Tu já ouviste esta estória antes? Sim Não B. Número total de respostas correctas às perguntas de compreensão __________ 136 Secção 3: Consciência Fonológica Bravo! Vamos fazer mais um exercício. Tens nessa folha várias imagens. Em cada série de imagens terás que descobrir aquela cujo nome começa com o mesmo som da palavra que vou pronunciar (o inquiridor entrega a folha que contém as imagens ao aluno). Diga: Indica a imagem cujo nome tem o mesmo som no início das palavras que ouviste. Exemplo: Eu digo “carro”. Tu deves indicar-me uma imagem cuja palavra começa com o mesmo som que ouviste /k/. Resposta: o aluno indica /casa/. Nota: [o inquiridor deve pronunciar os nomes das três imagens, em cada linha, e o aluno vai identificar a imagem que começa com o mesmo som da palavra em causa]. 1. Cana [casa, pasta, pá] /k/ Correcto Incorrecto Não respondeu 2. Milho [borracha, mão, apagador] /m/ Correcto Incorrecto Não respondeu 3. Sal [camisa, afiador, sapato] /s/ Correcto Incorrecto Não respondeu 4.Bola [gato, banana, galinha] /b/ Correcto Incorrecto Não respondeu 5. Pilão [cão, enxada, pato] /p/ Correcto Incorrecto Não respondeu 6. Gelo [boi, panela, janela] /j/ Correcto Incorrecto Não respondeu 7. Galo [cabrito, gato, saia] /g/ Correcto Incorrecto Não respondeu 8. Família [faca, prato, panela] /f/ Correcto Incorrecto Não respondeu 9. Lápis [manga, laranja, borracha ] /l/ Correcto Incorrecto Não respondeu 10. Tinta [tomate, pasta, afiador] /t/ Correcto Incorrecto Não respondeu Número total de respostas correctas: / 10 137 Secção 4: Conceitos sobre materiais impressos Entregue o livro ao aluno(a), pegando o livro na posição vertical, com a dobra apontada ao aluno e a parte oposta virada para si. Assinale o resultado por cada passo efectuado com um X na caixinha. Bravo! Vamos fazer mais um exercício. Desta vez, vamos usar um livrinho de estórias.Podemos começar? Nota: para as questões 8 e 9, o inquiridor deve entregar um lápis ao aluno, para apontar a letra e/ou a palavra, conforme o caso. 1. Mostra-me a frente do livro. Correcto Incorrecto Não respondeu 2. Abre o livro na página onde começa a história. Correcto Incorrecto Não respondeu 3. Mostra-me onde devo começar a ler esta história. Correcto Incorrecto Não respondeu 4. Em que direcção se lê? Correcto Incorrecto Não respondeu 5. Quando eu termino de ler esta linha, onde vou para continuar a ler? Correcto Incorrecto Não respondeu 6. Em que página estás? Correcto Incorrecto Não respondeu 7. Agora passa para a página ‘5’. Correcto Incorrecto Não respondeu 8. Por favor, com este lápis, mostra-me uma letra. Correcto Incorrecto Não respondeu 9. Por favor, com este lápis,mostra-me uma palavra. Correcto Incorrecto Não respondeu 10. Mostra-me onde termina a história. Correcto Incorrecto Não respondeu Total Correcto: ____________ Total Incorrecto: ___________ Total Não respondido: _______ 138 Secção 5: Conhecimento sobre os sons das letras Passe ao aluno a página de letras e leia as seguintes instruções: Nesta página, estão todas as letras do abecedário. Por favor, me diz os SONS do maior número de letras que puderes . Por exemplo, o som desta letra é [aponte para J] é “J”. Vamos praticar: diz-me o som da letra [Aponte para n]: Percebeste o que vamos fazer? Quando eu disser “começar”, por favor, diz-me o som das letras da melhor maneira que puderes. Lê as letras ao longo da página, iniciando pela primeira. [Aponte para a primeira letra na linha depois do exemplo]. Eu vou ficar calado (a) a ouvir-te, mas, se precisares de alguma ajuda, podes pedir-me. Pronto? Começar Inicie o cronómetro quando o aluno pronunciar o som da primeira letra. Siga as letras com a sua caneta (na tabela abaixo) e marque claramente com uma barra (/), o som da letra que for pronunciado erradamente pelo aluno. Quando o aluno se corrigir, conte esta como correcta. Fique calado (a), com a excepção de situações como: se o aluno hesitar mais de 3 segundos, diga o som da letra, aponte para a próxima letra e diga: “Por favor, continua!”. Marque a letra, cuja resposta você deu ao aluno, como incorrecta. APÓS 60 SEGUNDOS marque a última letra lida com uma chaveta ( ] ). Diga: Obrigado! E passe para a secção seguinte. Regra para interromper o exercício:Se o aluno não fornecer nenhuma resposta correcta na primeira linha, diga:“Muito Obrigado!”, pare o exercício, marque no quadro abaixo e passe para o próximo exercício. L i h R s P E o N T (10) Se a criança não fornecer nenhuma resposta correcta na primeira linha, pare aqui. I e T d K T a d E D (20) H o E m U R j g R u (30) W R b Z i F m T S r (40) Q t C n P V f V a E (50) W S q A m C O t j P (60) E z E s O F h u A t (70) R G h B y I g M I L (80) J Q N o E K Y r P X (90) N A c D d V O J e N (100) Marque um X no quadradinho, se o aluno não deu nenhuma resposta correcta na primeira linha: 139 1. Caso o aluno pronuncie os sons de todas as letras, em menos de 60 segundos, por favor anote o número de segundos que o aluno levou a completar o exercício (Número de segundos): ______ 2. Anote o número de sons de letras CORRECTOS, pronunciados durante 60 segundos: _________ 3. Anote o número de sons de letras ERRADOS, pronunciados durante 60 segundos: __________ Secção 6: Leitura de sílabas Nesta página, estão algumas sílabas. Por favor, lê o maior número de sílabas que puderes . Por exemplo, esta sílaba [aponte para na] lê-se “na”. Percebeste o que vamos fazer? Quando eu disser “começar”, por favor, lê as sílabas da melhor maneira que puderes. Lê as silabas ao longo da página, iniciando pela primeira. [Aponte para a primeira sílaba]. Eu vou ficar calado (a) a ouvir-te, mas, se precisares de alguma ajuda, podes pedir-me. Pronto? Começar Inicie o cronómetro quando o aluno ler a primeira sílaba. Siga as sílabas com a sua caneta (no material do avaliador) e marque claramente com uma barra (/), a sílaba que for lida erradamente pelo aluno. Quando o aluno se corrigir, conte esta como correcta. Fique calado (a), com a excepção de situações como: se o aluno hesitar mais de 3 segundos, leia a silaba, aponte para a próxima silaba e diga: “Por favor, continua!”. Marque a sílaba,cuja resposta você deu ao aluno, como incorrecta. APÓS 60 SEGUNDOS marque a última sílaba lida com uma chaveta ( ] ). Diga: Obrigado! E passe para a secção seguinte. Regra para interromper o exercício:Se o aluno não fornecer nenhuma resposta correcta na primeira linha, diga“Muito Obrigado!”, pare o exercício, marque X no quadradinho abaixo (onde se lê: Marque um X se o aluno não deu nenhuma resposta correcta na primeira linha) e passe para a secção 9 “Escrita”. na ti ma lo ra go su ci po fa 10 Se a criança não fornecer nenhuma resposta correcta na primeira linha, pare aqui. ca gi bu ha yo si le ja ve pe 20 Fa Na Po Ti Ma Ci Go Ra Su Ca 30 Gi Bu Ha Ve Ja Yo Bu Xi Pe Ze 40 Qua go Wi Ye Ki qua wi ye ki Go 50 140 Marque um X se o aluno não deu nenhuma resposta correcta na primeira linha. 1. Caso o aluno leia todas as sílabas, em menos de 60 segundos, por favor anote o número de segundos que o aluno levou para completar o exercício (Número de segundos): ________ 2. Anote o número de sílabas CORRECTAMENTE lidas durante 60 segundos: _________ 3. Anote o número de sílabas ERRADAMENTE lidas durante 60 segundos: __________ Secção 7: Leitura de palavras Entregue ao aluno, o cartão de palavras e leia as seguintes instruções: Nesse cartão, estão algumas palavras. Por favor, lê em voz alta o maior número de palavras que puderes Inicie o cronómetro quando o aluno ler a primeira palavra. Siga as palavras com a sua caneta (na tabela abaixo) e marque claramente com uma barra (/), a palavra que for lida erradamente pelo aluno. Quando o aluno se corrigir, conte esta como correcta. Fique calado (a), com a excepção de situações como: se o aluno hesitar mais de 3 segundos, leia a palavra, aponte para a próxima palavra e diga: “Por favor, continua!”. Marque a palavra,cuja resposta você deu ao aluno, como incorrecta. O(a) aluno(a) começa a ler as palavras da lista, da esquerda para adireita. O inquiridor marca a palavra lida como correta ou incorreta ou não respondida com um “X”. Se o(a) aluno(a) hesitar durante a leitura, por mais de cinco segundos, ou faz um esforço para lê-la por cinco segundos, o inquiridor deve pedir ao (a) aluno(a) para ler a próxima palavra na lista. O inquiridor nunca deve corrigir a palavra dita pelo(a) aluno(a). Além disso, o inquiridor nunca deve ler a palavra correctamente para o(a) aluno(a). O inquiridor continua a perguntar ao(a) aluno(a) sobre as palavras na lista. APÓS 60 SEGUNDOS marque a última palavra lida com uma chaveta ( ] ). Diga: Obrigado! E passe para a secção seguinte. Regra para a interrupção: O inquiridor pára com a secção quando o(a) aluno(a) lê incorretamente ou não responde a cinco palavras seguidas e passa para a secção 9 “Escrita”. Contudo, se o aluno erra três palavras e em seguida, por exemplo, lê uma palavra correcta, o inquiridor continua com o exercício até que ele(a) leia cinco palavras seguidas incorretamente ou conclua o exercício. No final deve somar o número de “X” de cada coluna. 141 CARTÃO DE PALAVRAS Corpus Correcto Incorrecto Não respondeu 1 Pano 2 Gola 3 Mota 4 Casa 5 Sala 6 Lápis 7 Milho 8 Papel 9 Lixo 10 Copo 11 Faca 12 Bola 13 Doce 14 Fruta 15 Tenda 16 Banana 17 Laranja 18 Caderno 19 Chávena 20 Casaco 21 Macaco 22 Garrafa 23 Tijolo 24 Cenoura 25 Boneca 26 Tangerina 27 Alfinete 28 Professor 29 Bicicleta 30 Gafanhoto TOTAL 142 Secção 8: Leitura fluente e compreensão de textos Mostre a folha com o texto. Diga: Aqui está um conto que quero que me leias; Quando eu te disser ‘começa’, começas a ler o conto em voz alta, iniciando pela primeira palavra. Por favor, lê o título e o resto do texto, da esquerda para a direita (mostre à criança como fazer). [Diga:]Começa por ler cada palavra, se encontras uma palavra que não sabes ou não reconheces, eu digo-te como se le. Por favor, lê o melhor que saibas. Entendes o que quero que tu faças? Podemos começar, por favor? 1. Active o cronómetro quando o aluno começa a ler a primeira palavra. Se, depois de três segundos, o aluno não conseguir pronunciar a primeira palavra do texto, diga a palavra em voz alta, marque como incorrecto, e nesse instante inicia o cronómetro novamente. 2. Siga a leitura do aluno na sua cópia, e marque as palavras incorrectas com uma diagonal ( / ). Se o aluno fizer uma auto-correccao, marque a palavra com um circulo e considere-a correcta. 3. Ao fim de um minuto, assinale com uma chaveta vertical logo após a última palavra que o aluno tentou ler. ( ] ). 4. Diga: Muito obrigado! Agora vou te fazer algumas perguntas sobre o que estiveste a ler, podes te referir ao conto se quiseres. 5. Se ao fim de um minuto, o aluno apenas tiver lido a primeira linha, [Diga:] Muito obrigado pela tua colaboracao até aqui! Vamos passar para a secção da escrita. (Não é necessário seguir com as perguntas de compreensão, visto que o aluno não consegue ler). 6. Faça as perguntas de compreensão, se o aluno tiver lido o texto. 7. Quando o aluno terminar com as perguntas, diga: Muito obrigado! Agora passamos para a secção da escrita. Instruções para anotar as gradeificações dos alunos. 1. Conte o número total de palavras lidas CORRECTAMENTE em um minuto. 2. Anote o número total de respostas correctas às perguntas de compreensão. 3. Faça as perguntas de compreensão de texto: a) Se o aluno ler até a palavra 10 da história, faça a pergunta número 1. b) Se o aluno ler até a palavra 23 da história, faça as perguntas1 e 2. c) Se o aluno ler até a palavra 44 da história, faça as perguntas 1, 2 e 3. d) Se o aluno ler até a palavra 66 da história, faça todas as perguntas. 4. Caso, ao final de 60 segundos, o aluno ão ter lido nada, não é necessário fazer as perguntas. Passe para a secção 9 “Escrita”. 143 TEXTO & PERGUNTAS O pato 2 Um dia, a Maria passeou até ao rio. 10 No rio, a Maria viu um pato a nadar. 19 Ela aproximou-se do pato. 23 Ela viu a cabeça, asas e patas do pato. 32 A Maria lançou um pão ao rio. 39 O pato comeu o pão. 44 A Maria olhou para o pato a fazer quá 53 e batendo as asas na água. 59 O pato nadou até próximo da Maria. 66 A Maria ficou contente. 70 (Texto adaptado do livro de leitura da 3ª grade de Português modalidade de Ensino Bilingue) 1. Para onde foi a Maria? (Resposta: Ao rio) Correcto Incorrecto Não respondeu 2. De que é que a Maria se aproximou? (Resposta: pato) Correcto Incorrecto Não respondeu 3. O que é que o pato comeu? (Resposta: pão) Correcto Incorrecto Não respondeu 4.Por que é que o pato aproximou-se da Maria? (Resposta: gostou do pão/para a agradecer/satisfeito por ter sido alimentado, etc). Admita quaisquer respostas do aluno (desde que tenham relação com o conteúdo do texto) Correcto Incorrecto Não respondeu A. Você já leu essa história antes? Sim Não B. Número total de palavras lidas CORRECTAMENTE em um minuto _____________ C. Número total de respostas CORRECTAS às perguntas de compreensão ____________ 144 Secção 9: Escrita Comece esta secção com um agradecimento: Bravo! Estamos quase no fim da nossa actividade. Para terminar, peço para escreveres o teu nome nesta linha (indique a linha em causa, na folha de exercícios para o aluno). [Depois disso, o inquiridor passa para o ditado.] Diga ao aluno: Passo a ler 10 palavras para ti. Presta atenção, ouve muito bem o que leio. Agora, leio, enquanto tu escreves as palavras que te vou ditar aqui (indique o sítio onde o aluno deve escrever o que você ditar). Entendeste? Podemos começar? [O inquiridor deve ler, as palavras, duas vezes, pausadamente.]Se o aluno não consegue escrever as primeiras três palavras, termine o exercício com os agradecimentos [ver no último rectângulo]. Escrita do nome do aluno A. Escreveu o nome Correcto Incorrecto Não escreveu B. Escreveu o apelido Correcto Incorrecto Não escreveu DITADO 1. Pano 2. gola 3. mota 4. casa 5. papel 6.Banana 7. laranja 8. caderno 9. tijolo 10. alfinete A. Número total de palavras escritas CORRECTAMENTE_____________ B. Número total de palavras escritas incorrectamente________________ C. Número total de palavras não escritas________________ Termine esta parte da conversa com o aluno da seguinte maneira: Assim terminamos esta parte da actividade. Agora vais fazer outras atividades. 145 II. Questões para o aluno Faça as seguintes perguntas ao aluno. Pode encorajar e explicar com palavras mais simples, acessíveis as crianças, contudo não oriente a resposta. O aluno deve responder sempre que possível: só excecionalmente escolher “sem resposta.” Por favor, circule a resposta. 1. a) És membro de algum clube extraescolar? Se sim, a quantos clubes vais? 0 / 1 / 2 / 3 o mais / sem resposta (NOTA PARA O ENTREVISTADOR, Se respondeu não, passe para a questão 2) b) Com que frequência participas nos clubes extraescolares? nunca /de vez em quando / uma vez por semana / 2 vezes por semana / 3 ou mais vezes por semana / sem resposta c) Gostas de ir aos clubes? muito / um pouco / não / sem resposta d) Ir ao clube te ajuda com o trabalho/atividades escolares? muito / um pouco / não / sem resposta e) As tuas atividades melhoraram depois que começaste a ir aos clubes? (Tens mais facilidade, consegues fazer melhor os trabalhos de escola?) muito / um pouco / não / sem resposta 2. a) Alguma vez sentes fome na escola? sim / não / sem resposta b) Estás com fome agora? sim / não / sem resposta (Questões para alunos nas escolas que recebem comida) c) Recebes comida na escola sempre que vens? sim / não / sem resposta Se não, porquê? _________________________________________________________________________________ d) Todos os alunos recebem comida sempre que vêm? sim / não / sem resposta Se não, porquê?_________________________________________________________________________________ e) Recebeste comida na escola hoje? sim / não / sem resposta f) Recebeste comida na escola ontem? sim / não / sem resposta g) A comida na escola sabe bem? sim / não / sem resposta (Gostas da comida? A comida é gostosa ?) h) Se não, porquê? ______________________________________________________________________________ 3. Quantas refeições comes por dia em casa? pôr numero ____/ sem resposta (Quantas vezes recebes comida por dia em casa?) 4.a) Quantas refeições comeste ONTEM em casa (fora da escola)? pôr numero ____/ sem resposta 146 (NOTA PARA O ENTREVISTADOR Segundo a resposta, ajuda a criança a responder às questões seguintes; se não teve uma das refeições mencionadas, escreve “nada”. O objetivo e de saber o que a criança comeu fora da escola. b) O que mata-bichaste ONTEM ? __________________________________________________________________________ c) O que almoçaste ONTEM ? _____________________________________________________________________________ d) O que jantaste ONTEM ? _______________________________________________________________________________ e) O que mais comeste ONTEM? ___________________________________________________________________________ 5. Tens problemas a prestar atenção na sala de aula? muitas vezes / as vezes / não / sem resposta 6. a) Depois de fazer as necessidades (usar expressões familiares à criança para ajudar), o que fazes? (NOTA PARA O ENTREVISTADOR, NÃO LER: Na resposta, o aluno menciona lavar as mãos?) sim / não / sem resposta b) O que fazes antes de comer? (NOTA PARA O ENTREVISTADOR, NÃO LER Na resposta, o aluno menciona lavar as mãos?) sim /não / sem resposta 7. (Se não falou de lavar as mãos, introduza o assunto de lavagem de mãos). Por que é importante lavarmos as mãos? (NOTA PARA O ENTREVISTADOR, NÃO LER Na resposta, o aluno menciona doença / ou germes/micróbios?) sim/não/ sem resposta 8. O teu professor fala as vezes de nutrição, de o que deves comer para ter uma dieta (alimentação) equilibrada e saudável? muitas vezes / as vezes / não / sem resposta (NOTA PARA O ENTREVISTADOR, Se respondeu não, passe para a questão 11) 9. O que diz teu professor sobre o que deves comer para ter uma dieta (alimentação) equilibrada e saudável? ___________________________________________________________________________________________ 10. Já tomaste comprimidos desparasitantes este ano/nesta grade? sim /não / sem resposta (Desparasitaste este ano/nesta grade?) 147 III. Altura e peso do aluno Pergunte ao aluno se sabe o seu peso. Explique que gostaria de o pesar. O aluno deve descalçar os sapatos. Convide o aluno a subir para a balança e ajude-o a ver o seu peso (aproximando à casa decimal/0.1 kg) Peso do aluno ________._____ kg Pergunte ao aluno se sabe a sua altura. Explique que gostaria de o medir. Convide o aluno a colocar-se em pé contra a parede onde a fita métrica está posicionada. Verifique que os seus calcanhares e ombros estão bem contra a parede e que a coluna dorsal está direita. Usando um livro/cartão direito e bem horizontal, colocado em ângulo reto na parede, veja a altura do aluno aproximando ao centímetro). Altura do aluno ___________ cm Pergunte ao aluno se tem alguma pergunta sobre a entrevista que acaba de fazer. Depois de responder às eventuais questões, agradeça e despeça-se do aluno. Observações do entrevistador ____________________________________________________________________________________________________ Como medir o peso com precisão 1. Utilize uma balança digital. Coloque a balança em chão firme (por exemplo azulejos, cimento ou madeira). 2. Peça à criança para tirar os sapatos. 3. Peça à criança para se colocar na balança com os dois pés no centro. 4. Registe o peso com aproximação à casa decimal (por exemplo, 25.1 kg). Como medir a altura com precisão 1. Peça à criança para retirar os sapatos, roupa volumosa, e acessórios de cabelo que possam interferir com a medição. 2. Faça a medição em chão firme e plano e contra uma superfície plana, como uma parede lisa. 3. Peça à criança para se encostar à parede com os pés planos e juntos. Certifique-se que a criança tem as pernas direitas, os braços ao lado e os ombros ao mesmo nível. 4. Peça à criança para olhar em frente e certifique-se que a linha de visão está paralela ao chão. 5. Faça a medição enquanto a criança tem a cabeça, os ombros, as costas e os calcanhares encostados à parede e os pés planos no chão (ver imagem). Dependendo da forma do corpo da criança, talvez nem todas as partes do corpo toquem a parede. 6. Use um objeto plano, como um livro, para formar um ângulo reto com a parede e baixe o objeto até tocar na coroa da cabeça da criança. 7. Certifique-se que os seus olhos estão ao mesmo nível do objeto. 8. Faça uma marca onde o objeto toca a parede. Depois use uma fita métrica para medir a altura desde o chão até à marca. 9. Registe com precisão a altura com aproximação ao centímetro. Adaptado do site dos Centres for Disease Control and Prevention: http://www.cdc.gov/healthyweight/assessing/bmi/childrens_bmi/measuring_children.html Termine a conversa com o aluno da seguinte maneira: Assim terminamos as nossas actividades. Foi muito bom ter conversado contigo e espero que também tenhas gostado. Muito obrigado, menino(a)! Continuação de bons estudos! 148 Annex 7: Questionnaire for teacher survey Questionário para professores/as Dados pessoais e administrativos (a serem preenchidos na sua totalidade por favor) Nome da escola ________________________________ Código da escola ________________________ Distrito ______________________________________ Nome do professor _______________________ Grau administrativo N1 A N2 B N3 C N4 D N5 E Sexo Masculino A Feminino B Idade 20-29 A 30-39 B 40-49 C 50-59 D 60-69 E Há quanto tempo trabalha nesta escola? Menos de 1 ano A entre 1-3 anos B entre 4-6 anos C mais de 6 anos D Que grade leciona ? 1ª grade A 2ª grade B 3ª grade C Outra grade (por favor indique qual é a grade) ___________________________ D 149 Há quanto tempo leciona nesta grade? Menos de 1 ano A entre 1-3 anos B entre 4-6 anos C mais de 6 anos D Tem alguma responsabilidade específica no âmbito do projeto de alimentação escolar? (assinale a resposta correta / as respostas corretas - é possível selecionar uma ou mais respostas): Membro do comité de alimentação escolar A Gestor de armazém B Professor formado em nutrição e saúde C Outra responsabilidade (por favor indica qual é a tarefa) ___________________________ D A. Perguntas sobre o projeto 1. Na sua opinião a organização da distribuição da comida até agora tem sido Excelente A Boa B Nem boa nem má C Má D Muito má E 2. Há aspetos que poderiam ser melhorados? Sim A Não (passeà pergunta 4) B 3. Em caso afirmativo, quais são os aspetos que poderiam ser melhorados? _________________________________________________________________________________ 4. Na sua escola forem criados um ou mais clube(s)de aprendizagem extracurricular(es)? Sim A Não (passe à pergunta 10) B 5. Em caso afirmativo, como descreveria a assiduidade dos seus alunos nos clubes? Todos os alunos da minha turma participam regularmente no clube A A maioria dos alunos da minha turma participa regularmente no clube B Alguns dos alunos da minha turma participam regularmente no clube C A participação no clube é irregular D Eu não sei E 150 6. Sabe quais são as atividades oferecidas nos clubes? Sim A Não B 7. Pode dar mais detalhes sobre as atividades dos clubes? _________________________________________________________________________________ 8. Está a notar efeitos positivos dos clubes no comportamento/desempenho dos alunos na sala de aula? Sim A Não B 9. Pode dar mais detalhes? ___________________________________________________________________ 10. A escola recebeu material de apoio aos clubes de aprendizagem no âmbito do projeto de alimentação escolar? Sim A Não (passe à pergunta 16) B 11. Foi consultado sobre as necessidades/ a escolha de material de apoio aos clubes? Sim A Não B 12. Se recebeu material de apoio aos clubes, este está a ser utilizado? Sim A Não (passe à pergunta 16) B 13. Utilizou algum dos materiais de apoio aos clubes hoje ou ontem? Sim A Não (passe à pergunta 16) B 14. Em caso afirmativo, qual material/quais materiais utilizou? ______________________________________ 15. Na sua opinião, o material de apoio aos clubes é: Excelente A Boa B Nem boa nem má C Má D Muito má E 151 16. Na sua turma, têm agora acesso a material didático suficiente para o ensino-aprendizagem eficiente? Sim A Não B 17. Tem beneficiado de qualquer formação ou apoio profissional no âmbito do projeto de alimentação escolar? Sim A Não (passeàSecção B) B 18. Em caso afirmativo, de que tipo de formação ou apoio profissional beneficiou? ___________________________________________________________________________________ 19. Se beneficiou, a formação ou o apoio profissional foi: Excelente A Boa B Nem boa nem má C Má D Muito má E B. Perguntas sobre os alunos 1. Durante o dia escolar, os seus alunos parecem estar com fome: Frequentemente A De vez em quando B Nunca C 2. No caso dos alunos parecerem estar com fome, quais são os sinais que o fazem pensar que as crianças podem estar com fome? _________________________________________________________________________________ 3. Os alunos têm problemas de atenção na sala de aula? Frequentemente A De vez em quando B Nunca C 4. Têm dificuldades em aprender? Frequentemente A De vez em quando B Nunca C 152 5. No caso de os alunos terem problemas de atenção ou aprendizagem, na sua opinião por que têm problemas em aprender? ______________________________________________________________________________ 6. Depois da introdução do projeto de alimentação escolar tem notado qualquer diferença no comportamento dos alunos dentro da sala de aula? Sim A Não (passeà pergunta 8) B 7. Em caso afirmativo, quais são as diferenças que tem notado? __________________________________________________________________________________ 8. Depois da introdução do projeto de alimentação escolar tem notado qualquer diferença nas práticas de higiene dos alunos (lavar as mãos depois de ir à casa de banho, lavar as mãos antes de comer, etc.)? Sim A Não (passeà pergunta 10) B 9. Em caso afirmativo, quais são as diferenças que tem notado? ___________________________________________________________________________________ 10. Tem notado outras diferenças depois da introdução do projeto de alimentação escolar na sua escola? Sim A Não (passeàsecção C) B 11. Em caso afirmativo, quais são as diferenças notadas? ___________________________________________________________________________________ 153 Perguntas em relação à nutrição 1. Tem beneficiado de algum treinamento de nutrição? Sim A Não (passe à pergunta 7) B 2. Em caso afirmativo, em que ano? 2013 A 2014 B 2015 C 2016 D 2017 E 3. Dos aspetos abaixo, indique quais são os aspetos que aprendeu (é possível selecionar uma ou mais respostas): Higiene A Grupos de alimentos B Higiene e conservação dos alimentos C Lavagem das mãos D Dieta equilibrada E 4. Com quem partilha esta informação ? Alunos A Pais e encarregados B Sua família C Conselho de escola D Comunidade E Outros (por favor indique com quem) ___________________________________ F 5. Acha que a formação em nutrição criou mudanças na sua vida ou na vida dos seus alunos? Sim A Não (passe à pergunta 7) B 6. Em caso afirmativo, quais são as diferenças notadas? ___________________________________________________________________________________ 154 7. Indica 2 exemplos de elementos de cada grupo de alimentos: Grupo de alimento Exemplo 1 Exemplo 2 Alimentos base Alimentos ricos em proteína (alimentos para crescer) Alimentos ricos em vitaminas/minerais (alimentos para proteger) Alimentos ricos em energia (ricos em calorias) 8. Indica 3 passos de lavagem das mãos: 1. 2. 3. C. Perguntas em relação à literacia 1. Está a beneficiar da / participar na formação em literacia no âmbito do projeto de alimentação escolar? Sim A Não B 2. Faça corresponder as competências com as definições (há uma definição extra): Competência Definição i. Consciência fonológica ii. Fonética iii. Fluência iv. Vocabulário v. Compreensão Definições A. Capacidade de responder a questões sobre o texto lido B. Capacidade de dar a definição correta duma palavra C. Capacidade de ouvir os diferentes sons nas palavras D. Capacidade de ler palavras ou frases rapidamente, com precisão e expressão E. Capacidade de escrever respostas às perguntas durante uma prova F. Capacidade de ouvir o som k e escrever a letra k, ou de ler uma palavra nova, descodificando som a som). 155 3. No ciclo das lições, qual é o propósito de Eu faço? Assinale a caixa que contém a resposta correta. Manter os alunos calados enquanto o professor demonstra novas atividades A Os alunos praticam palavras novas ou trabalhos novos enquanto o professor mostra B Mostrar uma capacidade nova aos alunos para que eles saibam o que estão a aprender e o que devem fazer C Propiciar aos alunos a oportunidade de praticar como ler/dizer palavras novas D 4. No ciclo das lições, qual é o propósito de Nós fazemos? Assinale a caixa que contém a resposta correta. Propiciar aos alunos a oportunidade de praticar a capacidade nova com o apoio do professor A Propiciar aos alunos a oportunidade de trabalhar em pares ou grupos B Manter ocupados os alunos C Propiciar aos alunos a oportunidade de ouvir e depois copiar o professor várias vezes D 5. Durante Vocês fazem o professor deve… (Assinale a caixa que contém a resposta correta): Ler em voz alta para que os alunos individualmente, em pares ouem grupos possam seguir A Preparar a atividade seguinte B Circular na sala para manter o controlo e ler com os alunos C Circular na sala para observar e ajudar os alunos individualmente, em pares ou em grupos D 6. Qual é a língua em que ensina / a sua língua de instrução? (assinale a resposta correta / as respostas corretas) Português A Xichangana B Xirhonga C Outra (por favor indica qual é a língua) ___________________________________ D 7. Com que frequência lê livros ou outros materiais nesta língua para preparar o seu ensino? (assinale a resposta que descreve melhor a sua situação) Todos os dias A Mais ou menos uma vez por semana B Mais ou menos uma vez por mês C Nunca D 8. Com que frequência lê para o prazer? (assinale a resposta que descreve melhor a sua situação) Todos os dias A Mais ou menos uma vez por semana B Mais ou menos uma vez por mês C Nunca (passeà questão 10) D 156 9. Em que língua lê por prazer? (assinale a resposta que descreve melhor a sua situação) Português A Xichangana B Xirhonga C Outra (por favor indique qual é a língua) ___________________________________ D 10. Como avalia as suas próprias capacidades de leitura em Português? (escolha a resposta que descreve melhor a sua situação) Leio fluentemente e compreendo tudo A Leio fluentemente e tenho alguma dificuldade em compreender algumas palavras B Leio e tenho dificuldade em compreender muitas palavras C Leio raramente / nunca e tenho muita dificuldade em compreender as palavras D 11. Como avalia as suas próprias capacidades de leitura em Xichangana /Xirhonga? (assinale a resposta correta conforme a sua língua de ensino) Leio fluentemente e compreendo tudo A Leio fluentemente e tenho alguma dificuldade em compreender algumas palavras B Leio e tenho dificuldade em compreender muitas palavras C Leio raramente / nunca e tenho muita dificuldade em compreender as palavras D 12. Antes de começar a participar no projeto e literacia de FFK, recebeu: (assinale a resposta correta “sim” ou “não”;caso assinale “sim” indica se foi útil) a) Uma formação sobre os cinco componentes da literacia (Se não recebeu uma formação, passe para a alínea c) sim / não b) Se recebeu a formação, foi útil? sim / não c) Materiais para melhorar o ensino da leitura e escrita (Se não recebeu materiais, passe para a alínea e) sim / não d) Se recebeu materiais, foram úteis? sim / não e) Um guião para facilitar a planificação das aulas(Se não recebeu um guiãopasse para a pergunta 13) sim / não f) Se recebeu um guião, foi útil? sim / não 13. Nas suas aulas, usa o guia de sugestões metodológicas? Todos os dias A Mais ou menos uma vez por semana B Mais ou menos uma vez por mês C Nunca D 157 14. Nas suas aulas, usa cartões de letras, sílabas e palavras? Todos os dias A Mais ou menos uma vez por semana B Mais ou menos uma vez por mês C Nunca D 15. Nas suas aulas, usa materiais para o professor ler em voz alta(escolha a resposta que descreve melhor a situação)? Mais ou menos três vezes por semana A Mais ou menos uma vez por semana B Mais ou menos uma vez por mês C Nunca D 16. Nas suas aulas, os livros do aluno são usados (escolha a resposta que descreve melhor a situação): Todos os dias A Mais ou menos uma vez por semana B Mais ou menos uma vez por mês C Nunca D 17. Durante a semana escolar quanto tempo têm os seus alunos para ler individualmente as histórias descodificáveis no Livro do Aluno ou nos livrinhos descodificáveis(escolha a resposta que descreve melhor a situação): 15 minutos por semana A 5 minutes por semana B 15 minutos por dia C 5 minutos por dia D 18. Como avalia o progresso dos alunos na leitura (escolha a resposta que descreve melhor a situação)? Avalio os alunos por meio de um teste no final de cada trimestre A Ouço os alunos ler individualmente para ajudar e monitorar a leitura B Peço aos alunos para ler um por um durante a aula C Muitos alunos não sabem ler : eu só avalio os alunos que sabem ler D 158 19. Como regista o progresso na leitura dos alunos (escolha a resposta que descreve melhor a situação)? Tenho um registo com os nomes de todos e anoto as etapas atingidas A Tenho um registo com os nomes de todos e dou notas no final do trimestre B Corrijo os cadernos dos alunos durante a aula C Completo os registos de progresso no final do ano D 20. Indique a afirmação que corresponde melhor à sua opinião (escolhendo a resposta A ou B)21 : # Resposta A Minha opinião Resposta B Minha opinião i É importante deixar os alunos levarem os livros para casa Os alunos não devem levar os livros da escola para casa ii É impossível para os alunos praticar a leitura em casa porque a maioria dos pais não sabe ler Os alunos devem praticar a leitura em casa, mesmo que os pais não sejam alfabetizados iii Os alunos da primeira grade são demasiado jovens para ter a responsabilidade de distribuir os livros Os alunos da primeira grade podem aprender a distribuir os livros iv Os alunos devem ser sentados em filas voltadas para a frente para participar no ensino-aprendizagem É necessário mudar a forma como os alunos se sentam consoante as diferentes atividades. v A preparação das aulas é essencial para bem lidar com a gestão da grade A preparação das aulas não ajuda muito com a gestão da grade vi Os alunos jovens aprendem melhor a leitura em frente ao quadro Os alunos jovens aprendem melhor a leitura quando têm um livro nas suas mãos e podem aprender ao seu próprio ritmo vii A prática independente não é útil para os alunos jovens porque fazem erros Poder fazer erros durante a prática independente é uma parte importante da aprendizagem 21Secção adaptada do Learning Gains Study no âmbito do EGRA desenvolvido no Malawi por USAID Malawi e RTI, 2016 159 Quer acrescentar mais alguma informação? __________________________________________________________________________________________ Agrademos a sua disponibilidade e tempo para preencher este questionário. Espera-se que os resultados contribuam para ajudar os professores e os alunos que fazem parte do projetode alimentação escolar. 160 Annex 8: Questionnaire for school survey The first two pages are shown. The full instrument is available on request. ID 1.1. Distrito ___________________________________________ 1.2. Posto Administrativo ___________________________________________ 1.3. Nome da Escola ___________________________________________ 1.4 Codigo da Escola 1.6. Data da entrevista 1.7. Tempo do início Hrs: Min : MANHÃ 1 DE TARDE 2 1.8. Tempo do término Hrs: Min : MANHÃ 1 DE TARDE 2 1.9. 1 SIM 2 NÃO 1.10. Entrevista completa? 1 SIM 2 NÃO 1.11 Quantos Professores tinha a escola no inicio do ano 2017? 1.12 Quantos Professores tinha a Escola no final deste ano lectivo? 1.13 Quantas Turmas tem a escola? QUESTIONÁRIO DA PARTICIPANTE - 2017 Secção 1: Informações sobre o Participante ( DIRECTOR DA ESCOLA) Estudo base do Projecto Alimentação Escolar - Fase 2 Informação sobre a Escola Consentimento informado escrito providenciado? Esta parte do questionário deve ser respondida pelo Director da Escola ou Director Pedagógico da Escola Apenas O questionario deve ser respondido na Escola, a onde as respostas podem ser verificadas. 161 SECÇÃO 2: CONSTRUÇÃO 2.1.1 A Escola tem latrina? SIM 1 NÃO 2 2.1.2. Quantas latrinas tem a Escola? 2.1.3 Quantas destas latrinas estão funcionais? 1 Nao Sei 2 2.1.4 Quantas latrinas foram: Construidas pelo projecto? 1 Reabilitadas pelo projecto? 2 2.1.5 Quantas latrinas são: para meninos? 1 para meninas? 2 para ambos (meninas e meninos)? 3 para professores? 4 2.1.6 Existe uma separação marcada (com imagem, escrita) entre latrinas para meninos e meninas? SIM 1 NÃO 2 2.2.1 Quantos lavatórios das mãos tem a Escola? anexos a latrina separados da latrina 2.2.2 Os lavatorios tem água: sempre 1 maioria das vezes 2 az vezes 3 poucas vezes 4 nunca 5 2.2.3 Quantos lavatorios precisam manutenção? 2.3.1 Qual é a frequencia de limpeza das latrinas ? Diaria 1 Atenção! Uma latrina dupla e igual duas latrinas 162 Annex 9: Questionnaires for EPF survey 9a Trainers Nome da EPF____________________________________________ Data ______________________________ Nº Nome completo do formador Género M/F Nível Académico e curso Área de Formação / Disciplina Instituição de Formação Possui a formação psicopedagógica? sim/não Anos de experiencia no EP* Anos de experiencia na EPF* Categoria (por exemplo DN1) Responsabilidades 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 163 9b Supply and demand of trainers by discipline Levantamento de formadores nas EPFs Nome da EPF Por favor, antes de preencher a tabela, observe o exemplo de preenchimento em baixo. Meta Especi￾alizado Não especia￾lizado Metodologias de Ensino de Ciências Sociais Ciências Sociais ( História e Geografia ou História ou ainda Geografia) 0 Língua Bantu de Moç. e Met. de Educação Bilingue Línguas ( Português, Língua Bantu ou Linguística) 0 Psicopedagogia Psicopedagogia (pode ser: Ciências da Educação, ou Psicologia escolar (Educacional) ou Ensino Básico) 0 Introdução à Metodologia de Pesquisa -acção e TIC’s; Informática (Tecnologia Educativa ou Engenharia Informática) 0 Metodologias de Ensino de Educação Visual; Artes visuais, ou Cénicas ou Ensino de desenho ou Arte e Cultura 0 Noções Básicas de Construção, Manutenção e Produção Escolar Construções e Obras. Para a Produção escolar formado em Agro-Pecuária 0 Metodologias de Ensino de Língua Portuguesa Língua Portuguesa, (Pode ser Linguística ou Literatura) 0 Metodologias de Ensino de Ciências Naturais; Ciências Naturais, (pode ser em Biologia e Química ou Biologia) 0 Metodologia de Ensino de Ofícios Artes visuais, ou Cénicas ou Ensino de desenho ou Arte e Cultura 0 Metodologias de Ensino de Educação Musical Música (pode ser em Canto e Música ou Comunicação e Artes) 0 Metodologias de Ensino de Matemática Matemática (pode ser em Matemática e Física) 0 Metodologias de Ensino de Educação Moral e Cívica; Educação Moral e Cívica 0 Técnicas de Expressão em Línguas Língua Portuguesa, (pode ser Linguística ou Literatura) 0 Metodologias de Ensino de Educação Física; Educação Física e Desportos 0 Organização e Gestão Escolar Planificação, Administração e Gestão da Educação (pode ser: Psicopedagogia) 0 Inglês Inglês 0 0 0 0 Meta Especializado EXPLICAÇÃO DE PREENCHIMENTO DISCIPLINA NA EPF ESPECIALIDADE DO FORMADOR Formadores Experiência na escola primária Formadores com 5 anos de experiência na escola primária Formadores sem 5 anos de experiência na escola primária Na meta preencha nº de formadores necessários por cada disciplina. Nº de formadores Total OBSERVAÇÕES / COMENTÁRIOS Nesta coluna preencha o námero de formadores qualificados que leccionam a diciplina. No caso de uma disciplina ser leccionada por um formador não qualificado, este formador não conta, mas escreva uma observação na coluna de Observações / Comentários