1 BASELINE STUDY: FY 2015 USDA MCGOVERN-DOLE FOOD FOR EDUCATION HOME GROWN SCHOOL FEEDING PROGRAM IN RWANDA 2016-2020 July 22, 2016 FUNDED BY: McGovern-Dole International Food for Education and Child Nutrition Program Agreement number: FFE-696-2015/007-00 LED BY: AUTHORS: Peninnah Mukiri Amos Kaburu DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov 2 | Page Abbreviations ADRA Adventist Development and Relief Agency CSB+ Corn Soya Blend DEO District Education Officer ECD Early Child Development EGRA Early Grade Reading Assessment FAD Food Assistance Division FAO Food and Agricultural organizations FAS Foreign Agricultural Services HGSF Home Grown School Feeding HQ Headquarter KII Key informant interview MAD Minimum Acceptable Diet MGD McGovern-Dole MINAGRI Ministry of Agriculture and Animal Resources MINEDUC Ministry of Education MINALOC Ministry of Local Government MINISANTE Ministry of Health NISR National Institute of Statistics Rwanda P4P Purchase for Progress PTA Parent-teacher association RAB Rwanda Agriculture Board REB Rwanda Education Board SMC School management committee ToR Terms of Reference UN United Nations UNICEF United Nations Children’s Fund USAID United States Agency for International Development USD United States Dollars 3 | Page USDA United States Department of Agriculture WFP World Food Programme WV World Vision 4 | Page Summary of Findings WFP Rwanda has been awarded USD 25 million through the United States Department of Agriculture (USDA) McGovern-Dole program to support school feeding over the next five years (2016-2020) in support of the Government of Rwanda's Home Grown School Feeding (HGSF) program. The program will be implemented in four districts; Nyaruguru, Nyamagabe, Rutsiro and Karongi districts, where poverty and food insecurity is high. This baseline study was commissioned in order to establish indicator baseline data and information to allow for regular monitoring of activity outputs and performance indicators; form the foundation for the planned midterm and final evaluations; and provide a situation analysis. The study was conducted through desk research, school based surveys and 13 key informant interviews. The school survey was conducted with a sample of 40 schools drawn from a sampling frame of the 104 target schools. School data was collected from a sample of 40 structured observations; 240 teachers (P1-P6) from selected randomly selected classes; Early Grade Reading Assessment (EGRA) test with 1200 randomly selected P1-P3 students; 1200 randomly selected P4-P6 students; 40 head teachers; 39 storekeepers and cooks; 40 members of Parent Teachers Association (PTA); and 29 members of school management committee (SMC). Field work for this study was conducted between June 7, 2016 and June 28, 2016. The baseline study focused on indicators that could be measured before project implementation. Some indicators could not be measured because their definition is linked to the implementation of project activities (see Annex 1). Baseline values for each indicator measured against its corresponding target, as per the project document, are summarized in Table 1. Following the baseline study, it is essential that WFP reviews and realigns the targets. In preparation for the midterm and final evaluation, there is also a need for a strong program monitoring component that collects and compiles data from each of the beneficiary school and related activities on a regular basis. All indicators, including policy-related indicators, require specific project records. 5 | Page Table 1: Summary of Baseline Indicator Values Against Target Result Indicator Baseline Indicator Value Target Increased Access to Preventative Health Interventions Number of students receiving deworming medication(s) 75,749 127,650 Increased Access to Requisite Food Preparation and Storage Tools and Equipment Number of target schools with access to improved food preparation and storage equipment (kitchens, cook area, storerooms, stoves and kitchen utensils 10 104 Increased Student Enrolment Number of students enrolled in school receiving USDA assistance (female) 41,404 66,378 Number of students enrolled in school receiving USDA assistance (male) 40,954 61,272 Number of students enrolled in school receiving USDA assistance 82,358 127,650 Increased Access to Clean Water and Sanitation Services Number of schools using an improved water source 62 412 Number of schools with improved sanitary facilities 94 412 Increased knowledge of Safe Food Prep and Storage Practices Percent of cooks and storekeepers who can identify at least three safe food preparation and storage practices 92% 95% Improved Knowledge of Health and Hygiene Practices Percent of students who can identify at least three key health and hygiene practices (female) 47% 80% Percent of students who can identify at least three key health and hygiene practices (male) 48% 80% Improved Attentiveness Percentage of students in classroom identified as attentive by their teachers 60% 80% Improved Student Attendance Number of students regularly (80%) attending USDA supported classrooms/school (female) 37,964 53,102 Number of students regularly (80%) attending USDA supported classrooms/school (male) 37,512 49,017 6 | Page Number of students regularly (80%) attending USDA supported classrooms/school 75,522 127,650 Reduced Health￾Related Absences Percent of students who miss more than 10 school days per year due to illness (female) 8% 20% Percent of students who miss more than 10 school days per year due to illness (male) 7% 20% More Consistent Teacher Attendance Number of teachers in target schools who attend and teach school at least 90 percent of scheduled school days per school year 986 653 Improved Literacy of School-Age Children Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (female) 57% 90% Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (male) 42% 90% Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text 49% 90% 7 | Page Table of Contents ABBREVIATIONS .................................................................................................... 2 SUMMARY OF FINDINGS ........................................................................................ 4 TABLE OF CONTENTS.............................................................................................. 7 1.0 INTRODUCTION................................................................................................11 1.1 Objectives of the Baseline Survey .................................................................................................11 2.0 STUDY APPROACH AND METHODS ..................................................................13 3.0 SITUATION ANALYSIS ......................................................................................18 3.1 School Feeding Program ........................................................................................................................18 3.2 School Gardens .......................................................................................................................................23 3.3 Deworming ..............................................................................................................................................24 3.4 Other organisations working with the primary schools ....................................................................26 3.5 Education promotion Activities at community level........................................................................30 3.6 Teachers reward and recognition program.......................................................................................31 3.7 Health Clubs .........................................................................................................................................31 3.8 Education Policies ..............................................................................................................................31 4.0 RESULTS FOR LITERACY INDICATORS (RESULTS FRAMEWORK #1) .................32 MGD 1.1: Improved quality of literacy instruction ..................................................................................32 MGD 1.2: Improved Attentiveness..............................................................................................................38 MGD 1.3: Improved Student Attendance...................................................................................................39 5. 0 RESULTS FOR HEALTH INDICATORS (RESULTS FRAMEWORK #2) .................41 MGD 2.1 Improved Knowledge of Health and Hygiene Practices ..........................................................41 MGD 2.2 Increased knowledge of safe food preparation and storage practices...................................41 MGD 2.4 Increased Access to Clean Water and Sanitation.....................................................................43 MGD 2.5 Increased Access to Preventative Health Interventions..........................................................46 MGD 2.6 Increased Access to Requisite Food Prep and Storage Tools and Equipment.....................47 6.0 ADDITIONAL FINDINGS ...................................................................................49 8 | Page 6.1 Schools with an ECD Centre and feeding programs ..................................................................49 6.2 Involvement of students in writing competition .............................................................................49 6.3 Schools with Access to a Hand Washing Facility.............................................................................49 7.0 LIMITATIONS OF THE STUDY ...........................................................................51 8.0 CONCLUSIONS AND RECOMMENDATION .........................................................53 ANNEXES 54 Annex 1: Summary of baseline indicators against target........................................................................55 Annex 2: Sampling document .....................................................................................................................62 Annex 3: Additional EGRA findings (that include P1& P2) and tables..................................................67 Annex 4: References .....................................................................................................................................74 Annex 5: Result Framework ........................................................................................................................75 TABLES TABLE 1: SUMMARY OF BASELINE INDICATOR VALUES AGAINST TARGET............................................................5 TABLE 2: SUMMARY OF PRIMARY DATA COLLECTION ACTIVITY IN SCHOOL, TARGET RESPONDENT AND SAMPLE SIZE ....................................................................................................................................................................14 TABLE 3: SPONTANEOUS MENTIONS BY PTA, SMC AND HEAD TEACHERS OF AREAS OF SUPPORT .....................21 TABLE 4: SUMMARY OF ANTICIPATED CHALLENGES ............................................................................................22 TABLE 5: SUMMARY OF SCHOOL GARDEN ACREAGE IN SAMPLED SCHOOLS.........................................................23 TABLE 6: MAPPING OF STAKEHOLDERS WORKING WITH PRIMARY SCHOOLS BY INTERVENTION ......................27 TABLE 7: LIST OF EDUCATION PROMOTION ACTIVITIES THAT PTA AND SMC ARE ENGAGED IN AT COMMUNITY LEVEL ..........................................................................................................................................................30 TABLE 8: ANALYSIS OF LITERACY INDICATOR BY GENDER...................................................................................34 TABLE 9: ANALYSIS OF LITERACY INDICATOR BY DISTRICT..................................................................................34 TABLE 10: PERFORMANCE ON READING SOUNDS AND SYLLABLES ......................................................................35 TABLE 11: AVERAGE OVERALL P3 PERFORMANCES AGAINST THE EXPECTED READING LEVEL ...........................36 TABLE 12: FREQUENCY OF CARRYING OUT DEWORMING ACTIVITIES IN SAMPLED SCHOOLS AS REPORTED BY THE HEAD TEACHER. ..........................................................................................................................................47 TABLE 13: DISTRIBUTION OF THE NUMBER OF STUDENTS THAT PARTICIPATED IN THE LAST COMPETITION BY SEX AND CLASS ............................................................................................................................................49 TABLE 14: DETERMINING NUMBER OF SCHOOLS REQUIRED ...............................................................................63 TABLE 15: LIST OF SURVEYED SCHOOLS, BY DISTRICT..........................................................................................64 TABLE 16: LIST OF KEY INFORMANT INTERVIEWS CONDUCTED .........................................................................66 9 | Page TABLE 17: AVERAGE READING SCORES AGAINST THE NUMBER OF TASKS ............................................................71 TABLE 18: AVERAGE READING SCORES BY CLASS.................................................................................................71 TABLE 19: AVERAGE SOUNDS READ BY DISTRICT AND DISAGGREGATED BY CLASS ..............................................71 TABLE 20: AVERAGE WORDS IN A TEXT CORRECTLY READ PER MINUTE BY DISTRICT AND DISAGGREGATED BY CLASS...........................................................................................................................................................71 TABLE 21: AVERAGE INVENTED WORDS CORRECTLY READ PER MINUTE BY DISTRICT AND DISAGGREGATED BY CLASS...........................................................................................................................................................72 TABLE 22: AVERAGE FAMILIAR WORDS CORRECTLY READ PER MINUTE BY DISTRICT AND DISAGGREGATED BY CLASS...........................................................................................................................................................72 TABLE 23: AVERAGE SYLLABLES READ BY DISTRICT AND DISAGGREGATED BY CLASS .........................................72 TABLE 24: OVERALL READING LEVELS BY DISTRICT............................................................................................72 TABLE 25: AVERAGE SOUNDS READ BY DISTRICT AND DISAGGREGATED BY CLASS..............................................72 TABLE 26: AVERAGE SYLLABLES READ BY DISTRICT AND DISAGGREGATED BY CLASS .........................................73 TABLE 27: AVERAGE FAMILIAR WORDS CORRECTLY READ PER MINUTE BY DISTRICT AND DISAGGREGATED BY GENDER.......................................................................................................................................................73 TABLE 28: AVERAGE INVENTED WORDS CORRECTLY READ PER MINUTE BY DISTRICT AND DISAGGREGATED BY GENDER.......................................................................................................................................................73 TABLE 29: AVERAGE WORDS IN A TEXT CORRECTLY READ PER MINUTE BY DISTRICT AND DISAGGREGATED BY GENDER.......................................................................................................................................................73 TABLE 30: AVERAGE READING SCORES BY GENDER............................................................................................73 FIGURES Figure 1: Summary of foods grown in school gardens .........................................................................................23 Figure 2: P3 detailed literacy performance by district: ........................................................................................37 Figure 3: Proportion of Non-readers in P3 by Task...............................................................................................37 Figure 4: Percent distribution of student across the attentive ladder .................................................................38 Figure 5: Percent of students who can identify at least three key health and hygiene practices by gender and grade ...........................................................................................................................................................41 Figure 6: Percent of cooks who cooks and store keepers who could mention at least 3 safe and food and storage practices.........................................................................................................................................43 Figure 7: Safe water access in schools..................................................................................................................44 Figure 8: Status of sanitation in sampled schools ................................................................................................45 Figure 9: Summary of schools with specific features of this indicator ................................................................48 Figure 10: Schools’ Access to a hand washing facility ..........................................................................................50 Figure 11: Average number of sounds, syllables and words read per minute .....................................................67 Figure 12: Average Reading Scores by Class.........................................................................................................68 Figure 13: Average Reading Scores by District .....................................................................................................69 10 | Page Figure 14: Average Reading Scores by Gender.....................................................................................................70 Figure 15: Comparison of reading Sounds and Syllables by grade .......................................................................70 11 | Page 1.0 Introduction The United States Department of Agriculture (USDA) through the McGovern-Dole program is a major funding mechanism for school feeding worldwide. Managed by the Food Assistance Division (FAD) under the Foreign Agricultural Services (FAS), the McGovern-Dole program strategic objectives are to improve literacy and increase the use of health and dietary practices. The program provides US produced agricultural commodities and financial assistance, and supports capacity development and enhanced monitoring and reporting, with a key emphasis on sustainability and government ownership. WFP Rwanda has been awarded USD 25 million through the McGovern Dole program to support the home grown school feeding (HGSF) program over the next five years (2016- 2020), a program designed to complement in support of the Government of Rwanda’s school feeding program. WFP will implement the HGSF program in close collaboration with the Ministry of Education (MINEDUC), Ministry of Agriculture and Animal Resources (MINAGRI) and Ministry of Local Government (MINALOC). The HGSF vision is to create school feeding models and generate evidence on their impact that could eventually be adopted by the government at a national scale. WFP in collaboration with government partners at national, district, and sector levels plans to provide approximately 7 million hot lunches and 7.76 million mid-morning and mid￾afternoon porridge meals annually to over 83,000 students. The program will be implemented in Nyaruguru and Nyamagabe districts in the Southern Province, and in Rutsiro and Karongi districts in the Western Province, districts that are combating poverty and food insecurity. World Vision (WV) is a sub-grantee on this project, whose interventions and will focus on literacy, health, and WASH. 1.1 Objectives of the Baseline Survey The grant agreement for this program incorporates a project monitoring plan that specifies 70 performance indicators against which progress will be measured. WFP Rwanda commissioned this baseline study to establish baseline information against which the project will regularly measure its performance. Research objectives are: i. Establish indicator baseline data and information for use to regularly monitor activity outputs and performance indicators for lower level results; 12 | Page ii. Form the foundation for the planned midterm and final evaluations to measure performance indicators for McGovern-Dole's two strategic objectives as well as the highest level results that feed into the strategic objectives; iii. Provide a situational analysis before the project begins, and the context necessary for the midterm and final evaluations to assess the project's relevance, effectiveness, efficiency, sustainability and impact. 13 | Page 2.0 Study Approach and Methods The baseline survey was conducted in the four program districts namely Nyaruguru, Nyamagabe, Rutsiro and Karongi. The population of the study was 104 schools selected for the HGSF program focusing on the primary section. A combination of secondary and primary data were used to create a situational analysis of school feeding in Rwanda and provide reliable measures for the indicators. Primary data collection utilized both quantitative and qualitative approaches. Qualitative data was collected through 13 key informant interviews (KII) with stakeholders (see Annex 2 for a detailed list of KII). The quantitative approach was a school based survey targeting the following categories of respondents: • Head teacher • Teachers (grade 1-6) • Students in grade 1-3 (EGRA) • Students in grade 4-6 • Cooks • Store keepers • Parents teachers association (PTA) • School management committee (SMC) Sampling of schools The baseline survey was conducted in 40 schools, 10 schools in each district. The schools were selected using systematic random sampling. Refer to Annex 2 for a detailed write up on sample size determination and sampling procedures of each cohort at school. Summary of school activities, target and sample size Table 2 in the next page summarizes the achieved sample for each cohort in the 40 schools. 14 | Page Table 2: Summary of primary data collection activity in school, target respondent and sample size Primary data collection activity Summary description of the activity Target respondent Achieved sample School facility checklist These were structured observations using a checklist Observation of school facilities and infrastructure 40 observations Early Grade Reading assessment (EGRA) A standard test in Kinyarwanda which was conducted with students in P1-P3. The students were picked randomly. 10 interviews per class (5 boys; 5 girls) 30 interviews per school The test was adapted to Rwandan context before conducting the study Primary school children in P1- P3 1,200 students 400 in each class (P1, P2 and P3) Student survey (health indicators) A short survey on health indicators and learning environment targeting P1-P6. 10 interviews per class (5 boys; 5 girls) 60 interviews per school Primary school children in P 1 - P6 1,200 students in P4- P6 1,200 students P1-P3; these were the same students who did EGRA Teachers’ Survey Interviews with teachers in sampled P1- P6 classes for student survey. This was Class teacher of P 1-P6 240 15 | Page combined with student attendance records document review 6 interviews per school Interviews with Store Keeper and Cook A survey with the school store keeper and cook for those schools that had these staff Store keeper and cook 39 interviews (29 store keepers and 10 cooks) Head teacher interview and document review (teacher attendance register, classroom register, school enrollment, training records, records on awards and recognition, records on health clubs, records on harvest, deworming records) A survey with the head teacher and review of documents in school Head teacher 40 Interviews with school leadership A short survey with school management committee (SMC) member where they existed- 1 member per school School management committee 29 A short survey with a Parents Teachers Association (PTA) member. All schools had a PTA-1 member per school PTA representative 40 16 | Page Desk research: A desk review was conducted using documents sourced from WFP, USDA website, publically available documents from the Government of Rwanda and other online resources. Desk research focused on each indicator in the performance management plan. Analysis of secondary data for each indicator used the guidelines provided under Food for Progress and McGovern-Dole (MGD) indicators and definitions. One of the major challenges encountered is that the secondary data was not disaggregated by student/teacher /school to meet the requirements of the indicator as defined by USDA. In addition, the definitions of most indicators according to the guidelines required specific information not available in secondary data. For example, while statistics exist on types of toilets in schools, the MGD indicators require specific information on improved sanitary facilities indicator. This limited the use of secondary data to respond to the objectives. To respond to all the baseline indicators, primary data has been used. Design of data collection tools, translation and piloting: Ipsos designed the quantitative and qualitative tools used in the study in collaboration with WFP. All tools were developed in English and later translated in Kinyarwanda except the EGRA tool which was already in Kinyarwanda before adaptation. The tools were then programmed onto tablets to facilitate electronic data collection. All tools were pre-tested in four schools in the target districts before commencing data collection. Table 2 summarizes the tools used in this study. The standard EGRA tool in Kinyarwanda was shared by World Vision. It was recommended by education development partners in Rwanda that the tool go through an adaptation process to ensure that the students have no prior exposure to the content. Adaptation focused on replacing the content of the test but maintaining the standard. A one-day adaptation workshop was conducted at WFP using four Kinyarwanda teachers from P1-P3. World Vision and WFP representatives were present during this activity. Prior to the adaptation workshop, informational interviews were held with education representatives from the US Agency for International Development (USAID) and the Education Development Center (EDC). The adaptation workshop was moderated by the Ipsos team lead for education. A pre-test was carried out before using the tool. In addition to the EGRA tool, students were asked additional questions to capture their awareness of health benefits. Below is a list of all tools used in this study: • School facility checklist 17 | Page • EGRA • Student tool (P4-P6) • Teachers interview tool/ attendance records review • Store keeper/cook interview tool • Head teacher interview /document review • Key informant interview tool Data collection team and training A centralized 4 days training was conducted in Kigali. The teams conducted a pilot in their respective district before engaging in data collection. The school based survey was conducted by four teams of 40 enumerators and four supervisors. Data collection teams were organized around a district. Key informant interviews were conducted by three moderators. Data collection teams were locals, fluent in Kinyarwanda. Teams that administered the EGRA had teacher training background and experience in teaching lower primary students. Ethical considerations in the study • A courtesy call was made to the district education office and Mayor’s office before commencing the activity. • The head teacher consented to the study before any activity is undertaken in school. • The teacher introduced the enumerators to the class before student selection took place. • Participation was voluntary. Individual consent from student and teachers was sought. • A warm up session with selected students was conducted before participation. • All the enumerators were friendly and respectful to the students. • EGRA tests were coded and individual performance of student was not shared with the teachers. Field dates Data for the baseline study was collected from June 7, 2016 to June 28, 2016. (School based survey data collection ended on June 20, 2016). 18 | Page 3.0 Situation Analysis 3.1 School Feeding Program Data from the key informant interviews show that there are three school feeding programs in the country: One Cup of Milk per Child program by MINAGRI, National School Feeding program by MINEDUC, and the Home Grown School Feeding Program by WFP. Overview of One Cup of Milk per Child Program: The One Cup of Milk per Child was initiated six years ago by the Government of Rwanda through MINAGRI in collaboration with the Ministry of Finance and Economic Planning. The program targets school children in ECD and P1 to P3 in selected schools within selected districts e.g. Nyamagabe, Muhanga, Kamonyi, Rulindo, Gacyenke. The program reaches 80,000 children with one litre of milk per week, provided in two servings. The program is being expanded to reach more children. Overview of National School Feeding Program Targeting secondary school students (9 and 12 years’ basic education), since 2004, the National School Feeding program is a community driven project, to which parents contribute the food that their children eat at school. The government supports families that are not able to contribute any food due to their economic situation. The program covers 35 schools in Nyaruguru and 38 schools in Nyamagabe. The parents’ contribution is through cash or cash equivalent of firewood or foodstuff such as beans, wheat, maize sorghum or porridge flour. Overview of Home Grown School Feeding Program This school feeding program is part of WFP’s portfolio globally. The Home Grown School Feeding (HGSF) Program is one of the seven elements of WFP’s Country Programme, a five￾year country program in which UN agencies design a common program to support the government in building capacity in various areas, including food assistance i.e. designing and managing food assistance programs. The pilot program was implemented in 24 schools in two most food insecure districts - Nyamagabe and Nyaruguru in the south in 2014-2015. During the pilot, hot lunches were provided to approximately 25,000 primary school children. The meals provided consisted of maize meal, beans, salt and vegetable oil. The maize and beans used in the program were locally grown and procured in Rwanda. This program was preceded by another model of school feeding until 2013. 19 | Page The USDA-funded HGSF program will support 83,000 children in 104 schools, including schools in two additional districts of Karongi and Rutsiro in the Western province. The program will continue to provide a hot meal comprising of maize, beans, salt and oil to selected schools in Nyamagabe and Nyaruguru. The program in Karongi in Rutsiro will introduce a fortified hot porridge made of Corn Soya Blend- or CSB+ and sugar, provided to students at mid-day. The HGSF program will be implemented by WFP in partnership with World Vision, the government, and the community. WFP will be providing maize, beans, salt, oil, CSB+ and sugar. The maize and beans will be purchased locally through additional funds raised by WFP, while USDA will provide CSB+ and fortified vegetable oil. The program will also provide training in food handling, stocking, record keeping for storekeepers and teachers, and nutrition and hygiene training for the cooks. It will also construct and rehabilitate kitchens, latrines and water systems in target schools; this will be done in two phases starting with the schools in the West in 2016 before continuing to those in the South in 2017. Government involvement began during inception phase and included input in the selection criteria for targeting schools as well as on deciding the types of meals to be provided. The government is also involved in supporting the project through dedicated human resources (consultants) at the national and district level. The USDA capacity development funds are being used by WFP to support these project specific consultants within the government apparatus to allow them to effectively manage the program and create a skill base, which would then be transferred to permanent government officials over the course of the project period. These personnel have been hired on WFP’s request, with and on the basis of the agreement between WFP and the Government and dedicated solely to supporting the school meals activities. They are not government agency bureaucrats and are hired specifically for this project. The relevant positions include a project coordinator and assistant at MINEDUC, advisor at MINAGRI, and a coordinator in each district. Currently, Nyamagabe and Nyaruguru have HGSF coordinators. Karongi and Rutsiro are expected to have recruited people by August. The government will also support the program by providing office space to allow all the different stakeholders to work together, technical assistance through the steering (national level) and the technical (district level) committees that will be involved in the governance of the program. Tasked with ensuring that the key program targets are being met, the steering committee is made up of the permanent secretaries of MINEDUC, MINAGRI, MINALOC, Country Director of WFP, Rwanda Agriculture Board (RAB), Rwanda Education Board (REB), 20 | Page and World Vision. At the district level, the technical committee and districts coordinators will be involved in the day to day implementation and documentation of the program. Quarterly update meetings of the technical committee will be used to discuss challenges and ways forward with district leadership. MINAGRI will support the schools in setting up school gardens in its field. The support will involve training on how to design the school garden and also in providing selected seeds. Government involvement will also include support to project monitoring. While WFP and WVI have the clear mandate of monitoring project implementation, district, sector and cell officials have been sensitized on the program, are invested in ensuring that it runs efficiently and will support WFP and WVI field officers with their monitoring activities. The Government involvement will facilitate capacity enhancement in support of nationally owned school feeding program. World Vision will implement activities in the area of literacy and water, sanitation and hygiene (WASH), whereby their role will be primarily to boost the level of reading in primary school students from P1 to P3 and improve the knowledge of and access to WASH. These activities are aimed at promoting reading, training of teachers to enhance teacher effectiveness within the schools, provision of teaching aid and other reading materials for students, and setting up libraries. In the area of WASH, World Vision will support schools with infrastructure such as sufficient number of latrines that meet national standards, water systems, and hand washing stations. Besides infrastructure, World Vision will also carry out communication on messages aimed at promoting knowledge and good WASH practices. Community involvement in the HGSF program will primarily be through Parent Teacher Associations (PTAs) and the HGSF committee made up of parents, teachers, store keepers, and cooks. The committee will be responsible for implementation of the program at the school level. The community will also be involved through the program’s engagement with small holder farmers. Through the Purchase for Progress (P4P) program, small holder farmers will be organized into cooperatives within the district and their capacity to produce the food required for the school feeding will be enhanced through trainings on financial literacy, post￾harvest handling and entrepreneurship. Finally, to enable the schools to cook the daily meals, the community will also provide firewood and labour. Sampled Schools with an ongoing school feeding program The baseline study sought to establish if the target schools have an ongoing school feeding program. Out of the 40 schools covered in this study, two schools had an ongoing school feeding program. One school was under the “One Cup of Milk per Child Program”. The other school was one of the 24 schools that participated in the WFP school feeding pilot program in 21 | Page Nyamagabe. Although, it had a school feeding program for students in secondary school, at the end of the WFP program, there was a gap for primary children from poor households that come from far. The head teacher set up a local arrangement, whereby the vulnerable children are provided with meals at school. The meals are not provided on daily basis but when the school can afford extra food. WFP and the government collaborated very closely to select the areas of implementation. Although the selection criteria for schools to be supported by the HGSF program were stringently applied to ensure no duplication of efforts, one school out of the 40 sampled schools ended-up with both the one cup of milk per child and HGSF programs. However, since the one cup of milk per child only provides a liter of milk in two servings over the course of a week to students in grades 1-3, it is seen as complementary to the mid-morning porridge. The second school had meals only for students in grades 7-12 i.e. no primary school students were receiving meals. School management support for the school feeding program All the head teachers and school management (PTA and school management committee) who were interviewed affirmed that they would support the school feeding program if introduced. According to the PTA members the program was perceived to have the following benefits: • Supports improvements in school enrollment and retention • Boosts nutrition • Improves learning • Assists parents who are unable to feed their children The sampled schools’ leadership were asked for their opinion on ways the school community (management and parents) would contribute to make such a program successful. Table 3 outlines areas of support mentioned by PTA, SCM and head teachers. The key among the list being provision of fuel, payment of cooks and building of the cooking facilities. The intended support by the school community indicates willingness to be involved and a strong sense of ownership of the program. Although nearly all sampled schools had a school garden, it was not mentioned as a resource that could be utilized to support school feeding. Table 3: Spontaneous mentions by PTA, SMC and head teachers of areas of support PTA School Management Committee Head teachers Base 40 23 40 22 | Page Provision of fuel 67% 57% 38% Payment of cooks 49% 25% 27% Building of cooking facilities 49% 46% 14% Provision of labour 38% 11% 19% Provision of food 23% 4% Buying utensils 23% 36% 3% Mobilize parents to support the program 8% 18% Provision of water 3% 4% Come up with community work 3% 4% Management of the program 11% Source: Interviews with school leadership The baseline study also sought to assess the challenges based on the previous program in the pilot districts. According to the education officials and program participants the key challenges are highlighted as follows: • Perceived discrimination of some grades and some schools. Targeting only a section of a school (grade 1 to 6) and excluding pre-school and secondary school grades or targeting some schools and excluding others within the same sector or district that face the same challenges is perceived to be unfair. Teachers fear that this could potentially lead to the relocation of students to the HGSF program schools, with a feeding program which is likely to resulting in some schools being over-crowded others being abandoned. • Not including the parents’ contributions to food provision may have an impact on the sustainability of the HGSF school feeding program. The schools’ leadership were asked for their opinion of the expected challenges when implementing a school feeding program to shed light on some of the priority areas when implementing the intervention. Table 4 outlines the list of these challenges. These are based on perceptions by the school leadership. A review of the facilities that had been done by WFP revealed that 46 percent of the target schools did not have storage and cooking facilities. Only 10 of the 40 sampled school had a cook in place. Although majority of the schools had a water source, supply was not always reliable, validating the concern of school leadership on access to water. Table 4: Summary of anticipated challenges PTA SCM Head Teachers Base 40 29 40 Lack of money for the food 85% 79% 48% Lack of utensils 28% 34% 48% Lack of cooking area 15% 14% 25% 23 | Page Lack of storage 15% 17% 15% Lack of water 13% 14% 10% Lack of cooks 10% 24% 23% Uncooperative parents because they don’t value the program 8% 3% 3% Lack of security for the food 3% 7% 8% Other challenges (below 5% listed below) 3% 11% Other challenges specified • Lack of fuel • Poor roads(Accessibility of schools) • Inadequate time ( head teachers combining management of the program with their existing roles) • Poor management of the program 3.2 School Gardens WFP intends to establish and maintain school gardens in the target schools. Interviews with the head teachers established that 35 out of 40 schools (88 percent) had a school garden. All schools except one had part of their land under cultivation for food crops. The size of the land varied from one school to the other. However, it was clear that schools have access to land resource. Table 5: Summary of school garden acreage in sampled schools Size of land Number of schools % 20 acres and below 17 43% 21-40 acres 6 15% Above 40 acres 12 30% No garden 5 13% Total 40 100% Note: Size was based on estimation not actual measurements As shown in Figure 1, vegetables are the main food crops grown in the school gardens and a limited variety of crops are grown in these gardens. Informal discussions with the head teachers during the interview revealed that the schools are limited in land utilization due to reliance on rain fed agriculture. Figure 1: Summary of foods grown in school gardens 24 | Page Note: Base is 34 because one school was not growing any food crops The baseline study sought to assess the estimated yield of each food crop grown based on the last harvest. Schools do not maintain records on the school garden activities. Therefore, there were no reliable records obtained from the yield estimates. From the discussions with the head teachers, it was noted that the harvest is not substantial compared to the claimed land size of the garden. According to the head teachers, in more than half of the 34 schools that were growing crops, the food produced in the school gardens do not benefit the students. The food crops are either sold (56 percent) or consumed by teachers (15 percent). It was established that in three out of 34 schools growing food crops students carried home the food crops produced in school gardens. 3.3 Deworming The current school based deworming activities in the country, including the four target districts, are carried out regularly (twice a year) by Ministry of Health (MINISANTE). The activity is carried out through health centers in the district under the supervision of the District Education and Health Officers. First, a record of all students (for the age group being targeted) is taken from the register, then nurses from the nearby health centers visit the school to administer the deworming medication, starting with those in the morning shift followed by those in the afternoon shift. In each school, the exercise takes about three days to a week to ensure that those who are absent are covered, before the health team moves to the next school. Although it is supposed to be a blanket intervention, there appear to be some discrepancies in 3% 6% 6% 24% 24% 26% 29% 35% 56% Sweet potatoes Wheat Mushroom Potatoes Bananas Carrots Maize Beans Vegetables Base: 34 25 | Page data provided by MINISANTE and the districts. In 2015, 89 primary schools in Nyaruguru and 77 in Nyamagabe were targeted. The community health mobilizers are used to sensitize the community about the exercises as well as inform them about the benefits of the medication. Awareness is raised through parents through their children before the exercise begins and using community platforms. Deworming is a component of the WFP program and the plan is to support MINISANTE through advocacy and awareness creation activities. While the ministry covers the entire country, the advocacy and awareness activities by WFP will be limited to its four program districts. Challenges faced by deworming in schools Based on the key informant interviews, challenges faced by schools include religious beliefs, (where religion is opposed to conventional medicine) and misconception associated with medication given for deworming. Key informants revealed that due to these issues some parents keep the children away from school during the period of the activity to avoid being dewormed. Sometimes children refuse them, therefore, he/she abstains himself/herself from coming to school during that whole week and he/she says it is the parents who have done that. That is the reason we involve the community health mobilizers. That way they can’t miss school within that activity, since community health mobilizers are very close to people*. DEO *Paraphrased to capture meaning Suggestions for improvement Related to the challenge above, there is the need for more sensitization and raising awareness around deworming prior to any deworming activity so that parents are well informed. Although findings reveal that WFP will be involved in advocacy and awareness creation, the scope will be limited to the four HGSF program target districts. “I think that if one thing can be improved, it is communications, ---on time… “Yes! and to explain it to the community and everyone know his part and do it, if it is the head of village he has to know it in order to inform people from this village. If it is that one of cell or if it is a teacher, director of a school the health centre…they have to know it before and make sure that all material needed are available because it is him the 1st implementer who has to take the 1st step (that of health centre) and others follow him, what I’m saying is that they have to improve the communication.” DEO 26 | Page 3.4 Other organisations working with the primary schools A review of the materials and discussions with key informants and head teachers established that there are a large number of players working with schools in different areas. Table 6 maps these stakeholders by intervention. WFP is the Government of Rwanda’s primary partner for all school feeding activities. The project was carefully designed and schools selected to ensure that no duplication of efforts takes place. If at all, there would be only complementary activities. In addition, the HGSF programme has established a Steering Committee and Technical Committee and the HGSF partners are members of the Education Sector Working Group, the Education Development Partners Group and the Agriculture Sector Working Group. 27 | Page Table 6: Mapping of stakeholders working with primary schools by intervention Program Organization Activities District Some of the schools where activities are conducted School feeding (Although the pilot program ended in December, these two schools still feel that WFP is current player) World Food Programme (WFP) Providing food and utensils Nyamagabe Rugogwe GS Nyaruguru Ruhinga PS Other nutrition activities ADENYA Providing mushrooms seeds Nyaruguru Ruhinga PS Sanitation (water and toilets) Grandshule planning Construction of toilet Karongi EP Gitanga World Vision Provision of hand washing materials called Kandangira ukarabe Nyamagabe Rugogwe GS Kiyumba PS Kibirizi PS Construction of toilets Nyamagabe Rugogwe GS REMA Provision of hand washing system called Kandagira ukarabe (Tread-on and wash), water tanks, whoes. Nyaruguru Ruheru GS PEPAPS Provision of water Nyaruguru Ruhinga PS Kiyonza GS EP Ruhororo Construction of toilets Nyaruguru Fugi GS Kiyonza GS Concern Provision of hand-wash system called Kandagira ukarabe (tread on and wash) Nyaruguru Kimina PS Health Poverty Action (HPA) Construction of toilets and room for girls Nyaruguru Kivuru PS EP Ruhororo 28 | Page School governance Concern Training on school management Nyaruguru Nyamagambe Zirambi PS Ruheru GS Kimina PS Rugerero GS Kiyonza GS Kivuru PS EP Gitanga Ep Kinyami GS Ngoma Training PTA members Nyamagabe Rugogwe GS Kiraro Protestant GS Training teachers Nyamagabe Nkore PS Training school administrators Nyamagabe Masagara PS Action Aid Training teachers and PTA Nyaruguru Ruheru GS VSO Training administrative staff on administration methodology Nyaruguru Ruheru GS ADRA Training of school administrator Karongi EP Kinyovu GS Shoba Provision of school materials, textbooks, book Ministry of Education Provision of books and text books Rutsiro EP Mwufe GS Kivumu EP Rutsiro REB Provision of books and text books Rutsiro EP Nyakarera Concern Provision of books Nyamagabe Kiyumba PS Nyampinga Provision of books Nyaruguru Kiyonza GS EDC Provision of books and text books Rutsiro EP Umubano 29 | Page HDP (Health Development and Performance) Training on sexual reproductive health Nyaruguru Kiyonza GS Renovation/construction of infrastructure in school e.g. classes, kitchens, stores World Food Programme (WFP) Construction of kitchen and store Nyaruguru Ruhinga PS Renovation of store and kitchen Nyaruguru Kivuru PS Construction of the Kitchen Nyaruguru EP Ruhororo World vision Construction of classrooms Nyamagabe Baro PS Rugogwe GS Kiyumba PS EP Kamegeri Action Aid Construction of new class rooms and new closed water holes Nyaruguru Gakaranka PS Construction of three new classrooms Nyaruguru Ruheru GS Langenlonshein Construction of classrooms Rutsiro EP Mpingamabuye Catholic church Construction of classrooms Nyamagabe Kibirizi PS District Construction of new class rooms Nyaruguru Kiyonza GS Training of teachers Concern Training on how to make a new curriculum Nyamagabe Kiyumba PS Training on literacy Nyaruguru Kivuru PS VSO Training on administration, teaching and learning process Nyaruguru Ruheru GS Training on teaching tools Nyaruguru Ruhinga PS 30 | Page UNFPA Comprehensive sexuality education Nyaruguru Kiyonza GS USAID Training on how to make a new curriculum Nyamagabe Baro PS Health education Ministry of Health Reproductive health for girls Nyaruguru Kimina PS HPA Sexual and reproductive health training Nyaruguru Kivuru PS Concern kandagira ukarabe Karongi EP Gitanga HDP (Health Development and Performance) Training on sexual and reproductive health Nyaruguru Zirambi PS Ruheru GS Rugerero GS Kiyonza GS Fugi GS Village health Training teachers and students in health Nyamagabe Rugogwe GS 3.5 Education promotion Activities at community level The program was keen to learn if the school leaders are involved in education promotion activities at the community level. The PTA and SMC were asked about education promotion activities they engage in outside the school. Table 7 summarizes the spontaneous mention of these activities. The main role that the PTA and SMC engage is in mobilizing parents to enroll children and increasing sensitization on girl child education. Members of SMC seem to be engaged in community libraries. Table 7: List of education promotion activities that PTA and SMC are engaged in at community level PTA SMC Base (those that are engaged in education promotion activities) 26 16 Following/encouraging parents to take children to school 62% 56% Sensitizing families and communities to promote girl child education 50% 81% Reading clubs 27% 31% Recognition of high performers 27% 38% Establishing community libraries 12% 50% Education to parents on technology 8% **Base is very small. List is interpreted qualitatively 31 | Page 3.6 Teachers reward and recognition program Out of all the 40 schools visited, all except two reported to have an existing teacher reward and recognition scheme. The scheme focuses on incentivizing teachers based on academic performance of the students. However, the scheme does not appear to be active in all schools. When asked how many teachers were awarded in 2015, 12 schools did not have any awardee and only 46 teachers received some award in 2015. 3.7 Health Clubs Thirty-five percent (35 percent) of the head teachers of visited schools reported that there was an existing Health Club in the school. Children in the Health Clubs are mainly involved in the training of health matters after which they sensitize other children on health issues. Most schools did not have any recent documentation verifying the membership. The data was used as it is with clarification from the head teacher or teacher involved. It was estimated that health clubs in the 14 schools had 658 student members; 360 girls and 298 boys. The average membership in a school is 47 students; 26 girls and 21 boys. 3.8 Education Policies Currently the only school feeding related policy is the national school feeding policy which is in the initial stages of formulation and is being drafted by MINEDUC in partnership with MINAGRI, with technical support from WFP. The objective of the policy is to ensure the effective and efficient implementation of a national school feeding program. This includes, but is not limited to, providing regulations, setting standard for service delivery, establishing appropriate institutions, and strengthening the governance, partnerships and multi-sectoral coordination of school feeding. 32 | Page 4.0 Results for Literacy Indicators (Results Framework #1) This section presents findings of the baseline literacy indicators. Not all indicators could be measured at baseline because they are based on program activities (see Annex 1). These will be tracked during the program implementation. MGD 1.1: Improved quality of literacy instruction Indicator Baseline Target Number of teachers in target schools who attend and teach at least 90% of scheduled school days per school year 986 653 Data for this indicator was collected through a review of teachers’ attendance records in the 40 sampled schools and then estimated for target schools. The records indicated if the teacher was present or not. These records did not specify if the teacher was present for the whole day or part of the day. Each teacher’s record was reviewed for the whole year in 2015. Where the teacher had taught for less than one year in the school, the base was the total days the teacher was in the school. Although the indicator requires both attendance and teaching, there were no records on class attendance. It is expected that teachers in school attend classes. Therefore, this baseline survey assumed that the numbers of days a teacher was in attendance is equivalent to the number of days taught. A total of 543 teachers’ records in the sampled schools were reviewed for the year 2015. At baseline, 68 percent of the teachers attended and taught at least 90 percent of scheduled school days in 2015 school year. Male teacher attendance was at 80 percent compared to female at 57 percent. According to DEOs in the four districts, the target schools have 1450 teachers. It can therefore be estimated that 986 teachers, attended and taught at least 90 percent of scheduled school days in 2015 school year. Indicator Baseline Target Percentage of students who by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. 49% 90% Percentage of students who by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (male). 42% 90% Percentage of students who by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (female). 57% 90% 33 | Page This indicator looks at the proportion of those children in P3 who have acquired the basic literacy skills. This indicator was measured using the scores of the literacy on the EGRA test. All the students were assessed using the same standardized procedures that allows for comparisons of the scores and therefore periodic monitoring to enable comparison with the base value. On each of the task, the learners were presented with 100 phonemes and syllables, 50 invented words and short familiar words, and given a short text of 66 words to read. They were assessed on their ability to read and also comprehend the text read. The students were given one minute to read as many words as possible at each of the five levels of the assessment test, which included: 1. Phonemic awareness (understanding the sounds letters making words in a particular language, naming the beginning sound of a word); the learners were presented with 100 letter sounds in Kinyarwanda. 2. Reading syllables (a consonant and a vowel together—e.g. ba); the learners were presented with 100 commonly found Kinyarwanda syllables (combination of the phonemes of consonants and vowels). 3. Reading short, familiar words; these are words commonly used in Kinyarwanda. The learners were presented with 50 words. 4. Reading invented words (a way to assess decoding skills). Invented words are words that lack meaning in a given language but follow the same morphological structure of the same language that include syllable formation and how syllables combine to make words. Invented words assess whether the children have acquired automaticity (ability to read any word in a given language). That means that, for a child who has mastered the principles of word formation in a given language, they would read any word in that language regardless of whether that word is new. This task presented such 50 words to the learners. 5. Reading words in a text. This is the measure of reading fluency. Fluency in reading is important in helping learners to comprehend facts. For fluency to occur, a child should read without skipping or omitting words in a text. Furthermore, the child should read a certain threshold of words. In addition, the child should not read words like a string of sentences and should not replace words in a text. Often, children who cannot read fluently will struggle to comprehend the facts in the text although the vice versa may not necessarily happen. The fluency reading was administered by giving the children a text in prose of 65 words and they were given a minute to read the text. 34 | Page This indicator was specific to the percentages of the children who could read a text and comprehend at the end of the grade of two. Since the baseline study was conducted in the middle of an academic year, the results present the scores for the students currently enrolled in grade 3, although learners in grades one and two also undertook the assessment. This indicator was measured using the last task on the EGRA test that involves reading words in a text. This is a composite indicator that involved those students who were able to not only read but also comprehend the inferred meaning from the text. This was analyzed by focusing on the proportion of the children who were able to answer the third comprehension question.. Overall, 49 percent of the children by the end of grade 2 can fluently read and understand the meaning of the same grade level text. These are the learners who were able to read and answer a question that required inferring meaning from the text. When comparisons are made across the two genders, girls outperform boys. On average, 57 percent of the girls read and comprehended compared to 42 percent of the boys. This was the situation in all the districts where the learners were tested. The analysis by gender are presented in Table 8 below. Table 8: Analysis of literacy indicator by gender Gender N % Boys 83 42% Girls 114 57% Total 197 49% Base: 200 children per gender As shown in Table 9 below, learning levels and outcomes are highest in Nyamagabe (65 percent), followed by Karongi (62 percent), Rutsiro at 37 percent and least in Nyaruguru (33 percent). The proportion who are competent in reading and comprehending is almost twice in Nyamagabe (65 percent) than in Nyaruguru (33 percent). In Nyamagabe and Karongi, more than a half of the learners can read a grade 2 text and comprehend an inferred question. On the other hand, 67 percent of the P3 students in Nyaruguru and 63 percent of the children in Rutsiro cannot read and comprehend a P2 text. Table 9: Analysis of literacy indicator by district District N % Nyamagabe 65 65% Karongi 62 62% Nyaruguru 33 33% Rutsiro 37 37% Total 197 49% Base: 100 children per district 35 | Page How learners in Grade 3 performed in the other EGRA Tasks The EGRA test also has other tasks. The tasks included the letter sounds, syllables, and familiar word reading and invented word reading as discussed earlier on. REB and the Language, Literacy and Learning (L3) initiative have been working on developing literacy-reading benchmarks. However, these benchmarks are only applicable to the words read and not specific on whether they are individual words or words in a text. The following facts emerge from these findings: 1. Even in the absence of benchmarks for the other literacy skills in Kinyarwanda, P3 students are able to read 16 letter sounds on average (per minute), decode 46 syllables per minute, decode 15 correctly invented words (per minute) and correctly read 21 familiar words per minute. This demonstrates the following 1. The P3 learners are struggling to read letter sounds compared to the syllables. Phonemic awareness is low compared to decoding syllables. However, it can be speculated that the children are struggling to read the sounds based on the nature of the Kinyarwanda language. Although the foundation of literacy is the grasping of sounds, this seems not to be the case with the target learners. It is evident that learners are struggling to read the letter sounds. In fact, the scores in syllable reading are higher than the scores in letter sound reading. This could be perhaps due to the nature of reading the sounds that is often accompanied by decoding by adding a vowel and therefore consonants rarely being read independently. For instance, whereas the learners decoded 46 syllables correctly per minute, the children could only decode and identify 16 letter sounds. Table 10: Performance on reading sounds and syllables No Literacy Skills Average Performance (per minute) No of tasks Benchmark 1 Reading Sounds 16 100 None 2 Syllables 46 100 None 2. Children have not mastered the word formation principles as measured by the number of invented words correctly read 36 | Page Ability to read invented words is a clear characteristic of developed automaticity and ability to decode any word in Kinyarwanda. However, when words are read (whether independently or in a text), children are decoding fewer invented words. The ability to read invented words reveals the extent to which children have mastered the principles in word formation in a language. However, the analysis shows that on average, P3 children read 21 words per minute (wpm) (familiar words), 15 wpm (invented words) and 20 wpm (words in a text). It is therefore possible that the children are able to identify and read the familiar words without necessarily mastering the principles behind those word formations. This comparison is presented in Table 11 below. Table 11: Average overall P3 performances against the expected reading level No Literacy Skills Average Performance (per minute) No of tasks Benchmark 3 Familiar Words 21 50 None 4 Invented Words 15 50 None 5 Words in a text 20 65 20-32 3. Girls in P3 are performing better than the boys in literacy skills Further, analysis by gender of the student reveals that the girls are performing better than the boys across the literacy skills. On average, girls read three more letter sounds than the boys, decoded 13 more syllables, read six more familiar words, five more invented words and seven more words in a text than the boys. Basically, on any given literacy task, the girls performed better than the boys. 4. The performance in any literacy skill vary across the districts; highest in Nyamagabe District and lowest in Nyaruguru District in any literacy skill The detailed performance for the P3 by district is presented in Figure 2 below. It is evident that the performance in Nyamagabe is the highest when making comparison across the four districts followed by Karongi, Rutsiro while the learning levels are the lowest in Nyaruguru. 37 | Page Figure 2: P3 detailed literacy performance by district: Base: Total=400, each district=100 P3 students 5. There are non-readers in P3 in all the literacy tasks This was measured by establishing the proportion of the children with 0 scores in each task. For instance, 5 percent of the P3 students cannot decode any letter sound, 14 percent of the same P3 students cannot identify any Kinyarwanda syllable. Furthermore, 27 percent of the students in this class cannot read any familiar word in Kinyarwanda while almost a third (29 percent) cannot read any invented word. When presented with words in a text, almost a quarter (24 percent) of all the learners in P3 cannot read a single word. This brings to the fore the reality that many children are struggling to learn the basic skills and are graduating to higher levels without acquiring the basic reading skills. This is illustrated in Figure 3 below. Figure 3: Proportion of Non-readers in P3 by Task 16 50 25 17 24 22 60 27 19 26 13 35 14 10 13 15 15 18 12 16 16 46 21 15 20 Sounds Syllables Familiar words Invented words Words in Text P3 Reading Scores by Task Karongi Nyamagabe Nyaruguru Rutsiro Total Average 5% 14% 27% 29% 24% Letter Sounds Syllables Familiar words Invented words Words in a text Base: P3 sampled students (400) 38 | Page MGD 1.2: Improved Attentiveness Indicator Baseline Target Percentage of students in classrooms identified as attentive by their teachers 60% 80% Data for this indicator was collected using the perception of teachers of sampled classes. A show card with a ladder of attentiveness was used as a projective technique to collect this information. The ladder was divided into four equal segments ranking the lowest segment to the highest segment. These segments represented four levels of attention as follows. • lowest segment of ladder (no attention), • second segment of ladder (low attention), • third segment of ladder (medium attention) and • fourth segment of ladder (high attention) The teacher was asked to place the students in the class on this ladder of attentiveness. For this indicator, the third and fourth segment was combined to represent attentiveness. An average score of all classes was used to calculate the percentage of attentive students. Considering the third and fourth segment as the attentive students, the percentage of students considered to be attentive by teachers is 60 percent. Figure 4: Percent distribution of student across the attentive ladder Note: The assessment of this indicator is based on a subjective approach that is founded on the perception of class teachers and the assumption that the teachers are aware of student attentiveness in class 1 15% 25% 36% 24% Base: All students in sampled classes (10, 397) Lowest ladder(No attention) Second ladder(low attention) Third ladder(medium attention) Fourth ladder(high attention) 39 | Page Indicator Baseline Target Number of students enrolled in schools receiving USDA assistance 82,358 127,650 Number of students enrolled in schools receiving USDA assistance (female) 41,404 66,378 Number of students enrolled in schools receiving USDA assistance (male) 40,954 61,272 The indicator counts students enrolled at schools that are directly benefiting from USDA assistance. For this indicator, USDA assistance to schools includes the provision of commodities for school feeding and/or the rehabilitation of school infrastructure. The focus of this indicator is 104 schools that have been pre-selected to participate in the home grown school feeding program. Data for this indicator is required to be disaggregated by gender. Based on the secondary data by WFP the total number of students enrolled in schools selected to receive USDA assistance is 82,358 students. This considers grades 1-6 only. Data was not disaggregated by gender in 13 schools. Based on schools that had gender disaggregation, the mean gender split is approximately 50 percent (50.2 percent female vs. 49.7 percent male). Applying the universal gender split, the estimated number of female students enrolled in these schools are 41,404 and 40,954 for male students. MGD 1.3: Improved Student Attendance Indicator Baseline Target Number of students regularly (80%) attending USDA supported classrooms/schools. 75,522 127,650 Number of students regularly (80%) attending USDA supported classrooms/schools (female). 37,964 53,102 Number of students regularly (80%) attending USDA supported classrooms/schools (male). 37,512 49,017 This indicator measures the number of boys and girls attending school regularly. According to the definition of this indicator, it requires the analysis of data for each student during normal school hours for a given school year and establish that each student was in school at least 80 percent of the school days. Desk research established that data on attendance is not summarized by student per year. Therefore, this indicator could not use secondary data at district or sector level. To respond to this indicator, it would have required analysis of more than 100,000 records of students using the attendance records at school level. Given time and resource constraints, alternate methodology was used. 40 | Page To obtain a proxy for this indicator at baseline, the sampled classes were used and their attendance reviewed for the first seven weeks in the second term (April 18 to June 3, 2016). Using the same standard of 80 percent attendance, the baseline study counted the number of students who have not been in school for seven or more days during this period. A total of 10,397 students’ records were analyzed for this period, which constituted 13 percent of the population records. Based on the reviewed records, 91.7 percent of the students regularly attended classes; 90.6 percent for boys and 92.7 percent for girls. When projected to the total population of target students of 82,358, it can be estimated that 75,522 students regularly attended USDA supported schools. When disaggregated by gender, 37,964 girls and 37,512 met the indicator. Indicator Baseline Target Percent of students who miss more than 10 school days per year due to illness (female) 8% 20% Percent of students who miss more than 10 school days per year due to illness (male) 7% 20% Similar to MGD indicator 1 described above, this data used students’ records for a period of seven weeks. An ordinary school year is 180 days. The indicator is measuring absence of 5.6 percent or more of the school days. This is equivalent to more than two days for 35 school days. Therefore, the enumerators counted the students in each sampled class who had missed more than two days during the last 35 school days due to illness. The percentage was calculated in each class and overall statistic was generated by gender. Based on 10,397 students’ records studied for seven weeks, overall, 7 percent of students missed more than 2 school days during this period because of illness. Girls recorded 8 percent and boys 7 percent. On the day the schools were visited, overall, 15 percent of the enrolled students in sampled classes were absent. The proportion was higher for boys at 17 percent compared to girls at 14 percent. 41 | Page 5. 0 Results for Health Indicators (Results Framework #2) This section presents the results of the indicators on health. MGD 2.1 Improved Knowledge of Health and Hygiene Practices Indicator Baseline Target Percent of students who can identify at least three key health and hygiene practices (female) 47% 80% Percent of students who can identify at least three key health and hygiene practices (male) 48% 80% Information for these two indicators was collected through interviews with students from P1 to P6. The students were asked to mention at least three key health and hygiene practices unprompted. Overall 48 percent of students could identify at least three key health and hygiene practices. There was no major variation across gender. However, there was a statistically significant difference between lower classes and upper classes (p value 0.00). Figure 5: Percent of students who can identify at least three key health and hygiene practices by gender and grade MGD 2.2 Increased knowledge of safe food preparation and storage practices Indicator Baseline Target Percent of cooks and store keepers who can identify at least three safe food preparation and storage practices. 92% 95% The data for this indicator was gathered through interviews with store keepers and cooks in sampled schools. A total of 29 store keepers and 10 cooks were interviewed. Although only two of the sampled schools had a school feeding program, eight school had cooks to serve the teachers. 48% 48% 47% 76% 76% 77% 19% 21% 17% Total Boy Girl Total Boy Girl Total Boy Girl Total (upper primary P4-P6) Lower primary(P1-P3) 42 | Page The respondents were asked to mention safe food preparation and storage practices unprompted. The baseline focused on the following: 1. Using clean containers to collect food from the store 2. Removing foreign matters from food before cooking, and then washing it with clean water thoroughly 3. Check expiry date of food before cooking 4. Hand washing before handling food and often during food preparation 5. Cleaning the kitchen and working area 6. Storing cooked food in covered cooking pots in a clean, safe place before serving the students 7. Safe storage of food • Taking measures to prevent food from contamination from pests and rodents • Storing food off the ground 8. Washing cooking utensils and materials with water and soap If a respondent mentioned three, he/ she was given a score of one. As shown in Figure 6, 92 percent could mention at least three safe food preparation and storage practices against a target of 95 percent. 43 | Page Figure 6: Percent of cooks who cooks and store keepers who could mention at least 3 safe and food and storage practices MGD 2.4 Increased Access to Clean Water and Sanitation Indicator Baseline Target Number of schools using an improved water source 62 412* *target that was agreed upon in signed project document – needs to be revised Indicator: Information for this indicator was gathered from the head teacher interviews and observations. For a school to fulfil the indicator criteria, it should comply with the following: 1. Main water source for the school was from the following sources: • Piped water into dwelling, plot, or yard • Public tap/standpipe • Tube well/borehole • Protected dug well • Protected spring • Rainwater collection 2. Water is normally available from the identified source(s) 3. Water was not unattainable from the identified source(s) in the past two weeks for a day or longer Based on sampled schools 60 percent (24/40) are using an improved water source. This translates to 62 schools at baseline, which had an improved water source. Percent who could name at least 3 safe food preparation and storage practices, 92% Percent who could not name at least 3 safe food preparation and storage practices, 8% Base: 39 sampled cooks and store keepers 44 | Page The study explored further the nature of water access in schools. Reliability of water source is where there is a slight challenge. Even though 35 schools had an improved water source, only 28 had water every day in the two weeks preceding the baseline survey. Figure 7: Safe water access in schools *Base: 40 sampled schools *The first bar represents the indicator as defined by USDA: a composite variable that has to meet the three criteria listed above. Indicator Baseline Target Number of Schools with Improved Sanitary Facilities 94 412* *target that was agreed upon in signed project document – needs to be revised This indicator measures whether there are adequate sanitary facilities at the school and whether that sanitary facility meets the improved sanitation standards defined in the Millennium Development Goals (MDGs). To be considered adequate, the school must have separate improved sanitation facilities available for the use of both males and females. In defining this indicator only toilets for students were used. An improved sanitation facility was defined as: • Flush or pour/flush facilities connected to a: • Piped sewer system o Septic system o Pit latrine • Pit latrines with a slab • Composting toilets • Ventilated improved pit latrines 24 35 33 28 Number of schools with improved water source (composite indicator - meeting the following three criteria) Number of schools whose main source of water is improved Number of schools where water is normally available from this source Number of schools where water was available all days in the previous two weeks 45 | Page To meet the criteria, the school had to fulfill the following two conditions: • Has separate boys’ and girls’ toilets • The type of toilet was classified as improved sanitation facility Based on this definition, out of the 40 schools visited at baseline 36 (90 percent) had improved sanitary facilities. Baseline value for this indicator for all target schools is 94 schools out of 104 program schools. Figure 8 summarizes the status of sanitation facilities in schools. Out of the 40 schools visited, three schools did not have separate toilets for teachers (one in Nyaruguru, one in Rutsiro and one in Karongi)1 . The teachers and students shared the same toilets. Two schools in Nyamagambe did not have separate toilets for boys and girls. Figure 8: Status of sanitation in sampled schools Looking at the number of users per toilet in sampled schools the toilet ratio i.e. number of students per stance was 77:1 for girls, 84:1 for boys and 80 overall. Number of users per toilet was much higher compared to the national toilet ratio in Rwanda which is 54 for girls, 56 for boys and 55 overall.2 Long ques were observed during break time. Toilets were not adequate to cater for the school population. 1 The specific names of schools will be shared with program staff in a separately. Consent form administered to head teachers promised that identity of the school names would be revealed in the report. 2 Ministry of Education; 2015 Educational Statistical Yearbook, June 2016. 36 14 3 2 1 1 Schools meeting the indicator￾Improved sanitation Schools where toilets were not clean (visible feacal matter) Schools with no separate toilet for teachers Schools with no separate toilet for girls Schools where girls toilets have no walls Schools where girls toilets have no doors Base: 40 Sampled schools 46 | Page MGD 2.5 Increased Access to Preventative Health Interventions Indicator Baseline Target Number of students receiving deworming medications 75,749 127,650 This indicator measures the number of students in a fiscal year that have received deworming medication(s), usually through the distribution of deworming tablets at school. This data was collected through interviews with the head teacher and review of documents of deworming exercise. The information was also collated from key informant interviews with the District Education Officers (DEOs). Based on reports from head teachers of sampled schools, all schools had the students dewormed. However, focusing on the past one year, a total of 29,839 students representing 92 percent of the enrolled students in the sampled schools had been dewormed. Total number of students enrolled in target schools are 82,358. Projecting to the total population enrolled in the target schools, baseline estimates that the number of students receiving deworming are 75,749. McGovern-Dole definitions recommends use of recommended medications, doses and treatment guidelines stipulated by the World Health Organization (WHO) for use in large￾scale school deworming programs. Baseline study was limited by access to the specific guidelines for the target district. However, there are indications that students in Rwanda are at high risk of worm infestations. The District Education Officers confirmed that deworming is supposed to be done twice a year. The baseline study assumed a standard of twice a year as recommended by WHO for soil-transmitted helminths using Albendazole (400mg); Mebendazole (500mg), or Levamisole (80mg).34 In addition, baseline was limited on the dosage used. Given that Ministry of Health is solely in charge of this initiatives, baseline assumed that the correct dosage was used. Based on findings from head teachers of sampled schools 65 percent (26/40) of the schools were conducting deworming exercise at least twice a year. None of the schools visited in Karongi districts met this criterion. Most of the schools visited in Rutsiro districts were carrying out deworming exercise four times in a year. 3 http://www.who.int/features/2015/rwanda-deworming-campaign/en/ 4 Food for Progress and McGovern-Dole Indicators and Definitions, Food Assistance Division, Offices of Capacity Building and Development 47 | Page Table 12: Frequency of carrying out deworming activities in sampled schools as reported by the head teacher. Frequency of dewormin g exercise Total Karongi Nyamagab e Nyarugur u Rutsiro Number of mentions % Number of mention s Number of mentions Number of mentions Number of mention s Every 3 months 7 18% 7 Twice a year 19 48% 10 8 1 Once a year 13 33% 10 2 1 Was a one￾off event 1 3% 1 Total 40 100 % 10 10 10 10 Caution: There was a contradiction with information obtained from KII with DEOs on frequency of deworming. DEOs reported a higher frequency of at least twice a year. MGD 2.6 Increased Access to Requisite Food Prep and Storage Tools and Equipment Indicator Baseline Target Number of target schools with access to improved food preparation and storage equipment (kitchens, cook area, storerooms, stoves and kitchen utensils) 10 104 Based on secondary data from information collected earlier by WFP in the 104 target schools, 48 schools (46 percent of total target schools) did not have cooking and storage facilities5 . However, this data does not clearly capture the whole indicator. Therefore, data for this indicator was collected from structured observations in sampled schools. To meet the indicator, the school had to have the following: • School has a kitchen • Cooking area is clean • Cooking area has an improved food preparation area (raised area for food preparation) • School has a fuel-saving stove • School has an improved storage are (store exclusively for food) 5 List of schools in the 4 district selected to participate in the program by WFP. 48 | Page Based on this criteria, only four of the sampled schools (10 percent) have access to improved food preparation and storage equipment (kitchens, cook area, storerooms, stoves and kitchen utensils Projected to all target schools, the baseline value for this indicator is 10 schools. Figure 9 illustrates that 58 percent of schools have an existing kitchen. Similar to available secondary data, all schools had space for constructing a kitchen. Figure 9: Summary of schools with specific features of this indicator Base: Sampled schools (40) 10% 58% 55% 37% 23% 20% Schools that meet the indicator Schools with a kitchen Schools with a fuel-saving stove Cooking area is clean Schools with an improved storage area (store exclusively for food) Cooking area has an improved food preparation area ( raised area for food preparation) 49 | Page 6.0 Additional Findings 6.1 Schools with an ECD Centre and feeding programs Out of the 40 schools visited 22 (55 percent) had an ECD center, however, only one school had a school feeding program for these children. The ECD center provided children with hot porridge and the feeding program was financed by parents. 6.2 Involvement of students in writing competition Out of the 40 schools visited, students in 13 of them participated in writing competition according to the head teachers. Although the ratio of the student is 1:1 between male and female, there were more girls participating in the writing competition compared to boys. Table 13: Distribution of the number of students that participated in the last competition by sex and class Grade Number of boys Number of girls Total P1 7 8 15 P2 6 6 12 P3 7 5 12 P4 7 10 17 P5 10 15 25 P6 7 17 24 44 61 105 6.3 Schools with Access to a Hand Washing Facility Although not a McGovern-Dole requirement, the baseline study sought to establish the schools’ access to hand washing facility. As shown in Figure 10, only 50 percent of the schools hand a functional hand washing facility. 50 | Page Figure 10: Schools’ Access to a hand washing facility % of schools with no hand washing facility, 43% % of schools with a hand washing facility but no running water, 7% % of schools with a hand washing facility with running water, 50% Base: 40 sampled schools 51 | Page 7.0 Limitations of the Study This baseline study was conducted within the limitations cited here below: 1. Timing of the EGRA exercise: The indicator requires data to be collected at the end of grade two. The ideal timing for this indicator should have been the end of the academic year or first week of the new term. By the time this study was commissioned and data collected, P3 students had covered seven weeks in second term. On the other hand, P2 students had not yet completed P2 syllabus. 2. Secondary data: The ToR anticipated that the baseline study would heavily rely on secondary data and minimal use of primary data. A review of available secondary data and corresponding indicators revealed that definitions of indicators required specific information, a gap that could not be filled through secondary data. Aggregated secondary data from schools at sector and district level were not in a format that could be utilized in this study, thus creating the need for this data to be collected from all the target schools. The study was limited to 40 sampled schools and estimates have been projected to the total population. 3. Incomplete secondary data: Data on teachers and students’ records were not comprehensive in indicating reason for absenteeism for all days. Given this limitation, WFP and World Vision have redesigned the class register to facilitate data collection on attendance for the midline and end line studies. 4. Missing secondary data: Schools maintain records on teacher attendance but don’t track if a teacher in attendance has taught or not. The indicator below requires data on both attendance and teaching. “Number of teachers in target schools who attend and teach school at least 90 percent of scheduled school days per school year” The baseline study made an assumption that any teacher in attendance taught. Monitoring records should track both attendance and teaching to facilitate the midline and final evaluations. 5. Factors contributing to variations in learning outcomes across the district and gender: Learning outcomes differ across the districts and gender. The baseline study did not manage to unveil the contributing factors and barriers to learning among boys and Nyaruguru district. A further study can inform the program on specific strategies to promote literacy and tailor make the literacy activities. 52 | Page 6. Research approval in Rwanda for a household survey was not feasible in the timeframe of the baseline study: The process of acquiring research approval for a household survey from the National Institute of Statistics in Rwanda takes at least three months, which was not possible given the timeline of the study. This limitation confined the baseline study to a school-based survey, with no access to the community or parents, except through the PTC. Due to this, the baseline value for the following indicator could not be established: Percent of parents in target communities who can name at least three benefits of primary education. WFP will conduct a follow-up survey with parents in September, 2016 and the baseline value for this indicator will be published in the next USDA semi-annual report. 7. The baseline value for the Minimum Acceptable Diet (MAD) was not established due to methodological limitations of applying MAD to school-going children in grades 1-6. MAD is internationally validated only for children aged 6-23 months. Consultations are currently ongoing between WFP and USDA on the feasibility of asking school-age children about their diet intakes and the reliability and accuracy of such information. 53 | Page 8.0 Conclusions and Recommendation Following the baseline study, the following conclusions can be drawn: • Findings from the sampled schools illustrate the absence of, and the need for, school feeding for students in grades 1-6. In addition, the baseline study revealed a very limited risk of overlap with other school feeding programs. • The school management committees support the school feeding program and are willing to make contributions towards the program. The plan to incorporate inputs from the school community is well aligned with the school leaderships’ expectations. • While majority of the target schools use an improved water source, reliability of water availability is the challenge. This program needs to incorporate a strategy that will ensure there is clean water available to the beneficiary schools. • Although most of the schools have improved sanitation facilities according to the definition of the indicator, the number of toilets are inadequate to cater for the student population. There is also a need to ensure that all target schools have separate girls and boys toilets and reserved toilets for teachers. • Nearly all target schools have school gardens. There is potential to utilise school gardens for increasing access to food in primary school. • Based on findings from EGRA, levels of literacy in the four program districts across genders are low, exemplified by the number of non-readers across all three grades tested in the four program districts. This intervention with a literacy improvement component is much-needed to promote learning in beneficiary schools. • The targets set by the program for some indicators don’t correspond to the current situation in the program schools and need to be revised. For example, the indicator for health related absence has been set at 20 percent. However, the baseline value established that only eight percent of girls and seven percent of boys are missing more than ten days of school per year due to illness. 54 | Page Annexes Below is the list of documents annexed in this report: 1. Summary of baseline indicators against target 2. Sampling document 3. Additional EGRA findings that include P1&2 and additional tables 4. References 5. Results framework 55 Annex 1: Summary of baseline indicators against target Result Indicator Baseline Indicator Value Target Increased Access to Preventative Health Interventions Number of students receiving deworming medication(s) 75, 749 127,650 Increased Access to Requisite Food Preparation and Storage Tools and Equipment Number of target schools with access to improved food preparation and storage equipment (kitchens, cook area, storerooms, stoves and kitchen utensils 10 104 Improved School Infrastructure Number of educational facilities (i.e. school buildings, classroom, and latrines) rehabilitated/constructed as result of USDA assistance (kitchens, cook areas) Zero at baseline Program activities have not started 104 Number of educational facilities (i.e. school buildings, classroom, and latrines) rehabilitated/constructed as result of USDA assistance Zero at baseline Program activities have not started 26 Increased Student Enrolment Number of students enrolled in school receiving USDA assistance (female) 41,404 66,378 Number of students enrolled in school receiving USDA assistance (male) 40,954 61,272 Number of students enrolled in school receiving USDA assistance 82,358 127,650 Increased Community Understanding of Benefits of Education Percent of parents in target communities who can name at least three benefits of primary education Research approval in Rwanda for a household survey was not feasible in the timeframe of the baseline. WFP decided that baseline data for this indicator will be collected through a separate survey in September. The baseline value for this indicator will be shared with USDA in 90% 56 the next semi-annual report. Increased Access to Clean Water and Sanitation Services Number of schools using an improved water source 62 412 Number of schools with improved sanitary facilities 94 412 Increased Knowledge of Nutrition Number of individuals trained in child health and nutrition as a result of USDA assistance (female) Zero at baseline Program activities have not started 145 Number of individuals trained in child health and nutrition as a result of USDA assistance (male) Zero at baseline Program activities have not started 63 Number of individuals trained in child health and nutrition as a result of USDA assistance Zero at baseline Program activities have not started 208 Increased knowledge of Safe Food Prep and Storage Practices Percent of cooks and storekeepers who can identify at least three safe food preparation and storage practices 92% 95% Improved Knowledge of Health and Hygiene Practices Percent of students who can identify at least three key health and hygiene practices (female) 47% 80% Percent of students who can identify at least three key health and hygiene practices (male) 48% 80% Improved Policy and Regulatory Framework Number of Child health and nutrition policies, regulations or administrative procedures in the following stages of development as result of USDA assistance (stage 5) Zero at baseline Program activities have not started 1 57 Number of educational policies, regulations, or administrative procedures in each of the following stages of development as a result of USDA assistance (stage 5) Zero at baseline Program activities have not started 1 Increased Capacity of Government Institutions Number of government staff trained or certified as a result of USDA assistance (female) Zero at baseline Program activities have not started 58 Number of government staff trained or certified as a result of USDA assistance (male) Zero at baseline Program activities have not started 86 Increased Engagement of Local Organizations and Community Groups Number of Parent- Teacher Associations (PTA) or similar "school" governance structures supported as a result of USDA assistance Zero at baseline Program activities have not started 104 Number of public-private partnerships formed as a result of USDA assistance Zero at baseline Program activities have not started 31 Value of public-private sector investments leveraged as a result of USDA assistance (private) Zero at baseline Program activities have not started 0 Increased Government Support Value of public-private sector investments leveraged as a result of USDA assistance (host government) Zero at baseline Program activities have not started 0 Percent increase in budget allocated by the Government of Rwanda to Home-Grown School Feeding Program Zero at baseline Currently Home Grown School Feeding program has no budget allocated to it 5% Increased Use of Health and Dietary Practices Percent of school-age children receiving a minimum acceptable diet (female) WFP dropped this indicator because of methodological limitations 65% Percent of school-age children receiving a minimum acceptable diet (male) WFP dropped this indicator because of methodological limitations 65% 58 Reduced Health￾Related Absences Percent of students who miss more than 10 school days per year due to illness (female) 8% 20% Percent of students who miss more than 10 school days per year due to illness (male) 7% 20% Increased Economic and Cultural Incentives (Or Decreased Disincentives) Number of school-aged children receiving daily school meals (Breakfast, snack, lunch) as a result of USDA assistance (female) Zero at baseline Program activities have not started 66,378 Number of school-aged children receiving daily school meals (Breakfast, snack, lunch) as a result of USDA assistance (male) Zero at baseline Program activities have not started 61,272 Number of school-aged children receiving daily school meals (Breakfast, snack, lunch) as a result of USDA assistance (new) Zero at baseline Program activities have not started 127,650 Number of school-aged children receiving daily school meals (Breakfast, snack, lunch) as a result of USDA assistance (continuing) Zero at baseline Program activities have not started 71,970 Improved Student Attendance Number of students regularly (80%) attending USDA supported classrooms/school (female) 37,964 53,102 Number of students regularly (80%) attending USDA supported classrooms/school (male) 37,512 49,017 Number of students regularly (80%) attending USDA supported classrooms/school 75, 522 127,650 Increased Access to Food (School Feeding) Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (female) Zero at baseline Program activities have not started 66,378 59 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (male) Zero at baseline Program activities have not started 61,272 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (new) Zero at baseline Program activities have not started 127,650 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (continuing) Zero at baseline Program activities have not started 71,970 Number of school-aged children receiving daily school meals (Breakfast, snack, lunch) as a result of USDA assistance (female) Zero at baseline Program activities have not started 66,378 Number of school-aged children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance (male) Zero at baseline Program activities have not started 61,272 Number of school-aged children receiving daily school meals (Breakfast, snack, lunch) as a result of USDA assistance (new) Zero at baseline Program activities have not started 127,650 Number of school-aged children receiving daily school meals (Breakfast, snack, lunch) as a result of USDA assistance (continuing) Zero at baseline Program activities have not started 71,970 Number of school-aged children receiving daily school meals (Breakfast, snack, lunch) as a result of USDA assistance Zero at baseline Program activities have not started 127,650 Reduced Short-Term Hunger Number of daily school meals (breakfast, snack, lunch) provided to school￾age children as a result of USDA assistance Zero at baseline Program activities have not started 74,808,333 Improved Attentiveness Percentage of students in classroom identified as attentive by their teachers 60% 80% 60 Increased Skills and Knowledge of School Administrators Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance Zero at baseline Program activities have not started 93 Number of school administrators and officials trained or certified as result of USDA assistance Zero at baseline Program activities have not started 139 Increased Skills and Knowledge of Teacher Number of teacher/educators/teaching assistants in target schools who demonstrate use of new techniques or tools as a result of USDA assistance Zero at baseline Program activities have not started 653 Number of teacher/educators/teaching assistants trained or certified as result of USDA assistance Zero at baseline Program activities have not started 816 Improved Literacy of Instructional Materials Number of target schools with supplemental reading materials available to students as result of USDA assistance Zero at baseline Program activities have not started 104 Better Access to School Supplies and Materials Number of textbooks and other teaching and learning materials provided as a result of USDA assistance Zero at baseline Program activities have not started 132,990 More Consistent Teacher Attendance Number of teachers in target schools who attend and teach school at least 90 percent of scheduled school days per school year 986 653 Improved Quality of Literacy Instruction Number of teachers in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Zero at baseline Program activities have not started 693 61 Improved Literacy of School-Age Children Number of individuals benefiting directly from USDA-funded interventions (new) Zero at baseline Program activities have not started 133,328 Number of individuals benefiting directly from USDA-funded interventions (continuing) Zero at baseline Program activities have not started 77,503 Number of individuals benefiting directly from USDA-funded interventions (male) Zero at baseline Program activities have not started 61,240 Number of individuals benefiting directly from USDA-funded interventions (female) Zero at baseline Program activities have not started 69,088 Number of individuals benefiting directly from USDA-funded interventions Zero at baseline Program activities have not started 133,328 Number of individuals benefiting indirectly from USDA-funded interventions Zero at baseline Program activities have not started 175,379 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (female) 57% 90% Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (male) 42% 90% Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text 49% 90% 62 Annex 2: Sampling document Detailed Sampling Procedure for School based Survey Sample size determination Early grade reading assessment (EGRA) exercise was used in determining the sample size of schools. This activity targeted students in grade 1 to 3 although the focus of this indicator is grade 3. (Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text) This baseline indicator is reported at total level and disaggregated by gender. Sampled schools for this indicator was used to collect data for other cohorts. This facilitated coordination of data collection activities. Sample size determination used the following formula. This formula is used in estimating the sample size for small populations. It takes into consideration population size, level of confidence and margin of error. n = Nz^2pq/( E^2(N-1)+ z^2pq) Where N=population size target students (83,000) z=confidence level (1.96) E=margin of error (2.8%) p=the proportion of a population with a particular characteristic (0.5) q=the proportion of a population not having this characteristic (0.5) Based on the above formula and parameters, the sample size calculation yields a sample of 1208. To distribute the sample equally in all schools a Figure of 1200 was used. Using the same formula and 95% confidence level with an interval of +/-5% a sample of 382 can be used. However, this cannot allow analysis by gender and by district. Therefore, Ipsos recommends a sample of 1,200 for this indicator. Sample distribution across districts Sample considerations 63 EGRA involved getting students from classes. Therefore, consideration in sampling was taken to ensure interruption of school learning activities was minimised by carrying out the study within one hour in a class. This time factored in sampling of students, warm up and assessment. School activities could also be interrupted by having many assessors in the school. Effectively there are 3-4 hours when this can be done. Lower grades attend school up to lunch time for those schools that do not have a double shift system. The teams were to complete the activities in one day. Based on these considerations, Ipsos proposed to have a sample of 10 students per grade; (5 girls and 5 boys). Therefore, in each school, a sample of 30 students. Distribution of the sample across the district was done using probability proportionate to population size as shown in Table 14. First the target population of each district was determined by summing up the population of schools in each district. The proportion of student population in each district to the total was calculated. Expected sample for each district was calculated by multiplying 1200 with the proportion of each district. To estimate the number of schools in each district the rationale of 30 interviews in each school was used. Number of schools was rounded to the nearest number. Based on this rationale 40 schools were sampled used for this study; 10 schools in each district. Table 14: Determining number of schools required (a) District Total number of students Proportion Sample Estimate of schools No. of schools Karongi 20028 24.3% 292 9.73 10 Nyamagabe 20887 25.4% 304 10.14 10 Rutsiro 19698 23.9% 287 9.57 10 Nyaruguru 21745 26.4% 317 10.56 10 82358 100.0% 1200 40 Student population data source: Sampling frame provided by WFP Sample size of students for health Data for this indicator was collected through a student survey. • Percent of students who can identify at least three key health and hygiene practices Data is disaggregated by gender. In addition, the first indicator required data disaggregation by classes. Although class 1-3 was included in collecting data for these indicators, it was expected that comprehension and recall may be lower. Therefore, focus of these indicators was grade 4-6. 64 Data was collected from 2,400 students; 1200 from grade 4-6 and 1200 from grade1-3. Given that a sample was drawn for EGRA which is a simple tool, Ipsos proposed to ask the sampled students additional few questions to capture this indicator. In grade 4, 5 and 6, data was collected from a sample of 10 students in each grade. Sample size for this indicator has been determined using the same rationale as EGRA sample. Selection of sampled schools Systematic random sampling was used to select the schools. A sampling frame was provided by WFP which comprised of all the 104 schools target beneficiary schools. These schools were grouped per district and given serial numbers. For each district a sampling interval was calculated. A random start was picked using True Random Number Generator. This was the first school to be picked on the list. The other 9 schools were selected by adding the sampling interval. Table 15: list of surveyed schools, by district Karongi District Nyamagabe District EP Kinyami GS Rugogwe EP Gitesi EP Kiyumba EP Gitanga GS Rususa GS Ngoma EP Baro EP Kinyovu GS Kiraro Protestant EP Biguhu A EP Kibirizi GS Nyabikeri EP Nyagisozi EP Nyamugwagwa EP Nkore GS Shoba Muramba EP Masagara EP Nyamabuye EP Kamegeri Rutsiro District Nyaruguru District EP Mujebeshi EP Ruhinga EP Mpingamabuye GS Ruheru GS Bitega B EP Zirambi EP Umubano EP Gakaranka EP Rundoyi EP Kimina EP Gasovu GS Rugerero EP Nyakarera GS Kiyonza EP Mwufe GS Kivuru GS KIVUMU GS Fugi EP Rutsiro EP Ruhororo *GS – Groupe Scolaire: primary and secondary school **EP – École Primaire: primary school 65 Selection of participating students Selection of participating students was done upon arrival in the school using systematic random sampling. The attendance register for that day was used. In each class two sampling frames were developed; one for girls and the other one for boys. A sampling interval for each frame was determined by dividing the total number of girls/boys by 5. Then a random start was obtained by which any number is picked randomly, that is between 1 and the sampling interval. The other four students were picked by adding the sampling interval. Each selected student was given a unique code which was used to identify the in the data In the event that there were multiple streams in a class one class was selected using simple random sampling. School Based surveys Entering the districts and school WFP assisted in contacting the district education offices and organizing for the first courtesy call at district education offices or Mayors office. Prior appointments were made with schools before the research activity day. The team leader organized the logistics to the school prior to the activity day. The team would ensure that they reached the school first thing in the morning. The team leader will introduce the team and the activity. Each team member would introduce their role that day without giving details that would bias the answers. The team leader would go through the consenting process with the head teacher. After getting consent, he/she would seek to understand the school program for that day so that they can effectively plan the activities within the school program. The team leader would request the head teacher to organize for a brief meeting with the teachers, cooks, storekeeper so that he can brief them and give them his/her blessings to cooperate in the survey. This was an opportunity to inform the teachers about the activity and the processes. It also helped to manage the misconceptions about the activities which can bias the responses. After introductions the team would commence sampling and data collection. After completing the school activities, the team leader will congregate his/her team and give a vote of thanks to the head teacher and the teachers. 66 Table 16: List of Key Informant Interviews Conducted Organization Designation Nyamagabe District DEO Karongi District DEO Rutsiro District DEO Nyaruguru District DEO Nyamagabe District HGSF District Coordinator Nyaruguru District HGSF District Coordinator WFP National Program officer WFP Head of Program Unit in WFP WFP Project Manager of HGSF WV Technical Program Manager WV Technical Program Officer MINAGRI Works at Rwanda Agricultural Board, in charge of One Cup of Milk per Child MINEDUC 67 Annex 3: Additional EGRA findings (that include P1& P2) and tables The following facts emerge from the analysis of the literacy scores for all the students including P1 and P2. 1. The learning levels in literacy in the four districts, across the classes and cutting across the two genders are low; The average learning levels across the four districts in all the classes and cutting across the two genders are low. On average, the learners could only identify 11 sounds correctly per minute with a standard deviation of nine, meaning that there is a high proportion of children who could not identify any letter sound at all. On the other hand, learners decoded an average of 25 syllables per minute, 11 familiar words per minute, 7 invented words per minute and 10 words in a text. This is against a list of 100 sounds, 100 syllables, 50 familiar and invented words and 65 words in a text given. It is evident that children are struggling to read. There are also wide variations among the learners as demonstrated by high standard deviations in the scores as shown in the Figure 11 below. Figure 11: Average number of sounds, syllables and words read per minute Base: 1200 students 2. Students in P3 have higher learning outcomes in literacy than their peers in P2 and P1; It is evident that the learning levels are comparatively higher in P3 and lowest in P1. This is perhaps expected considering that the P3 have more content exposure than their peers in P2 and P1 respectively. However, even the P2 for whom the test has been developed based on their level, it is evident that their learning levels are low. For instance, on decoding the sounds, 11 25 11 7 10 Reading Sounds Reading Syllables Familiar Words Invented words Words in a text 68 P3 on average 16 sounds in a minute compared to their peers in P1 who on average, decoded 6 sounds in a minute. When it comes to reading words in a text, the average number of words read in a text in P3 was 20wpm (words per minute) compared to 1 wpm in P1. The higher performance is also replicated in other literacy tasks that include reading the syllables, familiar words and invented words. Figure 12 below breaks down the average performance of the classes by task. Figure 12: Average Reading Scores by Class Base: 400 students in each class 3. Learning outcomes are highest in Nyamagabe and lowest in Nyaruguru district; When the learning levels in the four districts are compared, learning levels are highest in Nyamagabe and lowest in Nyaruguru while Rutsiru and Karongi fall in between. This occurs across all the literacy task and thus, this cannot be happening by chance. Nyamagabe has the highest literacy levels, followed by Karongi, Rutsiro and Nyaruguru has the lowest. However, the differences between the highest and the lowest vary from one task to the other. For instance, the average syllables read in Nyamagabe was 34 syllables per minute compared to 20 syllables correctly read in Nyaruguru. This is illustrated in Figure 13. 6 6 1 1 1 11 25 9 6 8 16 46 21 15 20 Reading Sounds Reading Syllables Familiar Words Invented words Words in a text P1 P2 P3 69 Figure 13: Average Reading Scores by District *Total refers to average 4. Girls demonstrate higher learning outcomes in literacy than the boys in the four districts; When boys and girls are compared, girls fair better than boys on all the literacy tasks. On average, girls identified 12 sounds compared to boys who identified 10 sounds, read 29 syllables compared to boys who read 22 syllables, read 12 familiar words than 9 read by boys, read 8 invented words compared to 6 read by the boys and read fluently (23 wpm) in a text compared to boys (8wpm). This is illustrated in Figure 14 below. 10 7 11 25 11 8 6 9 21 10 8 6 9 20 9 13 9 14 34 14 11 9 12 26 11 0 5 10 15 20 25 30 35 40 Words in a text Invented Words Familiar Words Syllables Sounds Average Scores per task by District Total Rutsiro Nyaruguru Nyamagabe Karongi 70 Figure 14: Average Reading Scores by Gender Base: Girls (600); Boys (600) 5. Learners in the target classes have a problem decoding the letter sounds in Kinyarwanda language Although the foundation of literacy is the grasping of sounds, this seems not to be the case with the target learners. It is evident that learners are struggling to read the letter sounds. In fact, the scores in syllable reading are higher than the scores in letter sound reading. This could be perhaps due to the nature of reading the sounds that is often accompanied by decoding by adding a vowel and therefore consonants rarely being read independently. For instance, the results in P1 are level at 6 sounds/syllables per minute but this takes a stead difference in P2 and P3 where the difference between the two tasks is 14 sounds/syllables and 30 sounds/syllables respectively. This is illustrated in Figure 15 below. Figure 15: Comparison of reading Sounds and Syllables by grade Base: 400 per class 6. There are high proportions of non-readers across the three classes in all the four target districts. It is evident that a high proportion of the children attending the target classes completely lack reading skills. When presented with a broad range of reading tasks that are timed, the P1 10 22 9 6 8 12 29 12 8 12 Sounds Syllables Familiar words Invented words Words in Text Boys Girls 6 11 16 6 25 46 P1 P2 P3 Sounds Syllables 71 learners have the highest proportion of non-readers. However, it is also observable that the P3 who have covered the entire test content record high proportions of the non-readers. For instance, almost a quarter (24 percent) of all the learners in P3 cannot read any word in a text while a third (29 percent) of those in the same class cannot decode an invented word. While focusing on the same group of learners (P3), 27 percent of them cannot read a familiar word. This brings to fore the reality that many children are struggling to learn the basic skills and are graduating to higher levels without the acquiring the basic reading skills. Table 17: Average reading scores against the number of tasks Analysis Type Sounds Syllables Familiar words Invented words Words in Text Mean 11 25 11 7 10 Number of tasks 100 100 50 50 65 Table 18: Average Reading scores by class Grade Sounds Syllables Familiar words Invented words Words in Text P1 6 6 1 1 1 P2 11 25 9 6 8 P3 16 46 21 15 20 Total 11 25 11 7 10 Table 19: Average sounds read by district and disaggregated by class District P1 P2 P3 Total Karongi 5 10 16 11 Nyamagabe 7 14 22 14 Nyaruguru 6 9 13 9 Rutsiro 4 10 15 10 Total 6 11 16 11 Table 20: Average words in a text correctly read per minute by district and disaggregated by class District P1 P2 P3 Total Karongi 1 8 24 11 Nyamagabe 1 12 26 13 Nyaruguru 1 6 13 7 Rutsiro 1 8 16 8 Total 1 8 20 10 72 Table 21: Average invented words correctly read per minute by district and disaggregated by class District P1 P2 P3 Total Karongi 0 2 17 10 Nyamagabe 1 8 19 12 Nyaruguru 1 6 10 11 Rutsiro 0 6 12 10 Total 1 6 15 7 Table 22: Average familiar words correctly read per minute by district and disaggregated by class District P1 P2 P3 Total Karongi 1 9 25 12 Nyamagabe 2 12 27 14 Nyaruguru 2 7 14 8 Rutsiro 1 9 18 9 Total 1 9 21 11 Table 23: Average syllables read by district and disaggregated by class District P1 P2 P3 Total Karongi 7 23 50 26 Nyamagabe 8 32 60 34 Nyaruguru 6 20 35 20 Rutsiro 4 10 15 21 Total 6 25 46 25 Table 24: Overall reading levels by District So un ds Syllable s Familiar words Invented words Words in Text Karongi 11 26 12 8 11 Nyamagabe 14 34 14 9 13 Nyaruguru 9 20 8 6 7 Rutsiro 10 21 9 6 8 Total 11 25 11 7 10 Table 25: Average sounds read by district and disaggregated by class District Boy Girl Total Karongi 10 11 11 Nyamagabe 13 15 14 Nyaruguru 8 10 9 Rutsiro 9 11 10 Total 10 12 11 73 Table 26: Average syllables read by district and disaggregated by class District Boy Girl Total Karongi 24 29 26 Nyamagabe 29 38 34 Nyaruguru 17 24 20 Rutsiro 18 25 21 Total 22 29 25 Table 27: Average familiar words correctly read per minute by district and disaggregated by gender District Boy Girl Total Karongi 10 13 12 Nyamagabe 12 15 14 Nyaruguru 6 9 8 Rutsiro 7 11 9 Total 9 12 11 Table 28: Average invented words correctly read per minute by district and disaggregated by gender District Boy Girl Total Karongi 7 9 10 Nyamagabe 8 11 12 Nyaruguru 4 7 11 Rutsiro 5 8 10 Total 6 8 7 Table 29: Average words in a text correctly read per minute by district and disaggregated by gender District Boy Girl Total Karongi 10 12 11 Nyamagabe 11 15 13 Nyaruguru 5 9 7 Rutsiro 6 10 8 Total 0 1 1 Table 30: Average Reading Scores by gender Gender Sounds Syllables Familiar words Invented words Words in Text Boys 10 22 9 6 8 Girls 12 29 12 8 12 Total 11 25 11 7 10 74 Annex 4: References Food Assistance Division, Office of Capacity Building and Development United States Department of Agriculture, Foreign Agricultural Service (2013). Monitoring and Evaluation Policy FAS, WFP(2015). Framework of Agreement between The Foreign Agriculture Services and the World Food Programme on the Donation of Agricutural c odities and the Reated Assistance Under the MacGovern -Dole International Food for Education and Child Nutrition. Republic of Rwanda, Ministry of Education (2014). National School Health Policy. Kigali Republic of Rwanda, Ministry of Education (2014). National School Health Strategic Plan 2013/2014-2017/2018. Kigali. Republic of Rwanda, Ministry of Education (June 2016). 2015 Educational Statistical Yearbook. Kigali Republic of Rwanda, (2013). Homegrown School Feeding Programme White Paper; “Developing local agriculture, nourishing young minds. Kigali Republic of Rwanda, Ministry of Local Government (January 2013). District Development Plan 2013/14-2017/18. Nyaruguru USAID, (March 2015). Overview of the Early Grade Reading Assessment (EGRA); Designing and Implementing Early Grade Reading Assessment World Food Program (2016). List of schools and summary information in the 4 district selected to participate in the Home School program by WFP. World Food Program. Rwanda FY2015 McGovern‐Dole Propo 75 Annex 5: Result Framework 76 77