Rwanda McGovern￾Dole FY 20 Project Baseline Evaluation September 2022 FY 20 McGovern-Dole Rwanda project implemented by World Food Programme Baseline Evaluation Program: McGovern-Dole International Food for Education and Child Nutrition Agreement Number: FFE-696-2020-013-00 Funding Year: Fiscal Year 2020 Implemented by:WFP Evaluation Authored by Al-Haddad, Robin Banwart, Lloyd Downen, Jeanne Finan, Padraic Kayiraba, Christine Mansaray, Sehdia Mueller, Monica Mujawase, Juventine Ravesloot, Bruce Tuyiringire, Justin DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov Ba s e lin e St u d y: USDA McGo ve r n -Dole Gra n t fo r WFP Ho m e -Gr o w n Sch o o l Fe e d in g Program in Rw a n d a (2020 t o 2025) 2 Ju n e 2022 Decentralized McGovern-Dole Baseline Evaluation Report WFP EVALUATION Report num ber: DE/RWCO/2021/025 Com m issioning office: WFP Rwanda Agreem ent Num ber: FFE-696-2020-013-00 Funding Year: Fiscal Year 2020 Project Duration: 2020-2025 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 Ke y pe rson n e l for t h e e va lu a t ion WFP Rw a n d a Evaluation Manager: Sarah Cruz Prepared by Bruce Ravesloot, Team Leader Jeanne Downen, Senior Evaluator Lloyd Banwart, Senior Analyst Padraic Finan, Analyst Justin Tuyiringire, Evaluator Daria Muteteri, Evaluator Juventine Mujawase, Evaluator Christine Kayiraba, Evaluator Robin Al-Haddad, Research Associate Sehdia Mansaray, Research Associate Monica Mueller, Quality Assurance Advisor 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 Ack n o w le d ge m e n t s The evaluation team gives sincere thanks to m any at World Food Programme in Rwanda and the Regional Bureau in Nairobi for their support to this evaluation, in particular Sarah Cruz, Evaluation Manager; Am y Blaum an, McGovern-Dole Project Manager; Jacques Sezikeye, McGovern-Dole Policy Officer, and Gabrielle Trem blay, Regional Evaluation Officer, with special thanks to Regional Evaluation Officer Ruth Musili for her support. We are also grateful to the num erous governm ent and partner representatives, school staff, parents and com m unity m em bers for their inputs to the evaluation. Dis cla im e r The opinions expressed in this report are those of the evaluation team , and do not necessarily reflect those of the World Food Programme or USDA. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this docum ent does not im ply endorsem ent by WFP or USDA of the opinions expressed. The designation em ployed and the presentation of m aterial in m aps do not im ply the expression of any opinion whatsoever on the part of WFP or USDA concerning the legal or constitutional status of any country, territory or sea area, or concerning the delim itation of frontiers. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 Co n t e n t s Ke y p e rson n e l for t h e e va lu a t ion ................................................................................................................... 2 Ackn ow le d ge m e n t s ......................................................................................................................................... 3 Discla im e r ......................................................................................................................................................... 3 List of Ta b le s .................................................................................................................................................... iii Exe cu t ive Su m m a ry ........................................................................................................................................ iv 1. In t rod u ct ion .................................................................................................................................................. 1 1.1. Evaluation features ...................................................................................................................................... 1 1.2. Context .......................................................................................................................................................... 2 2. Su b je ct of t h e b a se lin e , t h e ory of ch a n ge a n d b a se lin e q u e st ion s ...................................................... 11 2.1 Subject of the baseline, theory of change, activities and intended outputs and outcom es .............. 11 2.2 Evaluation question and evaluation criteria ............................................................................................ 14 3. Eva lu a t ion a p p roa ch a n d m e t h od ology for b a se lin e d a t a colle ct ion .................................................. 16 3.1. Evaluation approach and m ethodology ..................................................................................................16 3.2. Baseline data collection m ethods and tools ........................................................................................... 16 3.3. Lim itations...................................................................................................................................................19 3.4. Quality assurance of the baseline data collection.................................................................................. 20 3.5. Ethical considerations................................................................................................................................21 4. Ba se lin e fin d in gs a n d d iscu ssion .............................................................................................................. 22 4.1 Relevance .....................................................................................................................................................22 4.2 Coherence ....................................................................................................................................................25 4.3 Effectiveness ................................................................................................................................................27 4.4 Efficiency ......................................................................................................................................................43 4.5 Im pact ...........................................................................................................................................................45 4.6 Sustainability................................................................................................................................................47 5. Con clu sion s a n d Le sson s ........................................................................................................................... 49 5.1. Conclusions.................................................................................................................................................49 5.2. Lessons Learned .........................................................................................................................................51 An n e x 1: Ed u ca t ion p roje ct s – other actors ................................................................................................ 53 An n e x 2: Ma p of Progra m m e Are a ............................................................................................................... 54 An n e x 3: Re su lt s Fra m e w ork of McGove rn Dole ........................................................................................ 55 An n e x 4: Re su lt s Fra m e w ork of LRP ............................................................................................................ 58 An n e x 5: Crit e ria for t h e McGove rn -Dole Progra m .................................................................................... 60 An n e x 6: Ba se lin e e va lu a t ion m a t rix ........................................................................................................... 63 An n e x 7: Docu m e n t s Cit e d ............................................................................................................................ 80 An n e x 8: Meth odology ................................................................................................................................... 82 An n e x 9: McGove rn Dole Pe rform a n ce in dica t ors ..................................................................................... 87 An n e x 10: Da t a colle ct ion t ools .................................................................................................................. 101 An n e x 11: EGRA Tool (d ra ft ) ........................................................................................................................ 112 An n e x 12: Sch ool Ce n su s Tool ..................................................................................................................... 116 An n e x 13: Sch ool a n d He a d Te a ch e r Tool .................................................................................................. 119 An n e x 14: Su m m a ry list of in t erviews a n d focu s groups ........................................................................ 126 An n e x 15: Ba se lin e a n d e va lu a t ion t im elin e ............................................................................................ 129 An n e x 16: Su m m a ry t e rm s of re fe re n ce .................................................................................................... 134 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 An n e x 17: Con flict of in t e re st form s .......................................................................................................... 137 An n e x 18: Ge n d e r a sse ssm e n t .................................................................................................................... 138 An n e x 19: Observa t ions on progra m m e t a rget s ...................................................................................... 148 Acron ym s ...................................................................................................................................................... 152 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 Lis t o f Ta ble s Table 1: Child nutrition indicators for Rwanda .......................................................................................................... 3 Table 2: Gender equality indicators............................................................................................................................ 9 Table 3: Overview of m ain data collection tools .....................................................................................................17 Table 4: EGRA target sam ple sizes: baseline, m idterm and endline (# students)............................................... 18 Table 5: Average student attendance rate (percent), as estim ated by head teachers, by sex and school category .......................................................................................................................................................................27 Table 6: Total students enrolled (2021), by grade, sex, and school category, as reported by head teachers. 28 Table 7: Percent of repeat learners (prior academ ic year), as reported by head teachers, by grade, sex, and school category ...........................................................................................................................................................29 Table 8: Percent of dropouts (prior academ ic year), as reported by head teachers, by grade, sex, and school category .......................................................................................................................................................................30 Table 9: Num ber of daily school m eals received (2021), by grade, sex, and school category, as identified by teachers .......................................................................................................................................................................31 Table 10: Percent of enrolled students identified as attentive by their teachers, by grade and school category .......................................................................................................................................................................32 Table 11: Percentage of students able to com pletely read a short story aloud, by sex and school category 33 Table 12: Percent of students who, by the end of two grades of prim ary schooling, dem onstrate that they can read and understand the m eaning of grade -level text, by sex and school category .................................. 33 Table 13: Percent of students who, by the end of two grades of prim ary schooling, dem onstrate that they can read and understand the m eaning of grade level text, by district and school category ............................ 34 Table 14: Percent of students who receive reading help from parents, by sex and school category .............. 34 Table 15: Student perform ance by words-correct-per-m inute (WCPM) categories (percent of 3rd grade students), by sex and school category .....................................................................................................................35 Table 16: Percentage of schools using im proved water sources, by water source type and school category 35 Table 17: Percent of students who can identify at least three key health and hygiene practices, by sex and school category ...........................................................................................................................................................36 Table 18: Percent of students to regularly practice key health and hygiene practices, by type of practice and school category ...........................................................................................................................................................36 Table 19: Support received from organizations, as reported by head teachers (percent of schools), by type of support and school category ................................................................................................................................37 Table 20: Percent of m eals provided by schools during the 2021 school year, by m eal type ........................... 38 Table 21. Percent of parents who contributed to school m eals (2021), by contribution type and school category .......................................................................................................................................................................38 Table 22: Percentage of parents who contributed in-kind to school m eals (2021), by type of contribution and school category ...................................................................................................................................................39 Table 23: Percent of parents that can nam e the benefits of prim ary education, as assessed by head teachers, by type of benefit and school category ...................................................................................................39 Table 24: Percentage of students receiving daily m eals (2021), by grade, sex, and school category............... 40 Table 25: Percent of schools providing additional food item s for school m eals, by type of food and school category .......................................................................................................................................................................41 Table 26: Average num ber of m eals provided in the last week that include foods in addition to the donated US com m odities, by type of food and school category.......................................................................................... 42 Table 27: Source of food for school feeding program m e, by source and school category ............................... 42 Table 28: Percent of schools that have a partnership with any farm ers groups for food in 2021 ................... 43 Table 29: Education sector work by other international actors ............................................................................ 53 Table 30: Sam ple size selection .................................................................................................................................85 Table 31: EGRA target sam ple sizes: baseline, m idterm and endline (# students)............................................. 86 Table 32: Sum m ary of key inform ant interviews conducted ...............................................................................126 Table 33: Focus group discussions conducted ......................................................................................................126 Table 34: Baseline study tim eline ...........................................................................................................................129 Table 35: Evaluation tim eline 2020-2025 ...............................................................................................................132 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 iv Ex e cu t ive Su m m a r y 1. Pu rpose a nd scope of t he eva lua t ion . This is the baseline evaluation for the United States Departm ent of Agriculture (USDA) McGovern-Dole International Food for Education and Child Nutrition Program m e (McGovern-Dole) Grant for Hom e -Grown School Feeding Programme (HGSF) in Rwanda (2020-2025). This study was com m issioned by the WFP Rwanda country office. The m ain objectives of the baseline are accountability to beneficiaries and donors, and learning. Findings will inform Phase II im plem entation and support the rollout of the national school feeding program m e. The baseline study focuses on program m e indicators (developed by the country office based on USDA perform ance indicators) for which the external evaluation firm is responsible and covers the period from 11 August 2021 to 18 February 2022. The baseline addresses 10 evaluation questions, adapted from the m idterm and endline evaluation questions presented in the TOR and detailed in an evaluation m atrix. The baseline study will system atically em ploy the standard evaluation criteria of relevance, coherence, effectiveness, efficiency, im pact, and sustainability. 2. Con t e xt . Rwanda ranks 160th out of 185 countries in the 2020 Hum an Developm ent Index.1 The m ultidim ensional poverty headcount ratio 2 was 28.7 percent in 2016.3 Food insecurity affects 18.7 percent of Rwandan households,4 and Western and Southern provinces, where the m ajority of the HGSF schools are located, have the highest prevalence of food insecure households.5 Undernutrition translates into a greater probability of starting school at a later age, grade repetition, and a higher proportion of school drop-outs. The prim ary school com pletion rate has increased steadily since 2015,6 though dropout rates have also increased.7 In 2019, the Governm ent of Rwanda introduced a National School Feeding Policy and strategy that form ed the basis for the National School Feeding Program m e (NSFP); a budget to im plem ent the program m e was approved in 2020, and the NSFP began rolling out to 3.3 m illion students in October 2021. 3. Su b je ct of t h e e va lu a t ion . The McGovern-Dole Program m e supports school feeding and related activities in 136 pre - and prim ary schools in seven districts of Rwanda: Karongi, Rutsiro, Nyam agabe, and Nyaruguru (“Group 1” schools), which were supported in the FY15 McGovern-Dole Program (“Phase I”), and Burera, Kayonza and Gasabo (“Group 2” schools) added in FY20 (“Phase II”). Activities in Group 2 schools will com m ence once the baseline study is com pleted. The baseline includes schools not participating in either phase as “control” schools. In Phase II, the program m e will expand to pre -prim ary students and include the provision of a hot m eal to all students consisting of USDA com m odities, and fresh foods funded by com plem entary funding from other donors. The McGovern-Dole Program strategic goals are to im prove literacy of school-age children (MGD SO1) and increase the use of health and dietary practices (MGD SO2) in the targeted areas. Over the life of program m e, 117,095 social assistance beneficiaries (students receiving m eals or take -hom e rations)8 will benefit from program m e activities, as well as 15,000 non-student beneficiaries (e.g., teachers, parents, local governm ent, farm ers, cooks, storekeepers, com m ittee m em bers, etc.), and 351,285 indirect beneficiaries. 9 1 UNDP. 2020. Hum an Developm ent Report 2020. 2 This refers to the percentage of the population at risk of suffe ring m ultiple deprivations—that is, those with a deprivation score of 20–33 percent. 3 World Bank database. 2021. Consulted 6 Decem ber 2021. 4 WFP Rwanda. 2018. Com prehensive food security and vulnerability analysis - Rwanda. Decem ber. Data collected March￾April 2018. 5 WFP Rwanda. 2018. Com prehensive food security and vulnerability analysis - Rwanda. Decem ber. Data collected March￾April 2018. 6 National Institute of Statistics of Rwanda. 2020. Rwanda Statistical Yearbook 2020. Secondary com pletion rates not reported. 7 Republic of Rwanda. Ministry of Education. Education Statistics. 2019. 8 “Social assistance beneficiaries” are defined by USDA as “stude nts that received school m eals and/or take -home rations.” See USDA Foreign Agricultural Service – Food Assistance Division. 2019. USDA Food Assistance Indicators and Definitions.” February. Page 98. 9 Statistics and descriptions per em ail com m unication from WFP Rwanda country office, 28 March 2022. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 v 4. Ma in u sers/int en ded a u dien ce . The intended users of the baseline study are WFP stakeholders at national, regional and corporate levels; USDA and other donors; governm ental and non-governm ental partners; and the targeted com m unities and beneficiaries. 5. Me t h odology a n d da t a colle ct ion m e t h ods for t h e ba se lin e . The baseline study is a m ixed-method evaluation that em ploys a student literacy assessm ent, qualitative interviews with program m e stakeholders, e-surveys, and docum ent review. It provides context for program m e im plem entation at the initial stage of Phase II, and establishes baseline indicator values for perform ance indicators that will serve as a basis for com parison and interpretation of program m e results at m idterm and endline. Quantitative results in program m e schools are tested for statistical com parison with control schools. Challenges to the study included the de layed start of the baseline; a low response rate to the census e-survey, and achieving a gender balance in FGDs and KIIs due to the high percentage of m ales in key roles. 6. Fin d in gs: Re le va n ce . The McGovern-Dole Program is seen to be relevant to the needs of both Group 1 and Group 2 schools. Group 1 teachers and cooks report that m alnutrition and hunger am ong students have decreased due to the programme and that this has strongly im proved attendance, handwashing behavior, and attentiveness, and ensured that children from poor fam ilies get at least one nutritious m eal per day. Group 2 teachers expect that the program m e will reduce dropout rates, increase attendance and perform ance, im prove educational quality, fight m alnutrition, increase student socialization, help poor fam ilies, and change parent m indsets on the need to contribute to school m eals. Com ponents of the programm e have been a m odel for Governm ent as it expands school feeding nationally. Most district officials report good collaboration with WFP. Te achers and school com m ittee m em bers indicate that the poorest fam ilies struggle to m ake cash contributions to the program m e due to poverty and large fam ilies. 7. Fin d in gs: Coh e re n ce . The programme aligns with national objectives, policies, strategies and plans to im prove literacy of school-aged children. Qualitative interviews with district officials acknowledged strong involvem ent by the MINEDUC and MINAGRI in the program m e. The programme has adopted and collaborated with the Governm ent on som e key infrastructure, such as the addition of girls’ sanitary room s and perm anent handwashing stations. The literacy com ponent aligns well with other literacy and education initiatives in the country. Activitie s have also aligned with national priorities to im prove health and dietary practices, principally during the COVID-19 pandem ic. The programme has m ade substantial contributions to national capacity to design, m anagem ent, and im plem ent school feeding program m es, particularly through its work with the NSF Technical Working Group to operationalize the school feeding policy. Additionally, program m e partners are working with the Rwanda Education Board to develop tools to train teachers on language proficiency as schools transition to English language instruction. 8. Fin d in gs: Effe ct ive n e ss. The McGovern-Dole Program in Phase 1 was effective in im proving attendance and reducing dropouts. Headm asters in Group 1 schools com pleted a high percentage of trainings or certificates in school m anagem ent through the HGSF program m e. However, there are som e deficits that show in the Group 1 baseline results. Interruptions from COVID-19 related school closures lim ited the effectiveness of literacy program m es. This m ay have had unintended downstream effect on students’ literacy levels in Group 1 as control group students scored better on literacy tests than those from Group 1 or Group 2 schools. 9. All Group 1 and Group 2 schools a re using im proved water sources. During the COVID-19 pandem ic, the program m e established perm anent handwashing stations at participating schools and girl’s sanitary room s, both of which district officials noted as having greatly contributed to im proved hygiene outcom es. 10. Fin d in gs: Efficie n cy. While the use of resources am ong Group 1 and Group 2 schools was generally effective, there is room for greater efficiency in the program m e. Overcrowding of program m e schools contributes to program m e inefficiency. The Governm ent constructed over 22,000 new schools in 2021 10 to address overcrowding, though the student-teacher ratio at Group 1 schools has rem ained high. Crowded schools com prom ise the ability for feeding program m es and results in reduced m eal sizes for som e students. 11. Fin d in gs: Im p a ct . The McGovern-Dole Program has had a positive im pact on its beneficiaries and the Phase 1 results, as observed at the Phase 2 baseline, provide indication of im pact pathways for Phase 2. Students are reportedly m ore attentive because of the school m eals and attendance for both boys and girls has increased. The program m e has provided jobs for cooks and security guards to the surrounding com m unities. Parents have m ore tim e to devote to incom e activities instead of preparing lunch as children receive m eals 10 Edwin Ashim we. 2021. The New Tim es. Construction of new classroom s at over 95%. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 vi at school. Teachers report decreased rates of m alnutrition and im proved hygiene practices am ong students and com m unities surrounding NSFP schools. Som e external factors that currently and m ay continue to dampen the program m e’s im pact are the econom ic fallout from the COVID-19 pandem ic and social attitudes towards girls’ education. The draft gender study includes recom m endations for WFP to address som e of these barriers. 12. Fin d in gs: Su st a in a b ilit y. An im portant contribution to sustainability and consistency of approach are the national School Feeding Operational Guidelines, developed with technical support from WFP to support the Governm ent’s school feeding strategy. The guidelines were approved and dissem inated by MINEDUC in 2021 and form the basis of trainings for key stakeholders in the NSFP. The ability of parents to contribute to the cost of school m eals rem ains a challenge. 13. Con clu sion s: Re le va n ce . The programme is relevant to intended beneficiaries as it is seen as contributing to reduced dropouts, im proved literacy, reduced m alnutrition, and im proved hygiene and sanitation. The capacity strengthening support given by WFP to MINEDUC and others is relevant to institutional needs and the goals of the NSFP. 14. Con clu sion s: Coh e re n ce . The programme aligns with national objectives, policie s, strategies and plans to im prove literacy of school-aged children. Program m e partners are also working with the Rwanda Education Board to develop tools to train teachers on language proficiency and m ethodologies to support their ability to teach subjects in English. 15. Con clu sion s: Effe ct ive n e ss. Literacy levels in Group 1 and Group 2 schools are lower than expected due to COVID-19 related school interruptions. Moreover, a higher proportion of students from control schools perform better on literacy tests com pared to students in program m e schools with possible reasons for this explored in the m ain report. Specifically for Group 1 schools, this baseline finds continued positive results in effective use of health and dietary practices, m ore productive school gardens, and continued com m unity outreach on nutrition education. 16. Con clu sion s: Efficie n cy. There is room for greater efficiency in Group 1 and Group 2 schools as overcrowding of treatm ent schools com prom ises the ability of feeding program m es to provide enough food to each child, especially when pre -prim ary children also receive food. 17. Con clu sion s: Im p a ct . Overall, the baseline still finds positive im pact for the Phase 1 program m e in term s of on-tim e attendance, im proved hygiene, reduced m alnutrition, and an im proved teaching and learning environm ent. Parents also indicate that school m eals free their tim e for incom e -earning activities. However, the pandem ic has reduced incom es and cash contributions to schools, m aking it difficult for som e schools to pay for cooks, firewood, and other supplies. 18. Con clu sion s: Su st a in a b ilit y. The national School Feeding Operational Guidelines developed with programme support are an im portant contribution to sustainability. Many of the individual NSFP readiness com ponents have the potential to sustain. School Readiness Scorecard reports that all program m e schools have established School Feeding Com m ittees and School Tender Com m ittees, m ost of which are operational. 19. The program m atic lessons learned from the baseline study are :  The country office should continue to support initiatives to address the issue of parent contributions for the poorest fam ilies.  Many schools report it a challenge to pay increased water bills.  Phase II targets should be reviewed with specific target setting for Group 1 and Group 2 schools.  Local ownership of nutrition education and interventions is critical.  Com plem entary funding is being used to benefit NSFP schools.  Supporting governm ent goals in the NSFP is key. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 1 1. تn t r o d u ct io n 1.1. Eva lu a t io n fe a t u r e s 20. Pu rpose a n d ra t ion a le. This is the report of the baseline study conducted for the United States Departm ent of Agriculture (USDA) McGovern-Dole International Food for Education and Child Nutrition Programme (McGovern-Dole) Grant for World Food Programme (WFP) Hom e -Grown School Feeding Programme in Rwanda (2020-2025). 11 The current programme (“Phase II”) is the successor to the previous programme im plem ented 2016-2021.12 The Phase II baseline study, com m issioned by the WFP Rwanda country office, is part of a five -year decentralized evaluation contract (2020-2025) where the baseline study, m idline and endline will be carried out as part of one package by an independent evaluation firm . The three exercises are designed to provide evidence -based independent m easures of agreed-upon programme indicators. 21. Object ives. Accountability and learning are the m utually reinforcing objectives of the baseline. The baseline will establish a starting point to m easure progress on programme objectives, enable accountability to stakeholders on whether programme activities appropriately m eet beneficiary needs, and if the programme achieves its goals. The baseline will aid learning by providing inform ation on the initial status, attitudes and challenges of beneficiaries and control schools that will help guide programme strategy and im plem entation. During Phase II of the McGovern-Dole Programm e, learning will be em phasized, and evaluation findings will be used to support the transition of McGovern-Dole Program supported schools into the national school feeding programme . Issues relating to gender equality and wom en’s empowerment (GEWE) will be m ainstream ed across the two objectives. 22. The baseline study will serve the following specific objectives 13, based on the term s of reference: • Confirm indicator selection and targets and establish baseline values for all perform ance indicators included in the proposal, including for com parison schools. The baseline study will also be used to revisit program m e targets in light of baseline findings where relevant and to review the results fram eworks or theory of change, • Be used for ongoing program m e m onitoring activities to regularly m easure activity outputs and perform ance indicators for lower-level results, • Measure perform ance indicators for McGovern-Dole strategic as well as the highest-level results that feed into the strategic objectives as part of the m id-term and final evaluations, and • Provide a situational analysis before the program m e begins and confirm the full evaluation design as prepared during the inception period. This analysis will inform program m e im plem entation and will provide im portant context necessary for the m id-term and final evaluations to assess the program m e’s re levance, effectiveness, efficiency, sustainability, and im pact. 23. Su b je ct a n d scop e . Phase II of the McGovern-Dole Programm e is im plem ented jointly with the Rwanda Ministry of Education (MINEDUC), Ministry of Agriculture and Anim al Resources (MINAGRI), National Child Developm ent Agency (NCDA), World Vision International (WVI), Gardens for Health International (GHI), Rwanda Biom edical Centre (RBC) and the Districts of Education for Karongi, Rutsiro, Nyam agabe, Nyaruguru, Burera, Kayonza and Gasabo districts. It supports school feeding and related activities in 136 pre - and prim ary schools in seven districts of Rwanda. As a whole these activities are intended to im prove outcom es in student learning and literacy, school enrolm ent and attendance, and health and dietary practices, and to strengthen governm ent capacity to im plem ent the national school feeding programme . The programme has com plem entary household- and com m unity-level interventions in support of these objectives, and also works with farm er cooperatives to enhance their capacity to provide nutritious food to the national school feeding programme . Section 2.1 has further programme details. 11 Hereafter referred to as “McGovern-Dole Program m e.” 12 Phase I of the WFP USDA McGovern-Dole International Food for Education and Child Nutrition Program support in Rwanda took place from 2016 to 2021 (an extension was granted to the original 2020 end date due to COVID-19). 13 The baseline study will be undertaken in parallel to a separately com m issioned gender analysis. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 2 24. The baseline study focuses on the USDA McGovern-Dole Program indicators, and investigation into the activities and context expected to influence indicator perform ance. Data collection in the field ran from 25 October to 25 Novem ber 2021, alongside rem ote interviews that extended into early Decem ber. 25. While the baseline evaluation team will include a gender focus in qualitative data collection and in the overall analysis, findings perm itting, WFP has separately com m issioned an independent gender analysis conducted in parallel with this baseline study. The analysis is designed as a form ative assessm ent, to include a com prehensive review as well as “deep dives” of gender dim ensions. The gender exercise was initially envisioned under the scope of the baseline, but since the term s of reference were form ulated, it was decided that this analysis instead be a parallel process undertaken by a gender consultant under a separate scope of work. 26. St a ke h old e rs a n d u se rs. The m ain users of the baseline study and later evaluations are WFP stakeholders at national, regional and corporate level; USDA and other donors; governm ental and non-governm ental partners; as well as the com m unities and beneficiaries the programme is intended to serve. Within WFP, the m ain stakeholders and users are the WFP Rwanda country office, Regional Bureau (Nairobi), the School-based Programme s Division in headquarters, the Office of Evaluation in headquarters, and the Executive Board. External stakeholders include the schools and com m unities affected by the programme , national and district governm ent, donors, im plem enting partners, the School Feeding Technical Working Group, and the United Nations Country Team . The baseline study will allow these stakeholders to understand the context and indicator perform ance at baseline and enable perform ance and higher-level results analysis at m idterm and endline. 27. Eva lu a t ion t e a m . The baseline study was conducted by TANGO International, a United States-based evaluation firm , including a core team of two international and four national evaluators, with analytical, research and quality assurance support from the firm ’s headquarters staff. The sam e core team of evaluators carried out the m idterm evaluation (2018) and the final evaluation (2021) of Phase I and thus was highly fam iliar with the programme context, objectives, evolution, and perform ance over tim e. 28. Eva lu a t ion approa ch. The baseline study is a m ixed-m ethod evaluation that em ploys a student literacy assessm ent, interviews with programme stakeholders and participants, e -surveys, and docum ent review. It provides context for programme im plem entation at the initial stage of Phase II and establishes baseline indicator values for perform ance indicators. This report deliberately om its com parisons of Phase II baseline data and Phase I endline data; however, it is useful to read this baseline report in conjunction with the endline evaluation. The baseline data and inform ation will serve as a basis for com parison and interpretation of programme results of the m idterm evaluation (March-May 2023) and endline evaluation (July-Septem ber 2025). 1.2. Context Geogra phy 29. Rwanda is a sm all, hilly, landlocked, and densely populated country in East Africa with just under 13 m illion people (2021 projection).14 It is highly vulnerable to drought and landslides and m oderately vulnerable to earthquakes and windstorm s.15 Poverty food, a nd nutrition security 30. Rwanda ranks 160th out of 185 countries in the 2020 Hum an Developm ent Index. 16 The poverty headcount ratio at the international poverty line of $1.90 a day (2011 purchasing power parity) has been steadily 14 National Institute of Statistics of Rwanda. 2021. Consulted 6 Decem ber 2021. 15 Rwandan Ministry of Disaster Managem ent and Refugee Affairs (MIDIMAR). 2015. The National Risk Atlas of Rwanda. 16 UNDP. 2020. Hum an Developm ent Report 2020. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 3 decreasing, from 69.1 percent of the total population in 2005 to 56.5 percent in 2016. 17 The m ultidim ensional poverty headcount ratio 18 was 28.7 percent in 2016, down from 32.9 percent in 2013. 19 31. Food insecurity affects 18.7 percent of Rwandan households; of these, 1.7 percent are severely food insecure.20 Undernourishm ent rose from 29.5 percent of the population in 2008 to 35.2 percent in 2019.21 Western and Southern province s, where the m ajority of the HGSF schools are located, have the highest prevalence of food insecure households (29.9 percent and 20.5 percent, respectively, followed by Northern and Eastern province s (17.8 percent and 16.2 percent, respectively).22 Prevalence of m oderate and severe food insecurity is significantly higher in households in Ubudehe23 category 1 – the poorest category (28 percent m oderately food insecure and 4 percent severely food insecure).24 Sim ilarly, the highest rates of m oderate and severe food insecurity are found in the poorest wealth quintile (34 percent and 4 percent, respectively).25 32. Table 1 presents key child nutrition indicators for Rwanda. Ta b le 1: Ch ild n u t rit ion in d ica t ors for Rw a n d a Total Male Fem ale Prevalence of stunting am ong children under 5 years of age (2020)a 33.1 37.0 29.2 Prevalence of wasting am ong children under 5 years of age (2020) a 1.1 0.9 1.4 Prevalence of underweight am ong children under 5 years of age b 7.7 9.0 6.3 Prevalence of overweight am ong children under 5 years of age (2020) a 5.6 5.8 5.4 Source s: a National Institute of Statistics of Rwanda. 2021. Rwanda Dem ographic and Health Survey 2019-2021. Septem ber. b World Bank database. 2021. Consulted 6 Decem ber 2021. 33. Micronutrient deficiencies in Rwanda are a concern; 37.9 percent of children under five and 17.2 percent of wom en of reproductive age (15-49 years) are anem ic. 26 The m ost com m on causes of anem ia are lack of iron in the diet and intestinal worm s. Prevalence of intestinal worm s is widespread, affecting 45.2 percent of the population (2014), especially school-aged children.27 WFP and partners carried out a knowledge, attitudes and practices (KAP) study on deworm ing in Rwanda in 2021.28 The study found that while there is very high ownership of deworm ing program s, there are significant gaps in training of teachers and health workers on soil transm itted helm inths (i.e., worm s) and schistosom iasis. While 54.1 percent of com m unity health workers 17 World Bank database. 2021. Consulted 6 Decem ber 2021. 18 This refers to the percentage of the population at risk of suffe ring m ultiple deprivations—that is, those with a deprivation score of 20–33 percent. 19 World Bank database. 2021. Consulted 6 Decem ber 2021. 20 WFP Rwanda. 2018. Com prehensive food security and vulnerability analysis - Rwanda. Decem ber. Data collected March-April 2018. 21 World Bank database. 2021. Consulted 6 Decem ber 2021. 22 WFP Rwanda. 2018. Com prehensive food security and vulnerability analysis - Rwanda. Decem ber. Data collected March-April 2018. 23 Since 2015, the Governm ent of Rwanda has adopted a system of classifying all Rwandan households in four “Ubudehe” categorie s that reflect their econom ic status. 24 WFP Rwanda. 2018. Com prehensive food security and vulnerability analysis - Rwanda. Decem ber. Data collected March-April 2018. 25 WFP Rwanda. 2018. Com prehensive food security and vulnerability analysis - Rwanda. Decem ber. Data collected March-April 2018. 26 National Institute of Statistics of Rwanda. 2021. Rwanda Dem ographic and Health Survey 2019-2021. Septem ber. 27 The END Fund. 2018. Partnering to End Neglected Tropical Diseases in Rwanda. 28 WFP, University of Global Health Equity, Rwanda Biom edical Centre. Agents of Change in the education, sensitization, and reduction of deworm ing in Rwanda. Policy Brief. Novem ber 2021. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 4 and teachers dem onstrated good knowledge of these infections, only 3.5 percent of teachers were trained on worm infections, and there is especially low knowledge of sym ptom s. Agricult ure 34. National agriculture is critical to food security and an im portant sector of the Rwandan econom y, though the percentage value -added of agriculture, forestry, and fishing, has been decreasing since the late 1990s; in 2020 it was 26.3 percent of Gross Dom estic Product (GDP).29 The annual growth rate in these sectors has rem ained fairly stable in the past five years, ranging from a high of 5.03 percent in 2019 to a low of 0.86 percent in 2020.30 The m ajority of households in Rwanda are sm allholder farm ers with sm all plots of land. About 24 percent of the land is owned by wom en, 31 who are m ainly engaged in production rather than better-paying value -added agricultural processing and m arketing activities. SDG 2/ SDG 17 Overview 35. Rwanda has integrated the Sustainable Developm ent Goals into its national developm ent agenda through the draft Vision 2050, National Strategy for Transform ation (NST 1, 2017-2024) and related strategies at different levels.32 The Voluntary National Review (VNR) in 2019 reported that Rwanda has developed com prehensive policies and strategies that contribute to ending hunger, achieving food security and im proving nutrition and has been m aking steady progress in its fight against hunger and m alnourishm ent. 33 Rwanda’s SDG m onitoring process includes Com prehensive Food Security and Vulnerability Analyses (CFSVA). The latest CFSVA (2018) shows continuing positive trends under SDG 2 in reducing poverty and m alnutrition and in agriculture sector growth. 36. Under SDG 17, the VNR reports that Rwanda has created inclusive partnerships with governm ents, the private sector, civil society and developm ent partners that have delivered visible changes on the ground, and that foster the country’s developm ent based on its m edium -term plan and long-term vision. NST I em phasize s the im portant role of the private sector. While Rwanda registered encouraging results overall in term s of total governm ent revenue as a proportion of GDP (Indicator 17.1.1), peaking at 24.1 percent in 2018, the Governm ent recognizes the need to enhance dom estic resource m obilization and sustain progress in the m edium and long term . Rwanda continued to im plem ent its policy of self-reliance by financing a large share of its budget from dom estic resources (Indicator 17.1.2). Rwanda is also encouraging its citizens living abroad to contribute to the national developm ent process (Indicator 17.3.2). As a result, the volum e of rem ittances as a proportion of total GDP has continued to increase, despite global econom ic uncertainties.34 Governm ent policies a nd priorities releva nt to the programme Overall 37. The Governm ent of Rwanda is guided by the national developm ent plan Vision 2050, which envisions Rwanda transform ing from an agrarian to a knowledge -based econom y, attaining upper m iddle -incom e country status by 2035 and high-incom e status by 2050. The country’s poverty reduction strategy, the National Strategy for Transform ation (2017-2024) prioritizes quality education for all as a prerequisite for a knowledge - based econom y. Other governm ent policie s relevant to the programme are the national Food and Nutrition Policy (2013-2018), which focuses on elim inating chronic m alnutrition, and the School Health and Nutrition Policy (2014) that declares that all schoolchildren shall study in a healthy environm ent in child-friendly schools. 38. Rwanda joined the global School Meals Coalition (SMC) in June 2021 and is one of 11 taskforce m em bers. The coalition brings together 64 m em ber governm ents, United Nations agencies, civil society, the private sector and academ ia to ensure that every child receives a healthy, nutritious m eal in school by 2030. Mem ber states like Rwanda sign a national com m itm ent to scale up or im prove dom estic program m es, to be accountable 29 World Bank database. 2021. Consulted 9 Decem ber 2021. 30 World Bank database. 2021. Consulted 9 Decem ber 2021. 31 Gender Monitoring Office. 2019. The State of Gender Equality in Rwanda. 32 United Nations Division for Sustainable Developm ent Goals. 2021. Consulted 6 Decem ber 2021 33 United Republic of Rwanda. 2019. 2019 Rwanda Voluntary National Review Report. June. 34 United Republic of Rwanda. 2019. 2019 Rwanda Voluntary National Review Report. June. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 5 for their goals, and to support other countries' program m es. WFP is the global lead for the m onitoring and accountability com ponent of the SMC. Education policies 39. The National School Feeding (NSF) Policy (2019) presented the initial fram ework for developm ent of Rwanda’s National School Feeding Programme (NSFP). The policy calls for com prehensive, universal pre -, prim ary and secondary school coverage with a focus on water, sanitation, and hygiene (WASH) and nutrition, plus unconventional procurem ent for local purchase of fresh, nutritious, foods to enhance nutrition, dietary diversity, and econom ic developm ent for rural sm allholder farm ers through linkage to the reliable NSFP m arket for local produce. The NSF TWG was established in late 2019 to coordinate school feeding stakeholders and investm ents at high level and align the programme with long-term governm ent strategy; the TWG m eets quarterly. Term s of reference have been approved for WFP to co-chair the TWG with MINEDUC, as well to co-chair the NSF Steering Com m ittee (NSFSC), which will oversee strategic direction of the NSFP and coordinate programming across Rwanda’s 30 districts (at this writing, the NSFSC is not yet operational/active). The National School Feeding Steering Com m ittee (NSFC) was recom m ended as a policy action by the Com prehensive National School Feeding Policy to enhance m ulti - sectoral governance and institutional coordination arrangem ents and enforce shared responsibility by all stakeholders. The NSFSC will bring together m inistries, governm ent intuitions and developm ent partners. Following the approval of the school feeding policy, the term s of reference establishing the NSFSC were approved by the Ministry of Education in August 2020. The m ain purpose of the NSFC is to provide strategic guidance for the developm ent and im plem entation of a national school feeding program m e in line with the national policy and strategy in addition to overseeing all individual school feeding interventions in the country. The NSFC is m ade up of 16 governm ent m inistries and institutions and six developm ent partners, Chaired by the MINEDUC and co￾chaired by MINAGRI and WFP. 40. In July 2019, WFP and MINEDUC spearheaded developm ent of draft School Feeding Operational Guidelines; these were approved in 2021 and dissem inated to all schools nationally. The School Readiness Scorecard indicates that 91.6 percent of all program m e schools are com plying with these guidelines.35 In 2021, WFP and MINEDUC also developed cook/storekeeper training m aterials; validation of the m aterials is expected by the end of 2022. Training and school com m ittees will be rolled out in phases in 2022.The School Readiness Scorecard reports that all program m e schools have established School Feeding Com m ittees; of these, 94.4 percent are operational, and 38.3 percent have been trained by local/district authorities. The Scorecard indicates that School Tender Com m ittees have been established in 99.1 percent of program m e schools, are operational in 64.5 percent of these schools, and 22.4 percent have received training from local/district authorities. According to MINEDUC and WFP and McGovern-Dole country-specific guidance, the process to operationalize Rwanda’s NSFP will require additional capacity building at local, district, and national levels, to include procurem ent, logistics, storage, safe food handling and hygiene (including to cooks and storekeepers); and system s strengthening. 41. The Education Se ctor Strategic Plan (2018/19-2023/24) prom otes equal educational access for girls, children from poor fam ilies, and people with disabilities, though equitable access to disadvantaged students rem ains a challenge.36 The Governm ent provides a subsidy equal to approxim ately 40 percent of the cost a m eal, while parents are expected to contribute about 60 percent of the cost, in addition to other school feeding costs such as cook salaries and firewood. 42. A draft National Literacy Policy was introduced in 2019 and is awaiting approval by Governm ent. The policy was developed with technical support from the USAID-funded Mureke Dusom e project and is supported by WVI and other non-governm ental organizations. The policy supports full literacy developm ent am ong all Rwandan citizens and em beds literacy skills within the right to education. The draft policy serves as an overarching docum ent for national literacy developm ent and addresses a policy gap around early literacy developm ent as the foundation for all future learning. The draft policy supports greater coordination and policy coherence and objectives across m ultiple sectors and governm ent institutions, led by MINEDUC.37 35 WFP Rwanda. 2022. School Readiness Scorecard Excel file: HGSF_SchoolLevel_Score card_19JAN2022. 36 Republic of Rwanda , Ministry of Education. 2018. Education Sector Strategic Plan, 2018/19-2023/24. 37 Republic of Rwanda, Ministry of Education. 2019. National Literacy Policy (draft). Septem ber 2019. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 6 43. Another governm ent programme relevant to school feeding is One Cup of Milk, initiated in 2010 to address m alnutrition and dairy sector developm ent. In addition, the Secondary School Feeding Programme was initiated in 2014 to cover three m eals per day, via cash transfer to boarding schools. This programme has expanded into the NSFP. In 2020, the Governm ent announced the plan to scale up the program m e to over 3.3 m illion pre -prim ary, prim ary and secondary students in all public and governm ent-aided schools. 38, 39 44. Sou rcing school m ea ls, The National School Feeding policy and strategy calls for local procurem ent of fresh, nutritious foods by schools to enhance nutrition and dietary diversity in school m eals while supporting the econom ic developm ent of agricultural cooperatives and sm all farm ers. Gender policies 45. GEWE is a key overarching principle in the Governm ent of Rwanda’s legal and policy fram eworks. Rwanda’s constitution recognizes wom en’s rights, gender equality, and wom en’s legal rights to land and inheritance. NST I includes indicators for gender progress in education, em ploym ent, health, and other areas. Rwanda was ranked seventh in the 2021 Global Gender Gap Index by the World Econom ic Forum . 40 However, there is less progress in other areas, and the 2020 Gender Inequality Index, which captures inequality in reproductive health, em powerm ent, and labor m arkets, ranks Rwanda 92nd out of 189 countries.41 46. Rwanda’s Revised National Gender Policy (2021) is the Governm ent’s m ain strategic guidance docum ent. The policy states its m ission as “to ensure that gender gaps across sectors are addressed through accelerating effectiveness of gender m ainstream ing, gender-responsive interventions, and gender accountability to position Rwanda as a global m odel in prom oting gender equality. The overall goal of this policy is to im prove gender equality and equity in various sectors while increasing wom en’s access to productive econom ic resources and opportunities and ensuring that wom en and m en are free from any form of gender-based violence and discrim ination.” The policy em phasizes the engagem ent of men and boys and accelerating gender m ainstream ing in the private sector.42 Rwanda has signed onto various com m itm ents arising from the Novem ber 2021 African Union (AU) Men’s Conference on Positive Masculinity, including the AU’s com m itm ent to scale up proven solutions to com bat violence against wom en and girls, and im plem enting and m onitoring the AU Maputo Protocol Scorecard and Index, a tool designed to assess progress on gender equality, wom en’s rights, and wom en’s em powerm ent com m itm ents under the Maputo Protocol.43 Educa t ion context 47. Rwanda is ranked in the lowest quarter of the World Bank’s Hum an Capital Index (HCI) (2018). The HCI (2018) reported that the future potential of Rwandan youth is 63 percent below what they could achieve with better health and education. The report attributed this in part to the high stunting levels and the poor quality of education, reflected in low overall test scores and high repetition rates. Undernutrition underm ines student educational perform ance through hunger, frequent illness, and lim ited learning capacity associated with deficient cognitive developm ent. This translates into a greater probability of starting school at a later age, grade repetition, drop-out, and ultim ately a lower level of education. The HCI analysis generated high-level governm ent discussions on school feeding that were followed by its endorsem ent of the NSFP in 2019, and budget expansion to im plem ent the NSFP. 48. Rwanda has im proved the quality, coverage, and access to basic education through the Education Sector Strategic Plan (2018/19-2023/24). The Governm ent provides 12 years of free, com pulsory, basic education to all children. In 2019, MINEDUC changed the language of instruction for lower and upper prim ary from 38 Global Child Nutrition Forum . 2020. National School Feeding Program in Rwanda: Virtual Conference Nov 9, 2020. 39 The New Tim es. 2021. Govt injects Rwf27bn towards school feeding. July 26. 40 World Econom ic Forum . Global Gender Gap Report 2021. Insight Report. March 2021. United Nations Developm ent Program m e. Hum an Developm ent Report 2020. 41 World Econom ic Forum . Global Gender Gap Report 2021. Insight Report. March 2021. 42 Republic of Rwanda, Ministry of Gender and Fam ily Prom otion. 2021. Revised National Gender Policy – February 2021. 43 African Union News. 2021. Men’s Conference on Positive Masculinity: “Galvanizing Positive Masculinity to end the scourge of violence in Africa.” 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 7 Kinyarwanda to English.44 The gross enrolm ent rate for pre -prim ary (nursery) school increased from 23.8 percent in 2016 to 29.9 percent in 2019 and the net enrolm ent rate increased from 17.5 percent to 24.6 percent over the sam e period. Nevertheless, these rates are very low com pared to the SDG target of 100 percent by 2030.45 The prim ary school com pletion rate has increased steadily in recent years, from 60.4 percent overall in 2015 to 95.4 percent in 2019.46 However, dropout rates increased from 5.7 percent to 7.8 percent in the sam e period.47 Fem ale enrolm ent in both lower and upper secondary has been slightly but consistently higher than m ale enrolm ent in all years from 2015-2019. Rwanda’s Gender Parity Index48 was 0.99 in 2018, indicating that girls are near parity with boys.49 49. In 2016, 0.83 percent of children enrolled in pre -prim ary education were identified as having a disability; in prim ary schools, children with disabilities represent 0.75 percent of the total num ber of children enrolled, a sim ilar percentage to the previous three years. This represents fewer than one percent of enrolled students and is significantly below the expected num bers within the population. Students with a disability represented only one percent of the total enrolled in secondary education in 2016. 50. WASH infrastructure in schools is poor. The Prim ary schools have one toilet per 52 per students (51 per fem ales and 49 for m ales) and one per 11 staff (12 for fem ales and 10 for m ales). 50). Only 54.1 percent of schools have piped water. MINEDUC estim ates that m enstruation accounts for an average of 50 days/girl/year in absences. The lack of physical and social accessibility and affordability of sanitary pads for m enstrual hygiene m anagem ent (MHM) In Rwanda causes 18 percent of wom en and girls to m iss work or school.51 Since 2015, MINEDUC has acknowledged the im portance of MHM, and MHM has becom e increasingly im portant in WASH activities, though financial resources and the lack of water on school prem ises are lim itations. WFP and World Vision have partnered to build girls’ sanitary room s in schools, providing water and soap, sanitary pads, and disposal facilities. The aim is to im prove hygiene and knowledge am ong adolescent girls as well as increase attendance and focus in class. A recent survey of girls aged 10-19 in schools in Rutsiro, Karongi, Nyam agabe, and Nyaruguru supported through WFP hom e -grown school feeding found that 92 percent use the MHM services provided, and 66 percent source their pads from the sanitary room s. Effects of COVID-19 on education Gender cont ext 51. Wom en and girls face inequities arising from traditional gender roles that prioritize fam ily and household duties for fem ales, as well as econom ic and health challenges. Fem ale -headed households com prise 23 percent of all households and are m ore likely to be poorer and m ore food insecure than those headed by m en. Around one -quarter of households in the original four Phase I McGovern-Dole Program supported districts are fem ale -headed. Between 51 percent and 77 percent of households in those districts are in the poorest wealth quintiles. Only 54 percent of fem ale -headed households had som e education versus 80 percent of m ale -headed households. 52 52. Challenges to gender inclusion go beyond ensuring that girls m atriculate in school, including traditional attitudes am ong parents on education for girls. For exam ple, girls are expected to com plete household chores before going to school. The draft gender assessm ent notes that dom estic work is a significant burden 44 MINEDUC. 2019. Com m uniqué: MINEDUC endorses the use of English language as a m edium of instruction in lower prim ary. Decem ber. 45 Republic of Rwanda, Ministry of Education. 2019. 2019 Education Statistics. 46 National Institute of Statistics of Rwanda. 2020. Rwanda Statistical Yearbook 2020. Secondary com pletion rates not reported. 47 Republic of Rwanda. Ministry of Education. Education Statistics. 2019. 48 ratio of girls to boys enrolled in all prim ary schools 49 UNESCO Institute for Statistics. 50 Republic of Rwanda, Ministry of Education. 2019. 2019 Education Statistics. 51 WFP Rwanda. 2021. Asse ssing the Usage of Menstrual Hygiene Managem ent Services and Factors Influencing their Usage am ong Adolescent Schoolgirls in Rwanda. Report. Policy Brief. Novem ber. 52 WFP Rwanda. 2018. Com p rehensive Food Security and Vulnerability Analysis. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 8 for girls and has a strong negative im pact on their ability to attend school or do their hom ework. Dom estic chores often m akes it hard for girls to reach school on tim e, though the McGovern-Dole Program endline evaluation showed that girls’ on-tim e arrival at school im proved when school m eals were served.53 A 2017 study on gender and education in Rwanda found that “despite high levels of knowledge of gender equality am ong respondents, attitudes am ong parents and com m unity m em bers continue to prom ote outdated traditional cultural perspectives on gender.”54 A m ajority of school staff are m ale , providing few role m odels for young girls.55 The gender analysis recently com pleted for WFP Rwanda is largely consistent with these findings and further concludes that widespread attitudes that educa tion is less im portant for girls are well entrenched and present challenges to McGovern-Dole Program goals around equal access to education for girls . 56 53. Rwanda has seen som e progress in term s of enabling girls’ participation in school. MINEDUC has worked with WFP to design m eals to m eet the nutritional needs of adolescent girls, and the education sector plan has a dedicated budget line to address education barriers for girls, including the provision of gender-sensitive WASH facilities – though the Phase I McGovern-Dole Program endline found that greater sensitivity in the siting and access to girls’ sanitary room s is needed, 57 and the current baseline study finds that at some schools the toilet and sanitary facilities were closed, due to schools’ inability to pay water bills (see discussions in Section 4 and Section 5.2). 53 WFP. 2021. WFP's USDA McGovern-Dole International Food for Education and Child Nutrition Programme 's Support in Rwanda 2016-2021. Endline evaluation. Sim ilarly, the evaluation found that household dutie s hinder wom en’s m em bership and participation in agricultural cooperatives, which are an enhanced focus of the Phase II program m e. 54 Governm ent of Rwanda/UNICEF. A Study of Knowledge, Attitudes and Practice s around Gender and Education in Rwanda. October 2017. 55 WFP. 2021. WFP's USDA McGovern-Dole International Food for Education and Child Nutrition Programme 's Support in Rwanda 2016-2021. Endline evaluation. 56 WFP Rwanda. 2021. Gender Assessm ent Hom e Grown School Feeding Program m e. Decem ber 2021. Draft 57 WFP. 2021. WFP's USDA McGovern-Dole International Food for Education and Child Nutrition Programme 's Support in Rwanda 2016-2021. Endline evaluation. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 9 Gender indicators 54. Table 2 presents key gender equality indicators for Rwanda. Ta b le 2: Ge n d e r e q u a lit y in d ica t ors Total Male Fem ale Proportion of ever-partnered wom en and girls aged 15 years and older subjected to physical, sexual or psychological violence by a current or form er intim ate partner in the previous 12 m onths na na 30.0 Physical violence na na 19.7 Sexual violence na na 10.3 Psychological violence na na 23.6 Proportion of wom en aged 20-24 years who were m arried or in a union before age 15 and before age 18 na na Before age 15 na na 0.3 Before age 18 na na 5.5 Proportion of wom en aged 15-49 years who m ake their own inform ed decisions regarding sexual relations, contraceptive use and reproductive health care na na 63.1 Proportion of individuals who own a m obile telephone 54.8 61.6 47.9 a National Institute of Statistics of Rwanda. 2021. Rwanda Dem ographic and Health Survey 2019-2021. Septem ber. Refugees 55. As of October 2021, there are 126, 988 persons of concern in Rwanda, the m ajority refugees from the Dem ocratic Republic of Congo (60.8 percent) and Burundi (38.9 percent). UNHCR Rwanda provides protection and assistance to refugees in six refugee cam ps and three urban areas. Maham a Cam p in Eastern Province hosts the m ost refugees (43.8 percent of the total); Kiziba, Kigem e, and Nyabiheke cam ps each host approxim ately 11-13 percent, and the rem ainder are in Gihem be and Mugom bwa cam ps, with sm aller num bers in the urban areas of Kigali, Nyam ata, and Huye.58 Key ext erna l event s which led t o significa nt cha nges in WFP’s work 56. The COVID-19 pandem ic is the m ain contextual developm ent that has im pacted WFP Rwanda’s work in the past two years. Decem ber 2021 cum ulative figures indicate 100,464 COVID cases and 1,344 deaths.59 Schools closed in March 2020. Upper prim ary (P4-6) and secondary students returned to school in Novem ber 2020, and lower prim ary students (P1-3) returned in January 2021. In response to the 2020 school closures, the Governm ent introduced online and distance learning m odalities, which allowed for continuity of learning; however, there were lim itations due to the availability of internet connectivity. WFP Rwanda and partners adapted in various ways to keep core McGovern-Dole Program activities running, albeit at a reduced level, during the COVID-19 pandem ic. These adaptations took place m ostly in 2020 and early 2021 and included distributing take -hom e rations to ensure that vulnerable schoolchildren had access to food during pandem ic￾related school closures, shifting reading clubs to house -to-house book lending, changing in-person training for school adm inistrators and teachers to self-learning m anuals, incorporating COVID-19 topics into health and hygiene trainings and m essaging m aterials, shifting to radio learning for hygiene and health m essaging, and distributing vegetables from school gardens to vulnerable fam ilies who could not obtain fresh produce due to m ovem ent restrictions. MINEDUC also required that perm anent handwashing stations be established at schools to prevent the spread of COVID-19, which catalyzed partner efforts to replace tem porary handwashing stations with perm anent ones. 57. In 2020, the Global Partnership for Education (GPE) awarded the Governm ent of Rwanda USD 10 m illion to support post-pandem ic school re -openings. Half of this grant provided a three -month top-up to 25 percent of the 3.3 m illion students supported by the NSFP. Another USD 1 m illion was allocated to the construction of 1,348 perm anent handwashing stations in schools, using a design developed jointly by WVI, WFP, MINEDUC and UNICEF. Most schools in the country now have perm anent handwashing stations, and m ore classroom s 58 UNCHR. 2021. Operational Data Portal. Consulted 7 Decem ber 2021. 59 World Health Organisation. Rwanda COVID-19 update. 9 Decem ber 2021. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 10 have been built in areas where the NSFP operates. Additionally, in 2021 the Ministry of Education, with funding from the World Bank, has constructed a total of 22,214 new classroom s, approxim ately 98.86 percent against its goal to have 22,505 new classroom s by the end of 2021. 60 Fea t ures of int erna t iona l a ssist a nce in t he a rea 58. The am ount of net official developm ent assistance (ODA) and official aid Rwanda received increased from USD 1.134 billion in 2015 to USD 1.21 billion in 2019.61 Net ODA as a percentage of gross national incom e is 11.9 percent (2019, latest data available). 62 Rwanda’s largest donors are the International Developm ent Association, the United States, the African Developm ent Fund, EU institutions, and Global Fund.63 The m ajority of these funds are directed toward econom ic infrastructure, health and population, social infrastructure, and education. 64 Rela t ed work of ot her key hum a nit a ria n/ developm ent a ct ors 59. In addition to WFP and its McGovern-Dole Program partners, other international actors with significant roles in im plem enting education programming in Rwanda include World Vision (USAID Hom es and Com m unities award), Chem onics, the United States Agency for International Developm ent, Save the Children, the Foreign, Com m onwealth and Developm ent Office (FCDO),65 UNICEF, and the World Bank. See Annex 1 for details of these programme s. In support of school health, a num ber of com plem entary WASH programme s were im plem ented in select schools to im prove infrastructure and build capacity and knowledge at the school level; som e activities were paused during the COVID-19 pandem ic, but infrastructure activities continued. Ot her WFP Rwa nda a ct ivit ies 60. The WFP Rwanda Country Strategic Plan (2019-2023) has several activities in addition to the McGovern-Dole Programm e. Under Strategic Objective (SO) 1, it provides food and nutrition assistance and basic livelihood support to refugees and returnees, and food or cash, nutrition support and other assistance to local Rwandan populations in need of assistance. Under SO2, it supports the design, im plem entation, and scale -up of national food security and nutrition-sensitive social protection programme s. SO3 activities provide capacity￾strengthening support to national programme s that im prove the nutrition status of targeted populations. SO4 provides support, education, and capacity strengthening services for sm allholder farm ers and value chain actors. SO5 delivers supply-chain services and expertise to enable all partners to provide assistance to affected populations. 60 Edwin Ashim we. 2021. The New Tim es. Construction of new classroom s at ove r 95%. 61 World Bank database. 2021. Consulted 9 Decem ber 2021. 62 World Bank database. 2021. World Bank. 2021. Net ODA received (% of GNI)- Rwanda. Consulted 9 Decem ber 2021. 63 OECD. Top Ten Donors of Gross ODA for Rwanda, 2018-2019 average, USD m illion. Consulted 9 Decem ber 2021. 64 OECD. Top Ten Donors of Gross ODA for Rwanda, 2018-2019 average, USD m illion. Consulted 9 Decem ber 2021. 65 Form erly, UK Departm ent for International Developm ent (DfiD), 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 11 2. Su bje ct of t h e ba se lin e , t h e ory of ch a n ge a n d ba se lin e qu e st ion s 2.1 Su bje ct of t h e ba se lin e , t h e ory of ch a n ge , a ct ivit ie s a n d in t e n de d ou t pu t s and outcom es Progra mme overview 61. The subject of this baseline study is the USDA McGovern Dole Grant for WFP Hom e -Grown School Feeding Programme in Rwanda (2020-2025). The McGovern-Dole grant supports direct im plem entation of school feeding, WASH, health and nutrition, education and infrastructure activities in 136 pre - and prim ary schools in Karongi, Rutsiro, Nyam agabe, Nyaruguru, Burera, Kayonza and Gasabo districts, reaching approxim ately 117,095 students (see m ap in Annex 2). 66 Teachers targeted to participate in the program m e at school level num ber 336 (280 prim ary teachers, and 56 pre -prim ary teachers, and school adm inistration m em bers targeted to be trained on teacher coaching are 495 (388 in Group 1 schools and 108 in Group 2).The programme also includes household and com m unity-level support for 18,256 parents and local capacity strengthening for cooks; storekeepers; school com m ittees – including School General Assem bly Com m ittees (SGAC), School Feeding Com m ittees, School Managem ent Com m ittees, School Audit Com m ittees, and School Tender Com m ittees; sm allholder farm ers; Water User Associations; and local and district-level governm ent. There is also a strong focus on technical assistance and capacity strengthening to central-level governm ent school feeding stakeholders. Progra mme goa ls a nd objectives 62. The McGovern-Dole Program strategic goals are im proved literacy of school-age children (MGD SO1) and increased use of health and dietary practices (MGD SO2) in the targeted areas. WFP and partners aim to achieve MGD SO1 by: prom oting teacher attendance and recognition; distributing school supplies and m aterials; im proving literacy instruction m aterials; increasing the skills and knowledge of teachers and adm inistrators; providing school m eals; developing partnerships with farm er groups to supply food to schools; establishing and m aintaining school gardens; increasing use of health and dietary practices; raising awareness on the im portance of education; and reducing health-related absenteeism int schools. WFP and partners will im plem ent activities to achieve MGD SO2 by raising awareness on good hygiene practices; enhancing food preparation and cooking practices; building and rehabilitating latrines and water collection system s; distributing deworm ing m edication; and building/rehabilitating kitchens, cooking areas and storeroom s and providing fuel-efficient stoves. A key intervention overarching across MGD SO1 and SO2 is strengthening the capacity of Governm ent at national, district and school leve ls to oversee and m anage the NSFP (see McGovern-Dole Program results fram ework in Annex 3). 63. The programme aim s to achieve the following objectives (see annexes 3 and 4 for the McGovern-Dole and Local Regional Procurem ent (LRP) results fram eworks): • Im prove literacy skills of pre - and prim ary students through com m unity and parent engagem ent, targeted teacher training and coaching, the provision of learning m aterials, and student reading com petitions [WVI]; • Increase enrolm ent, reduce dropout, alleviate short-term hunger, and im prove student learning, concentration, and access to nutritious food by providing on-site, hot school m eals daily [WFP]; 66 Phase II transitions the four Phase I districts re presenting 108 schools from McGovern-Dole to National School Feeding Programme (NSFP) support. Three final districts representing 28 new schools were added to McGovern-Dole support in FY20 in order to install best practices through m odel schools, dem onstrating optim ized nutrition and efficient local sourcing, in vulnerable regions ahead of handover. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 12 • Im prove health and dietary practices of students through Social Behavior Change Com m unication, hygiene education activities and im proved water system s and latrine/handwashing facilities [GHI; RBC; WVI]; • Strengthen capacity of Governm ent to im plem ent the national school feeding programme through system s building, policy and strategy developm ent, and targeted support to Governm ent at the central and decentralized levels [WFP]; • Enhance capacities of farm ers in producing sufficient nutritious food for the national school feeding programme , while also im proving household food security and nutrition through targeted capacity building, enhanced financial inclusion, and connecting farm ers to schools to supply for school feeding [WFP]; and • Increase engagem ent and capacity of com m unities to delivery and m anage the national school feeding programme through targeted capacity building and infrastructure developm ent [WFP]. Act ivit ies a nd beneficia ries 64. Phase II of the McGovern Dole Program builds on Phase I and includes the full transition of direct school feeding to Governm ent, 67 , with Group 1 and Group 2 schools being transitioned to the NFSP in phases. At the tim e of the baseline study, Phase II activities had not yet started, as this is contingent on USDA approval of the baseline report. Thus, Group 2 schools in particular have not yet benefitted from McGovern-Dole support. 65. Phase II will provide a daily m eal and continue to im plem ent activities in the 108 prim ary schools in Karongi, Rutsiro, Nyam agabe, Nyaruguru that were supported in the FY15 McGovern-Dole Program (“Group 1” schools), and expand these services to an additional 28 pre - and prim ary schools in three new districts: Burera, Kayonza and Gasabo (“Group 2” schools). Phase II also expands the program m e to include pre - prim ary students. Phase II will thus reach a total of 136 schools. Phase II expands the program m e to pre - prim ary students and includes a hot m eal for all students. Key differences in Phase II are the provision of a new m eal, different from Phase I, that is the sam e across all schools. The hot m eals in Phase II will be m ade up of in-kind, locally, and regionally procured and fresh foods purchased through cash transfers to schools; this is a change from Phase I, which provided two food baskets, only to prim ary students. Other differences include a change from a blanket activity package for all schools to two packages; a significant reduction in literacy activities; an increase in WASH activities, especially infrastructure; and a stronger linkage/alignm ent of capacity strengthening to the NFSP rather than just to hom e -grown school feeding: Phase II will aim to m odel best practices to inform the further developm ent of the NSFP. This includes WFP support to strengthening the capacity of agricultural cooperatives and further developing the link between sm allholder farm ers and the school m eals m arket, supplying fresh, nutritious local foods.68 Phase II includes a procurem ent plan overseen by school-level tender com m ittees as a m odality for sourcing fresh foods until schools fully transition to the NSFP. 66. The 136 programme schools were selected based on a num ber of key poverty, food security, and education indicators. The Southern Province, including Nyaruguru and Nyam agabe districts, has som e of the highest num bers of fem ale -headed households (28.7 percent versus 26.7 percent nationally) and people with disabilities (4.4 percent versus 4.2 percent nationally) in the country, com pounding already high rates of poverty and food insecurity. In addition, MINEDUC requested that the programme include m odel schools located in all five Rwandan provinces. Sector selection was done in consultation with district officials and MINEDUC. Selection criteria included poverty rates, percentage of population in Ubudehe categories 1 and 67 WFP Rwanda. 2021. Phase II proposal for WFP's USDA McGove rn-Dole International Food for Education and Child Nutrition Programme 's Support in Rwanda. Draft. 68 WFP. 2021. WFP's USDA McGovern-Dole International Food for Education and Child Nutrition Programme 's Support in Rwanda 2016-2021. Endline evaluation. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 13 2,69 proxim ity and com plem entarity with other WFP/Governm ent of Rwanda programme s, com m unity support, and likelihood of success. 67. The McGovern-Dole Program will reach 117,095 pre - and prim ary students in grades 1-6, (49 percent girls, 51 percent boys) and 820 adults (including 280 teachers, 405 cooks and 136 storekeepers) who participate in the programme at school level. Household and com m unity-level interventions will directly benefit 18,256 parents. Through local capacity strengthening, 136 SGACs and 386 school adm inistration m em bers will directly benefit. Over the life of programme, 117,095 social assistance beneficiaries (students receiving m eals or take -hom e rations)70 will benefit from programme activities, as well as 15,000 non-student beneficiaries (e.g., teachers, parents, local governm ent, farm ers, cooks, storekeepers, com m ittee m em bers, etc.), and 351,285 indirect beneficiaries. 71 GEWE a nd equity 68. The draft operational plan for the McGovern-Dole Program m akes specific m ention of three GEWE and equity measures: 72 • Activity 2.1 (Construction of disability-inclusive latrines and girls’ sanitary rooms), as amended,73 states, “To prom ote hygiene, World Vision will construct Ventilated Im proved Pit (VIP) disability￾inclusive latrines in schools which need them m ost. Thirty disability-inclusive VIP latrines will be constructed: 20 in Group 1 schools and 10 in Group 2 schools. For every set of im proved latrines constructed, a girl’s sanitary room will also be constructed. In case s the schools where the latrine to be constructed already has a girls’ sanitary room , the girls’ sanitary room will be constructed at another school that does not have it.” • Activity 2.4 (Teaching girls on good menstrual hygiene) states, “Twenty-eight percent of fem ale students in the current McGovern-Dole Program are of adolescent age. In the current project phase, WFP is supporting National Early Childhood Development Programme to develop adolescent nutrition guidelines which WFP will translate into sim ple learning m aterials for adolescent girls. The Governm ent also plans to begin distributing iron-folic acid supplem entation to adolescent girls in schools. To em power these girls, WVI will use the girls’ sanitary room s as safe spaces to distribute supplem entation through teachers while also teaching girls about m enstrual health and hygiene lessons and nutrition education. These lessons will be based on existing m aterials used in com m unity hygiene clubs, to be adapted for these activities, in addition to the adolescent nutrition guideline learning m aterials.” This activity is consistent with the findings and recom m endations of a recent study on MHM in Rwanda com m issioned by the country office 74 (see discussion in Section 1.2 under “Education context”). • Activity 6.3 (Enhanced financial inclusion)states, “WFP will ensure gender-sensitive programming with special attention to the position of wom en as sm allholder farm ers and within group structures, prom oting them in leadership positions.” 69. The draft plan also describes equal representation of m en and wom en in programme -supported governance structures, such as teacher and student m em bership on School Feeding Com m ittees (Activity 5.8). 69 Ubudehe is “a long-standing and cultural value of m utual assistance which was adopted by the Governm ent in 2000 as part of the strategies to address poverty reduction… Ubudehe can be understood as a socio-e conom ic stratification system in which poor Rwandans are supported with social protection schem es.” Ubudehe classifies households by socio￾econom ic status. Categories 1 and 2 are the bottom two incom e categories. The New Tim es. 2020. New Ubudehe categorie s: What you need to know. June 25. 70 “Social assistance beneficiaries” are defined by USDA as “stude nts that received school m eals and/or take -home rations.” See USDA Foreign Agricultural Service – Food Assistance Division. 2019. USDA Food Assistance Indicators and Definitions.” February. Page 98. 71 Statistics and descriptions per em ail com m unication from WFP Rwanda country office, 28 March 2022. 72 Plan of Operations and Activities. Excerpt of Phase II proposal shared with the evaluation team . 73 Am ended text reflects a revision to the activity as written in the Phase II proposal. Revised text provided by WVI staff in the process of revising this baseline report. 74 WFP Rwanda. 2021. Asse ssing the Usage of Menstrual Hygiene Managem ent Services and Factors Influencing their Usage am ong Adolescent Schoolgirls in Rwanda. Report. Policy Brief. Novem ber. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 14 70. The finalization of the Phase II program m e design will also take into account findings and recom m endations from the form ative research on gender aspects of WFP Rwanda hom e -grown school feeding that is being conducted in parallel with this baseline study. The purpose of the analysis is “to gather the necessary data to not only render the program m e gender-transform ative in all its activities and outcom es, but further, to enable it to becom e a platform to create the necessary change for achieving gender equality and educational outcom es. The gender analysis could potentially inform the design of a gender-transform ative social and behavioral change strategy.” 75 Funding 71. The USDA McGovern-Dole program provides USD 25 m illion over five years. The programme receives com plem entary funding from other donors, including for food purchase. WFP estim ates that an additional USD 7.5 m illion will be m obilized in addition to the McGovern-Dole Program grant. These financial resources will enable the full im plem entation of the programme , including local and regional purchase of select com m odities to supplem ent the USDA-provided LRP funding, the construction of key WASH infrastructure, and the provision of cash transfers to schools for the purchase of fresh foods. Theory of cha nge 72. The theory of change put forward by this programme posits that: If the programme can leverage governm ent com m itm ent toward universal school feeding, as well as com m unity-level support to the sam e, and if the programme can provide the right accom panim ent, tools and resources at all levels, then increased com m unity and institutional capacity for operating and m anaging the NSFP will be achieved along with enhanced literacy and quality of education. This will result in children who are better educated, better nourished and better prepared to achieve Rwanda’s national developm ent goals. Moreover, this will result in a sustainable and resilient NSFP, with sustained m ultiple benefits for education, nutrition, agriculture and local econom ic developm ent.” No graphic to accom pany this theory of change was provided. 73. WFP will further develop its theory of change and associated assum ptions m atrix in the first six m onths of the programme , including finalizing the results fram ework to align with and m easure progress along the theory-of-change pathways. 76 Developing the theory of change is not within scope of this baseline study. At m idterm and at endline, the evaluation scope does include a review of the programme theory of change to use as a fram ework against which to assess actual progress along change pathways, and to identify risks and opportunities regarding achieving expected results. 2.2 Eva lu a t io n q u e st ion a n d e va lu a t ion crit e ria 74. The baseline study and subsequent m idterm and endline evaluations system atically em ploy the standard evaluation criteria of relevance, coherence, effectiveness, efficiency, im pact and sustainability. GEWE is m ainstream ed throughout. The baseline study will ground evaluation analysis at m idterm and endline on 1) quality of program m e design, 2) quality and results of WFP output and outcom e m onitoring tools and data (to the extent these are available), including progress data from the Readiness of Schools to Transition Scorecard, and 3) WFP’s targeting for the overall indicator set. 75. Aligned with the evaluation criteria, the three evaluation exercises will address the key evaluation questions outlined in the approved Evaluation Plan for the McGovern-Dole Program (see Annex 5). These are indicative questions designed in order to provide the background to the evaluation team . The evaluation team has categorized the questions relevant to the baseline study in an evaluation m atrix (see Annex 6). Collectively, the questions aim at highlighting the existing circum stances and the baseline status of perform ance indicators. The baseline evaluation m atrix is organized by the OECD-DAC criteria.77 It includes the following elem ents: 75 WFP Rwanda . 2021. Concept Note: Gender Analysis of the Hom e Grown School Feeding Program m e. October 2021. 76 In an em ail com m unication 28 March 2022, the WFP Rwanda country office confirm ed that the results fram e work was finalized and included in the agreem ent. 77 The Organisation for Econom ic Co-operation and Developm ent – Developm ent Assistance Com m ittee (OECD-DAC) has defined six evaluation criteria – relevance, coherence, effectiveness, efficiency, im pact and sustainability – and two 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 15  General baseline observations for the project outcom e dom ains to inform overall WFP adaptive m anagem ent, as appropriate ;  Specific baseline evaluation questions adapted from the m idterm and evaluation questions presented in the TOR, since the TOR did not include baseline evaluation questions; and  Perform ance Monitoring Plan (PMP) indicators for which the external evaluation process is the prim ary source. These indicators can be found under the effectiveness section of the m atrix. 76. The evaluation questions relevant to the baseline study (and presented in Annex 6) are as follows: 77. EQ 1: To what extent is the McGovern-Dole program m e appropriate to the needs of the target beneficiaries of m en, wom en, boys and girls? [TOR, Relevance, EQ1 part 1] 78. EQ 2: To what extent are the activities (design and im plem entation) appropriate in the context of COVID-19? [TOR, Relevance, EQ4] 79. EQ 3: To what extent are the activities aligned with governm ent capacity building gaps within the national school feeding program m e? [TOR, Relevance EQ1 part 2] 80. EQ 4: To what extent is the McGovern-Dole program m e aligned with overall USDA objectives as well as strategies, policies, and norm ative guidance, and with Governm ent’s relevant stated national policies, including sector policies and strategies? [TOR, Relevance EQ2] 81. EQ 5: To what extent is the McGovern-Dole program m e aligned with United Nations fram eworks and relevant developm ent partners? To what extent is it aligned with WFP’s overall strategy and related guidance? [TOR Relevance EQ3, part 1] 82. EQ 6: To what extent is it aligned with WFP’s overall strategy and related guidance? [TOR, Relevance EQ3, part 2] 83. EQ 7: To what extent is the M&E system designed to adequately respond to the needs and requirem ents of the project? Is the M&E system designed to sufficiently capture changes in the lives of the beneficiaries? [TOR, Effectiveness EQ3] 84. EQ 8: To what extent are McGovern-Dole program m e project com ponents (e.g., school feeding, literacy, food safety, WASH and hygiene, nutrition education, agricultural m arket support, etc.) likely to be sustained at national and local levels, com m unities, and by other partners? (considering handover to the Governm ent) [TOR, Sustainability, EQ 1, part 1] 85. EQ 9: What is the dem onstrated capacity at central and sub-national levels to m anage school feeding program m es in Rwanda (WFP and governm ent program m es)? [TOR, Sustainability, EQ 4] 86. EQ 10: To what extent are local com m unities (SGACs, School Feeding Com m ittees, Procurem ent Com m ittees, farm ers’ groups, etc.) involved in and contributing toward school feeding and education activities? [TOR, Sustainability, EQ 5] principles for their use. These criteria provide a norm ative fram ework used to determ ine the merit or worth of an intervention (policy, strategy, program m e, project or activity). They serve as the basis upon which evaluative judgm ents are made and are com m only applied in evaluations of international program s. See OECD DAC evaluation criteria. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 16 3. Eva lu a t io n a p p r o a ch a n d m e t h odology for ba se lin e da t a colle ct ion 3.1. Eva lu a t io n a p p r o a ch a n d m e t h o d o lo gy 87. The baseline, m idline and endline evaluations for the McGovern-Dole programme will be carried out through a representative sam ple of programme schools in all districts of intervention and a representative sam ple of schools with no WFP hom e grown school m eals program m e. The evaluation of McGovern-Dole program m e activities will be carried out using the sam e representative sam ple of schools for baseline, m idline and endline and will include target beneficiaries on m en, wom en, boys and girls to ensure that GEWE data will be collected in all three exercises. 88. The baseline study used a m ixed-m ethod approach, utilizing secondary and prim ary collected data to generate an accurate reflection of current programme sites relative to non-programme (counterfactual) sites, and to ensure inform ation from different m ethods and sources is triangulated to enhance the validity, reliability, and credibility of findings. 89. While routine programm e m onitoring disaggregates data by gender, the McGovern-Dole Program design has not incorporated gender considerations in a robust m anner to date . The evaluation incorporates GEWE considerations in the evaluation activities. For exam ple, the quantitative data were disaggregated by gender. The qualitative topical outlines incorporated questions on girls’ participation in McGovern-Dole Program activities (e.g., school gardens, literacy activities). Qualitative inquiry also included questions on access to education for girls, especially those from very poor fam ilies or who have disabilities, and teacher and parent attitudes about higher education for girls and its influence on girls’ perceptions. 90. It is worth noting that although the program m e supports WFP’s Gender Policy, the original design does not include a specific approach to address gender equality and wom en’s em powerm ent. It is anticipated that the form ative gender assessm ent com m issioned by WFP in parallel with the baseline study, supplem ented by baseline findings, will inform the integration of gender into program m e design and im plem entation. In addition, following the gender assessm ent, other appropriate gender assessm ent activities will be included in the m idterm and endline evaluations, building on baseline analysis and identified perform ance m arkers. Having gender ‘deep dives’ in all three evaluation phases will help to reinforce the im portance of this crosscutting issue and to redress the lim ited consideration it has been given in program m ing so far. 3.2. Ba s e lin e d a t a co lle ct io n m e t h o d s a n d t o o ls Overview 91. The baseline approach com bined a desk review; a school census; a school and head teacher survey; a student EGRA survey; qualitative fieldwork including sem i-structured interviews with key inform ants, focus groups (e.g., with students, SGACs, School Managem ent Com m ittees (SMCs)); observation of programme activities at schools and in group activities (e.g., SGACs, SMCs);78 and exam ination and triangulation of quantitative data from WFP and partner m onitoring reports and databases available at the tim e of the baseline. Table 3 presents an overview of the data collection tools em ployed at baseline. The data collection m ethods and tools have been designed in accordance with UN-SWAP criteria 2c and 2d.79 78 Direct observation was em ploye d where it could be done without disruption to norm al activities of classroom s, cooking, distribution of m eals, etc. 79 UN-SWAP criterion 2c: Are a diverse range of data sources and proce sse s em ployed (i.e., triangulation, validation) to guarantee inclusion, accuracy and credibility? Criterion 2d: Do the evaluation m ethods and sam pling fram e address the diversity of stakeholders affecte d by the intervention, particularly the m ost vulnerable, where appropriate? UNEG. 2018. UN-SWAP Evaluation Perform ance indicator Technical Note. April 2018. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 17 Ta b le 3: Ove rvie w of m a in da t a colle ct ion t ools Da t a colle ct ion tool Typ e of d a t a colle ct e d Com m e n t s Docum ent review Secondary data on programme background and context; perform ance indicator data collected by WFP; process m onitoring data analyzed by WFP, i.e., analysis of Readiness of Schools to Transition Scorecard The evaluation team was provided with a docum ent library set up by the country office and supplem ented this with additional research. Annex 7 lists the docum ents cited in this report. School census School statistics (available in school recordkeeping docum ents) Rem ote /online survey adm inistered to head teachers or their designates for all schools in the programme , as well as the control schools identified School and head teacher survey (ODK) MGD indicators The local team conducted in-person interviews of head teachers. Additional qualitative data were collected by the local team from KIs and/or focus groups (gender-disaggregated where possible) with teachers, students, cooks, storekeepers, and school com m ittees. EGRA tool MGD indicators The local team adm inistered the EGRA to 3rd graders in program m e schools and control schools. Topical outlines (interview guides) All: Qualitative data on all evaluation questions and to validate and help interpret indicator data Agricultural cooperatives: Qualitative data to validate operational and perform ance data collected as part of WFP’s routine m onitoring, to explore factors that affect cooperative perform ance, and assess readiness to supply schools Topical outlines for: --WFP Kigali and field staff* --Governm ent m inistries* --District governm ent (District Education Officials) * --Im plem enting Partners* --Donors* --United Nations Agency Partners (UNICEF)* --Schools (head teachers, teachers, SGACs, School Managem ent Com m ittees), School Feeding Com m ittee s, School Tender Com m ittees) --Agricultural cooperative partners. *Interviews in categories marked * conducted remotely by the international consultant team, others by local team 2 Focus groups to be disaggregated by gender where feasible. 92. For a detailed discussion of the m ethodological approach, see Annex 8. See the evaluation m atrix (Annex 6) for a detailed description of data sources, data validity, quality, and reliability, and data collection tools and how these link to the baseline eva luation questions; these were developed on the basis of the USDA perform ance indicators for which the external evaluation firm is responsible (see Annex 9). The evaluation questions are translated into data collection tools: topical outlines (interview guides) for interviews and focus groups (Annex 10), the EGRA tool (Annex 11), a census survey (Annex 12) and a school and head teacher survey (Annex 13). The topical outlines were based on Phase I tools and updated for Phase II based on experience, new information, and feedback from WFP and USDA review. All tools were adjusted based on reviewer com m ents and pre -test results before they we re finalized and deployed. Qua ntita tive 93. The quantitative data to be used for all three evaluation exercises prim arily com prise perform ance indicator data to be collected by WFP and partners (such as regular perform ance m onitoring data and School 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 18 Readiness Scorecard data collected by WFP Rwanda field m onitors), in addition to the indicators collected by this evaluation team (see Annex 9), beginning at baseline. The baseline study collected data for assigned indicators (see Annex 6) to enable perform ance and higher-level results analysis at m idterm and endline. Baseline results will confirm if evaluation questions and indicators are relevant to the programme and country-specific issues in Rwanda, and will also provide im portant context necessary for the m id-term and final evaluation to assess the programme against OECD-DAC criteria –relevance, coherence,80 efficiency, effectiveness, im pact, and sustainability. 94. The quantitative sam ple draws from three categories of schools, as follows: • WFP Phase II Group 1 – Participating schools that received support in WFP McGovern Dole Phase 1 (108 schools in four districts). The sam ple contains 21 Group 1 schools. • WFP Phase II Group 2 – Participating schools that did NOT receive support in WFP McGovern Dole Phase 1 (28 schools in three districts). The sam ple contains 10 Group 2 schools. • Control group – Schools selected to represent a counterfactual for com parison in future rounds. These school will NOT be supported by WFP McGovern Dole (10 purposefully selected schools). The sam ple contains 10 control schools. 95. Table 4 shows the target sam ple sizes for the num ber of students to receive the EGRA test. A sam ple size of 220 per stratum is sufficiently large to enable com parisons am ong strata/groups (i.e., Group 1, Group 2, and the control group); future EGRA survey rounds will thus use the sam e target sam ple size for each group) and be adm inistered in the sam e 31 program m e schools and 10 control schools as the baseline . Note that program m e im plem entation in Group 1 schools will phase out two or three years after this baseline, so in effect, the m idterm evaluation for the program m e will serve as an endline for Group 1 schools, and the endline evaluation will serve as an ex-post evaluation. As such, the hypothesis and evaluation/research questions for the latter, in the phased-out schools, will differ from those in the schools that continued the program m e. Selection of students at the school level at baseline was done by random ly selecting grade students in the chosen schools; the sam e m ethod will be used for the m idterm and endline surveys. Annex 8 describes the rationale and calculations for the school and student sam ple size s. Ta b le 4: EGRA t a rge t sa m p le size s: ba se lin e , m idt e rm a n d e n d lin e (# st u d e n t s) Ta rge t sa m p le size Ba se lin e Midt erm En d lin e Grou p 1 462 220 220 Grou p 2 220 220 220 TOTAL (MGD) 662 440 440 Con t rol 220 220 220 96. Quantitative data (EGRA results and school and head teacher survey data) were collected on Android tablets using Open Data Kit (ODK) and Tangerine (RTI) data collection software. Final versions of the tools were coded for application onto the Android devices and translated into Kinyarwanda. The EGRA tool in Kinyarwanda was adapted from the Phase I endline tools. As in Phase I1, the Phase II tool was m odified to ensure that the students would have no prior exposure to the content. Adopting the sam e standard as in previous evaluations, it was adm inistered to 3rd graders. 97. After data collection was com pleted, all data were downloaded from the evaluation firm ’s field server into an Excel sheet (csv. file) and stored on its secure servers for analysis. The csv file was checked question by question to ensure that the data were correctly downloaded. Sm all technical procedures were im plem ented to support the transfer of data to the analyzing platform , Stata; e.g., rem oving or replacing non-ASCII characters. The data were then converted from the csv file to a Stata dataset. The final step in data cleaning 80 While coherence was not included in the TOR, it has been added to the evaluation m atrix because coherence with governm ent policy and other organizations is an im portant part of the program m e, specifically because the program m e focuses on stre ngthening governm ent capacity in school feeding. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 19 was to prepare the dataset for analysis, which involves ensuring that variable nam es and labels were properly downloaded and renam ing indicator variables for analysis. 98. All data analysis was perform ed using the latest version of STATA (Stata 17). The analysis of was statistically powered to provide accurate point estim ates for the three school categories (Group 1, Group 2, and control). Significance tests were perform ed to determ ine whether the estim ates for Group 1 and Group 2 schools (i.e., Phase II program m e schools) differ from that of control schools. Qua lita tive 99. The qualitative work involved 47 key inform ant interviews (37M, 10F) and 48 focus groups (148M and 121F participants). See Annex 14 for a sum m ary of interview and focus groups conducted. Interviews and focus groups were conducted in the sam e schools and districts as the quantitative sam ple , and included key stakeholders including district education officials, head teachers and teachers, SGAC and SMC com m ittee m em bers, cooks and cleaners, and students. Focus groups were organized according to roles and m ost were m ixed m ale and fem ale respondents. The interviews were com plem ented by inform ation obtained through observation by the qualitative team (e.g., condition of infrastructure such as kitchens and toilets). Qualitative interviews were also conducted with key WFP staff and im plem enting partners. 100.Topical outlines for the interviews and focus groups were developed to reflect the key program m e objectives and inform ation needs (see Annex 10 for data collection tools). Evaluation team m em bers m ade sum m ary notes from KIIs, FGDs and sm all group m eetings and shared these with each other for discussion and analysis. Audio recordings, with respondent perm ission, were m ade in som e cases, especially for rem ote interviews. Team m em bers applied a real-tim e analysis process through regular debriefs am ong the TANGO team to form ulate findings across qualitative sources every tim e new interview batches were added. The qualitative data were analyzed using a m atrix m ethod that categorized key questions and grouped responses according to Group 1 and new and control schools to allow for com parison between the groups. 101.Sem i-structured them atic analysis was applied to the literature review. Docum ents were reviewed against pre -identified m arkers associated with the evaluation questions, the them atic focus areas, and em erging hypotheses. Va lida t ion/ Tria ngula t ion 102.Prim ary quantitative data from the EGRA and school and head teacher survey were triangulated with qualitative results and inform ation from the review of secondary literature, and integrated into the findings section of this report. 103.On Novem ber 25, when the field m ission was com plete and analysis was underway, TANGO presented a debriefing to WFP stakeholders to serve as an inform al validation of prelim inary findings. This was followed by an iterative review process (see Annex 15) involving written and verba l feedback on draft report versions from DEQAS and the ERG, to ultim ately finalize the report. 3.3. Lim it a t io n s 104.The following issues arose in data collection and analysis: 105.De la ye d t im in g. Per USDA’s Monitoring and Evaluation Policy, the baseline is to be com pleted within the first six m onths and prior to the start of activity im plem entation, with a final report subm itted to USDA. The start date of this baseline study was delayed due to pandem ic-related school closures. In addition, there were unforeseen delays in field data collection due to travel distances for the final school sam ple selection being longer than planned for, control school changes, the change from an e -survey to phone surveys for panel schools, and key inform ants not being available to me et with the local data collection team . The planned com pletion date was 12 Novem ber: the EGRA and school qualitative work was com pleted on 17 Novem ber; local KIIs were com pleted on 26 Novem ber; and the school census conducted by phone was com pleted on 6 Decem ber. These delays were largely absorbed by the evaluation team and the country office in the preparation of the first draft. 106.Low re sp on se t o e -survey. The baseline approach included a school census e -survey adm inistered to head teachers or their designates for all schools in the program m e. The census e -survey had a low response rate ; only two e -surveys were received despite m ultiple follow-up em ails and deadline extensions. To ensure tim ely 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 20 collection of survey data for the schools m ost relevant to the baseline and future evaluation exercises – the program m e schools in the sam ple and the control schools (i.e., together, “panel” schools) - the local team , by special exception, adm inistered the surveys for the panel schools by phone , and was thus successful in com pleting the questions in the census e -survey for all panel schools. 107.No st u den t s a t t wo con t rol sch ools. At two of the ten control schools, no students were present on the day of the local team ’s visit. After consultation with the evaluation team leader and approval by the evaluation manager, the local team replaced these schools with two new ones and succeeded in collecting the needed data at the replacem ent control schools. The sam e control schools will be visited in future rounds (at m idterm and endline). All head teachers of the final selection of control schools confirm ed interest and availability to participate in future rounds. 108.Ge n d e r b a la n ce of KIIs/ FGDs. While attem pts were m ade to select an equal num ber of m ales and fem ales for KIIs and FGDs, this was not possible. As noted above, the team interviewed 37 m ales and 10 fem ales, and the total FGD participants were 148 m ales and 121 fem ales. The prevalence of m ales in these exercises is explained in part sim ply by the availability of respondents, but also by the reality that m en predom inate in the staff/official posts relevant to this evaluation. The sum m ary of KIIs by category in Annex 14 reflects this proportionality. For instance, 13 of the school-based staff interviewed were m ale and 5 were fem ale; this largely reflects the predom inance of m en in head teacher roles. Sim ilarly, of the 15 district governm ent officials interviewed, 14 were m ale, again reflecting the gender distribution in Governm ent. The evaluation team understands from the prelim inary results of the draft form ative gender assessment that these proportions are consistent with the Rwandan context, and as noted earlier, the findings of this baseline and the gender assessm ent will consider how this lim itation can be m itigated in the m idterm and final evaluations. 109.Use of Ph a se I e n d lin e sa m p le d a t a . As noted earlier, where applicable, this report presents Phase I endline values in the indicator perform ance tables. These values are provided for context only: the Phase I and Phase II data were not treated to enable statistical com parison – this is beyond the scope of the baseline study and would require a significantly m ore arduous analytical process. Any com parisons of Phase I endline to Phase II baseline are guided prim arily by the qualitative data, with quantitative data utilized as reference and triangulation points where appropriate. 110.None of the issues described above necessitated changes to the baseline study analysis plan. 3.4. Qu a lit y a ssu ra n ce of t h e ba se lin e da t a colle ct ion 111.The quality assurance (QA) process for the baseline study has followed the processes, tem plates and quality assurance checklists defined by the WFP Decentralized Evaluation Quality Assurance System (DEQAS). The team leader is responsible for the overall quality of all evaluation processes and products, and as such m aintained regular com m unications with WFP Rwanda and the evaluation team to ensure ongoing clarity on expectations for roles, responsibilities, and tim elines, especially with regard to any changes to the plan outlined in the baseline inception report. This included a weekly check-in call between the evaluation m anager and team leader, at m inim um , to review progress and adapt evaluation plans as needed. In addition, a staff m em ber of the evaluation firm who was not directly involved in data collection was assigned to guide the team through the QA process and ensure com pliance with DEQAS standards as well as those internal to the firm . The internal QA m anager engaged with the evaluation team in an iterative review process for all deliverables before these were shared with WFP. 112.Team experience and training was another im portant QA com ponent. The core evaluation team has collaborated in a highly effective m anner across the Phase I m idterm and evaluation activities, as well as in other assignm ents. The team leader ensured that the team received refresher training on evaluation protocols and procedures, as well as regarding the updated topical outlines, gender issues, and the use of structured checklists. This training included facilitator and enum erator roles and responsibilities, rules, behaviors and ethics, respondent selection, use of field control sheets, and a detailed review of all survey tools. 113.On Novem ber 25, after data collection was nearly com plete and analysis was underway, the team leader led a debriefing session for WFP stakeholders at country office and RB level to present prelim inary results and analysis. This session served as an opportunity for m utual reflection and discussion of prelim inary findings; inputs were considered in the ongoing analysis and preparation of the draft evaluation report. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 21 114.In accordance with DEQAS, draft versions of the baseline study inception and evaluation reports were reviewed by the WFP Evaluation Com m ittee and Evaluation Reference Group established by the country office for this evaluation, to ensure the independence and im partiality of the evaluation at all stages. The team leader and evaluation m anager have instituted a prelim inary step to this form al process, whereby the evaluation m anager receive s prelim inary report drafts and gives inform al feedback during a regular weekly call; the evaluation team then considers this feedback in finalizing the “Draft Zero” reports for wider circulation. This iterative com m ent-and-revision process results in near-final versions of all report deliverables, which are then reviewed by USDA prior to finalizing the report. 3.5. Et h ica l co n s id e r a t io n s 115.The baseline study was conducted in conform ance with the 2020 United Nations Evaluation Group (UNEG) Ethical Guidelines. All team m em bers reviewed and signed pledges of ethical conduct, confidentiality agreem ents, and conflict of interest statem ents with regard to this study. The team leader com m unicated with the team regularly via em ail and WhatsApp for progress updates and to discuss any issues that m ight arise in these areas. No conflicts of interest or issues of ethical conduct arose at any point during the baseline study. 116.Considerations and protocols regarding inform ed consent, protecting privacy, confidentiality, and anonym ity of participants, ensuring cultural sensitivity, respecting the autonom y of participants, ensuring fair recruitm ent of participants (including wom en and socially excluded groups) and ensuring that the evaluation results in no harm to participants or their com m unities, were reviewed with the team before data collection. The data collection instrum ents include text regarding the privacy of responses and inform ed consent; all data collection exercises were preceded by this narrative and an explanation that participation in the baseline study is voluntary and that responses would not be attributed to specific respondents. Voluntary inform ed consent was obtained in all cases (e -surveys and interviews). All evaluation products have been prepared to om it personally identifying inform ation (PII), and the description of persons interviewed individually or in focus groups has been presented in this report in sum m ary form to avoid disclosure of PII. 117.Equal num bers of boys and girls were adm inistered the EGRA by a Rwandan team of three fem ale and one m ale evaluator. Where possible, the fem ale evaluators adm inistered the EGRA to girls, and the m ale evaluator adm inistered the EGRA to boys. Care was taken to optim ize the diversity of respondents interviewed (e.g., in term s of gender and age) in order to gather and represent inform ation from a variety of perspectives and in an inclusive way, though as noted in the lim itations section, achieving gender balance was constrained by availability and the existing gender discrepancies in staff and official position-holders. 118.The local team followed local COVID protocols in all field interactions, such as m asking, social distancing, and handwashing. 119.In addition to following UNEG guidelines, all staff, consultants, and officers m ust com ply with TANGO’s policies and procedures, including TANGO’s Code of Ethics and Conduct. TANGO consultants are trained internally on ethical research safeguards, and child and youth protection, based on current UNICEF guidance and client policies and standards, where available.81 120.The evaluation te am ensured appropriate ethical considerations we re in place for all interviews, particularly for sensitive populations, through transparent practices including: inform ing all interviewees of the purpose and duration of the interview, how they were identified to participate in the interview, inform ing interview participants of their rights, and providing guarantees that specific interview findings will rem ain confidential and that all inform ation provided will be used to assess the programme – with no dire ct attribution to the interviewee. Finally, all interviewees were inform ed that they m ay choose not to participate and asked to give their verbal consent to participate. Photographs required verbal consent. No interviews or photos were refused in the course of the baseline study. TANGO has ensured that data collection was efficient and respectful of people’s tim e and covered only the topics necessary to address the study questions. The ethical and safeguarding issues described here were m onitored throughout the evaluation process, including during fieldwork. No such issues arose during the baseline study, 81 UNICEF Ethical Research and Children 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 22 4. Ba s e lin e fin d in gs a n d d is cu s s io n 4.1 Re le va n ce Eva lua t ion quest ions releva nt a t ba seline EQ 1: To what extent is the McGovern-Dole programme appropriate to the needs of the target beneficiaries of men, women, boys and girls? [TOR, Relevance, EQ1 part 1] EQ 2: To what extent are the activities (design and implementation) appropriate in the context of COVID-19? [TOR, Relevance, EQ4] 121.These evaluation questions are discussed in the course of the baseline obse rvations m ade below, organized by McGovern-Dole strategic objectives. Ba seline observa t ions Improved literacy of school-age children (MGD SO1) 122.The McGovern-Dole Program is seen by head teachers and teachers in both Group 1 and Group 2 schools to be relevant to their needs. The qualitative interviews with head teachers and teachers revealed that Group 1 and Group 2 schools have a shared idea that the programme will im prove attendance and reduce the student dropout rate. Teachers and school com m ittee members interviewed see the McGovern-Dole Program as an opportunity to support the Governm ent in prom oting education. Group 2 schools in particular voiced expectations that the programme will increase attendance and perform ance, im prove educational quality, reduce the dropout rate , fight m alnutrition, increase student socialization, help poor fam ilies, and change parent m indsets about the im portance of education. District officials in Group 2 schools noted the programme ’s relevance, saying that both the programme and WFP are working to prom ote the quality of education and seek solutions to sustainable education. They also said that decreasing the dropout rate is a m ajor priority for the district, and that they have seen this happen in secondary schools where m eals are provided. 123.Phase II of the McGovern-Dole Program provides educational m aterials to foster a richer learning environm ent in classroom s, along with teaching aids and adm inistrative tools for teachers. Baseline study interviews validate this is as a highly relevant activity. Head teachers and teachers in Group 2 and control schools told the qualitative team that they lack school m aterials and infrastructure, including books, teaching aids, electricity, and com puters for record-keeping, reinforcing the relevance of these programme activities. KIIs with WFP staff note that while the Governm ent has recently built thousands of new classroom s and around 600 new schools, m any of these schools in Group 2 m ay not yet be fully equipped. Classroom s are often overcrowded (exceeding the governm ent standard of 45 students to one teacher), which creates pressure on existing facilities and equipm ent (e.g., toilets, desks, chairs), though this pressure is a lesser issue in Group 1 schools given their previous involvem ent in the programme . It should be noted that, at the tim e of the baseline study, McGovern-Dole activities had not yet started in Group 2 schools. Governm ent schools receive capitation grants to help purchase m aterials and equipm ent, but som e schools said the grant com es very late and thus affects the im plem entation of m any activities or is not sufficient to cover m aintenance costs. 124.Teachers in Group 1 schools told the qualitative team that one of the positive im pacts of the McGovern-Dole Program is that student perform ance and concentration have increased, resulting in higher grades; indeed, the Phase I endline evaluation found that school feeding im proved attendance, concentration, perform ance, enrolm ent and retention rates, all factors that are relevant to the challenges faced by Group 2 schools in particular. 82 Neither Group 2 nor control schools were receiving school m eals at the tim e of the survey, and teachers in Group 2 and control schools interviewed said that lower prim ary students attend irregularly and thus perform poorly. Som e teachers describe parents as “irresponsible” and as not investing in their children’s education or the programme . Other teachers say that there is inadequate collaboration between 82 WFP Rwanda. 2021. WFP's USDA McGovern-Dole International Food for Education and Child Nutrition Program m e's Support in Rwanda 2016-2021. Endline evaluation. 25 August (draft). 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 23 schools and parents and that parents do not follow up on their children’s education and do not com e when teachers call m eetings. Teachers in Group 2 and control schools told the qualitative team that parents of children with disabilities m ay not want to bring their children to school or contribute to school m eals because they think it is a waste of m oney. Head teachers believe that involvem ent from local governm ent will support the sensitization of parents, especially given the programme is still new. This supports the relevance of the programme ’s sensitization activities with parents and com m unities. 125.Teachers in Group 2 and control schools told the qualitative team that fam ily duties and a need to earn im m ediate incom e negatively affect children’s school enrolm ent and attendance. A programme partne r pointed out that m any young people are unable to find jobs after finishing their education. This m akes parents reluctant to invest in education and instead m ay prefer to have children help at hom e or work to help m eet the fam ily’s im m ediate need for incom e. Teachers also that poverty discourages students from participating in school as they cannot afford uniform s, pens, or notebooks, which m akes children m ore likely to drop out. Boys may drop out to look for work due to poverty. 126.In all schools, interviewed respondents explained that girls arrive late or do not attend school because of their responsibility for household chores. In addition, teachers in Group 2 and control schools reported that girls are asham ed to com e to school without m enstrual pads, which increases dropout by girls. This issue was addressed in Group 1 schools by the construction of separate latrines and sanitary room s for girls, which im proved regular attendance by girls, according to head teachers. This is consistent with the findings of the gender assessm ent, which found that "girl participants in the assessm ent indicated that the girls’ room s in schools have allowed them to access the m enstrual hygiene products and support they need during m enstruation, helping to reduce absenteeism in school and im proving their overall com fort level in this regard." The draft gender assessm ent states that “the inclusion of an intentional gender equality and wom en’s em powerm ent (GEWE) strategic approach in the HGSF program m e has not yet been m ainstream ed or im plem ented, thus the [following] gender assessm ent conducted at baseline is intended to help identify a way forward." The gender study recom m ends that WFP, am ong other actions, engage parents, com m unities, and key leadership in dialogue on gender issues and barriers, develop social behavior change cam paigns to target key barriers, and im prove data collection and analysis to include power dynam ics and root causes of gender inequity. The assessm ent notes that addressing gender barriers will require the com bined efforts of all stakeholders in the education sector, and that failing to address them can unintentionally reinforce existing norm s. The full set of recom m endations is included with the report in Annex 18. Increased use of health and dietary practices (MGD SO2) 127.Group 1 head teachers, teachers and cooks told the qualitative team that m alnutrition and hunger decreased as a result of the McGovern Dole programme . There is an expectation in Group 2 as well that the programme will reduce m alnutrition am ong students. In Group 2 and control schools, teachers interviewed by the qualitative team said that students receive few m eals at hom e , which affects their perform ance in school. Teachers in Group 2 and control schools stated that children from poor fam ilies m iss classes m ore often, and when asked why they were absent, the children say they were hungry. Teachers com m ented that students from poor fam ilies m ay eat only once a day. 128.Cooks in Group 1 schools said the purpose of the programme is to fight m alnutrition, increase attendance, and to ensure that children from poor fam ilies are given at least one good meal a day. Group 1 cooks shared that they have seen m ore regular and on-tim e attendance, increased handwashing, greater attentiveness in class, and decreased m alnutrition, and that children from poor fam ilies receive a m eal. The cooks also confirm ed that they were trained in hygiene and m eal preparation, while the Group 2 cooks interviewed have yet to receive training. 129.School gardens are present in all of the Group 1 schools visited. This is consistent with the School Readiness Scorecard (2021) indicating that 97.2 percent of all program m e schools have a school garden.83 The m ain objective of the school garden com ponent of the program m e , which is m anaged by GHI, is nutrition education. In addition to providing technical assistance for school gardens, GHI provides training to teachers, parents, students, and teachers on how to prepare balanced m eals. GHI key inform ants stated that GHI also views the school gardens as an enabling space to raise awareness and knowledge about nutrition in students’ homes, in part by acting as a source of vegetable seeds to enable fam ilies to diversify their m eals. Qualitative 83 WFP Rwanda. 2022. School Readiness Scorecard Excel file: HGSF_SchoolLevel_Score card_19JAN2022. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 24 data confirm ed that GHI seeds were used for school gardens and were distributed to students and neighboring fam ilies for hom e gardens. GHI further se es that cooking dem onstrations at school help parents understand the im portance of nutrition. Parents’ understanding is enhanced when they see the food item s they have contributed cooked as part of a nutritious school m eal. 130.Most of the school gardens at schools visited by the qualitative team are contributing vegetables to school m eals, though the am ount varies am ong schools. Group 1 schools have expanded the size of school gardens and som e schools are using vegetables from their own gardens rather than from the m arket. In Group 1 schools, cooks said that they get a variety of vegetables (eggplant, carrot, cabbage, greens) from the gardens but m any buy from outside m arkets as the gardens do not produce enough to m eet the needs of the school. Group 2 cooks said that their school gardens have just started and have not yet produced vegetables. The qualitative team observed that m ost Group 2 schools at the schools visited do not have school gardens, and most do not provide vegetables in their m eals. Further, it was observed that the students do not have m uch knowledge about nutrition. National school feeding programme (NSFP) readiness 131.Com ponents of the McGovern-Dole Program have been a m odel for the Governm ent to expand school feeding nationally, according to key stakeholders in the programme . For exam ple, WFP is collaborating with MINEDUC to develop training m aterials for cooks, storekeepers, school procurem ent com m ittees, school feeding com m ittees at school, sector and district level, and the National School Feeding Technical Working Group. These m aterials will include both print m aterials (flip books, training guides, posters) and audio-visual m aterials. The first phase of print m aterials for cooks and storekeepers was created in 2021 and is now finalized to be rolled out to schools. The next phases will be created in 2022, in close collaboration with MINEDUC and other key stakeholders. 132.At the district level, district education and agriculture officials offered their views about how involved stakeholders have been in programme design. The m ajority of district officials interviewed reported that collaboration with WFP was good; a few district officials in both Group 1 and Group 2 stated that com m unication between WFP and local officials could be im proved. All districts said that the data from Phase I was used to design Phase II. Overall, district officials in Group 2 said that WFP is collaborating with them to m ake sure the programme responds to needs. Group 1 and Group 2 district officials confirm ed that they were involved in the selection of schools for program m e im plem entation. 133.In August-Septem ber 2021, WFP and MINEDUC conducted district level m eetings to inform schools and local governm ent about the NSFP. Dissem ination m eetings and orientation trainings with district level education officials and district leadership were conducted in all 30 districts with key content focused on the School Feeding Operational Guidelines (approved by MINEDUC in May 2021). Also from August-October 2021, School Feeding Com m ittees were established in all schools and districts across the country, in alignm ent with the school feeding policy and operational guidelines. Mem bers of these com m ittees are dissem inating inform ation on the NSFP through other com m unity initiatives and platform s like Um uganda. WFP printed English and Kinyarwanda sum m aries of the School Feeding Operational Guidelines for all schools and distributed hard copies to m ore than 5,000 schools. 134. GHI also participated in the developm ent of the national school feeding strategy, GHI contributed its experience from Phase I to inform the design of the national program m e on social behavior change com m unication m aterials, nutrition education, kitchen gardens, training of cooks on nutrition, the revision of nutrition indicators, and engaging parents. During the national strategy developm ent GHI advocated for a nutrition-sensitive com ponent, em phasizing that the programme needs to ensure that schools have the infrastructure, capacity and tools to produce and consum e nutritious foods at school. 135.The McGovern-Dole Program and the National School Feeding policy support building the capacity of sm allholder farm ers in cooperatives to supply food directly to schools for the school feeding programme. This strategy is designed to m eet school needs for a diverse diet of nutritious foods while increasing the m arket and incom e for sm all agricultural cooperatives that are located close to the schools they supply. In Phase II, WFP will continue to build capacity of sm allholder farm ers in m anagem ent, production, storage, and m arketing. The farm er cooperatives interviewed in Group 1 areas acknowledge the value of the training received from WFP in good agricultural practices, increased production, post-harvest handling and storage techniques, management, and m arketing. The cooperatives acknowledge that a purpose of the McGovern￾Dole Program and the NSFP is to supply schools with food for school m e als, and cooperative m em bers have 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 25 children enrolled in schools with m eal programme s. Cooperatives in Group 2 areas anticipate that the programme will provide capacity building and m arket linkages for their agricultural production. 4.2 Co h e r e n ce Eva lua t ion questions releva nt a t ba seline EQ 3: To what extent are the activities aligned with government capacity building gaps within the national school feeding programme? [TOR, Relevance EQ1 part 2] EQ 4: To what extent is the McGovern-Dole programme aligned with overall USDA objectives as well as strategies, policies, and normative guidance, and with Government’s relevant stated national policies, including sector policies and strategies? [TOR, Relevance EQ2] EQ 5: To what extent is the McGovern-Dole programme a ligned with United Nations frameworks and relevant development partners? To what extent is it aligned with WFP’s overall strategy and related guidance? [TOR Relevance EQ3, part 1] EQ 6: To what extent is it aligned with WFP’s overall strategy and related guidance? [TOR, Relevance EQ3, part 2] 136.These evaluation questions are discussed in the course of the baseline observations m ade below, organized by McGovern-Dole strategic objectives. Ba seline observa t ions Improved literacy of school-age children (MGD SO1) 137.The McGovern-Dole Program contributes to national developm ent goals on education, food and nutrition security, and capacity strengthening. It aligns with national objectives, policies, strategies, and plans, including governm ent plans to increase the literacy rate to 99 percent and increase the pre -prim ary enrolm ent rate by 2050, as well as reduce poverty. McGovern-Dole Program stakeholders confirm that governm ent officials see the programme as an im portant contributor to the developm ent of the nationa l school feeding programme . During Phase I of the McGovern-Dole Program, WFP supported the Governm ent to develop the policy, strategy, and national guidelines for a national scale -up of the school feeding programme . 138.McGovern Dole Program interventions align with and are com patible with other early education and literacy initiatives in Rwanda. These include Soma Umenye (2016-2021), a national early-grade reading activity for one m illion children in grades 1-3, funded by United States Agency for Internationa l Developm ent (USAID); the Hom es and Com m unities project (started Novem ber 2021), funded by USAID and im plem ented by World Vision, focusing on pre -prim ary and early prim ary grades. and its predecessor USAID-funded early-grade literacy project, Mureke Dusom e (2015-2020). It also includes the Building Learning Foundations project funded by the United Kingdom Governm ent partners with MINEDUC and REB, which focuses on strengthening the quality of English language and m ath teaching for 2.6 m illion children in grades 1-3 in public and governm ent prim ary schools. 139.The McGovern-Dole Program also aligns with the draft National Literacy Policy (2019) and programm e activities directly support the literacy policy principles. For exam ple, the policy principle s that literacy skills are an inherent part of the right to education, and that form al literacy instruction is m ost effective when supported by a rich literate environm ent, are supported by the McGovern-Dole Program activities to train teachers in m odern teaching m ethods and provide classroom m aterials, including books, for a richer learning environm ent, The McGovern-Dole Program supports the objectives of the draft national policy firstly by providing a nutritious m eal that prom otes regular attendance, attention, and concentration, all of which contribute to im proved learning and literacy. Group 2 district officials say that they have partnerships with WVI, USAID-funded projects, and Building Learning Foundations that support teacher training and provide books to schools, and acknowledged strong involvem ent by MINEDUC and MINAGRI in the McGovern Dole programme . Group 1 district officials cited WVI for its large role in im proving literacy and felt that this work was sufficient to m eet district needs and that other partners or agencies were not needed. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 26 Increased use of health and dietary practices (MGD SO2) 140.De worm in g. Intestinal worm s prevent the absorption of nutrients, and is a recognized problem am ong school-age children in Rwanda. 84 Deworm ing is the responsibility of WFP’s partner, the Rwanda Biom edical Centre (RBC) and is done with schools through health center workers. Teachers are also involved in recording the num ber of students and distributing deworm ing pills. Teachers largely gave favorable reviews of the training received in im plem enting the programme . There is not uniform knowledge am ong teachers as to whether this deworm ing programme is WFP-led or from the Governm ent. 141.Ha n dwa sh in g. MINISANTE launched a national handwashing strategy in 2019 to prom ote handwashing at critical health points for all fam ilies by 2024.85 Handwashing is a feature of the McGovern-Dole Program to prom ote health and good hygiene practices in schools. In response to the COVID-19 pandem ic, MINEDUC worked with World Vision and UNICEF to develop a design for a perm anent low-cost handwashing station. Schools were able to construct handwashing stations during the pandem ic using this design. During Phase II, World Vision will train teachers, parents and com m unity leaders to build m obile handwashing stations using locally available m aterials, which will com plem ent the perm anent handwashing stations. Governm ent and McGovern-Dole Program partners also worked together in Phase I to develop a design for low-cost kitchen infrastructure for schools. National school feeding programme (NSFP) readiness 142.The McGovern-Dole Program is aligned with national priorities to operationalize a nationally funded school m eals programme in all schools in Rwanda. The Governm ent received technical assistance from WFP during Phase I of the McGovern-Dole Program that supported the roll-out of a national school m eals programme through technical support and appropriate capacity strengthening. WFP continues in Phase II to help local and national governm ent institutions to design, m anage, and im plem ent a national school feeding programme while transitioning out of direct im plem entation of school feeding programme s. WFP continues to work closely with the NSF Technical Working Group (TWG) to lay the foundation for operationalizing the school feeding policy. In July 2019, WFP and MINEDUC spe arheaded the developm ent of draft School Feeding Operational Guidelines, and in 2020, cook/storekeeper training m aterials. The McGovern-Dole Program also aligns with the School Meals Coalition form ed in 2021 to im prove the quality of school m eals and strengthen school m eal system s globally. In Septem ber 2021, the Governm ent of Rwanda affirm ed its m em bership to the School Meals Coalition, com mitting to scaling up the NFSP. 143.At national level, a McGovern-Dole Program partner sees that school feeding is gradually influencing food and nutrition policy by creating greater awareness of school m eals as a vehicle for im proved nutrition at school and at hom e. WFP sees increasing dialogue between MINEDUC, MINAGRI, and MINISANTE on nutrition-sensitive interventions, attributed in part to the school feeding program m es. WFP notes that in Phase II, WFP will have a seconded staff in the NCDA covering nutrition and school feeding, which will help strengthen the links am ong the national nutrition agenda, the national school feeding agenda, and the McGovern-Dole Program . In addition, in Phase II WFP will bring in an additional senior advisor on public procurem ent to help refine the procurem ent m odel in the National School Feeding Program . 144.At the district level, McGovern-Dole Program activities have been aligned with Imihigo. Imihigo are form al public service com m itm ents signed by districts and other governm ent bodies to deliver key specific outputs that contribute to Rwanda’s developm ent. The Imihigo contracts are planned and evaluated annually. The inclusion of programme activities into Imihigo will help ensure coherence with other activities, and accountability at the local level. 84 United Nations Rwanda. 2017. Rwanda Com m on Country Analysis. 85 WFP. McGovern-Dole program m e proposal Phase II. 2020. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 27 4.3 Effe ct ive n e s s Eva lua t ion quest ions releva nt a t ba seline EQ 7: To what extent is the M&E system designed to adequately respond to the needs and requirements of the project? Is the M&E system designed to sufficiently capture changes in the lives of the beneficiaries? [TOR, Effectiveness EQ3] 145.The key lesson learned around the M&E system at baseline stage is that Phase II targets should be reviewed with specific target setting for Group 1 and Group 2 schools. The baseline values prove an opportunity to reflect on the feasibility of Phase II targets for the indicators under the evaluation team ’s responsibility – i.e., indicators for which we have values at this tim e. The prim ary and critical observation is that Phase II targets are set at the all-schools level, i.e., the sam e target for Group 1 and Group 2 schools; targets need to be set for each group separately with clear docum entation of calculations and underlying assum ptions. Group 1 schools are generally further ahead due to their participation under Phase 1. Even in the few cases where baseline values are sim ilar due to high school staff turnover, Group 1 schools m ay still have an advantage due to the im proved infrastructure already in place and institutionalization of practices. In addition, the evaluation team has m ade additional observations by indicator for consideration by WFP in its target setting review. These are detailed in Annex 19. Ba seline observa t ions 146.For the quantitative analysis reported in this section, significance tests were perform ed to determ ine whether the estim ates for program m e schools (Group 1 and Group 2) differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. The tables note a ssociations found to be statistically significant at the p<0.10, p<0.05, and p<0.01 levels. The narrative highlights results that are statistically significant at the p<0.10 level or higher. Improved literacy of school-age children (MGD SO1) 147.At t e n d a n ce . Table 5 presents the average student attendance for 2021, as reported by head teachers. Unsurprisingly Group 1 schools, which participated in school feeding programme s during the first phase of the McGovern-Dole Program and continue in Phase II, show a higher num ber of students attending for both m ale and fem ales. Ta b le 5: Ave ra ge st u d e n t a t t e n d a n ce ra t e (p e rce n t ), a s e st im a t e d b y h e a d t e a ch e rs, b y se x a n d sch ool ca t egory Total 2021 WFP Phase II Group 1 2021 WFP Phase II Group 2 2021 Control Group 2021 Fem ales – Prim ary 82.4 94.0*** 68.9 71.8 Males – Prim ary 81.9 92.7*** 69.2 71.9 Source : Head teacher baseline survey 2020 Significance tests were performed to determine whether the estimates for programme schools differ from the estimates for control schools: the tests compared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. Number of schools, by type: Group 1 – 21 schools; Group 2 – 10 schools; control group – 10 schools. 148.The disaggregation of vulnerable groups is not well captured in school data outside of counting fem ale versus m ale enrolm ent and attendance, and incom e levels. Som e district officials told the qualitative team that they largely see the em phasis of “Education for All” as addressing gender issues, such that there is no need to focus on specific issues of gender or m arginalized groups. There is recognition am ongst school com m ittee m em bers and teachers that girls often stay at hom e due to dom estic duties and boys m iss school when seeking work. In general, there are no specific gender initiatives – or vulnerability centered initiatives – to encourage attendance. According to teachers interviewed in the qualitative survey, disabled students are engaged in the sam e way by teachers, and any initiative would have been organized internally by teachers. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 28 149.En rolm e n t . Table 6 shows the student enrolm ent for the 2021 school year, disaggregated by grade and sex. Overall, there was a significant num ber m ore of m ale and fem ale students attending McGovern-Dole schools (Group 1) than control schools. Group 2 and control schools also show som e significant differences in all grades, especially for m ale students. Ta b le 6: Tot a l st u d e n t s e n rolle d (2021), b y gra d e , se x, a n d sch ool ca t e gory, a s re p ort e d by head teachers All Schools WFP Phase II Group 1 2021 WFP Phase II Group 2 2021 Control Group 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 1st Grade 6377 3059 3318 3308 1521*** 1787* 1836 931* 905** 1233 607 626 2nd Grade 5527 2674 2853 2929 1435*** 1494 1485 716*** 769 1113 523 590 3rd Grade 3941 1935 2006 2282 1123*** 1159** 980 484 496* 679 328 351 4th Grade 3563 1793 1770 1974 1001*** 973* 920 463 457* 669 329 340 5th Grade 3279 1695 1584 1871 969*** 902** 887 456 431* 521 270 251 6th Grade 2449 1335 1114 1375 736*** 639** 702 400 302* 372 199 173 EDC 2207 1166 1041 1392 720 672* 389 216* 173 426 230 196 Total Students 27343 13657 13686 15131 7505 7626 7199 3666 3533 5013 2486 2527 Source : Baseline census e -survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. Num ber of schools, by type: Group 1 – 21 schools; Group 2 – 10 schools; control group – 10 schools. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 29 150.Re p e a t le a rn e rs. Table 7 shows the breakdown of repeat learners, by grade, for the 2020 school year. The gender-disaggregated data by school category show that groups 1 and 2 have m ore repeat learners than do control schools for grades 1 through 6. Ta b le 7: Pe rce n t of re pe a t le a rn e rs (p rior a ca d e m ic ye a r), a s re p ort e d b y h e a d t e a ch e rs, b y gra d e , se x, a n d sch ool ca t e gory All Schools WFP Phase II Group 1 2020 WFP Phase II Group 2 2020 Control Group 2020 Total students 2020 Fem ale Students 2020 Male Students 2020 Total students 2020 Fem ale Students 2020 Male Students 2020 Total students 2020 Fem ale Students 2020 Male Students 2020 Total students 2020 Fem ale Students 2020 Male Students 2020 1st Grade 28.2 27.6 28.7 25.0 24.2* 25.7** 37.4 37.4** 37.4** 21.4 20.1 22.6 2nd Grade 23.2 19.9 26.4 22.2 20.4*** 24.1* 27.7 21.8** 33.5*** 16.8 13.3 20.2 3rd Grade 18.5 15.4 21.3 19.2 15.3** 22.9*** 20.4 17.7** 23.1*** 10.3 10.0 10.5 4th Grade 18.5 16.6 20.3 19.0 18.1*** 19.7*** 20.4 15.8*** 24.6*** 10.7 11.2 10.2 5th Grade 22.6 21.7 23.6 25.4 24.3*** 26.5*** 20.1 19.8*** 20.5** 16.3 15.0 17.8 6th Grade 6.5 6.5 6.6 8.2 8.1*** 8.3** 4.3 4.7** 4.0 2.8 1.1 5.6 EDC 2.9 2.9 3.0 2.4 2.4 2.5 5.8 5.4 6.3 0.0 0.0 0.0 Tot a l St u d e n t s 20.4 18.7 22.1 19.8 18.3 21.3 23.6 21.4 25.9 14.8 13.5 16.1 n (sch ools) 41 21 10 10 Baseline census e -survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 30 151.Dropou t s. Table 8 shows the data on dropouts, by grade, for the 2020 school year. The dropout rate varies across grades but generally ranges from 2 to 5 percent, with som e statistically significances between program m e and control schools. In all cases where the differences are statistically significant, program m e schools have higher dropout rates than control schools. Ta b le 8: Pe rce n t of d rop ou t s (p rior a ca de m ic ye a r), a s re p ort e d b y h e a d t e a ch e rs, b y gra de , se x, a n d sch ool ca t e gory All Schools WFP Phase II Group 1 2020 WFP Phase II Group 2 2020 Control Group 2020 Total students 2020 Fem ale Students 2020 Male Students 2020 Total students 2020 Fem ale Students 2020 Male Students 2020 Total students 2020 Fem ale Students 2020 Male Students 2020 Total students 2020 Fem ale Students 2020 Male Students 2020 1st Grade 2.1 2.2 2.0 1.8 2.0 1.7 3.0 3.1* 2.9* 1.4 1.3 1.6 2nd Grade 2.2 1.9 2.5 1.9 1.4 2.4 2.4 2.1 2.8 2.8 3.4 2.2 3rd Grade 3.7 3.1 4.3 3.7 3.3 4.2* 3.8 2.4 5.1** 3.4 4.3 2.7 4th Grade 3.5 3.0 3.9 2.8 2.6* 2.9 5.1 4.2*** 6.0** 2.2 1.5 2.9 5th Grade 4.2 3.5 5.0 3.5 2.3 4.8* 5.3 5.2 5.4* 4.1 3.8 4.6 6th Grade 3.4 3.2 3.6 3.8 3.6** 4.1** 3.1 3.1*** 3.0* 0.7 0.0 1.9 EDC 1.8 1.4 2.2 2.5 2.2 2.9 1.0 0.4 1.7 0.0 0.0 0.0 Tot a l St u d e n t s 2.9 2.6 3.2 2.7 2.4 3.0 3.6 3.1 4.0 2.3 2.2 2.3 Baseline census e -survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 31 152.Da ily m e a ls. Table 9 shows daily m eals received by students for the 2021 school year, disaggregated by grade and sex. Again, as expected, Group 1 demonstrates a significantly higher num ber of m eals being provided to its students (in com parison to the control) for grades 1 through 6 by both m ale and fem ale students. The control group shows higher m eal num bers than Group 2 for all grades, however these differences are only significant for grades 1 (fem ales and m ales), 2 (fem ales and m ales) and ECD (m ales). Ta b le 9: Nu m b e r of d a ily sch ool m e a ls re ce ive d (2021), b y gra d e , se x, a n d sch ool ca t e gory, a s id e n t ifie d b y t e a ch e rs All Schools WFP Phase II Group 1 2021 WFP Phase II Group 2 2021 Control Group 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 1st Grade 4148 1920 2228 3308 1521*** 1787*** 212 84** 128* 628 315 313 2nd Grade 3589 1778 1811 2929 1435*** 1494*** 151 82** 69*** 509 261 248 3rd Grade 2832 1384 1448 2282 1123*** 1159*** 233 108 125 317 153 164 4th Grade 2720 1363 1357 1974 1001*** 973*** 341 169 172 405 193 212 5th Grade 2553 1335 1218 1871 969*** 902*** 363 196 167 319 170 149 6th Grade 1889 1017 872 1375 736*** 639*** 299 179 120 215 102 113 ECD 1175 629 546 679 352 327 136 83 53* 360 194 166 Total Students 18906 9426 9480 14418 7137 7281 1735 901 834 2753 1388 1365 Source : Baseline census e -survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. Num ber of schools, by type: Group 1 – 21 schools; Group 2 – 10 schools; control group – 10 schools. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 32 153.Sa t isfa ct ion w it h sch ool m e a l con t e n t a n d q u a lit y. Students told the qualitative team that they like the rice that is served but do not like the m aize as m uch because they eat it often at hom e. Schools that accept food contributions from parents have found a problem with cooking a variety of donated beans that require different tim es for cooking as a result not all beans in the m ix are well cooked. In two Phase I schools, students said there is not enough food to satisfy their hunger. This could be due to using som e of the food for nursery students (several teachers reported that food is saved for nursery schools, though they are not counted in the final num ber of students fed). It is noted that Phase II will include food for pre -prim ary students, which should address this issue. 154.At t e n t ive n e ss. Table 10 reports on student attentiveness, as assessed by teachers. Overall, close to three - quarters (71.9 percent) of all students were identified as attentive. The findings show statistical differences only between Group 1 schools and control schools, for grades 2, 3 and 6. In these cases, attentiveness rates for students in the Group 1 program m e schools are higher than for control schools by 7-13 percentage points. These results are consistent with interview findings: teachers in Group 1 schools told the qualita tive team that one of the positive im pacts of the McGovern-Dole Program is that student perform ance and concentration have increased, resulting in higher grades. Qualitative interviews also suggest that regular attendance and attentiveness of students are linked with school m eals. Ta b le 10: Pe rce n t of e n rolle d st u de n t s ide n t ifie d a s a t t e n t ive by t h e ir t e a ch e rs, by gra de a n d sch ool category All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Schools All Grades 71.9 74.9 71.7 58.0 1st Grade 69.8 73.9** 67.7 63.3 2nd Grade 71.9 74.5* 71.2 67.0 3rd Grade 71.6 73.6 70.0 69.2 4th Grade 71.5 74.8 67.6 68.5 5th Grade 72.3 75.7* 75.4 62.0 6th Grade 74.2 76.3 78.0 66.1 Baseline census e -survey 2020 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 155.Lit e ra cy re su lt s. To highlight the im portance of local reading clubs and the availability of books and support, endline percentages from Phase I of the McGovern-Dole Program m e are shown in Table 11 and Table 12; when reading clubs m et regularly prior to COVID-19; this com parison to endline is discussed first, followed by discussions of specific findings from each table . 156.The findings in Table 11 and Table 12 show that reading and com prehension scores at the Phase I endline were high for both fem ale and m ale students. Covid-19 m easures such as school closures and the closure of reading clubs m ay have contributed to a regression of students’ progress since the endline survey. This observation is supported by com m ents from teachers to the qualitative team that the reading clubs were very effective in m otivating students to read and to parents to support reading tim e. This regression can be seen in the reading and com prehension scores of both m ale and fem ale students of Group 1 in Table 11 and for Group 1 m ale students in Table 12. 157.Table 11 presents the results on students’ abilities to read a short story aloud. The proportion of fem ale students who can com pletely read the story given is slightly higher than m ale students, regardless of school category (m ale -fem ale differences were not tested for statistical significance). Overall, m ore than three quarters (75.8 percent) of students from control schools reported the ability to com pletely read a short story aloud com pared to only two-thirds (66.17 percent) of students in Group 1 and 68.01 percent of students in Group 2. A factor that possibly explains higher perform ance on this indicator in control schools is that the latter have fewer students than programme schools, so teachers are better able to follow up on individual 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 33 students, whereas in programme schools, classroom s are m ore crowded, m aking it m ore difficult for teachers to follow and support the progress of each student. Qualitative findings also indicated apparent strong leadership from head teachers in control schools. Ta b le 11: Pe rce n t a ge of st u de n t s a ble t o com ple t e ly re a d a sh ort st ory a lou d, by se x a n d sch ool category Ba se lin e 2021 All Sch ools WFP Ph a se II Grou p 1 WFP Ph a se II Grou p 2 Con t rol % n % n % n % n Males 65.9 452 64.3* 235 65.8 108 74.1 109 Fem ales 70.4 452 68.1* 235 70.2 114 77.7 103 Tot a l 68.9 904 66.17** 470 68.01* 222 75.8 222 Source : EGRA baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. Num ber of schools, by type: Group 1 – 21 schools; Group 2 – 10 schools; control group – 10 schools. 158.The findings in Table 12 indicate that overall, a higher proportion of students from the control group (68.7 percent) than those from Group 1 (60 percent) or Group 2 (59.9 percent) could read and understand grade - leve l text. Disaggregating results by sex shows that a lower percentage of m ales in Group 1 (58.29 percent) could read and understand grade -level text com pared to control schools (68.9 percent. Ta b le 12: Pe rce n t of st u de n t s wh o, by t h e en d of t wo gra des of prim a ry sch oolin g, dem onstra t e t h a t t h ey ca n rea d a n d un derst a n d t he m ea n in g of gra de -level t ext, by sex a n d sch ool ca t egory Ba se lin e 2021 All Sch ools WFP Ph a se II Grou p 1 WFP Ph a se II Grou p 2 Con t rol % n % n % n % n Males 62.2 452 58.29* 235 61.11 108 68.9 103 Fem ales 61.5 452 61.7 235 58.77 114 68.8 109 Tot a l 62.1 904 60.0** 470 59.9** 222 68.7 212 Source : EGRA baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. Num ber of schools, by type: Group 1 – 21 schools; Group 2 – 10 schools; control group – 10 schools. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 34 159.Table 13 disaggregates the reading and com prehension results by district. There are only two statistically significant different results across school categories: in Nyaruguru district, a larger proportion of students in control schools (85 percent) could read and understand grade -level text com pared to Group 1 schools (55.8 percent). The data for Nyam agabe district show an opposite finding: in this case, Group 1 schools perform ed better on this indicator than control schools (67.8 percent versus 43.8 percent, respectively). Ta b le 13: Pe rce n t of st u de n t s wh o, by t h e en d of t wo gra des of prim a ry sch oolin g, dem onstra t e t h a t t h ey ca n rea d a n d un derst a n d t he m ea n in g of gra de level t ext, by dist rict a n d sch ool ca t egory Baseline 2021 All Sch ools WFP Ph a se II Grou p 1 WFP Ph a se II Grou p 2 Con t rol Sch ools n n n n Karongi 70.2 242 68.4 177 n/a n/a 75.4 65 Nyaruguru 60.4 134 55.8*** 113 n/a n/a 85.0 20 Nyam agabe 64.2 106 67.8* 90 n/a n/a 43.8 16 Rutsiro 41.1 90 41.1 90 n/a n/a n/a n/a Gasabo 72.7 44 n/a n/a 72.7 22 72.7 22 Kayonza 66.9 112 n/a n/a 66.6 90 68.2 22 Burera 55.6 176 n/a n/a 51.8 110 62.1 66 All Districts 62.1 904 60.0 470.0 59.9 222 68.7 212 Source : EGRA baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 160.Som e of the factors lim iting the effectiveness of literacy program m es relate to interruptions due to COVID￾19 related containm ent m easures and school closures. Reading clubs were disrupted in Phase I, reducing students' ability to borrow books to read at hom e, but were replaced by house -to-house book lending, and by school lending once schools reopened. The qualitative survey found that teachers feel the lending activities are less effective than reading clubs in prom oting reading for several reasons: teachers are not sure that students have read the book, students m ay not be as m otivated to read at hom e, students are occupied in household chores and do not have a lot of tim e to read, and parents do not rem ind children to read. This m ay have had unintended effects on students’ literacy levels in Group 1. While the reading club closures occurred during Phase I, they still had a downstream effect in Phase II; as discussed above, as the overall proportion of students from the control group reported a significantly higher proportion of students reading and understanding grade level texts and being able to com pletely read a short story com pared to those from Group 1 or Group 2 schools. 161.Table 14 shows that overall and also by sex, about three -quarters of students reported receiving reading help from their parents. There are no statistically significant differences across school types. Ta b le 14: Pe rce n t of st u de n t s wh o re ce ive re a din g h e lp from pa ren t s, by sex a nd sch ool ca t egory Baseline 2021 All Sch ools n WFP Ph a se II Grou p 1 n WFP Ph a se II Grou p 2 n Con t rol n Male s 77.0 452 77.4 235 78.7 108 74.3 109 Fem ale s 76.3 451 78.3 235 76.1 113 71.8 103 Tot a l 76.7 903 77.9 470 77.4 221 73.1 212 Source : Baseline census survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 35 162.Table 15: reports the percent of students (total and disaggregated by gender) who can read in different words-correct-per-m inute ranges (WCPM). Overall, about half of all students were able to read at the highest level of 56 WCPM or m ore. The results show some statistically significant associations between fem ales in both groups com pared to control schools, m ore notably in higher WPCM categories. Statistical tests were not perform ed to com pare m ales and fem ales. Ta b le 15: St u d e n t p e rform a n ce b y w ord s-correct-per-m in u t e (WCPM) ca t e gorie s (p e rce n t of 3rd grade st u d e n t s), b y se x a n d sch ool ca t e gory Baseline 2021 All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Schools # WCPM Total M F Tot a l M F Tot a l M F Tot a l M F 1 to 15 0.8 0.0 1.1 0.0 0.0 0.0 1.3 0.0 2.6* 0.0 0.0 0.0 16 to 30 2.4 1.0 3.9 2.7 1.2 4.3 2.2 0.0 3.5 1.8 0.0 3.8 31 to 40 9.3 6.4 12.2 8.7 6.4 11.1 9.0 4.6 13.2 10.8 8.2 13.5 41 to 56 35.0 35.6 34.3 34.0 34.9 33.2* 32.4 36.1 28.9* 39.6 36.6 42.7 >56 words 52.5 56.6 48.5 53.8 57.0 50.0* 55.0 58.3 51.7* 47.2 54.1 39.8 n 904 452 452 470 235 235 222 108 114 212 109 103 Source : EGRA baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. Increased use of health and dietary practices (MGD SO2) 163.Table 16: shows that all Group 1 and Group 2 schools were using som e form of im proved water source com pared to 90 percent of control schools. Analyzing by type of water source, the only significant difference across school types was piped water, used by 85.7 percent of Group 1 schools com pared to 50 percent of control schools. All three categories of schools were predom inantly using rainwater, and public taps though to a m uch lesser extent. No schools used spring water (whether protected or unprotected). Ta b le 16: Pe rce n t a ge of sch ools u sin g im prove d wa t e r sou rce s, by wa t e r sou rce t ype a n d sch ool category Baseline 2021 All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Schools Any im proved water source 97.9 100.0 100.0 90.0 Piped water 70.7 85.7** 60.0 50.0 Public Tap 9.8 4.8 20.0 10.0 Protected Spring (im proved) 0.0 0.0 0.0 0.0 Rainwater (im proved) 90.3 90.5 90.0 90.0 Unprotected spring (not im proved) 0.0 0.0 0.0 0.0 n 41 21 10 10 Source> Baseline head teacher survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 164.Having a reliable water source on site is critical to both school m eal preparation and hygiene . Connecting schools to piped water and was a key infrastructure activity during Phase I; most WASH activities have been paused until the com pletion of the baseline study for Phase II. During Phase II, the program m e will continue to increase access to safe drinking water and prom ote the adoption of hygiene practices through the construction of latrines, perm anent handwashing stations, and sanitary room s. 165.The introduction of girl’s sanitary room s has been a significant feature of the McGovern-Dole Program, designed to provide a safe space for girls to address their m enstrual hygiene needs at school and thus be able to attend school regularly. The Governm ent has recognized the im portance of the sanitary room s, and 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 36 in 2020 announced a new draft law that would m ake having a girl’s safe room m andatory for all schools from prim ary to university level. The qualitative team noted that where schools have shut off water taps due to problem s paying water bills, the sanita ry room s m ay have less access to water. 166.Table 17 reports the percentage of students who can identify at least three of the program m e -targeted health and hygiene practices. The results indicate that overall, only 13.4 percent can do so. The disaggregation by school category shows that Group 1 schools perform substantially better on this indicator than control schools, though the percentages in Group 1 are still low, at around 19 percent of students who can nam e three key health and hygiene practices. Ta b le 17: Pe rce n t of st u de n t s wh o ca n ide n t ify a t le a st t h re e ke y h e a lt h a n d h ygie n e pra ct ice s, by sex a n d sch ool ca t egory Baseline 2021 All Sch ools n WFP Ph a se II Grou p 1 n WFP Ph a se II Grou p 2 n Con t rol Sch ools n Male students 13.3 452 18.29*** 235 9.3 108 6.4 109 Fem ale student 13.7 452 19.1*** 235 7.9 114 7.8 103 Tot a ls 13.4 904 18.7*** 470 8.6 222 7.1 212 Source : EGRA baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 167.Table 18 shows the percent of the students who indicated that regularly practice the selected key health and hygiene habits. These results are subject to self-reporting bias, as students m ay over-report the frequency of these behaviors, wishing to provide the desired response. Overall, the m ost reported practice was bathing at 91% for all schools. Group 1 showed better (albeit still low) results than the control group for handwashing before eating (Group 1, 17.78 percent versus control 13.2 percent), brushing teeth (11.06 percent versus 6.1 percent) and handwashing with soap after using the toilet (8.08 percent versus 4.7 percent). Both Group 1 and Group 2 values for drinking and using clean water, while low (8.9 -percent and 7.2 percent respectively) were still higher than the control group (2.8 percent). Ta b le 18: Pe rce n t of st u de n t s t o re gu la rly pra ct ice key h e a lt h a n d h ygie n e pra ct ice s, by t ype of pra ct ice a nd sch ool ca t egory National School Feeding Programme readiness 168.For Group 1 schools in Phase II, there is m ore NSFP readiness com pared to Group 2 schools. WFP Rwanda is largely seen as doing well in their consultations with the districts for input. There is a noticeable im provem ent Baseline 2021 All Sch ools WFP Ph a se II Grou p 1 WFP Ph a se II Grou p 2 Con t rol Other personal hygiene (i.e. bathing) 91.0 903 90.4 470 88.7 221 94.8 212 Handwash with soap after toilet 6.5 903 8.08* 470 4.9 221 4.7 212 Handwash before eating 15.8 903 17.78** 470 14.0 221 13.2 212 Brush teeth 8.5 903 11.06*** 470 5.4 221 6.1 212 Eating a balanced diet 17.7 903 16.6 470 22.2 221 15.6 212 Avoid open defecation 0.0 903 1.0 470 0.0 221 0.0 212 Other – Drink/Use clean water 0.1 903 8.9*** 470 7.23** 221 2.8 212 Source : EGRA baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 37 in com m unication since Phase I, though one district was dissatisfied with the incorporation of its suggestions for the Phase II design. Group 1 District officials and teachers say there is good support from WFP, but m ore involvem ent from the com m unity and parents is needed. Teachers expect the effectiveness and efficiency of the program m e to im prove with governm ent intervention. Several schools that are newly im plem enting the NSFP said the Governm ent has built kitchens for the program m e, though kitchens were not yet fully equipped. 169.Table 19: presents self-reported data on the kinds of support schools receive from outside organizations (i.e., including but not lim ited to WFP). The data show that all Group 1 schools, 90 percent of Group 2 schools and 70 percent of control schools reported receiving school feeding support from som e organization. The NFSP has begun its roll-out, which is reflected in the 70 percent of control schools receiving school feeding. Notably, more than a third (38.1 percent) of Group 1 schools received support for nutrition activities other than school feeding, com pared to none of the Group 2 or control schools. All Group 1 schools com pared to 90 percent of Group 2 schools and 80 percent of control schools received support for deworm ing. The overwhelm ing m ajority of Group 1 schools (90.5 percent) also received support for sanitation (including water and toilets), com pared to less than a third (30 percent) of Group 2 schools and only 20 percent of control schools. Over half of Group 1 schools (52.4 percent) received support for school governance com pared to 30 percent of Group 2 schools and no control schools. A greater proportion of Group 1 schools (85.7 percent) com pared to Group 2 schools (80 percent) and control schools (70 percent) received support for teacher training. Group 1 teachers interviewed for the baseline study stated that the McGovern-Dole Program m e has done well in providing needed skills to teachers and students and the surrounding com m unity. However, due to a shortage of professional staff, teachers in Group 1 and Group 2 said they m ust cover m any subjects, indicating that this affected the quality of their teaching. Ta b le 19: Su p p ort re ce ive d from orga n iza t ion s, a s re p ort e d b y h e a d t e a ch e rs (p e rce n t of sch ools), b y t ype of su pport a n d school ca t egory Baseline 2021 All Sch ools WFP Ph a se II Grou p 1 WFP Ph a se II Grou p 2 Con t rol Sch ools School feeding 90.2 100.0** 90.0 70.0 Other nutrition activities 19.5 38.1** 0.0 0.0 Deworm ing 92.7 100.0** 90.0 80.0 Sanitation (water and toilets) 58.5 90.5*** 30.0 20.0 School governance 34.1 52.4*** 30.0* 0.0 Provision of school m aterials, textbooks, books 100.0 100.0 100.0 100.0 Renovation/construction of infrastructure in school e.g. classes, kitchens, stores 85.3 95.2 70.0 80.0 Training of teachers 80.4 85.7** 80.0 70.0 Health education 22.0 38.0 10.0 0.0 n (sam ple) 41 21 10 10 Source : Head teacher baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 38 170.Table 20 presents the e -survey data on the kinds of m eals schools provide. It shows that all Group 1 schools provide a school m eal com pared to 80 percent of Group 2 schools and only 70 percent of control schools. Of those schools that provide a school m eal, all Group 1 and Group 2 schools provide lunch, whereas 85.7 percent of control schools do. Only control schools provide breakfast, and no schools provide snacks. Ta b le 20: Pe rce n t of m e a ls provide d by sch ools du rin g t h e 2021 sch ool ye a r, by m e a l t ype All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Schools Provide a m eal 87.8 100*** 80.0 70.0 n 41 21 10 10 Provide breakfast 2.7 0.0* 0.0 14.3 Provide lunch 97.2 100.0* 100.0 85.7 Provide snacks 0.0 0.0 0.0 0.0 n 36 21 8 7 Source : Baseline census e -survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 171.Table 21 shows the percentage of parents contributing to the cost of school m eals as reported by head teachers for the 2021 school year. There are no statistically significant differences in Group 1 or Group 2 schools com pared to control schools in the percentages of parents contributing in-kind item s. There are large statistically significant difference s in the percentage of parents m aking cash contributions: 63.0 percent in Group 1 schools and 2.2 percent in Group 2 schools, versus 24.2 percent in control schools. There is also a large and significant difference in the percentage of Group 1 school parents who do not contribute (25.0 percent). This contrasts with parents who do not contribute at all in Group 2 schools (66.7 percent) and in control schools (57.8 percent). An im portant takeaway is that a high percentage of parents do not contribute to school m eals in Group 2 and control schools in particular. Ta ble 21. Pe rce n t of p a re n t s wh o con t rib u t e d t o sch ool m e a ls (2021), b y con t ribu t ion t yp e a n d sch ool ca t egory All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Group Cash 38.8 63.0*** 2.2** 24.4 In-Kind 19.1 15.4 30.5 15.3 Do not contribute 43.1 25.0*** 66.7 57.8 n (schools) 41 21 10 10 Source : Baseline census e -survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 172.Table 22 (next page) breaks down the parents’ in-kind contributions by parents by type. Overall, the m ost contributed item was food (26.8 percent of parents), followed by firewood (31.7 percent). There were large and significant differences in two item s: no parents in Group 1 schools contributed food, while 40 percent of parents in control schools did, and no parents in Group 2 schools gave firewood, com pared to 30 percent in control schools. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 39 Ta b le 22: Pe rce n t a ge of pa re n t s wh o con t ribu t e d in -kin d t o sch ool m e a ls (2021), b y t yp e of con t ribu t ion a n d sch ool ca t egory All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Group In-Kind: 19.1 15.4 30.5 15.3 Labour 9.7 14.3 10.0 0.0 Food 26.8 0.0*** 70.0 40.0 Water 0.0 0.0 0.0 0.0 Firewood 31.7 47.6 0.0* 30.0 Other 4.8 4.7 0.0 10.0 n (schools) 41 21 10 10 Source : Baseline census e-survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 173.Head teachers and district officials in both Group 1 and Group 2 told the qualitative interviewers that poverty and large fam ilies are two reasons that parents do not give cash. Som e fam ilies are already contributing to school m eals in secondary schools and contributing to prim ary school m eals will be an extra burden on these fam ilies. Som e other fam ilies do not give cash, especially in Group 2 schools, because they do not recognize the im portance of school m eals to learning. This issue was addressed in Group 1 schools by intensive awareness-raising by district and school officials, and key inform ants stated that a sim ilar approach will be em ployed with Group 2 schools. In addition, a head teacher in a Group 1 school observed that parents are told that education is free and so m ay not understand why contributions are requested. In Phase I, Group 1 schools would accept non-cash contributions in the form of labor, food item s, or firewood. Group 1 schools noticed that parents preferred non-cash contributions, which m ade it difficult for schools to efficiently plan and prepare m eals; consequently, most Group 1 schools now request a m onetary contribution. 174.Table 23 reports head teachers’ perceptions of what prim ary education benefits parents are able to nam e. Increasing earning potential is by far the m ost com m on response, regardless of school category. There are statistically significant differences between program m e schools and control schools in a few areas. An interesting finding is that head teachers in control schools were m uch m ore likely to state that parents would nam e increased opportunities, the ability to read, write and count, knowledge for daily life, and com m unity engagem ent as educational benefits, than their counterparts in Group 1 schools. Ta b le 23: Pe rce n t of pa re n t s t h a t ca n n a m e t h e be n e fit s of prim a ry edu ca t ion , a s a ssessed by h ea d t e a ch e rs, by t ype of be n e fit a n d sch ool ca t e gory Baseline 2021 Total WFP Phase II Group 1 WFP Phase II Group 2 Control Increased health 45.9 54.5 32.2 41.7 Im proved nutrition 52.2 49.6 46.5* 63.5 Increased opportunities 50.6 45.6* 50.0 61.8 Increased earning potential 77.4 76.4 76.0 80.8 Ability to read/write/count 54.3 46.0** 57.8 68.0 Knowledge for daily life 43.8 34.1** 52.5 55.5 Increased socialization 45.3 39.5 52.0 51.0 Strengthening relationships 52.3 44.4 58.0 63.0 Increased engagem ent with com m unity 20.1 7.04* 37.8 30.0 n 41 21 10 10 Source : Head teacher baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 40 175.Table 24 shows a breakdown of the percentage of enrolled students who received daily m eals, as reported by head teachers. All schools in Group 1 have reported 100 percent of students receiving m eals for grades 1 through 6. Group 2 also show significant results for grade 1 and grade 2, however in the opposite direction. It is noted that the McGovern-Dole Program m e has not yet started in Group 2 schools. Ta b le 24: Pe rce n t a ge of st u d e n t s re ce ivin g d a ily m e a ls (2021), b y gra d e , se x, a n d sch ool ca t e gory All Schools WFP Phase II Group 1 2021 WFP Phase II Group 2 2021 Control Group 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 Total students 2021 Fem ale Students 2021 Male Students 2021 1st Grade 65.0 62.8 67.1 100.0 100.0*** 100.0*** 11.5 9.0** 14.1* 50.9 51.9 50.0 2nd Grade 64.9 66.5 63.5 100.0 100.0*** 100.0*** 10.2 11.5** 9.0*** 45.7 49.9 42.0 3rd Grade 71.9 71.5 72.2 100.0 100.0*** 100.0*** 23.8 22.3 25.2 46.7 46.6 46.7 4th Grade 76.3 76.0 76.7 100.0 100.0*** 100.0*** 37.1 36.5 37.6 60.5 58.7 62.4 5th Grade 77.9 78.8 76.9 100.0 100.0*** 100.0*** 40.9 43.0 38.7 61.2 63.0 59.4 6th Grade 77.1 76.2 78.3 100.0 100.0*** 100.0*** 42.6 44.8 39.7 57.8 51.3 65.3 EDC 53.2 53.9 52.4 48.8 48.9 48.7 35.0 38.4 30.6* 84.5 84.3 84.7 Source : Baseline census survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 41 176.The NSFP has been newly introduced and was being rolled out when the qualitative study was conducted. Som e of the schools interviewed by the qualitative team im plem enting the new NSFP are using different strategies to provide at least som e m eals to poor students. For exam ple, one school rolling out the NSFP is using only governm ent m oney to fund school m eals as parents have not started contributing, though this strategy risks suggesting to parents that their contribution is not needed, and is quickly exhausting funds for m eals. Another school in the NSFP told the qualitative team that it provided m eals for all students for two weeks, then stopped because parents did not contribute, and now only 25 students out of 984 have m eals at school. Another school in the NSFP decided to provide students from the poorest fam ilies with a m eal because they know the fam ilies cannot afford a contribution. Som e head teachers observed that m any parents feel the program m e is unaffordable because they cannot contribute cash at school and at the sam e tim e feed children at hom e. One school in NSFP said that governm ent funds were received but m eals had been provided only six tim es since the start of term because parents did not contribute so m ost of the tim e there was no firewood to cook with. Two schools said that only P6 students received m eals, and only those whose parents contributed, leaving half to two-thirds of P6 students without a m eal at school. 177.Table 25 presents data (from schools reported by head teachers to provide additional food item s) from head teachers on the kinds of food the schools contribute to school m eals. Fruits and vegetables are the most com m on contribution regardless of school category. There are som e differences between Group 1 and control schools: all Group 1 schools provide fruit, com pared to about three -quarters of control schools, and 14.3 percent of control schools provide legum es, while Group 1 schools do not. Overall, schools supplem ent little in the way of legum es and anim al proteins such as m ilk and m eat. Ta b le 25: Pe rce n t of sch ools providin g a ddit ion a l food it e m s for sch ool m ea ls, by t ype of food and sch ool ca t egory All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Schools Fruits 91.7 100.0*** 87.5 71.4 Vegetables 80.6 71.4 100.0 85.7 Legum es 2.7 0.0* 0.0 14.3 Anim al Proteins (m ilk, m eat) 8.3 4.8 12.5 14.3 n 36 21 8 7 Source : Head teacher baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 42 178.Table 26 presents data (from schools reported by head teachers to provide additional food item s) from head teachers on the frequency of m eals that included school-supplem ented foods in school m eals in the week before the survey. Fruit and vegetables were m ost frequently included, 3-5 tim es per week on average. Serving legum es and anim al proteins was rare. Ta b le 26: Ave ra ge n u m b e r of m e a ls provide d in t h e la st we e k t h a t in clu de foods in a ddit ion t o t h e d on a t e d US com m od it ie s, b y type of food a n d sch ool ca t egory All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Schools Fruits 4.4 5.0*** 3.9 3.4 Vegetables 3.9 3.6 4.6 4.2 Legum es 0.1 0.0* 0.0 0.7 Anim al Proteins (m ilk, m eat) 0.3 0.2 0.3 0.7 n 36 21 8 7 Source : Head teacher baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 179.Table 27 sum m arizes head teachers’ reports of the sources of food for school feeding in their school (from schools reported by head teachers to provide additional food item s). As expected, all Group 1 schools indicated they obtain food from WFP or another NGO, and no Group 2 or control schools do. Few Group 1 schools (4.7 percent) receive governm ent support, while the vast m ajority of Group 2 schools (87.5 percent) and control schools (85.7 percent) do. Additionally, m ost Group 1 schools (90.4 percent) obtain food from kitchen gardens, while far fewer Group 2 schools (12.5 percent) or control schools (28.5 percent) do. A higher proportion of control schools (85.7 percent) reported parents provide food for school children com pared to 75 percent of Group 2 schools and only 14.3 percent of Group 1 schools. Ta b le 27: Sou rce of food for sch ool fe e d in g p rogra m m e , b y sou rce a n d sch ool category All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Schools WFP/NGOs provided 58.3 100.0 0.0 0.0 Governm ent 38.9 4.7*** 87.5 85.7 Kitchen garden 61.1 90.4*** 12.5 28.5 Local m arkets 66.7 71.4 62.5 57.1 Parents provided 41.7 14.3*** 75.0 85.7 n 36 21 8 7 Source : Head teacher baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 43 180.The head teacher survey also collected inform ation (from schools reported by head teachers to provide additional food item s) on school partnerships with farm ers groups. As shown in Table 28, 14.3 percent of control schools have such partnerships com pared to only 4.7 percent of Group 1 schools and none in Group 2, which is unsurprising given that the work with farm ers’ groups only started toward the end of Phase I. Ta b le 28: Pe rce n t of schools t h a t h a ve a pa rt n ersh ip wit h a n y fa rm ers grou ps for food in 2021 All Schools WFP Phase II Group 1 WFP Phase II Group 2 Control Schools Schools 5.5 4.7 0.0 14.3 n 36 21 8 7 Source : Head teacher baseline survey 2021 Significance tests were perform ed to determ ine whether the estim ates for program m e schools differ from the estim ates for control schools: the tests com pared Group 1 and control, and Group 2 and control. Associations found to be statistically significant are indicated by * p<0.1, ** p<0.05 and *** p<0.01. 181.The m ajority of cooperatives interviewed by the qualitative team are not selling their production to schools. Their reasons vary from wanting to sell produce in bulk while schools only want to buy a lim ite d am ount of produce, to a focus on local m arkets and buyers, to waiting for WFP to initiate the connection. When parents contribute food to schools, this has also m ade it difficult to sell to schools. District officials believe a priority for Phase II of NSFP should be m obilization and building the capacity of farm ers, schools, and local partners to ensure the sustainability of the programme . 182.Cooperative m em bers told the qualitative team that they have gained leadership and professional business skills through training from WFP. 4.4 Efficie n cy Eva lua t ion quest ions releva nt a t ba seline 183.There are no specific evaluation questions associated with this OECD-DAC criterion at baseline; the section discusses baseline observations. Ba seline observa t ions Improved literacy of school-age children (MGD SO1) 184. In a bid to curb overcrowding and long distances travelled by students going to and returning from schools, the Ministry of Education in June 2020 em barked on a plan to construct 22,505 new classroom s (17,414 prim ary schools, 3,591 for secondary schools and 1,500 classroom s for nursery schools). The construction was funded jointly by both the Governm ent or Rwanda and the World Bank’s credit financing. World Bank also allocated funds to other activities like training of teachers and construction of Teacher Training Center schools. In total World Bank’s credit financing for Rwanda Quality Basic Education for Hum an Developm ent project Phase 1 was worth US$200 m illion. 86 185.The Government has addressed the issue of overcrowded schools by building 22,000 new schools in 2021, 87 though at the tim e of the baseline there were not enough teachers to staff those schools and classroom s rem ain crowded. Key inform ants with school staff, district representatives and WFP flag the im portance of efficient budget allocation through m atching infrastructure investm ents with funding for school staff recruitm ent. On average programme schools have m ore students than control schools: because programme schools are in the sam e areas as control schools, students will often choose to attend the programme schools rather than the control schools to receive food, often resulting in classroom overcrowding. Teachers and head teachers have discussed the overpopulation of students as an issue. The num ber of teachers has not increased to accom m odate the increased num ber of students. Som e of the control schools are also newly constructed schools that were built with the intention of reducing the num ber of students in a nearby school. Students at tim es chose to attend these schools because they are closer to their hom es. 86 Data in this paragraph supplied via em ail by WFP Rwanda country office 28 January 2022. 87 Edwin Ashim we. 2021. The New Tim es. Construction of new classroom s at over 95%. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 44 186.Programme schools are governm ent-aided and m ay also benefit from internationally funded projects in literacy, education, health, and related areas. MINEDUC provides textbooks and other support to governm ent schools, but control schools typically do not receive external financial or m aterial support, only support from the Governm ent. Within the national governm ent there is good horizontal com m unication regarding education initiatives, according to programme partners. District officials who participated in Phase I were satisfied with the status of com m unication on the programme , though a few officials in both Group 1 and Group 2, inform ed the qualitative team that felt that com m unication between WFP and local officials could be strengthened. Increased use of health and dietary practices (MGD SO2) 187.Cooks told the qualitative team that they think there are too few supporting staff, and in general do not think the food is sufficient; lower prim ary tends to receive enough, but upper prim ary does not. Cooks are paid by contributions from parents; so, the num ber of cooks depends on what parents have contributed. The food is described as nutritious with a m ix of rice (three tim es per week), beans and vegetables daily, and m aize two tim es per week. Teachers say cooks follow directions properly and use the official ratio when cooking. However, because nursery students receive food, all students do not receive the correct m eal size. Trainings for cooks, when given, have focused on hygiene and preparation of food for students. 188.According to the School Readiness Scorecard, 63.6 percent of all program m e schools have signed form al contracts with cooks.88 About half of the cooks in Group 1 schools interviewed by the qualitative team have contracts. Other cooks have been told their contracts are com ing; in the m eantim e, if a better paid opportunity com es along for cooks, it is likely that they will leave. All cooks interviewed report being paid on tim e or inform ed of the scheduled paym ent; this is som ewhat lower than the result of the School Readiness Scorecard, which indicates cooks in 88.8 percent of all program m e schools are paid on tim e . 189.Cooks are m ost often m en and wom en are cleaners; the explanation given by cooks is that this is due to the am ount of tim e and labor it takes to cook for schools. Many wom en are m others with dem ands at hom e and leaving the school very late is a safety concern for wom en. While wom en m ust deal with m any dem ands on their energy, tim e and m obility, the underlying reasons for the dom inance of m ale cooks are m ore related to traditional gender roles and access to paid jobs. The Phase II proposal also describes equal representation of m en and wom en in program-supported governance structures, such as teacher and student m em bership on School Feeding Com m ittees (Activity 5.8). However, the gender assessm ent found that m en and wom en do not participate equally in decision-m aking processes and leadership roles at the com m unity level, and that women’s ideas and perspectives are not highly valued by m en and even by boys. The gender assessm ent recom m ends that the program take several actions to address this, including equal representation on com m ittees and in paid roles be m andatory; ensuring that m eaningful capacity is built around core gender concepts and that key leadership figures are engaged in a m eaningful way on gender issues and barriers, and a m ore robust engagem ent on gender issues across a num ber of forum s. 190.Schools buy vegetables from outside m arkets to m eet m eal requirem ents, particularly in the dry season. Most of the Group 1 schools interviewed by the qualitative team are able to contribute vegetables to the school m eal, but this is prim arily in the rainy season as schools rarely have irrigation. Managem ent of the garden is seen by school personnel as an additional responsibility for teachers and students. At larger schools, students are often sent hom e to gather m ulch for the school garden. 191.At the tim e of the baseline, schools are rarely linked with farm er cooperatives; connecting schools to farm ers for the purchase of fresh foods (m ilk, avocado, dodo and their substitutes) will begin in 2022 through com plem entary funding. Schools in the McGovern-Dole Program typically go to the m arket them selves or to individual farm ers to supplem ent produce from school gardens. Schools in the NSFP typically have food supplied by a tender. National school feeding programme (NSFP) readiness 192.In the qualitative survey, the students benefitting from the NSFP are those whose parents contribute to the programme , though nursery students also receive food. Parent contributions are still low though it has reached up to 50 percent in som e districts; shifting parents’ m indsets on the programme is a challenge 88 WFP Rwanda. 2022. School Readiness Scorecard Excel file: HGSF_SchoolLevel_Score card_19JAN2022. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 45 reported by all districts and head teachers. Teachers believe governm ent officials need to help m obilize parents for contribution. 193.The Governm ent received technical assistance from WFP during Phase I of the McGovern-Dole Program that supported the roll-out of a national school m eals programme through technical support and appropriate capacity strengthening. During Phase II, WFP is piloting activities that have the potential to im prove the efficiency of the McGovern-Dole Program and potentially of the NSFP if the Governm ent adopts the m odels. One exam ple, according to WFP, is WFP’s pilot of a whole grain fortified m aize m eal product in 18 schools in 2021. The whole grain m aize m eal is a cost-neutral way to increase the nutritional content of the school m eal, providing m ore protein and m icronutrients for the sam e cost. 194. Where support has occurred, the construction of kitchens in schools and agriculture/school garden related trainings have largely taken place. There is perm anent staff at the district level who coordinate HGSF and NSFP programme s who have supported collaboration efforts. Teachers believe training has been very helpful and a big drive to the success of the programme . Most com m ittee m em bers report they lack training to be effective in their roles. 195.Teachers confirm ed to the qualitative team that m eals are encouraging students to com e to school. They find the tim e spent on providing food though, to be a challenge. More basic infrastructure for programme im plem entation is needed, according to teachers. 196.Most district officials interviewed by the qualitative team reported that District School Feeding Steering Com m ittees have yet to m eet since the NSFP is in its initial im plem entation phases. However, one Group 1 district stated there are SFCs at district, sector, cell, and village level, elected in the past two m onths. They said that they are m eeting whenever needed as the NSFP gets underway and m obilization is still in progress. A Group 2 district official said that the first m eeting was headed by the vice m ayor for social affairs and that SFCs have been established at the sector level, indicating the nascent institutionalization of the NSFP at district level. The Group 2 district said one of the decisions taken at the m eeting was to help poor parents to contribute through non-m onetary m eans, and to ask schools to give m eals to all children regardless of whether parents have contributed. District officials com m ented that having the NSFP introduced after COVID￾19 is a m ajor challenge. 197.Procurem ent of item s for NSFP im plem entation can be challenging for schools though food deliveries have largely been consistent. During the COVID-19 pandem ic, there were interruptions for som e schools in m eals due to school closures, supply chain interruptions, or in obtaining firewood for cooking fuel. Most schools said that they had no interruptions in school feeding. Som e schools reported that they adjusted to interruptions by serving porridge only to the upper prim ary schools since the lower grade received take - hom e rations. 198.MINEDUC’s School Data Managem ent System (SDMS) will require schools to enter school feeding data into a national system . Schools will be responsible for collecting new indicators for the NSFP, and the system will allow schools to access funds allocated to the by Governm ent for the school feeding program m e. In the qualitative survey, both Group 1 and Group 2 districts were not fam iliar with the SDMS system , with one district official noting that no new staff have been allocated for this task. 4.5 Im p a ct Eva lua t ion quest ions releva nt a t ba seline 199.There are no specific evaluation questions associated with this OECD-DAC crite rion at baseline; the section discusses baseline observations. Ba seline observa t ions Improved literacy of school-age children (MGD SO1) 200.Students are reported to be m ore attentive as a result of receiving m eals at schools. School attendance for boys and girls has increased, which m eets district official and school expectations for Phase II. With the increase in attendance, teachers report that the ratio of student to teacher is too high for them to teach as effectively as possible. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 46 201.District officials reported that the closure of school due to COVID-19 pandem ic restrictions increased the num ber of m issed classes and dropouts; one Group 1 district reported an increase in dropouts from nine percent to 14.3 percent. Som e district officials said that it has been a challenge to bring the dropouts back to school. The capacity of schools has been im pacted following COVID-19 re -openings when trained teachers were getting jobs at other schools. Parents’ ability to contribute was reduced as fam ilies lost incom e due to the pandem ic. This increased poverty and the ability of very poor parents to feed their children, who no longer received m eals at school. Parents also could not afford school m aterials so were not sending their children to school. There was an increase in the am ount of household duties requiring children to stay hom e and help their caregivers. Increased use of health and dietary practices (MGD SO2) 202.Officials and teachers expect a decrease in rates of m alnutrition and im proved hygiene for students and com m unities surrounding NSFP schools. Teachers report that students and the surrounding com m unities have im proved their sanitation and hygiene practices. Teachers told the qualitative team that having lunch at school m akes the students feel m ore socially engaged and econom ically on par with each other. Students say that they feel fuller during lunch and are gaining weight. 203.Districts shared that the handwashing stations established by schools during the COVID-19 pandem ic have supported im proved hygiene at schools. The water bill for schools, however, has increased with the increase in handwashing. Water is m anaged by private water providers, and m any schools are finding it a challenge to pay the increased water bills. Som e have turned off water taps, or delay repairing water taps, as a result. Programme partners recognize that this is a challenge and have taken up this issue with the national School Feeding Technical Working Group to look for solutions to reduce water bills. 204.As m entioned, during the COVID-19 pandem ic, fam ilies were affected econom ically such that student’s attendance and access to healthy food were negatively im pacted. 205.The establishm ent of com m unity seed banks and nurseries is a positive unintended consequence arising from extending the work of school gardens to the com m unity during the COVID-19 pandem ic, according to GHI. In addition, the provision of vegetable seeds to the com m unity from the school has contributed to a shift in perception am ong parents that schools are m ore than just a place where a child studies, but a place where parents can get inputs that enable them to diversify the food they grow. 206.The draft gender assessm ent report state s that “all girls interviewed for this assessm ent reported that the girls’ room s in their schools were very helpful to them and were generally well stocked and accessible.” The report further says that girls reported reduced absenteeism related to m enstruation, and that “the im plem entation of girls’ room s in HGSF partner schools is considered to be of significant benefit to girls.” The provision of sanitary pads at schools also reduced stress on girls, as fathers are som etim es reluctant or angry about spending m oney on pads. National school feeding programme (NSFP) readiness 207.The NSFP was launched in October 2021, so it is too early to assess im pact. Teachers in Group1 schools told the qualitative team that parents have m ore tim e to do other activities thanks to the NSFP. Where this tim e would have been devoted to preparing food, tim e is now being devoted to dom estic duties or to farm work, where this is not the sam e for control schools where fewer students are eating school m eals. There has been a positive e conom ic im pact on the surrounding com m unities through em ploym ent of cooks and security guards, and firewood sales. 208.The expectations of district officials for the McGovern-Dole Program around strengthened farm er capacity are largely for WFP to train farm ers in im proved production and storage techniques; professional and leadership training for cooperatives; and ensuring farm ers are linked to reliable m arkets or buyers to whom they can sell. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 47 4.6 Su st a in a bilit y Eva lua tion questions releva nt at ba seline EQ 8: To what extent are McGovern-Dole programme project components (e.g., school feeding, literacy, food safety, WASH and hygiene, nutrition education, agricultural market support, etc.) likely to be sustained at national and local levels, communities, and by other partners? (considering handover to the Government) [TOR, Sustainability, EQ 1, part 1] EQ 9: What is the demonstrated capacity at central and sub-national levels to manage school feeding programmes in Rwanda (WFP and government programmes)? [TOR, Sustainability, EQ 4] EQ 10: To what extent are local communities (SGACs, School Feeding Committees, Procurement Committees, farmers’ groups, etc.) involved in and contributing toward school feeding and education activities? [TOR, Sustainability, EQ 5] 209.These evaluation questions are discussed in the course of the baseline observations m ade below, organized by McGovern-Dole strategic objectives. Ba seline observa t ions Improved literacy of school-age children (MGD SO1) 210.While it is too early in the program m e to assess sustainability, in the qualitative interviews, respondents shared som e concerns about factors that m ay affect the sustainability of program m e activities. For Group 2 schools in particular, schoolteachers and district officials agree there is a lack of basic infrastructure and m aterials for students within the classroom . This is a less reported issue for Group 1 schools who have had infrastructure im provem ents due to the program m e. 211.Teachers shared that the pandem ic backslid progress for students who were pulled away by household duties or a need to seek em ploym ent in greater frequency. Schools placed children in rem edial program m ing and school calendars were altered to include longer hours and or weekends for students to catch up. Reading clubs were stopped due to Covid-19, but to rem edy this, teachers began to lend books to students. Siblings additionally filled in gaps left by a lack of schooling. 212.The Group 1 schools in the McGovern-Dole Program have expanded the school gardens and som e schools are producing vegetables for school m eals, a positive indication of sustainability. GHI key inform ants be lieve that parents close to schools will continue learning from the schools, but recognize that GHI will need to work to support sustainability in schools and households after the program is phased out. Increased use of health and dietary practices (MGD SO2) 213.There was a sm aller reported use of water purification by teachers. Water is turned off with a likely result being a reduced use of sanitary room s by girls. Rainwater harvesting is being used by schools. National school feeding programme (NSFP) readiness 214.The district’s expectations from WFP are not as high as that of parents or teachers. They believe WFP will provide solutions to issues but are also concerned about and strategizing to address challenges ahead in the phase out period. District officials noted several capacity developm ent activities that WFP has provided to the districts to im plem ent the NSFP, including joint field visits between district and WFP staff; technical m eetings to exchange ideas, the training of farm er cooperatives to im prove production in order to supply food to schools, and the construction of m odern kitchens as m odels for NSFP schools. One district m entioned that WFP has shared NSFP operational guidelines. Only one Group 1 district said there was no support in im plem enting the NSFP and was critical of WFP activities in m ost categories. 215.Several Group 1 district officials com m unicated a need for additional capacity strengthening, including technical support to the district to facilitate the im plem entation of the NSFP and the transition from the McGovern-Dole Program in a way that support sustainability. Also m entioned were a need to train teachers to provide support; training for School Tender Com m ittees, cooks, and storekeepers, and increasing productivity of farm ers. District officials for Group 2 schools highlighted sim ilar capacity strengthening needs. 216.While there is opportunity for Group 2 schools to learn from Group 1 schools, there is concern from district officials and schools over the sustainability of the program m e and transition given the need for parent’s 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 48 contribution to be higher. District officials, teachers and SFC/STCs believe parents are used to having WFP pay for or m anage program m es. For som e Group 2 schools there are assum ptions that WFP will provide eve rything and there is no need for parents to contribute anything. 217.The district approach to increasing the ratio of the parent contribution to 60 percent is through sensitization and m obilization to create greater understanding and ownership of the program m e am ong parents. All stakeholders recognize that contributions are difficult for very poor fam ilies, often due to poverty and the inability to pay for several children at one tim e. Teachers and district officials note that poor fam ilies struggle to support students financially. The introduction of a scaled contribution based on social categories has been suggested by teachers and a district official; one district official pointed out that incom e -based contributions are already applied in the m andatory national m edical insurance schem e. 218.Other specific concerns around sustainability voiced by district officials include m ore infrastructure such as kitchens and water, technical support to districts to im plem ent the program m e, capacity building of local partners and beneficiaries, building com m unity ownership, receiving parent contributions in full, and supporting students from very poor fam ilies. The Governm ent provides capitation grants to schools to help pay for basic costs, but the am ount is insufficient to m eet all of the school expenses. Som e district officials expressed concern that school budgets were not sufficient to m aintain the infrastructure that has been built by the program m e. 219.The difficulty of obtaining firewood to cook the m eals in som e areas, and concern over the environm ental im pact of using firewood was raised by a program m e partner as a sustainability issue for Phase II. Program m e partners are discussing innovations to see if the program m e can introduce green initiatives for cooking. There are also prelim inary studies on the prom otion of reusable m enstrual pads in collaboration with a local university. 220.Som e program m e partners suggest that to ensure sustainability of activities under the McGovern-Dole Program , the program m e could advocate for som e key activities to be integrated into Imihigo perform ance contracts. This is seen as helping to ensure sustainability, especially at the district and sector level, for handwashing stations and girls’sanitary room s. 221.Key to the Governm ent’s strategy for the NSFP is the ability for schools to purchase ingredients for school m eals locally, ideally from farm er cooperatives. WFP is working with sm allholder farm ers in cooperatives to im prove production, m anagem ent and m arketing and their readiness to supply the NSFP over the long term . Local procurem ent of fresh foods by schools is seen as a m eans of im proving nutrition and dietary diversity in a sustainable m anner. When asked about their expectations for the McGovern-Dole Program in strengthening local farm er capacity to supply high quality food to local school m eal programme s, district officials in both Group 1 and Group 2 em phasized working with farm er cooperatives to build their capacity in production, leadership and m anagem ent, post-harvest handling and storage, and how to link with buyers. They see the m ain challenges to increasing production as lim ited land, poor and rocky soils, and bad roads that lim it m arket access. Group 1 district officials observed that farm ers are not yet ready to supply schools. Schools have not yet started buying food from cooperatives. 222.Challenges to farm ers’ capacity include production issues due to clim ate change, poor roads, difficulty in accessing m arkets, and sm all am ounts of land. Som e of the farm er cooperatives visited by the qualitative team do not see schools as viable m arkets and are struggling to access m arkets consistently. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 49 5. Co n clu s io n s a n d Le s s o n s 5.1. Co n clu s io n s Releva nce 223.The McGovern-Dole Program is relevant in its reach to intended beneficiaries. Education officials, particularly district officials and head teachers, are optim istic that the programme will contribute to the reduction of student dropout rates, which is a m ajor priority for school districts. Due to irregular attendance by lower prim ary students, m any students were said to perform poorly. Qualitative interviews further indicate that education officials expect the programme to im prove student literacy perform ance and educational quality, im prove m alnutrition, increase student socialization, and change parent m indsets. The programme ’s activities to provide teaching m aterials and teacher training are relevant to the needs of teachers. 224.There have been m arked increases in use of health and dietary practices as a result of the programme . Headteachers, teachers, and cooks from Group 1 and Group 2 schools indicated that m alnutrition and hunger am ong students decreased as a result of the McGovern-Dole Program. Cooks from Group 1 schools, shared that they have seen im proved and earlier attendance, increased handwashing behaviors, greater attentiveness in class, decreased m alnutrition rates, and that children from poor fam ilies are able to get at least one m eal. Additionally, school gardens are present in all Group 1 schools and are contributing vegetables to school m eals. 225.Qualitative interviews also indicate that collaboration with district education and agriculture officials is sufficient. The m ajority of district officials interviewed reported that collaboration with WFP was good; while one district reported that collaboration was good at the national level, com m unication could be im proved at the district level. Overall, district officials in Group 2 said that WFP is collaborating with them to m ake sure they respond to needs of com m unities, and district officials from Group 1 and Group 2 confirm ed that they were involved in the selection of schools. Coherence 226.The McGovern-Dole Program aligns with national objectives, policies, strategies and plans to im prove literacy of school-aged children. Qualitative interviews with district officials acknowledged strong involvem ent by the MINEDUC and MINAGRI in the programme . The governm ent has adopted som e of the activities of the McGovern-Dole Program as resources allow, such as girls’ sanitary room s and handwashing stations due to the favorable results from the programme . In addition, WFP interventions are com patible with other literacy and education initiatives in the country including those im plem ented by World Vision International (WVI), USAID-funded projects, and Building Learning Foundations in Group 1 and Group 2 districts. 227.Programme s have been aligned with national priorities to im prove health and dietary practices, principally during the COVID-19 pandem ic in which schools constructed handwashing stations. Additionally, while deworm ing is the responsibility of the Rwanda Biom edical Centre, initiatives are done in coordination with schools and teachers have been involved in recording the num ber of students and distributing pills to children. Moreover, the McGovern-Dole Program has m ade substantial contributions to the capacity of local and national institutions in the design, m anagem ent, and im plem entation of school fe eding programme s, particularly through its work with the NSF Technical Working Group to operationalize its school feeding policy. 228.Rwanda has transitioned from Kinyarwanda to English as the language of instruction in schools. While it is too early to assess the im pact of that transition on learning, there are recognized challenges to this transition, included a low level of proficiency in English am ong m any teachers, especially in rural areas. Programme partners are working with the Rwanda Education Board to develop tools to train teachers on language proficiency and m ethodologies to support their ability to teach subjects in English. Effect iveness 229.All Group 1 and Group 2 schools were using im proved water sources, specifically piped water, com pared to fewer control schools. Additionally, the introduction of girl’s sanita ry room s has been a significant feature of the McGovern-Dole Program. Overall, a higher proportion of students in Group 1 schools com pared to students in control schools were also said to be able to identify at least three key health and hygiene practices. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 50 Qualitative interviews with teachers and headm asters also confirm ed im proved use of health and dietary practices at programme schools. 230.NSFP readiness for Group 1 schools is greater com pared to Group 2 schools. District officials and teachers report good support from WFP, but m ore involvem ent from the com m unity and parents are needed. Parent contribution rem ains a challenge for m any of the Group 1 and Group 2 schools, especially for cash contributions. Programme im plem entors and stakeholders recognize that the cash contribution is a m ajor challenge to parents, schools, and to the sustainability of the programme . Som e fam ilies do not understand the need to feed children at school, or they m ay live close to the school and prefer to feed children at hom e rather than pay for a school m eal. Other fam ilies m ay not contribute to school m eals due to poverty, and an inability to afford contributions when the fam ily has m any children in school. For the first group of fam ilies, the programme partners and schools are continuing to sensitize parents to the need for children to eat at school, the im portance of education, and the im portance of supporting the programme . For the second group, programme partners are exploring ways to enable poor parents to contribute labor instead of cash. A third possibility is to advocate for the Governm ent to increase its budget for school m eals, but stakeholders point out that the education budget is already the second largest in governm ent, so this m ay not be feasible in the short term . 231.At baseline, students attending schools that previously received school feeding are m ore likely to attend school regularly than those who attend schools without school feeding. All Group 1 schools are providing a school m eal com pared to fewer Group 2 schools and control schools who provide school m eals. All Group 1 schools were reported to be receiving organizational support for school feeding, while fewer Group 2 schools and control schools received the sam e. Particularly, Group 1 schools received m ore support for nutrition activities, sanitation im provem ents (including water and toilets), and teacher training com pared to Group 2 schools and control schools who received sim ilar support. While all Group 1 schools indicated they obtain food from WFP or another NGO, few received governm ent support. Qualitative interviews with teachers indicate that they expect the effectiveness and efficiency of the programme to im prove with governm ent intervention. Schools that have a better understanding of the programme are those where secondary schools have been engaged in providing school m eals previously. 232.Not all cooks have received training since m any were recently hired when schools reopened, replacing those who left during the COVID-19 pandem ic. Those who did receive training told the qualitative team that the quality of training by WFP is good and has allowed them to prepare m eals well. During the survey cooks were not able to answer many questions about good practices, and often handwashing was the m ain, if not only, practice m entioned. However, cooks who were trained were observed to be practicing good hygiene and food preparation m ethods, and kitchens were observed to be clean and organized: in sum , cooks were practicing what they were taught. It was also noted during the endline survey that cooks m ay not have been able to articulate what they had learned but were observed to be practicing good hygiene and food preparation m ethods. Only about half of the cooks interviewed in Group 1 schools had contracts, m aking them m ore likely to leave for other paid opportunities. All cooks who were interviewed by the qualitative team confirm ed that they were paid on tim e. Efficiency 233.While the use of resources am ong Group 1 and Group 2 schools was generally effective, there is room for greater efficiency. Overcrowding of programme schools contributes to inefficiency. Because Group 1 and Group 2 schools are located in the sam e districts as control schools, students often choose to attend the programme schools over control schools in order to receive food. This results in classroom overcrowding, com prom ising the ability for feeding programme s to m eet the need with sufficient quantities of food, particularly for upper prim ary classes. With the increases in attendance, teachers report that the student to teacher ratio is too high for them to teach effectively. While nursery school students are receiving food, they are not officially included in the programme counts, decreasing m eal sizes for older students, particularly children from the poorest fam ilies whose fam ilies cannot afford to contribute . Im pa ct 234.Som e external factors m ay affected the programme ’s im pact, nam ely the econom ic fallout from the COVID￾19 pandem ic. Parents’ ability to contribute to the programme was reduced as fam ilies lost incom e due to the pandem ic. In som e cases, parents could not afford school m aterials, particularly for large fam ilies, thus were no longer sending their children to school. Increased rates of poverty and food insecurity due to the 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 51 pandem ic has m eant that students from the poorest fam ilies are no longer able to receive school m eals, and negatively im pacting their access to healthy food. Som e students who walk hom e to take lunch, often do not return to school. Girls, in particular, face increased care responsibilities at hom e as the am ount of household duties increased due to the pandem ic. To address the inability of poor fam ilies to m ake a cash contribution to school m eals, the introduction of a scaled contribution based on incom e categories has been suggested by teachers and district officials. 235.Officials and teachers report decreased rates of m alnutrition and im proved hygiene for students and com m unities surrounding NSFP schools. Qualitative interviews indicate that teachers feel that students and the surrounding com m unities have im proved their sanitation and hygiene practices, particularly due to the handwashing stations installed at schools during the COVID-19 pandem ic. The gender assessm ent notes that social attitudes and gender norm s rem ain a substantial barrier to the education of girls and m akes recom m endations as to how to incorporate m easures into the program m e to inform the design of a gender￾transform ative social and behavioral change strategy to achieve gender equality in education. Sust a ina bilit y 236.The School Readiness Scorecard reports that all program m e schools have established School Feeding Com m ittees, m ost of which are operational; 99.1 percent have established School Tender Com m ittees. About 65 percent of which are operational. Com m ittee training is lagging, though this is likely a factor of the phased roll-out. 89 An im portant contribution to sustainability and consistency of approach are the national School Feeding Operational Guidelines, developed with technical support from WFP to support the Governm ent’s school feeding strategy. The guidelines were approved and dissem inated by MINEDUC in 2021 and form the basis of trainings for key stakeholders in the NSFP; m ost program m e schools are com plying with those guidelines, according to WFP’s School Readiness Scorecard. 237.Many of the individual NSFP readiness com ponents have the potential to sustain. Capacity developm ent activities, in particular, have been critical to the NSFP program m e including joint field visits between district and WFP staff; technical m eetings for exchanging of ideas; the training of farm er cooperatives to im prove production in order to supply food to schools; and the construction of m odern kitchens as m odels for NSFP schools. However, several Group 1 district officials have com m unicated a need for additional capacity strengthening initiatives such as technical support to the district to facilitate the im plem entation of the NSFP and the transition from the McGovern-Dole Program in a way that supports a sustainable nationally owned school feeding program m e. Also m entioned were a need to train teachers to provide support, training for School Tender Com m ittees, cooks and storekeepers, and increased productivity of farm ers. 238. WFP supplies USDA com m odities (rice, oil) and local and regional com m odities (beans, m aize m eal, salt) to program m e schools. As noted in Section 4.4, beginning in 2022, WFP will use com plem entary funding to connect schools to farm ers for the purchase of fresh foods. Connecting schools and local sm all holder farm ers requires building the capacity of farm ers, schools, and local partners to ensure the sustainability of the programme . The evaluation team recognizes that this is a challenging task but believes that WFP’s work in this regard will contribute substantially to sustainability of the NSFP. 5.2. Le s s o n s Le a r n e d 239.This section discusses program m atic lessons from the baseline study, followed by one lesson on m ethodology. Progra mma tic lessons 240.The country office should continue to support initiatives to address the issue of parent contributions for the poorest fam ilies. The issue of parents’ ability and willingness to contribute to the cost of school m eals has been a recurrent them e in the endline and baseline surveys. It is especially challenging for the poorest fam ilies and fam ilies with m any children. The Governm ent and WFP have recognized this challenge and are investigating ways to address it. At the tim e of this report, WFP key inform ants noted that the World Bank has written a concept note for the Governm ent indicating that it would dedicate part of their future financing for social protection to support school feeding in Rwanda. The proposed support would provide additional 89 WFP Rwanda. 2022. School Readiness Scorecard Excel file: HGSF_SchoolLevel_Score card_19JAN2022. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 52 funds from the social protection budget to support Ubedehe 1 parents in paying the school feeding contribution. This would help overcom e the potential discrim ination against the poorest students in receiving school m eals and avoid com prom ising other developm ent outcom es for children from poor fam ilies. According to a WFP key inform ant, the World Bank proposal is a product of WFP having engaged m ore closely with the World Bank. WFP acknowledges that this type of support m ay require a new governm ent budget line item or policy and so will take tim e to com e to fruition. 241.Ma n y sch ools re port it a ch a lle n ge t o pa y in cre a sed wa t e r bills. As a result of COVID-19 preventative m easures, the Governm ent now requires that all schools have perm anent handwashing facilities. Many schools find it challenging to pay the increased water bills from the newly installed handwashing stations, resulting in som e schools having turned off water taps. This com prom ises the sustainability of im proved hygiene practices and uptake of prom oted behavior changes. While water bills are not a program m e responsibility, program m e partners have taken the issue to the National School Feeding TWG. To address this challenge in the future, WFP and program m e partners should advocate for the Governm ent to consider subsidizing water bills given its facilities requirem ent. 242.Loca l ow n e rsh ip of n u t rit ion e d u ca t ion a n d in t e rve n t ion s is crit ica l. Stronger collaboration and partnership with local leadership are needed to im plem ent nutrition interventions in schools. To ensure that nutrition knowledge and interventions reach not only the schools but also the household level, building relationships with schools is param ount. This includes identifying key stakeholders (including parents) and the role they play in intended outcom es, and providing supervision and m onitoring to increase the ownership of nutrition interventions. GHI observed during Phase I that understanding and ownership of nutrition education and interventions im proved over tim e, and the organization adapted its approach by going beyond the schools to support the establishm ent of com m unity vegetable seed banks accessible to fam ilies. 243.Com p le m e n t a ry fu n d in g is b e in g u se d t o b e n e fit NSFP sch ools. In Rwanda, the USDA-funded McGovern￾Dole Program has also attracted com plem entary donor funding. The USDA and com plem entary funding are being used to develop m aterial for McGovern-Dole Program schools that in turn benefits the NSFP, according to WFP key inform ants. For exam ple, the McGovern-Dole Program is funding the provision of capacity￾strengthening m aterials for the WFP supported school feeding program m es, after which WFP is able to use com plem entary grant m oney to provide the m aterials to a broader range of non-program m e schools. The McGovern-Dole Program enables WFP to develop an approach or a capacity strengthening package , which m ay then be replicated by MINEDUC with its own funds or additional donor resources. 244.Su p p ort in g gove rn m e n t goa ls in t h e NSFP is ke y. The endline found that WFP has strengthened its credibility in school feeding with the Governm ent by ensuring that its work aligns with Governm ent goals, policy and strategy. In Phase II, WFP is continuing to support Governm ent goals in the NSFP by conducting a review of local food procurem ent issues and challenges to docum ent lessons and inform future strategies for the NSFP. WFP has also seconded a staff person to the NCDA to strengthen collaboration between nutrition goals and school feeding program m es. Eva lua t ion m et hodology lessons 245. Th e e va lu a t ion t e a m will sh ift from a sch ool ce n su s e -su rvey su rvey t o a pa n el su rvey in fu t u re rounds. As noted in the lim itations section, the baseline approach included a school census e -survey adm inistered to head teachers or their designates for all schools in the program m e as opposed to the in￾person interview approach taken during the Phase 1 evaluations. This was a cost-saving m easure introduced in consultation with WFP in the proposal review process. Due to the low response rate of the e -survey and indications that this is unlikely to im prove in future rounds, the m idterm and endline evaluations will switch back to an in-person survey conducted either on site or by phone in the panel schools only. The approach will be further developed in the m idterm evaluation inception phase . 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 53 An n e x 1: Ed u ca t io n p r o je ct s Μother actors Ta b le 29: Ed u ca t ion se ct or w ork b y ot h e r in t e rn a t ion a l a ct ors Act or Proje ct De script ion USAID/ Chem onics The USAID-funded Som a Um enye Project (2016-2021) is a national early-grade reading intervention to im prove Kinyarwanda reading skills for prim ary grades 1-3 am ong 1 m illion children in public and governm ent-aided schools. The project focuses on classroom instruction techniques, teacher m anuals, and pre -service teacher training programme s. USAID/ Save the Children USAID’s Mureke Dusom e Project (2015-2020), im plem ented by Save the Children, is a nationwide early grade literacy project that supports partnerships between schools and the broader com m unity to im prove Kinyarwanda literacy am ong prim ary students through com m unity m obilization, and reading clubs. FCDO/ Voluntary Service Overseas The FCDO-funded Building Learning Foundations project is im plem ented by Voluntary Service Overseas and seeks to im prove learning outcom es by enhancing the quality of English and Math teaching in prim ary grades 1-3 in all public and Governm ent-aided prim ary schools. The project focuses on teacher developm ent, leadership and system s strengthening. UNICEF UNICEF has supported m odelling and scaling‐up Child‐Friendly School standards, which were adopted as the national quality guidelines for school infrastructure and software inputs. UNICEF also supported the Learning Achievem ent in Rwandan Schools (LARS) Assessm ent, to im prove the quality of education and m easure learning outcom es in literacy and num eracy. UNDAP The joint United Nations Developm ent Assistance Programme (UNDAP) 2018‐2023 focuses on increased and equitable access to quality education, health, nutrition and WASH services. World Bank The World Bank's Quality of Basic Education for Hum an Capital Developm ent project will support the technical review of tools, fram eworks, and m ethodologies, and pilot a new system to inform policy developm ent and instructional practices. 1 Term s of Reference: Evaluation of USDA McGovern Dole Grant for WFP Hom e -Grown School Feeding Programme in Rwanda from 2020 to 2025 WFP Rwanda Country Office. 2021. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 54 An n e x 2: Ma p o f p rogramme a rea Source : WFP Rwanda. 2021. Subm itted as part of Phase II proposal for McGovern Dole Grant for WFP Hom e -Grown School Feeding Programme in Rwanda (2020 -2025). 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 55 An n e x 3: Re s u lt s Fr a m e w o r k o f McGo ve r n Do le 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 56 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 57 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 58 An n e x 4: Re s u lt s Fr a m e w o r k o f LRP 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 59 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 60 An n e x 5: Cr it e r ia fo r t h e McGove rn -Dole Progra m Focu s Are a Ke y Qu e st ion s – Mid -t erm eva lua t ion Ke y Qu e st ion s – Fin a l Eva lu a t ion Re le va n ce 1. To what extent is the McGovern-Dole Programm e appropriate to the needs of the target beneficiaries on m en, wom en, boys and girls? To what extent has the design of capacity strengthening activities aligned with and/or enhanced governm ent capacity building gaps within the national school feeding programme ? 2. To what extent is the McGovern-Dole Programm e aligned with overall USDA objectives as well as strategies, policies and norm ative guidance; and Governm ent’s relevant stated national policies, including sector policies and strategies? 3. To what extent is the McGovern-Dole Programme aligned with fram eworks of UN agencies and relevant developm ent partners? To what extent is it aligned with WFP's overall strategy and related guidance? 4. To what extent are the changes m ade to activities (design and im plem entation) due to external shocks and other factors (e.g. Covid￾19) relevant for beneficiaries? 1. To what extent is the McGovern-Dole Programm e appropriate to the needs of the target beneficiaries on m en, wom en, boys and girls? To what extent has the design of capacity strengthening activities aligned with and/or enhanced governm ent capacity building gaps within the national school feeding programme ? 2. To what extent is the McGovern-Dole Programme aligned with overall USDA objectives as well as strategies, policies and norm ative guidance; and Governm ent’s relevant stated national policies, including sector policies and strategies? 3. To what extent is the McGovern-Dole Programm e aligned with fram eworks of UN agencies and relevant developm ent partners? To what extent is it aligned with WFP's overall strategy and related guidance? 4. To what extent were the changes m ade to activities (design and im plem entation) due to external shocks and other factors (e.g. Covid-19_ relevant for beneficiaries? Effe ct ive n e ss 1. To what extent at the m id-term point progress has been m ade towards reaching the overall objectives of the McGovern-Dole Programm e (outlined in attachm ent A of the Agreem ent) for various beneficiary groups (for m en, wom en, boys and girls) and by type of activity? 2. What were the m ajor factors influencing the achievem ent or non￾achievem ent of the objectives and outcom es of the McGovern-Dole Programm e by the tim e of the m id-term evaluation? What, if any, unexpected outcom es resulted from programme im plem entation? 3. To what extent has the M&E system been adequately designed to respond to the needs and requirem ents of the project? Has the M&E system been sufficiently able to capture changes in the lives of the beneficiaries? 4. To what extent have the inform ation supplied by the m onitoring and Beneficiary/Stakeholder Com plaint and Feedback m echanism s been 1. To what extent were the objectives and results of the McGovern-Dole Programm e achieved for various beneficiary groups (by gender where applicable) and by type of activity? 2. To what extent have the intended results and overarching programme objectives been achieved? What were the particular features of the McGovern-Dole Programm e and context that m ade a difference for m en, wom en, boys and girls? What was the influence of other factors? What unexpected outcom es resulted from programme im plem entation? 3. To what extent have the findings of the baseline evaluation been im plem ented to contribute to the achievem ent of the expected outcom es? 4. To what extent has the M&E system been adequately designed to respond to the needs and requirem ents of the project? Has the M&E system been sufficiently able to capture changes in the lives of the beneficiaries? 5. To what extent have the m onitoring and Beneficiary/Stakeholder Com plaint and Feedback m echanism s been utilized for McGovern-Dole 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 61 utilized for the McGovern-Dole Programm e corrective m easures as well as for WFP’s learning agenda? What specific lessons have been identified through these m echanism s? 5. To what extent did external shocks and other factors (e.g. COVID-19) affect project im plem entation and perform ance? Programm e corrective m easures as well as for WFP’s learning agenda? What specific lessons have been identified through these m echanism s? 6. To what extent did external shocks and other factors including factors related to COVID-19 affect project im plem entation and perform ance and how were these m itigated? Efficie n cy 1. Were the activities im plem ented in line with the McGovern-Dole Programm e im plem entation plan and in a tim ely m anner (programme delivery, logistics and M&E arrangem ents)? What factors im pacted the delivery process (cost factors, WFP and partners perform ance, external factors)? 2. Were the activities undertaken as part of McGovern-Dole Programme cost-efficient? 3. What factors im pacted the efficiency and cost efficiency of the programme im plem entation? What m easures can support enhancem ent of the McGovern-Dole Programm e efficiency for the rem aining im plem entation period? 1. To what extent are the transfer cost, cost per beneficiary, logistics, programme deliveries and M&E arrangem ent aligned with project design? What factors im pacted the delivery process and the programme ’s achievem ents (cost factors, WFP and partners perform ance, external factors)? 2. Were the activities undertaken as part of McGovern-Dole Programm e cost￾efficient? 3. What factors im pacted the cost efficiency of the project im plem entation? Im p a ct 1. What are the m edium -term effects on beneficiaries’ lives, m en, wom en, boy and girl - through com parison of targeted and non-targeted schools against the programme objectives? 2. What are the gender-specific m edium-term im pacts? Did the intervention influence the gender equality wom en’s empowerment (GEWE) context? If yes, how? 3. What are the internal factors contributing to the achievem ent or non￾achievem ent of the expected outcom es (factors within WFP’s control): the processes, system s and tools in place to support the operation design, im plem entation, m onitoring and evaluation and reporting; the governance structure and institutional arrangem ents (including issues related to staffing, capacity and technical backstopping from RB/HQ); and internal partne rship and coordination approaches and arrangem ents; etc.? 4. What are the m edium -term effects on sm allholder farm ers’ lives through the support received under the McGovern-Dole Programme? 1. What intended and unintended im pact has the McGovern-Dole Programme m ade on m en, wom en, boy and girl beneficiaries (through com parison of targeted and non-targeted schools against the programme objectives) and stakeholders (including Governm ent, authorities, com m unities)? 2. What were the internal factors contributing to the achievem ent or non￾achievem ent of the expected outcom es (factors within WFP’s control): the processes, system s and tools in place to support the operation design, im plem entation, m onitoring and evaluation and reporting; the governance structure and institutional arrangem ents (including issues related to staffing, capacity and technical backstopping from RB/HQ); and internal partnership and coordination approaches and arrangem ents; etc.? 3. What were the external factors leading to the im pact (factors outside WFP’s control): the external operating environm ent; the funding clim ate; external incentives and pressures; etc.? 4. What are the overall effects on sm allholder farm ers’ lives through the support received under the McGovern-Dole Program? Su st a in a b ilit y 1. To what extent were the McGovern-Dole Program im plem entation arrangem ents include considerations for sustainability (handover to the governm ent) at national and local levels, com m unities and other 1. To what extent was the McGovern-Dole Program im plem entation in line with the transition plan/strategy agreed with and endorsed by the Governm ent (including handover to the governm ent at national and local 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 62 partners for all project com ponents (school feeding, literacy, food safety, WASH and hygiene, nutrition education, agricultural m arket support, etc) agreed with and endorsed by the Governm ent and national stakeholders? To what extent progress has been m ade against the overall transition process against the project plan and handover plan/strategy agreed with and endorsed by the Governm ent? 2. To what extent progress has been m ade towards institutionalization of the m easures planned as part of the technical assistance to the Governm ent that is expected to support the sustainability of the intervention (including policy work, to system s, institutional capacity etc.)? What progress has been m ade since the project design stage (through strategic engagem ent, advocacy and other efforts with Governm ent and relevant stakeholders) in supporting the transition of school feeding im plem entation from the McGovern-Dole Programme beyond WFP’s intervention to the national school feeding programme , to the extent it can be evaluated by the m id-term evaluation (national budget for the national school feeding programme and other funding sources)? 4. What is the dem onstrated capacity at central and sub-national levels to m anage school feeding programme s in Rwanda (WFP and governm ent programme s)? 5. To what extent are local com m unities (SGACs, School Feeding Com m ittees, Procurem ent Com m ittees, farm ers’ groups, etc.) involved in and contributing toward school feeding and education activities? 6. Based on available evidence, to what extent are the benefits of the programme likely to continue beyond WFP’s intervention for the targeted beneficiaries (m en, wom en, boys and girls)? levels, com m unities and other partners for all project com ponents (school feeding, literacy, food safety, WASH and hygiene, agricultural m arket support, etc.)? Have adjustm ents to the transition plan/strategy identified during m id￾term evaluation and throughout the programme been factored in the McGovern-Dole Program im plem entation and im pacted success of the handover process? Has the overall transition process been conducted as per the McGovern-Dole Program m e plan and transition plan/strategy agreed with and endorsed by the Governm ent? 2. To what extent has the package of technical assistance activities and m easures undertaken during the project duration been institutionalized into the Governm ent’s policies, strategies and system s and is likely to support the sustainability of the intervention (including policy work, support to system s, institutional capacity etc.)? What progress has been m ade since the project design stage (through strategic engagem ent, advocacy and other efforts with Governm ent and relevant stakeholders) in supporting the transition of school feeding im plem entation from the McGovern-Dole Program beyond WFP’s intervention national school feeding programme , to the (national budget for the national school feeding programme and other funding sources)? 4. How effective has the transition process been? (criteria for effective transition to be defined by the project team at the start of the programme ) 5. What is the dem onstrated capacity at central and sub-national levels to m anage school feeding programme in Rwanda (WFP and governm ent programme s)? 6. To what extent are local com m unities (SGACs, School Feeding Com m ittees, Procurem ent Com m ittees, farm ers’ groups, etc.) able to m anage and coordinate school feeding and education activities (WFP and governm ent school-feeding related activities)? 7. Based on available evidence to what extent are the benefits of the programme likely to continue beyond WFP’s intervention for the targeted beneficiaries (m en, wom en, boys, and girls)? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 63 An n e x 6: Ba s e lin e e va lu a t io n m a t r ix 246.As noted in Section 3.1, the baseline evaluation m atrix is organized by the OECD-DAC criteria. It includes the following elem ents: --General baseline observations for the project outcom e dom ains to inform overall WFP adaptive m anagem ent, as appropriate ; --Specific baseline evaluation questions adapted from the m idterm and evaluation questions presented in the TOR, since the TOR did not include baseline evaluation questions; and --PMP indicators for which the external evaluation process is the prim ary source. These indicators can be found under the effectiveness section of the m atrix. 247.The m ain report headings and subheadings align with the OECD-DAC criteria and evaluation questions under each. This structure will also be used to further develop the evaluation m atrix in the m idterm and endline evaluation phases using the evaluation questions presented in the TOR as a starting point. Key for Data validity, quality, and reliability colum n: Strength of evidence rating: weak/lim ited fair strong 90 Where relevant, all indicator values calculated by TANGO will be based on the m ost current USDA indicator definitions and guidance. Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion Re le va n ce a) Ba se lin e Ob se rva t ion s Im proved literacy of school￾age children (MGD SO1) Observations on status of:  School capacity  School infrastructure  Docum ent review  School census  Programme docum ents and needs assessments strong Com parison of prim ary qualitative findings, and quantitative 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 64 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion  Com m unity and household support  National and local governm ent support  Linkages between schools, com m unities and governm ent agencies  Qualitative data collection  Sem i-structured observation  Quantitative data collection  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff findings and secondary data Contextual analysis only to the extent that findings allow Increased use of health and dietary practices (MGD SO2) Observations on status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  Capacity of agriculture cooperatives  Linkages between schools and cooperatives  Docum ent review  School census  Qualitative data collection  Sem i-structured observation  Quantitative data collection  Programme docum ents and needs assessments  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong National school feeding program m e (NSFP) readiness Observations on status of:  National and local governm ent capacity  Linkages between NSFP stakeholders, particular focus on bridging linkages between central and decentralized levels  Docum ent review  Qualitative data collection  Programme docum ents and needs assessments  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong b) Sp e cific b a se lin e e va lu a t ion q u e st ion s (a d a p t e d from TOR m id t e rm a n d e n d lin e q u e st ion s, a n d in clu d e d a s re le va n t ) 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 65 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion EQ 1: To what extent is the McGovern-Dole program m e appropriate to the needs of the target beneficiaries of m en, wom en, boys and girls? [TOR, Relevance, eval Q1 part 1]  Degree to which design was based on needs assessment and analysis.  Level of differentiation in needs.  Evidence that design was aligned with the understanding of differentiated needs at the tim e.  Docum ent review  Qualitative data collection  Programme docum ents and needs assessments  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong NB Gender assessm ent in draft form at tim e of baseline Triangulation of docum ent review against perspectives of interviewees to the extent possible at baseline. EQ 2: To what extent are the activities (design and im plem entation) appropriate in the context of COVID-19? [TOR, Relevance, eval Q4]  COVID-19 preparedness and response m easures taken by the program  Degree to which these m easures align with national and global leading practice  Docum ent review  Qualitative data collection  Programme docum ents  Program m e staff strong Triangulation of docum ent review against perspectives of interviewees to the extent possible at baseline. c) PMP In d ica t ors No indicators under TANGO responsibility - - - - - Coh e re n ce a) Ba se lin e Ob se rva t ion s Im proved literacy of school￾age children (MGD SO1) Observations on status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  School census  School surveys  Qualitative data collection  Sem i-structured observation  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong Com parison of prim ary qualitative findings, and quantitative findings and secondary data 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 66 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion  Linkages between schools, com m unities and governm ent agencies Contextual analysis only to the extent that Increased use of health and findings allow dietary practices (MGD SO2) Observations on status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  Capacity of agriculture cooperatives  Linkages between schools and cooperatives  School census  School surveys  Qualitative data collection  Sem i-structured observation  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong National school feeding program m e (NSFP) readiness Observations on status of:  National and local governm ent capacity  Linkages between NSFP stakeholders, particular focus on bridging linkages between central and decentralized levels  Qualitative data collection  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff  Readiness scorecard strong b) Sp e cific b a se lin e e va lu a t ion q u e st ion s (a d a p t e d from TOR m id t e rm a n d e n d lin e q u e st ion s, a n d in clu d e d a s re le va n t ) EQ 3: To what extent are the activities aligned with governm ent capacity building gaps within the national school feeding programme?  Degree to which design was based on assessment and analysis of the NSFP, specifically gap analysis  Evidence that design was aligned with the  Docum ent review  Qualitative data collection  Programme docum ents  Gap analysis (if available)  Governm ent officials strong Com parison of prim ary qualitative findings, and quantitative findings and secondary data 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 67 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion [TOR, Relevance eval Q1 part 2] understanding of governm ent capacity-building gaps  Program m e staff Triangulation of docum ent review against perspectives of interviewees to the extent possible at baseline. EQ 4: To what extent is the McGovern-Dole program m e aligned with overall USDA objectives as well as strategies, policies, and norm ative guidance, and with Governm ent’s relevant stated national policies, including sector policies and strategies? [TOR, Relevance eval Q2]  Degree to which design was based on assessment and analysis of the NSFP, and analysis of current USDA guidance.  Evidence that design was aligned with the understanding of governm ent policies and m ost current USDA guidance.  Docum ent review  Qualitative data collection  Programme docum ents  Governm ent officials  Program m e staff strong Com parison of prim ary qualitative findings, and quantitative findings and secondary data Triangulation of docum ent review against perspectives of interviewees to the extent possible at baseline. EQ 5: To what extent is the McGovern-Dole program m e aligned with United Nations fram eworks and relevant developm ent partners? To what extent is it aligned with WFP’s overall strategy and related guidance? [TOR Relevance eval Q3, part 1]  Degree to which design was based on assessment and analysis of UN and developm ent partner fram eworks, strategies and program s, and analysis of coherence and com plem entarity.  Evidence that design was aligned with UN and developm ent partner  Docum ent review  Qualitative data collection  Programme docum ents  Program m e staff  Partner representatives (to the extent available) strong Com parison of prim ary qualitative findings, and quantitative findings and secondary data Triangulation of docum ent review against perspectives of interviewees to the 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 68 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion initiatives, policies, and strategies. extent possible at baseline. EQ 6: To what extent is it aligned with WFP’s overall strategy and related guidance? [TOR, Relevance eval Q3, part 2]  Degree to which design was based on assessment and analysis of current WFP guidance?  Evidence that design was aligned with WFP guidance, policies and strategies.  Docum ent review  Qualitative data collection  Programme docum ents  Program m e staff strong Com parison of prim ary qualitative findings, and quantitative findings and secondary data Triangulation of docum ent review against perspectives of interviewees to the extent possible at baseline. c) PMP In d ica t ors No indicators under TANGO responsibility - - - - - Effe ct ive n e ss a) Ba se lin e Ob se rva t ion s Not applicable at baseline - - - - - b) Sp e cific b a se lin e e va lu a t ion q u e st ion s (a d a p t e d from TOR m id t e rm a n d e n d lin e q u e st ion s, a n d in clu d e d a s re le va n t ) EQ 7: To what extent is the M&E system designed to adequately respond to the needs and requirem ents of the project? Is the M&E system designed to sufficiently capture changes in the lives of the beneficiaries?  Incorporation of lessons from the MGD Phase I endline  Alignm ent with WFP corporate M&E guidance  Evidence of M&E tailoring to the context and specific needs of the Phase II program m e  Docum ent review  Qualitative data collection  Programme docum ents  Program m e staff strong Com parison of prim ary qualitative findings, and quantitative findings and secondary data Triangulation of docum ent review 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 69 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion [TOR, Effectiveness eval Q3]  Feasibility check on WFP capacity to im plem ent the M&E plan against perspectives of interviewees to the extent possible at baseline. c) PMP In d ica t ors Im proved literacy of school￾age children (MGD SO1) Percent of students who, by the end of two grades of prim ary schooling, dem onstrate that they can read and understand the m eaning of grade level text  Docum ent review  EGRA student assessment  KIIs and FGDs with school staff and parents  Programme docum ents; Assessm ent reports; ACRs; APPs; perform ance indicator data  Students G3  School staff  Parents  Program m e staff strong NB EGRA adm inistered to 3rd graders; results will be for 3rd graders only Analyzed as per EGRA m ethodology, Survey data triangulated with qualitative data for trend and contribution analysis Average student attendance rate in USDA supported classroom s/schools  Docum ent review  School census  School survey  KIIs and FGDs with school staff and parents  Programme docum ents; ACRs; APPs; perform ance indicator data collected by WFP  School records: attendance registers collected by head teachers and school directors, WFP Monitoring tools  Head teachers fair Som e attendance data m ay be estim ated by head teachers School data triangulated with qualitative data for trend and contribution analysis 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 70 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion  Teachers  Parents  Program m e staff Percentage of children with whom a caregiver or older sibling was engaged in two or m ore direct actions to prom ote learning in the past week  Docum ent review  EGRA student assessment  KIIs and FGDs with school staff and parents  Programme docum ents; GHI project reports; ACRs; APPs; perform ance indicator data collected by WFP  Students G3  School staff  Parents  Program m e staff strong Survey data triangulated with qualitative data for trend and contribution analysis Observations on status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  Linkages between schools, com m unities and governm ent agencies  Docum ent review  School census  School surveys  Qualitative data collection  Sem i-structured observation  Programme docum ents; ACRs; APPs; perform ance indicator data collected by WFP  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong Additional contextual analysis only to the extent that findings allow Increased use of health and dietary practices (MGD SO2) Num ber of m eals provided that include fruits, vegetables, legum es and/or anim al source proteins in  Docum ent review  School census  School surveys  Programme docum ents; WFP project reports; ACRs; APPs; strong Survey data triangulated with qualitative data for trend and 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 71 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion addition to the donated US com m odity  EGRA student assessment  Qualitative data collection  Sem i-structured observation perform ance indicator data collected by WFP  Head teachers  Teachers  Parents  Students  Program m e staff contribution analysis Num ber of schools that are using nutrition and food safety guides developed for cooks and food store managers  Docum ent review  School census  School surveys  Qualitative data collection  Sem i-structured observation  Programme docum ents; WFP project reports; ACRs; APPs; perform ance indicator data collected by WFP  Head teachers  Cooks  Store m anagers  Program m e staff strong Survey data triangulated with qualitative data for trend and contribution analysis Num ber of individuals who dem onstrate use of new safe food preparation and storage practices as a result of USDA assistance  Docum ent review  School census  School surveys  Qualitative data collection  Sem i-structured observation  Programme docum ents; ACRs; APPs; perform ance indicator data collected by WFP  Head teachers  Cooks  Store m anagers  Program m e staff strong Survey data triangulated with qualitative data for trend and contribution analysis 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 72 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion National school feeding program m e (NSFP) readiness Num ber of policies, regulations, or adm inistrative procedures in each of the following stages of developm ent as a result of USDA assistance  Docum ent review  Review of policies, regulations and procedures  Qualitative data collection  WFP m onitoring data (readiness scorecard)  Governm ent records (MINEDUC) and WFP and GHI project records  Programme docum ents; ACRs; APPs; perform ance indicator data  Governm ent staff  Program m e staff strong Trend and contribution analysis; triangulated across listed sources Num ber of National School Feeding Steering Com m ittee m eetings supported  Docum ent review  Qualitative data collection  Sem i-structured observation  WFP m onitoring data  Meeting agendas and m inutes  Programme docum ents; WFP project reports; ACRs; APPs; perform ance indicator data collected by WFP  Governm ent staff  Program m e staff strong Trend and contribution analysis; triangulated across listed sources Num ber of District School Feeding Steering Com m ittee m eetings supported  Docum ent review  Qualitative data collection  WFP m onitoring data strong Trend and contribution analysis; 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 73 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion  Sem i-structured observation  Meeting agendas and m inutes  Governm ent staff  Program m e staff triangulated across listed sources Num ber of National School Feeding Technical Working Groups m eetings supported  Docum ent review  Qualitative data collection  Sem i-structured observation  WFP project reports, MINEDUC reports  Meeting agendas and m inutes  WFP m onitoring data  Governm ent staff  Program m e staff strong Trend and contribution analysis; triangulated across listed sources Com m unity awareness of the NSFP and strategy  School census  School surveys  Qualitative data collection  School staff  Parents fair More likely to be subject to respondent bias Trend and contribution analysis; triangulated across listed sources Household/parent contribution to school  School census  School surveys  Qualitative data collection  Sem i-structured observation  School staff  Parents strong Trend and contribution analysis; triangulated across listed sources Perform ance of agricultural cooperatives providing com m odities for the NSFP  Docum ent/m onit oring data review  Qualitative data collection  Sem i-structured observation  WFP m onitoring data  Cooperative staff  Cooperative members strong WFP m onitoring data triangulated with qualitative data for trend and contribution analysis, with 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 74 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion  Head teachers  Program m e staff specific focus on readiness to supply schools Efficie n cy a) Ba se lin e Ob se rva t ion s Im proved literacy of school￾age children (MGD SO1) Observations on current status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  Linkages between schools, com m unities and governm ent agencies  School census  Qualitative data collection  Sem i-structured observation  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong Com parison of prim ary qualitative findings, and quantitative findings and secondary data Contextual analysis only to the extent that findings allow Increased use of health and dietary practices (MGD SO2) Observations on status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  Capacity of agriculture cooperatives  Linkages between schools and cooperatives  School census  Qualitative data collection  Sem i-structured observation  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong National school feeding program m e (NSFP) readiness Observations on status of:  National and local governm ent capacity  Qualitative data collection  School staff  Parents and com m unity duty bearers strong 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 75 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion  Linkages between NSFP stakeholders, particular focus on bridging linkages between central and decentralized levels  Governm ent officials  Program m e staff b) c) Sp e cific b a se lin e e va lu a t ion q u e st ion s (a d a p t e d from TOR m id t e rm a n d e n d lin e q u e st ion s, a n d in clu d e d a s releva n t ) Not applicable at baseline - - - - - d) e ) PMP In d ica t ors No indicators under TANGO responsibility - - - - - Im p a ct a) Ba se lin e Ob se rva t ion s Im proved literacy of school￾age children (MGD SO1) Observations on status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  Linkages between schools, com m unities and governm ent agencies  School census  Qualitative data collection  Sem i-structured observation  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong Contextual analysis only to the extent that findings allow Increased use of health and dietary practices (MGD SO2) Observations on status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  School census  Qualitative data collection  Sem i-structured observation  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 76 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion  Capacity of agriculture cooperatives  Linkages between schools and cooperatives National school feeding program m e (NSFP) readiness Observations on status of:  National and local governm ent capacity  Linkages between NSFP stakeholders, particular focus on bridging linkages between central and decentralized levels  Qualitative data collection  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff strong b) Sp e cific b a se lin e e va lu a t ion q u e st ion s (a d a p t e d from TOR m id t e rm a n d e n d lin e q u e st ion s, a n d in clu d e d a s re le va n t ) Not applicable at baseline - - - - - c) PMP In d ica t ors No indicators under TANGO responsibility - - - - - Su st a in a b ilit y a) Ba se lin e Ob se rva t ion s Im proved literacy of school￾age children (MGD SO1) Observations on status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  Linkages between schools, com m unities and governm ent agencies  Docum ent review  Qualitative data collection  Sem i-structured observation  Assessm ents and evaluations; Readiness of Schools to Transition Scorecard  School staff  Parents and com m unity duty bearers fair Longer-term m onitoring data would be needed to ascertain sustainability over the long term. Contextual analysis only to the extent that findings allow 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 77 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion  Evidence of continued program m e im plem entation / m aintained infrastructure where WFP has pulled back (if available)  Governm ent officials  Program m e staff Increased use of health and dietary practices (MGD SO2) Observations on status of:  School capacity  School infrastructure  Com m unity and household support  National and local governm ent support  Capacity of agriculture cooperatives  Linkages between schools and cooperatives  Evidence of continued program m e im plem entation / m aintained infrastructure where WFP has pulled back (if available)  Docum ent review  Qualitative data collection  Sem i-structured observation  Assessm ents and evaluations; Readiness of Schools to Transition Scorecard  School staff  Parents and com m unity duty bearers  Governm ent officials  Program m e staff fair Longer-term m onitoring data would be needed to ascertain sustainability over the long term. National school feeding program m e (NSFP) readiness Observations on status of:  National and local governm ent capacity.  Linkages between NSFP stakeholders, particular focus on bridging linkages between central and decentralized levels.  Evidence of continued program m e im plem entation /  Docum ent review  Qualitative data collection  Assessm ents and evaluations; Readiness of Schools to Transition Scorecard  School staff  Parents and com m unity duty bearers fair Longer-term m onitoring data would be needed to ascertain sustainability over the long term. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 78 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion m aintained infrastructure where WFP has pulled back (if available).  Governm ent officials  Program m e staff b) Sp e cific b a se lin e e va lu a t ion q u e st ion s (a d a p t e d from TOR m id t e rm a n d e n d lin e q u e st ion s, a n d in clu d e d a s re le va n t ) EQ 8: To what extent are McGovern-Dole program m e project com ponents (e.g., school feeding, literacy, food safety, WASH and hygiene, nutrition education, agricultural m arket support, etc) likely to be sustained at national and local levels, com m unities, and by other partners? (considering handover to the Governm ent) [TOR, Sustainability, eval Q 1, part 1]  Perceptions on sustainability of outputs and outcom es.  Levels of preparation/anticipation for sustainability and transition strategies.  (Expected) readiness of Governm ent.  Docum ent review  Qualitative data collection  Programme docum ents  Program m e staff  Governm ent representatives fair Longer-term m onitoring data would be needed to ascertain sustainability over the long term. EQ 9: What is the dem onstrated capacity at central and sub-national levels to m anage school feeding program m es in Rwanda (WFP and governm ent program m es)? [TOR, Sustainability, eval Q 4]  Dem onstrated capacity at Governm ent levels  Evidence of WFP assessment of governm ent capacity  Level of understanding of evolving Governm ent capacity over the course of the project  Docum ent review  Qualitative data collection  Programme docum ents  Program m e staff  Governm ent representatives fair Longer-term m onitoring data would be needed to ascertain sustainability over the long term. EQ 10: To what extent are local com m unities (SGACs, School Feeding Com m ittees, Procurem ent Com m ittees,  Level of com m unity engagement  Level of com m unity organization  Docum ent review  Qualitative data collection  Programme docum ents  Program m e staff fair Longer-term m onitoring data 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 79 Are a s of In q u iry / Eva lu a t ion Qu e st ion s In d ica t ors u n d e r TANGO re spon sibilit y for da t a colle ct ion 90 Da t a colle ct ion methods Sou rce s of da t a /in form a t ion Da t a va lidit y, qu a lit y, a n d re lia bilit y Da t a a n a lysis methods/ t ria n gu la t ion farm ers’ groups, etc.) involved in and contributing toward school feeding and education activities? [TOR, Sustainability, eval Q 5]  WFP understanding of the risks and opportunities of engaging com m unity stakeholders  Beneficiaries and com m unity stakeholders would be needed to ascertain sustainability over the long term. c) PMP In d ica t ors No indicators under TANGO responsibility - - - - - 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 80 An n e x 7: Do cu m e n t s Cit e d Go ve r n m e n t o f Rw a n d a Gender Monitoring Office. 2019. The State of Gender Equality in Rwanda. Governm ent of Rwanda/UNICEF. 2017. A Study of Knowledge, Attitudes and Practices around Gender and Education in Rwanda. MINEDUC. 2019. Com m uniqué: MINEDUC endorses the use of English language as a m edium of instruction in lower prim ary. National Institute of Statistics of Rwanda. 2021. Size of the resident population. National Institute of Statistics of Rwanda. 2021. Dem ographic and Health Survey 2019-20. National Institute of Statistics of Rwanda. 2020. Rwanda Statistical Yearbook 2020. Republic of Rwanda, Ministry of Education. 2019. 2019 Education Statistics. Republic of Rwanda, Ministry of Education. 2018. Education Sector Strategic Plan, 2018/19-2023/24. Republic of Rwanda, Ministry of Gender and Fam ily Prom otion. 2021. Revised National Gender Policy – February 2021. Rwandan Ministry of Disaster Managem ent and Refugee Affairs (MIDIMAR). 2015. The National Risk Atlas of Rwanda United Republic of Rwanda. 2019. 2019 Rwanda Voluntary National Review Report. WFP WFP Rwanda. 2021. Gender Assessm ent Hom e Grown School Feeding Program m e. Decem ber 2021. Draft. WFP Rwanda. 2021. Concept Note: Gender Analysis of the Hom e Grown School Feeding Program m e. October 2021. WFP Rwanda. 2021. Assessing the Usage of Menstrual Hygiene Managem ent Services and Factors Influencing their Usage am ong Adolescent Schoolgirls in Rwanda. Report. Policy Brief. Novem ber. WFP Rwanda. 2021. WFP's USDA McGovern-Dole International Food for Education and Child Nutrition Programme 's Support in Rwanda 2016-2021. Endline evaluation. 25 August (draft). WFP Rwanda. 2021. Phase II proposal for WFP's USDA McGovern-Dole International Food for Education and Child Nutrition Programme 's Support in Rwanda. Draft. WFP Rwanda. 2021. WFP Rwanda Country Brief, WFP Rwanda. 2021. WFP Rwanda Country Strategic Plan (draft). WFP Rwanda. 2018. Com prehensive food security and vulnerability analysis - Rwanda. Decem ber. Ot h e r Un it e d Na t io n s Age n cie s UNEG. 2018. UN-SWAP Evaluation Perform ance indicator Technical Note. April 2018. United Nations Developm ent Programme . Hum an Developm ent Report 2020. United Nations Division for Sustainable Developm ent Goals. 2021. Rwanda -Voluntary National Review 2019. UNCHR. 2021. Operational Data Portal. World Health Organization. 2021. Rwanda COVID-19 update. Ot h e r Sou rce s The END Fund. 2018. Partnering to End Neglected Tropical Diseases in Rwanda. Global Child Nutrition Forum . 2020. National School Feeding Program in Rwanda: Virtual Conference Nov 9, 2020. The New Tim es. 2021. Govt injects Rwf27bn towards school feeding. July 26. OECD. Top Ten Donors of Gross ODA for Rwanda, 2018-2019 average, USD m illion USDA Foreign Agricultural Service – Food Assistance Division. 2019. USDA Food Assistance Indicators and Definitions. February. World Bank. 2021. Poverty headcount ratio at $1.90 a day (2011 PPP) (% of population). World Bank. 2021. Net official developm ent assistance and official aid received (constant 2018 US$) - Rwanda, Tanzania 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 81 World Bank database. 2021. World Bank. 2021. Net ODA received (% of GNI)- Rwanda. World Bank. 2021. Multidim ensional poverty headcount ratio (% of total population) – Rwanda. World Bank database. 2021. Persistence of last grade of prim ary, total (% of cohort) World Bank database . 2020. Agriculture, forestry, and fishing, value added (% of GDP). World Bank database . 2019. Adolescent fertility rate (births per 1,000 wom en ages 15-19) - Rwanda World Bank database . 2019. Prevalence of undernourishm ent (% of population) - Rwanda World Econom ic Forum . 2021. Global Gender Gap Report 2021. Insight Report. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 82 An n e x 8: Me t h odology 248.This annex describes the m ain qualitative m ethods the evaluation team used to explore the evaluation questions and to triangulate findings with secondary inform ation and quantitative data. 249.The data collection m ethods described were adm inistered across three strata, referred to by group number or by their full designation. The strata are: • Grou p 1: WFP McGove rn Dole Ph a se I sch ools (108 sch ools from fou r d ist rict s) • Grou p 2: Sch ools a d d e d t o t h e origin a l 108 (28 sch ools from t h re e d ist rict s) • Grou p 3: Con t rol sch ools (10 p u rp ose fu lly se le ct e d sch ools t o p rovid e a cou n t e rfa ct u a l) 250.See sam pling section below for details regarding how the sam ple was drawn, including control schools. Docum ent review/ Review of seconda ry da t a 251.The evaluation team reviewed a range of WFP and external docum ents to provide context and additional data and inform ation relevant to the baseline study, and to help explain and triangulate findings from prim ary qualitative and quantitative data. Annex 7 lists the docum ents cited in this report. Tthe report of the form ative gender assessm ent com m issioned by WFP Rwanda is in revision stages at this writing, so references to this are to the draft version of the report. Census survey of hea d t ea chers 252.A census survey was adm inistered at baseline to all three groups. It collected statistical data available in school records such as the num ber of teachers, students, attendance, enrolm ent, dropouts, etc. 253.The census survey was com pleted prior to any in-person data collection. The census was program m ed using Open Data Kit (ODK) survey software and adm inistered via e -m ail and/or phone with head teachers com pleting on their own tim e. It included key McGovern-Dole Program indicators in a structured survey questionnaire and took approxim ately 15-30 m inutes to com plete. The survey tool was based on tools used in the Phase I program m e; it was reviewed and com pared to the list of indicators in the m ost recent Term s of Reference. 254.The census provided the evaluation team with inform ation on key dom ains (e.g., enrolm ent, teacher num bers, teacher to student ratios, a ttendance of girls and boys, etc.). The census was also the first step in establishing inform ation and perspectives on program m e achievem ents and strengths, and program m e gaps or areas for im provem ent prior to further prim ary data collection. 255.The acceptable overall non-response rate for the census survey was set at 30 percent based on WFP Rwanda experience with phone surveys and TANGO’s global experience with rem ote surveys. In addition, the evaluation team ensured a 100 percent response rate for the panel schools, i.e., the program m e schools in the sam ple. As noted in the lim itations section of the m ain report, there was a low response rate for the census surveyTo ensure tim ely collection of survey data for the schools m ost relevant to the baseline and future evaluation exercises – the program m e schools in the sam ple and the control schools (i.e., together, “panel” schools) - the local team , by special exception, adm inistered the surveys for the panel schools by phone , and was thus successful in com pleting the questions in the census e -survey for all panel schools. Ea rly Gra de Rea ding Assessm ent 256.Students were assessed using the Early Grade Reading Assessm ent (EGRA) tool. Reading skills were assessed by adm inistering EGRA to students in the 31 program m e schools and 10 counterfactual schools.91 Based on a standardized m ethod for m easuring changes in reading outcom es, analysis of the EGRA data showed changes over tim e in literacy. Qualitative data inform ed relationships between the literacy outcom es and other trends. As in Phase I EGRA surveys, the evaluation team added questions to the EGRA to capture data on a few additional indicators, such as hygiene practices. 91 The EGRA endline data from Phase I will not be used because they would not be com parable to Phase II, which uses a different panel (sam ple) of schools. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 83 257.The country office and evaluation team agreed during the inception phase that the EGRA will be adm inistered to third-grade (P3) students), ensuring students have had at least two years of instruction. In order to assess any changes in reading levels and in health and nutrition practices, TANGO believed that children in P3 would be representative of the other grades across the program m e tim efram e, as these students had had several years of program m e benefits at point of m easurem ent (baseline, m idterm , and endline). Selection of this grade also provided continuity across Phase I and Phase II results and was consistent with other EGRA approaches. It was neither advisable nor necessary to sam ple students in each grade; sam pling all grades would have been cost-prohibitive and against EGRA leading practice. School surveys 258.The Phase I program m e, the school survey (which was referred to as the ‘headm aster survey’ in Phase I) included data collected by the evaluators during school visits. For Phase II, these data were covered under the census survey, which was conducted rem otely for greater efficiency. The census survey was the first data collection activity undertaken in each evaluation phase (baseline, m idterm and endline) and focused on school perform ance statistics. The tim e needed to com plete the survey rem otely was estim ated to be under 30 minutes. 259.For all selected sam ple schools, TANGO developed a structured survey to assess perform ance against school￾level program m e indicators not already captured by the census or EGRA surveys. This survey was conducted through in-person interviews on Android tablets using the ODK data collection software. Survey respondents included the head teachers, grade teachers, and cooks. 260.School surveys were conducted in program m e and control schools. Key inform a nt int erviews (KIIs) a nd focus group discussions (FGDs) 261.KIIs and FGDs were guided by interview guides that were the sam e across the three evaluation exercises. These were based on the Phase I tools, considering learning from the final evaluation, and m odified to ensure responsiveness to the Phase II evaluation questions and the interests of stakeholders. Tools were specifically tailored to consider new contextual inform ation and m odified if needed based on rem ote interviews with staff early in the data collection process. Per standard practice, all KIIs and FGDs were conducted by a team of two people, with one leading the interview or focus group and the other taking notes. Efforts were m ade to assign team m em bers KIIs and FGDs in accordance with gender, language, and cultural considerations. Phone interviews we re often conducted by only one person to sim plify the scheduling and for technical ease and to m axim ize num ber of interviews. Where technically feasible, audio recordings were m ade for the evaluation team ’s reference. All KIIs and FGDs, as well as the e -survey, followed inform ed consent protocols. 262.The sam ple for the qualitative work were conducted at the schools selected for the EGRA assessm ents, including both program m e and control schools. The evaluation team conducted qualitative deep dives at 10 schools total across the three groups of schools: Group 1 program m e schools (4 x), Group 2 program m e schools (4x), and control schools (2x). This purposive sam ple was drawn in consultation with WFP to ensure basic representation of key school characteristics across the total. The sam ple will rem ain the sam e for all evaluations. Every EGRA survey was accom panied by a head teacher key inform ant interview to provide a qualitative data point for all 31 program m e and 10 sam pled schools, in addition to the deep dives in the eight program m e and two control schools. 263.It was estim ated that the baseline would have 15-25 national and subnational stakeholder interviews, and each of the deep dive school sites would have between 3-5 qualitative activities (key inform ant interviews, in￾depth interviews and focus group discussions) across school staff and students, and parents. Interviewee types at any given site were selected using purposive sam pling, with the goal of covering the full range of categories across the sam ple as a whole. 264.KIIs were held with individuals from the following stakeholder categories, as m uch as possible with equal representation of m ales and fem ales from each responding group: • WFP Rwanda Country Office staff; WFP Regional Bureau Nairobi staff • Sub-grantees/ partners: World Vision, GHI • Governm ent officials at national and district authorities) levels: MINEDUC, MINAGRI, NCDA, Ministry of Gender and Fam ily Prom otion (MIGEPROF), RBC, Rwanda Education Board (REB), Rwanda Cooperative Agency (RCA) 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 84 • Donors: USDA, MasterCard and/or the WFP HQ Private Partnerships team • Other stakeholders: USAID, UNICEF, World Bank • District Education Coordinators and Officers • District School Feeding Com m ittees • School Feeding Com m ittees • School Tender Com m ittees • Head teachers • Mayors 265.FGDs were held with the following stakeholders at school and com m unity level, as m uch as possible with equal representation of m ales and fem ales from each responding group. FGDs were single -sex, and conducted by interviewers/facilitators of the sam e sex to the extent logistically feasible: • Teachers • School General Assem bly Com m ittees (SGACs) (som e FGDs were with parents only) • School Managem ent Com m ittees • School Feeding Com m ittees • School Tender Com m ittees • Cooks and cleaners • Storekeepers • Com m unity leaders • Mem bers of sm allholder agricultural cooperatives working with WFP 266.Final num bers on persons interviewed are sum m arized in Annex 14. Observa tion 267.The evaluation team also gathered inform ation via direct observation of program m e activities at service delivery points (i.e., schools) and in group activities (e.g., SGACs). Direct observation with structured checklists were em ployed where possible without disruption to norm al activities of classroom s, cooking, distribution of m eals, etc. In addition, photo evidence was used for kitchens, stores, water points and toilets to enable calibration of observations across schools as to the condition of these facilities. Da t a collect ion from fa rm er coopera t ives 268.The evaluation team visited a purposive sam ple of one cooperative per district for qualitative data collection to validate operational and perform ance data collected as part of WFP’s routine m onitoring, to explore factors that affect cooperative perform ance, and assess readiness to supply schools. Sa m plin g Progra mme schools 269.The schools where TANGO undertook in-depth qualitative data collection were nested within the quantitative sam ple to carry the random ized selection of schools across the com plete m ixed-method approach and to enable effective triangulation across data points. 270.To identify which program m e schools would participate in the EGRA assessm ent, TANGO proposed a sim ple random selection of schools within Group 1 and Group 2; Table 30: shows the sam ple size selection of schools by strata. Given the sm all populations (<108 schools per group) of these groups, a 20 percent sam ple from each group ensured sufficient representation. However, TANGO proposed the num ber of schools selected in Group 2 be adjusted upward from 5.6 to 10. Schools from each group were selected using sim ple random sam pling, ensuring each school had an equal probability of being selected relative to other schools in the stratum . Strata statistical weights were applied when indicator values were aggregated beyond the strata level (for exam ple if Group 1 and Group 2 results were aggregated for one ‘total’ program m e estim ate, this estim ate was estim ated using statistical weights). 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 85 Ta b le 30: Sa m p le size se le ct ion Tot a l s ch ools in p opu la t ion 20 percent of a ll sch ools Tot a l select ed sch ools for sa m ple Min im u m EGRA sa m ple size (st u d e n ts) Grou p 1 108 21.4 21 462 Grou p 2 28 5.6 10 220 TOTAL (MGD) Unknown Unknown 31 682 Con t rol Unknown Unknown 10 220 271.TANGO applied the form ula below to derive a m inim um sam ple size of P3 students to assure statistical accuracy in com parisons across strata (groups 1, 2, and 3,) as well as across survey rounds (baseline, m idterm , endline). where: Va ria b le Assu m e d va lu e De script ion n = Deff = 2 Design effect for com plex sam ple design (assum ed to be = 2) Zα = 1.282 Z value associated with desired significance level for confidence (90%, one -tailed) Zβ = 0.842 Z value associated with desired significance level for power (80%, one - tailed) P1 = 50.0% estim ated level of an indicator m easured as a proportion at the tim e of the first survey or within a com parison group P2 = 65.0% expected level of the indicator either at a later survey round or different com parison group. (P2 - P1) is the m agnitude of change or difference across subgroups that the sam ple is powered to detect (in this case, a difference of 30%, or 15 percentage points). NR 10.0% Non-response rate 272. The above form ula com putes a m inim um required sam ple size of 210 students to enable statistically accurate com parisons for a single group (stratum ). TANGO rounded up the sam ple size to 220 for logistical ease – whereby 11 m ale and 11 fem ale grade students were interviewed in 31 programme schools and 10 counterfactual/control schools (refer to Table 30: above). A sam ple size of 220 per stratum is sufficiently large to enable com parisons among strata/groups (i.e ., Group 1, Group 2, and the control group); future EGRA survey rounds will thus use the sam e target sam ple size for each group (see Table 31) and be adm inistered in the sam e 31 program m e schools and 10 control schools as the baseline . Note that programme im plem entation in Group 1 schools will phase out two or three years after this baseline, so in effect, the m idterm evaluation for the programme will serve as an endline for Group 1 schools, and the endline evaluation will serve as an ex-post evaluation. As such, the hypothesis and evaluation/research questions for the latter, in the phased-out schools, will differ from those in the schools that continued the programme . 273.Selection of students at the school level at baseline was done by random ly selecting grade students in the chosen schools; the sam e m ethod will be used for the m idterm and endline surveys. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 86 Ta b le 31: EGRA t a rge t sa m p le size s: ba se lin e , m id t e rm a n d e n d lin e (# st u d e n t s) Ta rge t sa m p le size Ba se lin e Midt erm En d lin e Grou p 1 462 220 220 Grou p 2 220 220 220 TOTAL (MGD) 662 440 440 Con t rol 220 220 220 Cont rol schools 274.The 10 control schools were selected as follows. The evaluation team asked each head teacher (from the selected programme schools) to identify/refer the three closest schools of the sam e level and type (e.g., governm ent or private), and provide the following inform ation for each “referred” school to the extent of their knowledge: • Wh a t is t h e sch ool’s loca t ion ? Dist a n ce from t h is program m e sch ool (i.e ., t h e re sp on d e n t ’s sch ool)? • Is t h e sch ool u rb a n , p e ri-u rba n , or ru ra l? • Is t h is a gove rn m e n t or p riva t e sch ool? • Wh a t is t h e e st im a t e d sch ool size (i.e ., st u d e n t p op u la t ion )? 275.The head teacher was also asked to provide the nam e and contact inform ation of the head teacher at each referred school nam ed, if known. 276.The evaluation team cross-checked the “'referred” schools with the total list of schools receiving WFP support (from the sam ple fram e they provided). Any “referred” schools receiving WFP support was rem oved from the list. This resulted in a list of 1-3 “referred” (i.e., potential control) schools per programme school. Of these, the evaluation team used random sam pling to select one potential control school per programme school. The final stage of selecting the 10 control schools was purposive, based on the following traits, in rank order: • Proxim it y t o a n ot her sch ool in t he sa m ple • Align m e n t w it h u rb a n / p e ri-u rb a n / ru ra l ch a ra ct e rist ic (i.e ., st rive for t h e sa m e ch a ra ct e rist ic in t he program m e a n d con t rol sch ool, e.g., t o a void pa iring a n u rba n sch ool wit h a ru ra l on e ) • Align m e n t of ca t e gory: gove rn m e n t vs p riva t e • Sim ila r sch ool size 277.Additional characteristics that were considered included num ber of teachers, enrolm ent and attendance rates, as they were available . 278.As part of control school selection, the proposed control group schools were asked for perm ission to conduct future data collection activities over the course of the programme . This inform ed consent-seeking process was co-designed and im plem ented with the country office. Following each data collection activity in control schools, these schools received a sum m ary brief developed by WFP outlining all EGRA results for McGovern￾Dole Program schools and non-McGovern-Dole Program schools, and included their school’s EGRA results (aggregated across students, not per student) for use in school decision-m aking. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 87 An n e x 9: McGove rn Dole Pe rform a nce indica t ors Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Percent of students who, by the end of two grades of prim ary schooling, dem onstrate that they can read and understand the m eaning of grade level text 4 Support Im proved Literacy Assessm ent report Early Grade Reading Assessm ent Tool and Literacy Boost Assessm ent Tool Tot a l Fem ale Male Baseline, Midterm , Final TANGO Average student attendance rate in USDA supported classroom s/schools 1.1 Provide Nutritious School Meals School records: attendance registers collected by head teachers and school directors, WFP Monitoring tools WFP analysis of school attendance records Tot a l Fem ale Male Biannual Teachers and head teachers; WFP Field Monitors TANGO Num ber of teaching and learning m aterials provided as a result of USDA assistance 4.2 Support Lower Grade Teachers WV project reports Monitoring form s n/a Biannual WV Num ber of teachers/educators/teaching assistants in target schools who dem onstrate use of new and quality teaching techniques or tools as a result of USDA assistance 4..2 Support Lower Grade Teachers Survey: interview Direct observations with standard form s Tot a l Fem ale (60%) Male (40%) Biannual WV Num ber of teachers/educators/teaching assistants trained or certified as a result of USDA assistance 4.2 Support Lower Grade Teachers WV project records, training records Training attendance form Tot a l Fem ale Male Biannual WV 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 88 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of school adm inistrators and officials in target schools who dem onstrate use of new techniques or tools as a result of USDA assistance 4.3 Support Teachers' professional developm ent Survey: interview Direct observations with standard form s Tot a l Fem ale Male Biannual Head teachers supervised by WV Num ber of school adm inistrators and officials trained or certified as a result of USDA assistance 4.3 Support Teachers' professional developm ent WV project records, training records Training attendance form Tot a l Fem ale Male Biannual WV Num ber of educational facilities (i.e. im proved water sources, latrines, etc.) rehabilitated/constructed as a result of USDA assistance 2,1 Construction of disability-inclusive VIP latrines and girls’ sanitary room s 2,2 Construction of water system s 2,6 Construction and establishm ent of hand washing stations WV project records WV analysis of project records Tot a l Biannual WV Classroom s Kitchens/Cook Areas Im proved Water Sources Latrines Permanent hand washing stations Tem porary hand washing stations Other school grounds or school buildings Num ber of students enrolled in school receiving USDA assistance 1.1 Provide Nutritious School Meals Governm ent records: MINEDUC student enrolm ent Annual reports Tot a l Annual WFP Pre -Prim ary Fem ale Pre -Prim ary Male 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 89 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? records, District Student Enrolm ent records, and School records Prim ary Fem ale Prim ary Male Num ber of policies, regulations, or adm inistrative procedures in each of the following stages of developm ent as a result of USDA assistance 3.11 Operationalize the national strategy on school gardens and increase sustainability of garden resources 5.4 Strengthening National Fram eworks and Institutions Governm ent records (MINEDUC) and WFP and GHI project records Review and analysis of sector policies and WFP/GHI records. Tot a l Baseline, Midterm , Endline evaluations TANGO Education (Stage 1-5 noted) Child Health & Nutrition (Stage 1-5 noted) Num ber of School General Assem bly Com m ittees or sim ilar school governance structures supported as a result of USDA assistance 2.4 -increased pupil and parents’ awareness on good hygiene practices 3.6- support school management com m ittees to becom e nutrition cham pions in their com m unities 3.10- increase parent and student engagem ent in garden activities 4.4- sensitize com m unity m em bers on the im portance of education School records, Project records Analysis of project reports and programme records n/a Bi-annual report WFP, WV 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 90 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of daily school m eals (breakfast, snack, lunch) provided to school￾age children as a result of USDA assistance 1,1 Provide Nutritious School Meals School reports and Cooperating Partners (CP) reports WFP analysis of reports n/a Bi-annual report, m onthly report by CP, daily collection by school WFP, Head Teachers Num ber of school-age children receiving daily school m eals (breakfast, snack, lunch) as a result of USDA assistance 1,1 Provide Nutritious School Meals School reports and CP reports WFP analysis of reports Tot a l Bi-annual report, m onthly report by CP, daily collection by school WFP, Head Teachers New, Fem ale Continuing, Fem ale New, Male Continuing, Fem ale Num ber of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance 1.1 Provide Nutritious School Meals School reports and CP reports WFP analysis of reports Tot a l Annual WFP, Head Teachers Com m unity Assets Household Assets Hum an Assets/Capital, Fem ale, New Hum an Assets/Capital, Fem ale, Continuing Hum an Assets/Capital, Male, New Hum an Assets/Capital, Male, Continuing 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 91 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of individuals who dem onstrate use of new child health and nutrition practices as a result of USDA assistance 3 Prom ote Nutrition and Dietary Practices WV and GHI project reports WV and GHI analysis of project records Tot a l Fem ale (55%) Male (45%) Annual WV, GHI Num ber of individuals who dem onstrate use of new safe food preparation and storage practices as a result of USDA assistance 5.7.1 Build Capacity of cooks and storekeepers WFP reports WFP analysis of reports Tot a l Fem ale (55%) Male (45%) Annual; Baseline, Midterm , Endline WFP; TANGO Num ber of individuals trained in safe food preparation and storage as a result of USDA assistance 5.7.1 Build Capacity of cooks and storekeepers WFP reports WFP analysis of reports Tot a l Fem ale Male Biannual WFP Num ber of individuals trained in child health and nutrition as a result of USDA assistance. 3 Prom ote Nutrition and Dietary Practices WV and GHI project reports WV and GHI analysis of project records Tot a l Fem ale (55%) Male (45%) Biannual WV, GHI Num ber of children under five (0-59 m onths) reached with nutrition-specific interventions through USG￾supported programme s 3,5 Child Growth Monitoring for children under 5 for pre -prim ary students GHI project reports GHI analysis of project records Tot a l Fem ale Male Annual GHI Num ber of schools using an im proved water source 2,2 Construction of water system s WV Project reports WV analysis of project records n/a Biannual WV 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 92 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of schools with im proved sanitation facilities 2,1 Construction of disability-inclusive VIP latrines and girls’ sanitary room s 2,7 Construction and establishm ent of hand washing stations WV Project reports WV analysis of project records n/a Biannual WV Num ber of students receiving deworm ing m edication(s) 2,10 Distribution of Deworm ing Medication and Prevention Education RBC reports, WV records WFP review and analysis of project records n/a Biannual WFP Num ber of individuals participating in USDA food security programme s 1 Provide Nutritious School Meals 2 Prom ote Im proved Health WFP reports WFP review and analysis of project records People in governm ent, Male Annual WFP People in governm ent, Fem ale Proprietors of USDA-assisted private sector firm s, Male Proprietors of USDA-assisted private sector firm s, Fem ale People in civil society, Male People in civil society, Fem ale Laborers, Male 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 93 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Laborers, Fem ale Producers, Sm allholder farm ers, Male Producers, Sm allholder farm ers, Fem ale Num ber of individuals benefiting indirectly from USDA-funded interventions 1. Provide Nutritious School Meals WFP reports WFP review and analysis of project records n/a Annual WFP Num ber of schools reached as a result of USDA assistance 1. Provide Nutritious School Meals WFP reports WFP review and analysis of project records n/a Biannual WFP Num ber of individuals participating in USDA food security programme s that include an LRP com ponent 6. Build Capacity of Farm er Groups to Supply Food to Schools WFP reports WFP review and analysis of project records n/a Biannual WFP Num ber of individuals benefitting indirectly as a result of USDA assistance 6. Build Capacity of Farm er Groups to Supply Food to Schools WFP reports WFP review and analysis of project records n/a Annual WFP Cost of com m odity procured as a result of USDA assistance (by com m odity and source country) 6. Build Capacity of Farm er Groups to Supply Food to Schools WFP reports WFP review and analysis of project records Tot a l Biannual Baseline, m idline, endline WFP 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 94 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Quantity of com m odity procured as a result of USDA assistance (by com m odity and source country) 6. Build Capacity of Farm er Groups to Supply Food to Schools WFP reports WFP review and analysis of project records Tot a l MML Beans Biannual Baseline, m idline, endline WFP Value of annual sales of farm s and firm s receiving USDA assistance 6. Build Capacity of Farm er Groups to Supply Food to Schools WFP reports WFP review and analysis of project records Tot a l Maize Beans Annual; Baseline, m idline, endline WFP; Volum e of com m odities sold by farm s and firm s receiving USDA assistance 6. Build Capacity of Farm er Groups to Supply Food to Schools WFP reports WFP review and analysis of project records Tot a l Annual Baseline, m idline, endline WFP Maize Beans Num ber of individuals in the agriculture system who have applied im proved m anagem ent practices or technologies with the USDA assistance 6. Build Capacity of Farm er Groups to Supply Food to Schools WFP reports WFP review and analysis of project records Total Annual WFP Fem ale Male Num ber of schools reached with LRP activities as a result of USDA assistance 6. Build Capacity of Farm er Groups to Supply Food to Schools WFP reports WFP review and analysis of project records Biannual WFP Not e: Out come t a rget s will be reviewed a ft er the ba seline survey 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 95 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of m eals provided that include fruits, vegetables, legum es and/or anim al source proteins in addition to the donated US com m odity 1. Provide Nutritious School Meals WFP project reports WFP review and analysis of project records n/a Biannual Baseline, m idline, endline WFP TANGO Num ber of school-aged children who receive 5 or m ore m eals per week that include fruits, vegetables, and/or anim al source proteins in addition to US com m odities 1. Provide Nutritious School Meals WFP project reports WFP review and analysis of project records Tot a l Fem ale Male Biannual WFP Num ber of school gardens established and m aintained 3.9 Establish and m aintain school gardens GHI project reports GHI analysis of project records n/a Biannual GHI Num ber of students benefiting from the establishm ent and m aintenance of school gardens 3.9 Establish and m aintain school gardens GHI project reports GHI analysis of project records Tot a l Fem ale Male Biannual GHI Num ber of growth m onitoring and prom otion interventions conducted at pre -schools as a result of GHI advocacy 3.5 Child Growth Monitoring for children under 5 for pre -prim ary students GHI project reports GHI analysis of project records n/a Biannual GHI Num ber of children under five (0-59 m onths) reached with growth m onitoring and prom otion interventions 3.5 Child Growth Monitoring for children under 5 for pre -prim ary students GHI project reports GHI analysis of project records Tot a l Fem ale Male Biannual GHI 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 96 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of schools which received seeds package 3.9 Establish and m aintain school gardens GHI project reports GHI analysis of project records n/a Biannual GHI Num ber of nurseries established at schools 3.9 Establish and m aintain school gardens GHI project reports GHI analysis of project records n/a Biannual GHI Percentage of children with whom a caregiver or older sibling was engaged in two or m ore direct actions to prom ote learning in the past week 4 Support Im proved Literacy GHI project reports GHI analysis of project records n/a Biannual Baseline, m idline, endline WV TANGO Num ber of students participating in reading com petitions facilitated as a result of USDA assistance 4.5 Organize Reading Com petitions WV project reports WV analalysis of project report Tot a l Fem ale Male Biannual WV Num ber of WASH com m ittees established at schools 2.9 Establishm ent of WASH com m ittees/reinforce Water Users Com m ittees WV project reports WV analysis of project report n/a Biannual WV Num ber of fem ale students trained on good m enstrual hygiene practices 2.4 Teaching girls on good m enstrual hygiene WV project reports WV analysis of project report n/a Biannual WV Num ber of Inform ation Education and Com m unication (IEC) hygiene m aterials distributed 2.5 Developm ent and distribution of IEC hygiene m aterials WV project reports WV analysis of project report n/a Biannual WV Num ber of students reached with health and hygiene m essages as a result of USDA assistance 2.3 Increase pupils’ and parents’ awareness on good hygiene practices WV project reports WV analysis of project report n/a Biannual WV 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 97 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of parents, teachers and students trained in soap m aking 2.7 Training teachers, parents and students in soap m aking WV project reports WV analysis of project report n/a Biannual WV Num ber of fuel-efficient stoves provided and rehabilitated 5.7.2 Enhance Kitchen and Stove Infrastructure WFP project reports WFP analysis of project reports n/a Biannual WFP Num ber of individuals directly benefiting from the provision and rehabilitation of fuel-efficient stoves 5.7.2 Enhance Kitchen and Stove Infrastructure WFP project reports WFP analysis of project reports n/a Biannual WFP Num ber of parents trained as part of School Feeding Com m ittees 6.1 Establish district school feeding com m ittees and provide capacity building at the district level WFP project reports WFP analysis of project reports n/a Biannual WFP Num ber of parents trained as part of School Tender Com m ittees 6.1 Establish district school feeding com m ittees and provide capacity building at the district level WFP project reports WFP analysis of project reports n/a Biannual WFP Num ber of students benefiting from newly constructed/rehabilitated latrines 2.1 Construction of disability-inclusive VIP latrines and girls’ sanitary room s WV project reports WV analysis of project reports n/a Biannual WV Num ber of students benefiting from newly constructed or enhanced water system s 2.6 Construction and establishm ent of hand washing stations WV project reports WV analysis of project reports n/a Biannual WV 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 98 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of students benefiting from kitchens, cook areas and storeroom s built or rehabilitated 5.7.2 Enhance Kitchen and Stove Infrastructure WFP project reports WFP analysis of project reports n/a Biannual WFP Num ber of Governm ent staff trained at national level 5.1 Provide capacity building and technical trainings at the national level WFP project reports WFP analysis of project reports n/a Biannual WFP Num ber of Governm ent staff trained at district level 6.1 Establish district school feeding com m ittees and provide capacity building at the district level WFP project reports WFP analysis of project reports n/a Biannual WFP Num ber of Governm ent staff trained at sector/cell level 6.2 Provide capacity building at the sector and cell levels and establish sector school feeding com m ittees WFP project reports WFP analysis of project reports n/a Biannual WFP Num ber of National School Feeding Steering Com m ittee m eetings supported 5.3 Mobilize National School Feeding Steering Com m ittee and Technical Working Group WFP project reports WFP analysis of project reports n/a Biannual Baseline, m idline, endline WFP TANGO Num ber of District School Feeding Steering Com m ittee m eetings supported 6.1 Establish district school feeding com m ittees and provide capacity building at the district level WFP project reports, district reports WFP analysis of reports n/a Biannual Baseline, m idline, endline WFP TANGO 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 99 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of National School Feeding Technical Working Groups m eetings supported 5.3 Mobilize National School Feeding Steering Com m ittee and Technical Working Group WFP project reports, MINEDUC reports WFP analysis of reports n/a Biannual Baseline, m idline, endline WFP TANGO Num ber of students who participated in school internal class com petitions on nutrition 4.5 Organize Reading Com petitions WV project reports WV analysis of project reports Total Biannual WV Num ber of com m unity level seed week events organized 3.10 Increase parent and student engagem ent in garden activities GHI project reports GHI analysis of project reports n/a Biannual GHI Num ber of schools with operational plan for school gardens 3.11 Operationalize the national strategy on school gardens and increase sustainability of garden resources GHI project reports GHI analysis of project reports n/a Biannual GHI Num ber of nutrition￾focused Parents’ Day Im plem ented at schools 3.1 Nutrition focused Parents’ Day Im plem ented at all schools GHI project reports GHI analysis of project reports n/a Biannual GHI Num ber of schools that are using nutrition and food safety guides developed for cooks and food store managers 5.7.1 Build Capacity of Cooks and Storekeepers WFP project reports WFP analysis of project reports n/a Biannual Baseline, m idline, endline WFP TANGO 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 100 Pe rform a n ce In d ica t or Re su lt / Act ivit y Da t a Sou rce Method/ Ap p roa ch of Da t a Colle ct ion or Ca lcu la t ion Disa ggre ga t ion Da t a Colle ct ion Wh e n ? Da t a Colle ct ion : Wh o? Num ber of m aternal and child nutrition com m unity events in which GHI shared nutrition and agriculture m essaging 3.7 Develop and distribute nutrition education m aterials to schools and neighbouring com m unities GHI project reports GHI analysis of project reports n/a Biannual GHI Num ber of cooking dem onstration sessions conducted during m aternal and child nutrition events 3.4 Integrate nutrition and agriculture awareness activities into existing m aternal and child nutrition cam paigns GHI project reports GHI analysis of project reports n/a Biannual GHI Num ber of nutrition￾focused clubs established in schools 3.2 Teachers continuously engaged in nutrition education GHI project reports GHI analysis of project reports n/a Biannual GHI Num ber of nutrition￾focused educational m aterials distributed 3.7 Develop and distribute nutrition education m aterials to schools and neighbouring com m unities GHI project reports GHI analysis of project reports n/a Biannual GHI Num ber of technical working groups and district coordination m eetings in which GHI shared lessons learned from the project and Maternal and Child Nutrition integration 3.3 Local authorities’ officials trained on agriculture and nutrition and coordination workshops conducted GHI project reports GHI analysis of project reports n/a Biannual GHI 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 101 An n e x 10: Da t a colle ct ion t ools 279.The qualitative data collection tools build on the tools from the Phase I m idterm and final evaluations to provide continuity from Phase I (FY 2016-2020) to Phase II (FY 2021-2025) for the key areas that the McGovern￾Dole Programm e supports. These draft tools have been m odified to reflect changes from Phase I to Phase II and will be reviewed and finalized with input from WFP Rwanda prior to data collection. Please note that these topical outlines/ interview guides are intended as a m enu of possible topics and not all questions in each section will be asked of each respondent. Questions are posed to individual respondents according to their knowledge of and involvem ent with the programme , the topics m ost relevant to their position, the tim e available for the interview, and other factors. WFP Count ry Office Illustrative list of interviewees: McGovern-Dole Programme Team, Strategic Outcome 2 Manager, Head of Programme, M&E Team, Health/ Nutrition Officer, Head Smallholder Agricultural Market Support unit, Admin/Finance Officer, Logistics Team, Gender Focal Point, Deputy Country Director, Country Director, SABER consultant 1. How is MINEDUC collaborating with WFP on Phase II of the McGovern-Dole Programm e? On the plans for a national school feeding programme (NSFP)? 2. How was MINEDUC involved in the design of HGSF activities? Describe their input; was it relevant and realistic? 3. How do Phase II HGSF activities align with the national school feeding policy? Has the WFP HGSF work fed into the developm ent of national, regional and district level structures to support school feeding? 4. How have gender issues been taken into account in the programme ? Is this approach appropriate and effective? 5. What are the m ain barriers to enrolm ent, retention and com pletion faced by girls and boys with disabilities? Do girls face different, or greater barriers than boys? 6. What influence has WFP had on Governm ent’s national school feeding approach with regard to disadvantaged students (girls, children from poor fam ilies, students with disabilities)? 7. What is your assessm ent of the perform ance of the WFP Rwanda country office and sub-national offices? 8. What are the com m unication and inform ation-sharing processes between WFP and MINEDUC? Between MINEDUC and District Adm inistrators and District Education Officers? How effective are these processes for collaboration, coordination and decision-m aking? 9. Extent to which WFP has incorporated national and local capacity strengthening opportunities and constraints into the Phase II design (probe: design phase, im plem entation, capacity developm ent/handover plan)? 10. What analysis has WFP done regarding the cost-efficiency and cost-effectiveness of the HGSF programme ? What are your im pressions of cost-efficiency and cost-effectiveness? 11. What capacity developm ent support is WFP providing to the NSFP in Phase II? How was the type of support WFP offers decided on (e.g., what wa s MINEDUC input)? Exam ples of changes expected as a result of that support? 12. Strengths of the McGovern-Dole Programm e to date? What has worked well and will be incorporated into Phase II? a. Capacity strengthening and technical support to national strategies, policies, and im plem entation b. Advocacy at national level c. Coordination and com m unication d. Training strategies for head teachers in school m anagem ent and teachers in current teaching m ethods e. Im proved student literacy, attentiveness, attendance, retention f. School infrastructure (kitchens, water, WASH) g. Increased use of health and dietary practices h. School gardens, nutrition, outreach to com m unities i. Com m unity participation in education j. Support to agricultural cooperatives and sm all farm ers to supply school m eals k. Parent contribution to cost of m eals 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 102 l. Gender equity, access to education for girls, children from very poor fam ilies, children with disabilities m . Other topics 13. Constraints/ challenges of the McGovern-Dole Programm e to date? How have these been addressed for Phase II? a. Capacity strengthening and technical support to national strategies, policies and im plem entation b. Advocacy at national level c. Coordination and com m unication d. Training strategies for head teachers in school m anagem ent and teachers in current teaching m ethods e. Im proved student literacy, attentiveness, attendance, retention f. School infrastructure (kitchens, water, WASH) g. Increased use of health and dietary practices h. School gardens, nutrition, outreach to com m unities i. Com m unity participation in education j. Support to agricultural cooperatives and sm all farm ers to supply school m eals k. Parent contribution to cost of m eals l. Gender equity, access to education for girls, children from very poor fam ilies, children with disabilities m . Other topics 14. Are the resources and expertise WFP has m obilized, adequate to im plem ent Phase II of the HGSF? 15. Do you see any further opportunities for cooperation with Governm ental and non-Governm ental partners (e.g., in health, education, gender equity)? 16. To what extent is WFP supporting partnerships with the private sector to address sustainability? 17. What are MINEDUC’s needs and concerns around the roll-out of the NSFP and its sustainability? 18. What aspects of the McGovern-Dole Programm e is MINEDUC adopting, or inte rested in adopting, for the NSFP? 19. What are WFP’s m ain contributions and priorities in support of the NSFP during Phase II? Min is t r y o f Ed u ca t io n – Na t io n a l Le ve l Illustrative list of interviewees: PS MINEDUC, HGSF Project Specialist MINEDUC Topical outline may also be used for donor interviews 1. How has the McGovern-Dole Programm e contributed to Governm ent’s strategy and im plem entation of the NSFP at the national level? a. At the district level? b. Exam ples of WFP input that influenced NSFP policies, strategies and im plem entation? c. What aspects of the programme have not been adopted for the NSFP, and why? 2. What capacity developm ent support is WFP providing to the NSFP? a. Which functions/offices require capacity strengthening to im plem ent the NSFP? b. What types of support? c. Relevance and quality of support? 3. Describe the current capacity strengthening needs of the NSFP: a. At the national level (e.g., on planning, budgeting, implementation, technical skills, management skills) b. At district and sector level (e.g., on coordination, supervision) 4. How can the programme m ost effectively contribute to the im plem entation and effectiveness of the NSFP? a. What are MINEDUC’s priorities for Phase II in this respect? 5. Schools will need to enter school feeding data into the School Data Managem ent System and will be responsible for collecting new indicators for the NSFP. a. Do schools have the training and staff capacity for this task? b. How will MINEDUC prepare schools to carry out m onitoring and data collection? c. How is WFP supporting MINEDUC and schools to im plem ent out this new requirem ent? 6. What are your expectations for (WFP-supported) m odel schools in Phase II? How will their experience be used across the broader education system ? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 103 7. What are the com m unication and inform ation-sharing processes between WFP and MINEDUC at national level? At district level? a. How effective are these processes for collaboration, coordination and decision-m aking? 8. Please com m ent on the regularity, frequency, and num ber of National School Feeding Steering Com m ittee m eetings. a. Please describe the process and results generated by the National SFSC m eetings (e.g., provide exam ples of decisions or how problem s were addre ssed) b. Please describe any capacity strengthening received by the National SFSC. 9. Please com m ent on the regularity, frequency, and num ber of National School Feeding Technical Working Group m eetings. a. Please describe the process and results generated by the National TWG m eetings (e.g., provide exam ples of decisions or how problem s were addressed) b. Please describe any capacity strengthening received by the National TWG from WFP. 10. What are your expectations for the McGovern-Dole Programm e in strengthening local farm er capacity to consistently supply high quality food to local school m eal programme s? a. What do you see as the challenges to farm ers? To schools? (e.g., national procurement laws, production challenges, links between farmers and local schools) b. How is WFP assisting MINEDUC to address challenges to sm all holder farm ers to supply school m eal programme s? What m ore can be done? c. In what areas do farm ers need capacity strengthening? 11. How are issues around gender/vulnerable children/disabled children taken into account in the NSFP? Is this approach appropriate and effective? Is it reaching the children from the poorest fam ilies, from poor fem ale -headed households, especially girls? 12. What barriers to enrolm ent, attendance and com pletion do children with disabilities face? Do girls face different or greater barriers than boys? How is this being addressed? 13. What role do the attitudes of teachers and parent play in setting expectations for girls to continue to secondary and tertiary education? 14. How is the COVID-19 pandem ic affecting the im plem entation of the NSFP? What are your m ain concerns for schools around COVID-19? Ca pa cit y St rengt hening/ Rea diness 15. How do you assess the readiness and capacity of MINEDUC and related m inistries to fully im plem ent the NSFP with regard to: a. Policy fram eworks b. Institutional capacity c. Budget allocation d. Coordination e. Monitoring, i.e., the ability to m onitor programme im plem entation to ensure quality and inform tim ely decision-m aking. f. Inspection and reporting g. Local procurem ent h. Dissem ination and com m unity engagem ent i. Which areas need the m ost support? How is/can WFP support MINEDUC in these areas? 16. How do you assess the readiness and capacity of districts and sectors (and related local officials) to fully im plem ent the NSFP with regard to: a. Planning b. Institutional capacity c. Coordination d. Budgeting e. Monitoring, i.e., the ability to m onitor programme im plem entation to ensure quality and inform tim ely decision-m aking. f. Inspection and reporting g. Local procurem ent h. Which areas need the m ost support? How is/can WFP support districts and sectors in these areas? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 104 17. Do you have any concerns around the transition of McGovern-Dole supported schools to the NSFP when the programme ends? What are the sustainability issues connected with the transition of these schools to the NSFP? Min is t r y o f Ed u ca t io n – Dis t r ict Le ve l Illustrative list of interviewees: District Director of Education, Mayor/Vice Mayor, Executive Secretary, Director Agricultural and Natural Resources Genera l Quest ions 1. How has your institution collaborated with the McGovern-Dole Programm e in the past? Wha t was your institution’s involvem ent in the design of Phase II of HGSF activities? Is the design relevant and realistic? 2. What capacity developm ent support is WFP providing to the district to im plem ent the NSFP? Please discuss WFP support to specific capacity and system s. What is the relevance and quality of the support? 3. What is the district’s needs and priorities related to capacity strengthening for the NSFP? 4. What are your expectations for the McGovern-Dole Programm e in strengthening local farm er capacity to consistently supply high quality food to local school m eal programme s? 5. What do you see as the challenges to farm ers? To schools? (e.g., national procurem ent laws, production challenges, links between farm ers and local schools) 6. How is WFP assisting to address challenges to sm all holder farm ers so they can supply school m eals? 7. To what extent are HGSF activities aligned with the priorities and developm ent plans of your institution? 8. What is your assessm ent of the perform ance of the WFP Rwanda country office and sub-national offices? 9. What are the com m unication and inform ation-sharing processes between WFP and your institution? How effective are these processes for collaboration, coordination, and decision-m aking? 10. Please com m ent on the regularity, frequency, and num ber of District School Feeding Steering Com m ittee m eetings. a. Please describe the process and results generated by the District SFSC m eetings (e.g., provide exam ples of decisions or how problem s were addressed) b. Please describe any capacity strengthening received by the District SFSC from WFP. 11. What is your institution’s needs and concerns around sustainability of the activities im plem ented under McGovern-Dole Programm e after activities phase out? 12. Are activities by other partners or other agencies sufficient to com plem ent the McGovern -Dole Programm e to enhance sustainability? What additional partnerships could be explored? 13. How are issues around gender/vulnerable children/disabled children considered in the program m e ? Is this approach appropriate and effective? Is it reaching the children from the poorest fam ilies, from poor fem ale -headed households, especially girls? 14. What im pact from the programme do you expect to see on newly enrolled schools and com m unities in Phase II? On (WFP-supported) m odel schools and com m unities? (this question is only relevant to interviews in the three new districts) 15. What should be the priorities for Phase II of the McGovern-Dole Programm e? How can the programme m ost effectively contribute to the scale -up of the NSFP? 16. What has been the effect of the COVID-19 pandem ic and related restrictions on schools this academ ic year? Ca pa cit y St rengt hening/ Rea diness 1. How do you assess the readiness and capacity of districts and sectors (and related local officials) to fully im plem ent the NSFP with regard to: a. Planning b. Institutional capacity c. Coordination d. Budgeting e. Inspection and reporting f. Local procurem ent 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 105 g. Which areas need the m ost support? How is/can WFP support districts and sectors in these areas? 2. Schools will need to enter school feeding data into the School Data Managem ent System and will be responsible for collecting new indicators for the NSFP. a. Do schools have the training and staff capacity for this task? b. How will MINEDUC prepare schools to carry out m onitoring and data collection? c. How is WFP supporting MINEDUC and schools to im plem ent out this new requirem ent? 3. Parents are expected to contribute up to 60 percent of the cost of the school m eal under the NSFP. How will the district/sector support schools and parents to m eet this goal? 4. Do you have any concerns around the transition of McGovern-Dole supported schools to the NSFP when the programme ends? What are the sustainability issues connected with the transition of these schools to the NSFP? 5. What is the district’s readiness to support district and sector School Feeding Com m ittees? What support is needed to help these com m ittees fulfil their responsibilities? Minist ry-specific Topics Min ist ry Ad d it ion a l/ Sp e cific Lin e of In q u iry Ministry of Agriculture and Anim al Resources (MINAGRI) --Alignm ent with PSTA4; alignm ent with Governm ent policy to use local/regional school m eal sources instead of international --Role the m inistry plays with sm allholder farm ers National Childhood Developm ent Agency (NCDA) --Role in NSFP --Role in pre -prim ary education Rwanda Biom edical Centre (RBC) --Assessm ent of changes to health and dietary practices; im pact of school infrastructure on health Ministry of Local Affairs (MINALOC) at district level --MINALOC district-level perspective on how the design and im plem entation of the programme involves the local authorities as key stakeholders Ministry of Gender and Fam ily Prom otion (MIGEPROF) --Alignm ent of NSFP with gender and education guidelines/ policy/ strategy -- Assessm ent of how issues of gender/vulne rable children been taken into account in NSFP and prim ary education strategies? Is this approach appropriate and effective? Gaps? The World Bank --Assessm ent of im pact of national school feeding policy, strategy and programme on future potential for hum an capital in Rwanda Im p le m e n t in g Pa r t n e r s , UNICEF Interviewees: WVI, GHI, RBC, UNICEF Topics for All Im plem ent ing Pa rt ners: 1. How is your organization collaborating with the McGovern-Dole Programm e in Phase II? 2. How was your organization involved in the design of Phase II HGSF activities? Is the design relevant and realistic? How has your role changed from Phase I? 3. What are the com m unication and inform ation-sharing processes between WFP and your institution? Within the School Feeding Technical Working Group? How effective are these processes for coordination and decision-m aking? 4. Do you participate in the National School Feeding Steering Com m ittee or Technical Working Group m eetings? 6. What challenges do you see to your responsibility to m anage and m onitor school feeding, in light of the im plem entation of the NSFP? What capacity strengthening support is needed? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 106 a. Please com m ent on the regularity, frequency, and num ber of NSFSC and TWG m eetings held. b. Please describe the process and results generated by the NSF Steering Com m ittee m eetings. c. The TWG m eetings? What capacity strengthening activities has your organization provided? 5. What factors have influenced collaboration and decision-m aking during this period (positively or negatively)? 6. What additional opportunities exist for collaboration/synergies with your own organization? 7. Do you see any further opportunities for cooperation with Governm ental and non-Governm ental partners (e.g., in health, education, gender equity)? 8. What analysis has been done regarding the cost-efficiency and cost-effectiveness of the programme ? What are your im pressions of cost-efficiency and cost-effectiveness? 9. Strengths of the McGovern-Dole Programm e to date? What has worked well? What lessons are being carried over by your organization to Phase II? 10. Constraints/ challenges of the McGovern-Dole Program m e to date? How are these addressed in the design of Phase II? 11. What is your institution’s needs and concerns around sustainability of the activities im plem ented under McGovern-Dole Programm e after activities phase out? 12. How are issues of gender and disadvantaged children being taken into account? Describe whether this approach is appropriate and effective. Describe whether it is adequate to address the issues faced by these children and their fam ilies? 13. What has been the effect of the COVID-19 pandem ic and related restrictions on your activities with schools this academ ic year? 14. What learnings from the McGovern-Dole Programm e have been adopted by the NSFP? 15. How can the programme m ost effectively contribute to the scale -up of the NSFP? What needs and opportunities do you see for capacity strengthening to support the NSFP? Topics for WVI 16. What are the m ain challenges to the NSFP in term s of health and hygiene infrastructure? What is the role of World Vision in the McGovern-Dole Programm e to supporting the NSFP in these challenges? 17. How can/is World Vision’s work to support girls and schools being integrated into the NSFP? How can/is WV building the capacity of governm ent and schools to adopt this work? 18. What were the m ain challenges around increasing awareness on WASH and literacy in Phase I? What changes has World Vision m ade to better addre ss these activities in Phase II?" 19. How is World Vision supporting teachers and students to transition from Kinyarwanda to English? What are the biggest challenges to this transition? 20. What are the m ain challenges for pre -prim ary students in the school environm ent? The m ain challenges to a transition to prim ary school? How are these being addressed? 21. What are the m ain challenges to sensitizing parents to the im portance of education, especially for girls, for children from very poor fam ilies, and for children with disabilities? How are these being addressed? What m ethods can be adopted by other schools? 22. What are the m ain capacity challenges faced by school procurem ent com m ittees (e.g., linking with suppliers, contracts, food quality, m anagem ent of storage and food safety)? How can the sustainability of the supply chains be strengthened? Topics for GHI: 23. How are lessons from Phase I on school gardens, nutrition and com m unity outreach inform ing the im plem entation of the NSFP? What are the challenges and gaps? What are the positive developm ents? 24. What are the m ain challenges to supporting the operationalization of the national strategy on school gardens? To sustainability? 25. How can capacity to integrate nutrition-sensitive knowledge and activities into the NSFP be strengthened? What are the prim ary needs? 26. How are the specific nutritional need of adolescent girls being integrated into school garden and nutrition activities? Into activities with com m unities? What are the challenges? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 107 27. How are the nutritional needs of pre -prim ary children addressed through school garden and com m unity outreach activities? What are the m ain challenges, especially to engaging parents? Topics for RBC: 28. What capacity strengthening support can be provided to schools and com m unities to reduce the incidence of worm infestations am ong children? 29. One of RBC’s new initiatives is to engage teachers to screen children for illness. Please describe if/how the McGovern-Dole Program m e is supporting this initiative. 30. Another of RBC’s initiatives is to develop tools and m aterials for the prevention of neglected tropical diseases and parasitic diseases, which will be distributed to schools. Please describe if/how the McGovern-Dole Programm e is supporting this activity. 31. RBC m ade som e adaptations to how it delivered services during the COVID-19 pandem ic (e.g., com m unity-based deworm ing cam paigns, adm inistration of deworm ing tablets in schools by teachers). Please describe if these adaptations are continuing, and any other effects of COVID-19 on how RBC now delivers services. Topics for NCDA: 32. How are NCDA and WFP collaborating with MINEDUC to ensure that students’ nutritional needs are adequately addressed as the NSFP is im plem ented in schools? a. What are the achievem ents to date? What are the m ain challenges? 33. What input did NCDA have into the design of Phase II of the McGovern-Dole Program m e? 34. What are the m ain challenges to the im plem entation of the NSFP for pre -prim ary students? For ECD students? 35. Under the NSFP, parents are expected to contribute up to 60 percent of the school m eal. a. What challenges does this pose for delivering nutritionally adequate m eals? b. How can these challenges be addressed by NCDA, WFP and MINEDUC? 36. What capacity strengthening is needed to support the NSFP to ensure adequate nutrition for pre -prim ary schools? For ECD students? a. How is this being addressed by governm ent? b. What support is WFP providing? Is support relevant and appropriate? 37. What role do school gardens play in providing inputs to nutritious m eals in McGovern-Dole Programme schools? In NSFP schools? 38. What is being done to sensitize parents and other com m unity m em bers to the im portance of nutrition at hom e? What are the m ain cha llenges? How are they being addressed? Topics for UNICEF: Not e: UNICEF support ed m odelling a nd sca ling up Child‐Friendly School st a nda rds, which were a dopted a s the na tiona l qua lity guidelines for school infra structure a nd soft wa re input s. UNICEF a lso support ed t he Lea rning Achievem ent in Rwa nda n Schools (LARS) Assessm ent t o im prove t he qua lit y of educa t ion a nd m ea sure lea rning outcomes in litera cy a nd numera cy. 39. Com plem entarity between WFP and UNICEF priorities and activities in the McGovern -Dole Program me? In the NSFP? Successes and challenges? 40. Effects of COVID-19 on UNICEF-WFP activities, and adaptations. Adaptations that will be carried over post-COVID and in NSFP. 41. Com m unication and inform ation-sharing processes between WFP and UNICEF on the McGovern-Dole Programm e and the NSFP. Effectiveness of these processes for collaboration, coordination and decision-m aking. 42. Effectiveness of cross-sector planning for education, WASH, HGSF and other programme s that affect the quality of education for children? At national level? At district level? Please give exam ples. 43. Changes to the educational environm ent in the WFP McGovern-Dole supported schools since 2015? What changes can be adopted (or should not be adopted) by the NSFP? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 108 44. How are issues around gender/vulnerable children/disabled children addressed in prim ary and pre - prim ary schools Is this approach appropriate and effective? Is it reaching the children from the poorest fam ilies, from poor fem ale -headed households, especially girls? 45. Concerns about sustainability of McGovern-Dole Programm e activities in WASH infrastructure and other areas after the programme ends? 46. Are activities by other partners or other agencies sufficient to com plem ent the McGovern-Dole Programm e to enhance sustainability? What additional partnerships could be explored? Sch o o l He a d Te a ch e r s a n d Te a ch e r s Illustrative list of interviewees: Head tea chers, teachers For Pha se I schools: 1. What do you see as the purpose of the McGovern-Dole Programme? 2. What records are kept on (1) adm inistration of m eals and (2) student attendance? (ask to see records; check if disaggregated by gender, disabled students or other vulnerable categories) 3. What changes have you noted in the school or students since school m eals began here? Differences between boys and girls? Differences am ong children from very poor fam ilies and disabled children? 4. Describe any positive or negative im pact of the programme on: (Phase I schools) a. the school? b. wider com m unity? c. the people who prepare the m eals? d. local farm ers/producers? e. Your own skills and effectiveness as head teacher or teacher? 5. Strengths of the McGovern-Dole Programme to date? What has worked well? (probe: literacy outcomes, WASH and kitchen infrastructure, school gardens logistics, relationship with the community) 6. Constraints/ challenges of the McGovern-Dole Program m e to date? How have these been addressed? (Probe on adequacy and frequency of parent contribution to cost of school meals) 7. What factors have influenced achievem ent/ non-achievem ent of results? What can be im proved? 8. Have food deliveries during the last year been regular and com plete (all item s received in the requested am ounts)? 9. Has the school had to interrupt feeding for any extended periods during the last year? Why? 10. Does the school follow the official ration scale and num ber and tim ing of m eals per day, as per School Feeding Operational Guidelines? If not, why not? 11. Does the school garden contribute food to the m eals? If yes, please describe what, how m uch, and when the garden contributes to school m eals. 12. Are there any other health activities in the school (deworm ing, m alaria prevention)? Who im plem ents them ? Quality and effectiveness? 13. What has been the effect of the COVID-19 pandem ic and related restrictions on schools this academ ic year? Ca pa cit y St rengt hening/ Rea diness 14. What has the McGovern-Dole Programm e done to support capacity strengthening of: a. school head teachers and teachers b. school com m ittees (Tender Com m ittee, School Feeding Com m ittee, SGAC) c. cooks in preparation for the transition to the NSFP? (Please describe) 15. Is this support adequate and relevant? Are there any gaps that need to be addressed? 16. Do you feel that the school is ready to transition to the NSFP in 2023? Do you have any concerns about the transition? Please explain. 17. Once the schools transition to the NSFP, parents are expected to cover 60 percent of the costs of a daily m eal. Do you think it will be possible for parents to m ake this contribution, and if not, why not? What are alternatives or solutions for parents who cannot m ake this contribution? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 109 For New a nd Cont rol Schools: 1. What are the m ain challenges faced by the head teachers and teachers in this school in achieving its education goals? 2. What are the m ain challenges faced by students to regular attendance, staying in school, and learning? 3. What are the specific challenges to attendance and learning faced by girls, boys, children from very poor fam ilies, disabled students? 4. What type of school feeding programme does your school participate in (describe activities, source of support)? 5. What do you see as the purpose of the McGovern-Dole Programm e? How do you expect to benefit from the programme ? 6. Is this school im plem enting the National School Feeding Programme ? If yes, how is the school involved (describe the school’s activities with the NSFP)? Sch o o l Fe e d in g Co m m it t e e s a n d Sch o o l Te n d e r Co m m it t e e s Pha se I schools: 1. What do you see as the purpose of the McGovern-Dole Programm e? What is the role of the [COMMITTEE TYPE] in the programme ? 2. Do parents contribute to the m eals (probe: fresh vegetables, condim ents, fuel; level of participation, difficulties) 3. What changes (good or bad) have you noticed since the introduction of fresh m eals and buying food locally? 4. What changes have you seen in the students since the project started? Differences between boys and girls? 5. Describe any positive or negative im pact of the programme on: a. the school? b. wider com m unity? c. the people who prepare the m eals? d. local farm ers/producers? e. Skills of school head teachers and teachers, [COMMITTEE TYPE], outreach to com m unity 6. What has worked well? 7. Constraints and challenges? How have these been addressed? 8. Suggestions on how to im prove the programme ? New a nd cont rol schools: 1. Do you have school feeding programme s at this school? If yes, which programme s? Please describe the activities and any support from governm ent, institutions, parents or com m unities. What is the role of the [COMMITTEE TYPE] in school feeding? In other activities? 2. What are the m ain challenges faced by the head teachers and teachers in this school in achieving its education goals? 3. What are the m ain challenges faced by students to regular attendance, staying in school, and learning? 4. What are the specific challenges to attendance and learning faced by girls, boys, children from very poor fam ilies, disabled students? 5. What type of school feeding programme does your school participate in (describe activities, source of support)? 6. What do you see as the purpose of the McGovern-Dole Programm e? How do you expect to benefit from the programme ? 7. Is this school im plem enting the National School Feeding Programme ? If yes, how is the school involved (describe the school’s activities with the NSFP)? Co o k s For Pha se I schools: 1. What do you see as the purpose of the McGovern-Dole Programme? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 110 2. What is your role in the programme ? Please describe any training or knowledge you received from the programme , and how you use that knowledge in your job. 3. Do you have a contract with the school? Do you get paid on tim e and on a regular schedule? If not, why not? 4. Do wom en face m ore challenges than m en to being hired as cooks? What are those challenges? How can they be addressed? 5. What changes have you noted in the school or students since the project started? Differences between boys and girls? Differences am ong children from very poor fam ilies, or disabled children? 6. Is the food sufficient? What kinds of foods do you serve (describe)? Do you think the foods are nutritious? Why or why not? 7. What changes (good or bad) have you noticed since the introduction of fresh m eals and buying food locally? 8. Do parents contribute to the m eals? What do they contribute (fresh vegetables, condim ents, fuel, work, cash)? 9. Does the school garden contribute food to the m eals? If yes, please describe what, how m uch, and when the garden contributes to school m eals. 10. Do pre -prim ary children and prim ary children get the sam e m eal (size, com position, and type of m eal)? 11. Were deliveries during the last school term regular and com plete (all item s received in the requested am ounts)? If not, why not? 12. Has the school had to interrupt feeding for any extended periods during the last year? Why? 13. Does the school follow the official ration scale and num ber and tim ing of m eals per day? If not, why not? 14. Has the initiative had an im pact on your lives? Has it affected how you are seen/ treated by the school or by the com m unity? Describe. 15. Thinking back on the past school term , do you have any suggestions on how the programme can strengthen your skills to do your job as a cook? For New a nd Cont rol Schools: Note: if new or control schools have cooks for a school feeding programme, questions 2-10 can be asked. Researchers should obtain details on type of programme, meal composition, frequency, who supports it and how. St u de n t s (Gra de 5 a n d High e r) For Pha se I schools (Group 1): 1. Did you eat breakfast today before com ing to school? What did you eat? Do you eat breakfast every day? Do you get hungry during classes? How does being hungry affect your attention? 2. What do you like m ost about the school m eals? 3. What do you like least about the school m eals? 4. Has the school m eal programme changed anything for you? (Probe: concentration, no hunger, more frequent or regular attendance, one extra meal a day, one less meal at home per day, extra burden of in-kind or cash contribution for parents) 5. Are there days when the school does not provide a m e al? On the days that there are no m eals, do you still com e to school? Do other children com e to school? 6. If you could change som ething about the school m eals, what would that be? For New a nd Cont rol Schools: 1. Did you eat breakfast today before com ing to school? What did you eat? Do you eat breakfast every day? Do you get hungry during classes? How does being hungry affect your attention? 2. Do you bring a snack or lunch to school? Do you go hom e for lunch? What kind of food do you bring? Do all students bring a snack or food to eat during school hours? 3. Note: if there is a school feeding programme at new or control schools, ask questions 1-5 above. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 111 Sm a llh o ld e r Fa r m e r s / Gr o u p s , Lo ca l Co o p e r a t ive s 1. How long has this group/cooperative existed? How m any m ale and fem ale m em bers? What are the leadership positions (get num ber/ratio of m ale and fem ale officers)? 2. What do you see as the purpose of WFP support to your cooperative? 3. Is the cooperative aware of school feeding programme s in nearby schools? Has the cooperative explored supplying those schools with food? 4. Does this cooperative have any agreem ents to supply food to schools? If yes, what type of agreem ent do you have with the school(s) regarding the am ount and price for agricultural products? What happ ens if you cannot supply the food? Do schools pay on tim e and if not, why? How is the food transported to the schools? 5. What vegetables, fruits or other food do you supply to schools for school m eals? How m uch food do you supply? To which schools? How does the food you produce vary by season? What different types of food are you producing to supply the schools? 6. How do you access credit or cash to build the resources needed to increase production, quality, and access to m arkets? Who provides the funding? 7. What surplus do you produce above your com m itm ent to schools? Who do you sell it to? What role has WFP played in producing a surplus, if any? 8. What capacity strengthening have farm er groups/cooperatives and its m em bers received from WFP? 9. How has this m ade a difference to the type of food you produce? How has it influenced how the group/cooperative is m anaged? How are products are processed and m arketed? How products are stored? 10. What are the benefits/advantages to the group/cooperative of participating in the programme (Probe: incom e; im proved skills in production, processing, food safety, m arketing, post-harvest losses, group cooperation, group m anagem ent, financial m anagem ent, business plans, sustainable agricultural practices) Benefits to individuals? What has worked well? 11. Constraints/challenges of participating in the programme ? How are these being addressed? 12. What training or support have you received from WFP (or partners)? Relevance and quality of training/support? Exam ples of changes you’ve m ade as a result of that support? 13. Do you plan to continue participating in the McGovern-Dole Programm e? Why or why not? 14. Do you participate in other programme s to strengthen your production, m arketing, and m anagem ent skills? If yes, what programme s, and what activities do you participate in (e.g., MINAGRI, Rwanda Cooperative Agency, MINICOM, NGOs)? 15. Suggestions on how to im prove the programme ? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 112 An n e x 11: EGRA To o l ΜNovem ber 2021 This tool is written in Kinyarwanda. Se ct io n 1: Kn o w in g t h e s o u n d o f a le t t e r / Ik icir o cya 1: Ku m e n ya in yu gu t i n ’ib ih e k a n e Ingero: B u ny Exa m p le s: J m z ts b K m b a r e nz c n l o t P nd u kw w E h r bw y v d shw i mbw S n u g R z n cy B A ry W O t sh r m w y p nt T J by o i p njw D N M nty f U bw D E K m p G m v F t m y E W nk b nj H y r g m w F s u ry V L ns c r e nsh h gw b o m b Se ct io n 2: Re a d in g s lla b le s / Ik icir o cya 2. Gu s o m a im ige m o Ingero : ko ru mbe Exa m p le s: ba m i ku no gwe wa vu su ndo byi de SA cu re tso do ti nzu zi sho 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 113 ZE fi ro ngu ve na bwi co hu nshu nwe m o du zwi ja po shu Kwa yi ko m yo he RU si jya fu ga ne bi mbo ta vo nsu bi do hu ko pfu gi nyo mbwi pu bo rya nu ta gi tso si gwe yo ntu zo be shya wo ba nda gu njwe nsho be rwa gi co RA m u ZE mwa ni pi yu ko gwe ru m byo ye nki tu zi Se ct io n 3: Re a d in g co m m o n w o r d s / Ik icir o cya 3. Gu s o m a a m a ga m b o a zw i cya n e In ge ro: umuti in za ra urwego Exa m p le s: ikaram u ibiryo igitoki ibara amano im boga kubyina isukari icunga inkweto ikigo ibishyim bo am avuta um upira urubuto gukaraba amatunda am azi indwara igi urugo igitondo umubu ishati igiceri avoka inka ururabo umugabo inshuti um utaka im bwa guteka ikibaho ikoti isahani Ishyam ba izina abana im yaka umurongo ibirayi ijwi ikanzu amababa um ukinnyi um uneke icupa intoki isahani Se ct io n 4: Dis co ve r in g n o n -e x is t e n t w o r d s / IKicir o cya 4. Gu t a h u r a a m a ga m b o y’a m a h im b a n o Ingero ` umumeho ish in i amahenko 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 114 Exa m p le s: Ishovu Utum eda ishyuperu agaceru im yefo um unzwegi kubeja ikene im byem o am adozu uruzefa varu injyerega akanem a inkwerezo ipfereco im eni intwapa uruceno nika ndapemu kuvenwa betini akaniti yeti Ngufa icuyu am agweca isebwene agahubi ntenewi icyuram i apuya im eriba ityuni am acyuvo inwazi am ayeti ipewo ivum ita gupofera inovi tashibu amabwi ikovu injyoku irobi m afadera insetu akote Se ct io n 5: Re a d in g a t e xt / Ik icir o cya 5. Gu s o m a u m w a n d ik o Um unsi um we, Cyuzuzo yadusuye iwacu m u rugo. Yadusabye gutuza akatuganiriza. Twese twahise tugira am atsiko. Twicara m u ntebe ducecetse.Yatubwiye ko gukina m u biziba bishobora kudutera indwara. Bityo, dukwiye kubyirinda. Yatwibukije kandi ko kurya indyo yuzuye bizaturinda kurwara. Tugom ba rero kubiharanira. Yadusabye kunywa am azi m eza asukuye. Bizatum a tugira ubuzim a bwiza. Yanatugiriye inam a yo kwirinda Ibiyobyabwenge. Ibiyobyabwenge byangiza ubuzim a. Yanatubwiye ko gukora im yitozo ngororam ubiri, bikom eza am agufwa. Yadusabye ko igihe turwaye, dukwiye kwivuza hakiri kare. Cyuzuzo yatwunguye byinshi! Ibibazo byo kum va um wandiko n’ibisubizo (ntibihabwa um unyeshuri) Com prehension questions and answers (not to be given) 1. Ni inde wadusuye iwacu m u rugo?  Ni Cyuzuzo 2. Kurya indyo yuzuye biturinda iki?  Biturinda indwara 3. Cyuzuzo yatugiriye inam a yo kwirinda iki?  Yatugiriye inam a yo kwirinda ibiyobyabwenge/ Kwirinda gukina m u biziba. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 115 4. Kuki ari byiza gukora im yitozo ngororam ubiri?  Bikom eza am agufwa 5. Ni iki abantu bagom ba gukora iyo barwaye?  Bakwiye kwivuza/ kujya kwa m uganga 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 116 An n e x 12: Sch o o l Ce n s u s To o l In s t ru ct ion s This school level census will collect school statistics from all schools in the programme as well as the control schools. The head teachers at selected schools under the USDA-funded McGovern-Dole Program im plem ented by WFP will be requested to fill out this census. The form will be delivered to head teachers rem otely. It is to be filled out by head teachers and returned electronically to the survey team . Em ail surveys will be followed up via phone for areas with poor internet availability and access. This survey is programmed using Open Data Kit (ODK) survey software and adm inistered via e -m ail and/or phone with head teachers. Collect on Android Tablets in English and/or Kinyarwanda. The paper/Microsoft word/.pdf docum ent is used for training, review and quality control purposes only. If you have any questions on this survey tool or the associated sam pling m ethodology, please contact Lloyd Owen Banwart at TANGO International at Lloyd@tangointernational.com Re s p o n d e n t The Head Teacher or her/his designate is the respondent to this survey: He a d Te a ch e r s u r ve y Q# Qu est ion Re sp on se Cod e Notes ID1 Date of survey response ID2 ID3 School Geographical Inform ation Province District Village Cell ID4 School Nam e ID7 School type Prim ary Groupe scolaire (prim ary and secondary) ID5 School funding m odel Mixed (governm ent-aided) Governm ent ID6 GPS Coordinates of school (if available) Conse nt Consent and Introduction This school census is being conducted on behalf of WFP, World Vision, Gardens for Health International and TANGO International. The census is part of a baseline study being carried out for WFP’s McGovern-Dole Programm e. The purpose of this survey is to gather school perform ance data at the beginning of the programme . The survey requests data about the school, its teachers and students. Your responses will not be used to generate either positive or negative im pressions about the school. Participation in this survey is com pletely voluntary. We very m uch appreciate your input to this im portant survey. Do you agree to participate in the school census? Consent Refused 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 117 Q# Qu est ion Re sp on se Cod e Notes A1 Respondent Nam e A2 Respondent Sex Male Fem ale A3 Position of the respondent Head Teacher Deputy Head Teacher Teacher Other (specify) A4 Which grades are in this school? [Select one P1 – P3 P4 – P6 P1 – P6 Other A5 Does the school have a school based ECD centre? Yes – Nursery Yes – ECDC No A6 Num ber of m ale teachers A7 Num ber of fem ale teachers A8 Num ber of cooks and storekeepers [total]? A9 Num ber of teachers who were present for 90 percent of scheduled school days in the past school year? A10 Average student attendance rate Girls – Pre -prim ary Prim ary A11 Boys – Pre -prim ary Prim ary Grade For each grade (1-6 and ECD) com plete the following questions. This should be done with the teachers of the grades present. All questions pertain to the 2021 and/or 2020 school year [TBD] Start with com pleting the following for 1st Grade and continue, in order, through 6th Grade. P1 P2 P3 P4 P5 P6 G1a Num ber of total enrolled m ale students in this grade in 2020 / 21 [will be asked separately for 2020 and 2021] G1b Num ber of total enrolled fem ale students in this grade in 2020 / 21 G4a Num ber of m ale students receiving daily school m eals in 2020 / 21 (breakfast, snack, lunch) G4b Num ber of fem ale students receiving daily school m eals in 2020 / 21 (breakfast, snack, lunch) P1 Under the National School Feeding Programme , at least 50% of parents are expected contribute at least 100% of the required school feeding contribution. Will parents in your school be prepared and able to m ake this contribution? Yes No P2 Num ber of repeat learners by grade and sex (prior academ ic year) P1 P2 P3 P4 P5 P6 Girls Boys 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 118 Q# Qu est ion Re sp on se Cod e Notes Dropouts by grade and sex (prior academ ic year) P1 P2 P3 P4 P5 P6 Note: figures m ay be affected by COVID-19 Girls Boys Main reasons for drop-out (open question) Teacher turnover in last academ ic year – what percent of teachers left for any reason? Num ber of unfilled teaching staff positions now? Num ber of unfilled cook positions now? Is the school using the nutrition and food safety guides developed for cooks and food store m anagers? Yes No Do Not Know P3 Parent contribution Percent of parents who contribute to the cost of school m eals with cash: average am ount per week P4 Percent of parents who contribute in-kind P5 If in-kind, type of contribution Labour Food Water Firewood Other P6 Percent of parents who do not contribute to cost of school m eals 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 119 An n e x 13: Sch o o l a n d He a d Te a ch e r To o l In s t ru ct ion s This school level survey is to be adm inistered by a researcher. Som e questions will be verified with direct observation. The researcher will be guided by a knowledgeable head teacher, and the head teacher will be the respondent for question. The survey will be adm inistered at selected schools under the USDA-funded McGovern-Dole Program im plem ented by WFP. This survey is adm inistered using Open Data Kit (ODK) Collect on Android Tablets in English and/or Kinyarwanda. The paper/Microsoft word/.pdf docum ent is used for training, review and quality control purposes only. If you have any questions on this survey tool or the associated sam pling m ethodology, please contact Lloyd Owen Banwart at TANGO International at Lloyd@tangointernational.com Re s p o n d e n t The respondent(s) of this survey can be any of the following: 1. Direct Observation 2. School Head Teacher 3. Cook / storekeeper [Question C5 only] He a d Te a ch e r Su r ve y Q# Qu est ion Re sp on se Cod e Notes ID1 Date of Interview ID2 Interviewer Nam e ID3 School Geographical Inform ation Province District Village Cell ID4 School Nam e ID7 School type Prim ary Groupe scolaire (prim ary and secondary) ID5 School funding m odel Mixed (governm ent-aided) Governm ent ID6 GPS Coordinates of school 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 120 Q# Qu est ion Re sp on se Cod e Notes Conse nt Consent and Introduction Hello, m y nam e is ___________, and I am a researcher conducting a survey on behalf of WFP, World Vision, Gardens for Health International and TANGO International. The purpose of this survey is to ……. The survey will include a series of questions between the school head teacher (you) and m yself. The topics covered will include …… In addition, trained enum erators will conduct a short (30 m inute) interview with 22 random ly selected 3rd grade students. This survey will only ask questions regarding …… This survey (and the participation of the students) is com pletely voluntary. Do you agree to participate in the school survey and consent to let the random ly selected 3rd grade students participate in the student survey? Consent Refused A1 Respondent Nam e A2 Respondent Sex Male Fem ale A3 Position of the respondent Head Teacher Deputy Head Teacher Teacher Other (specify) A4 Which grades are in this school? [Select one] Pre -prim ary P1 – P3 P4 – P6 P1 – P6 Other A6 Have YOU received any trainings or certifications as a result of the HGSF programme ? Yes No Do not Know (DNK) new Describe the trainings or certifications. A7 Note: these num bered categories contained questions that are now in the school census; the num bers are left in until tools are finalized to facilitate analysis A8 A9 A10 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 121 Q# Qu est ion Re sp on se Cod e Notes A11 Are there organizations supporting your school in the following activities? [Read Responses] [Select all that apply] School feeding Other nutrition activities Deworm ing Sanitation (water and toilets) School governance Provision of school m aterials, textbooks, books Renovation/construction of infrastructure in school e.g., classes, kitchens, stores Training of teachers Health education Other activities (specify) A12 If yes; which organizations? School feeding Other nutrition activities Deworm ing Sanitation (water and toilets) School governance Provision of school m aterials, textbooks, books Renovation/construction of infrastructure in school e.g., classes, kitchens, stores Training of teachers Health education Other activities (specify) A13 Based on your level of knowledge about the com m unity and the pupil’s parents, what PERCENT of parents do you feel can nam e at least three of the below benefits of prim ary education? 1. Increased Health Percent 2. Im proved Nutrition 3. Increased opportunities 4. Increased earning potential 5. Ability to read/write/count 6. Knowledge for daily life 7. Increased socialization 8. Strengthening relationships 9. Increased engagem ent with com m unity 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 122 Q# Qu est ion Re sp on se Cod e Notes A14 Are any of the following com m ittees active in your school? School General Assem bly Com m ittee Yes No DNK School Feeding Com m ittee Yes No DNK School Tender Com m ittee Yes No DNK School Managem ent Com m ittee Yes No DNK Other (specify) • New If yes, how often in a school year do they m eet? SGAC SFC STC SMC Other Weekly Monthly Quarterly Other (specify) A15 A16 A17 Across all com m ittees/structures active in the school, have they been trained in any of the following under the HGSF programme ? [Read responses] [Check all that apply] SGAC SFC STC SMC Other School governance Im proved school management School infrastructure School garden Nutrition / school feeding Health and hygiene Other A18 A18_p A19 Does your school have a school m eals programme for its students? Yes – pre -prim ary Yes – lower prim ary Yes – upper prim ary Yes - both No DNK A20 How m any days in a week are these m eals provided? A21 How m any m eals in a day are provided? A22 Which m eals does the school provide? [Check all that apply] Breakfast Lunch Snacks How m any m eals were provided in the last week that include foods in addition to the donated US com m odities? [Check all that apply] Num ber of m eals that included: 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 123 Q# Qu est ion Re sp on se Cod e Notes Fruits Vegetables Legum es Anim al Proteins (m ilk, m eat) A23 Where is the food for school children obtained? [Check all that apply] WFP Farm er groups Governm ent NGOs provided Parents provided Local m arkets Kitchen garden Other A24 Does the school have a partnership with any farm ers groups for food? NOTE: understood this is not part of the current HGSF m odel, but there m ay be other partnerships Yes No DNK A25 Where does the m oney for purchasing food com e from ? [Check all that apply] Donor organizations Parents School budget Donations Other A26 Please describe the seasonal m enu in this school (include all ingredients) Term 1 Term 2 Term 3 Is the school using the nutrition and food safety guides developed for cooks and food store m anagers? Yes No DNK Grade This section intentionally left blank in draft tool P 1 P2 P3 P4 P5 P 6 G1a G1b G4a G4b *For the following 6 questions, go to each classroom and have the teacher(s) provide an estimate. P1 *What PERCENTAGE of ENROLLED students (both m ale and fem ale) in P1 grade can be identified as attentive by their teachers? P2 *What PERCENTAGE of ENROLLED students (both m ale and fem ale) in P2 grade can be identified as attentive by their teachers? P3 *What PERCENTAGE of ENROLLED students (both m ale and fem ale) in P3 grade can be identified as attentive by their teachers? 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 124 Q# Qu est ion Re sp on se Cod e Notes P4 *What PERCENTAGE of ENROLLED students (both m ale and fem ale) in P4 grade can be identified as attentive by their teachers? P5 *What PERCENTAGE of ENROLLED students (both m ale and fem ale) in P5 grade can be identified as attentive by their teachers? P6 *What PERCENTAGE of ENROLLED students (both m ale and fem ale) in P6 grade can be identified as attentive by their teachers? C1a Does the school have water source? Piped water Public tap Tubewell or borehole Protected dug well Rainwater Protected spring Cart with tank Tanker-truck Bottled water Unprotected dug well Surface water Unprotected spring Other New Take a photo of the water source Q1b If yes, was/were any of these water source(s) im proved through the support of the HGSF programme ? Yes No DNK Q2 Is water norm ally available from this source? (Norm ally is m ore than 4 of the 5 school days each week and all year). Yes No DK C3 Does this school have supplem ental reading m aterials available to students obtained because of USDA and World Vision assistance? Yes (observed) Yes (not observed) No Don’t Know C3_P Take a photo of the reading m aterials (or sam ple of them ) C4 Are the school cooks and/or storekeeper available? Yes – Cook(s) Yes- Storekeeper is available No -cook and storekeeper both not available 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 125 Q# Qu est ion Re sp on se Cod e Notes C5 What are safe food preparation and storage practices? [If C4 is yes -cook, then ask only to the cook(s). If cook not available, we ask storekeeper] [Do not read out] [Select all that apply] 1. Food m ust be handled and prepared with utmost cleanliness, including proper hand washing before preparing food 2. All staff handling food in school m ust receive training on basic hygiene 3. Contact between raw foodstuffs and cooked food m ust be avoided 4. Food should be cooked thoroughly 5. Food m ust be kept at safe tem peratures 6. Safe water and safe raw ingredients m ust be used in food preparation 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 126 An n e x 14: Su m m a ry list of int e rvie w s and focus groups 280.Table 32 presents sum m ary inform ation regarding interviews conducted to com ply with USDA requirem ents for excluding personally identifying inform ation in the evaluation report. Table 33 sum m arizes the focus groups conducted. Ta b le 32: Su m m a ry of ke y in form a n t in t e rvie w s con d u ct e d Tit le # KIIs WFP Rw a n d a WFP Rwanda staff 7 KIIs (4M, 3F) (including 2 pending as of 9 Dec) Sch ool-based School-based staff (head teachers, deputy head teachers, cooks, storekeepers) 18 KIIs (13M, 5F) Dist rict Gove rn m e n t District staff/ officials (directors of HGSF, education, and agriculture; agronom ists) 15 KIIs (14M, 1F) Na t ion a l Gove rn m e n t National governm ent staff/officials 2 KIIs (2M, 0F) (including 2 pending as of 9 Dec) Pa rt n e r NGOs Partner NGO staff 4 KIIs (4M, 0F) Fa rm e r coop e ra t ive s Cooperative president (1 KII; other cooperatives interviewed by FGD) 1 KII (1F) TOTAL 47 KIIs (37M, 10F) Ta b le 33: Focu s grou p d iscu ssion s con d u ct e d Focu s Grou p Typ e #M #F Dist rict /se ct or Sch ool/ ot h e r Da t e 1 Teachers 3 3 Nyaruguru/Ruh eru GS Ziram bi 26 Oct 2021 2 Teachers 1 2 Nyaruguru/Ruh eru EP Gahotora 27 Oct 2021 3 School Feeding Com m ittee 3 2 Nyaruguru/Ruh eru EP Gahotora 27 Oct 2021 4 Cooks 2 1 Nyaruguru/Ruh eru EP Gahotora 27 Oct 2021 5 P5 students 2 3 Nyaruguru/Ruh eru EP Gahotora 27 Oct 2021 6 P5 students 10 10 Nyaruguru/ Nyabim ata EP Kabere 27 Oct 2021 7 P5 students 3 3 Nyaruguru/ Ruheru EP Mukaka 27 Oct 2021 8 Farm er Cooperative 1 12 Nyaruguru Twitezim bere Kiyonza 28 Oct 2021 9 Teachers 2 4 Nyam agabe/ Kibirizi GS Kiraro P 29 Oct 2021 10 School Tender Com m ittee 4 1 Nyam agabe/ Kibirizi GS Kiraro P 29 Oct 2021 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 127 Focu s Grou p Typ e #M #F Dist rict /se ct or Sch ool/ ot h e r Da t e 11 Cooks 3 0 Nyam agabe/ Kibirizi GS Kiraro P 29 Oct 2021 12 P5 students 3 3 Nyam agabe/ Kibirizi GS Kiraro P 29 Oct 2021 13 Farm er cooperative 7 2 Nyam agabe/ Cyanika Abibum biyeh amwe Cyanika 29 Oct 2021 14 Farm er cooperative 3 3 Rutsiro Isuka nibwo bukungu 01 Nov 2021 15 Teachers 3 2 Rutsiro/ Kivum u GS Rwinyoni 02 Nov 2021 16 School Feeding Com m ittee 3 3 Rutsiro/ Kivum u GS Rwinyoni 02 Nov 2021 17 Cooks 2 1 Rutsiro/ Kivum u GS Rwinyoni 02 Nov 2021 18 P5 students 3 2 Rutsiro/ Kivum u GS Rwinyoni 02 Nov 2021 19 Cooks 3 0 Rutsiro/ Manihira GS Rwam iko 02 Nov 2021 20 P5 students 3 3 Rutsiro/ Manihira GS Rwam iko 02 Nov 2021 21 P5 students 3 3 Karongi/ Gashari EP Kaduha 03 Nov 2021 22 P5 students 4 4 Karongi/ Gashari EP Gashari 03 Nov 2021 23 Teachers 2 3 Karongi/ Gitesi GS Gashubi 04 Nov 2021 24 School Feeding Com m ittee 2 0 Karongi/ Gitesi GS Gashubi 04 Nov 2021 25 Cooks 1 1 Karongi/ Gitesi GS Gashubi 04 Nov 2021 26 Farm er cooperative 2 0 Karongi/ Murundi Duharanire Isuku Murundi 05 Nov 2021 27 P5 students 4 4 Karongi/ Ruganda EP Gisayura 08 Nov 2021 28 P5 students 4 4 Karongi/ Ruganda EP Nyabisiga 08 Nov 2021 29 Teachers 2 3 Gasabo/ Rutunga GS Kayanga 09 Nov 2021 30 School Feeding Com m ittee 1 1 Gasabo/ Rutunga GS Kayanga 09 Nov 2021 31 P6 Students 3 3 Gasabo/ Rutunga GS Kayanga 09 Nov 2021 32 Farm er cooperative 2 2 Gasabo/ Rutunga Zam uka Rutunga 09 Nov 2021 33 Teachers 5 3 Kayonza/ Rwinkwavu GS Abadahigwa 10 Nov 2021 34 School Feeding Com m ittee 1 3 Kayonza/ Rwinkwavu GS Abadahigwa 10 Nov 2021 35 P5 students 8 4 Kayonza/ Rwinkwavu GS Abadahigwa 10 Nov 2021 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 128 Focu s Grou p Typ e #M #F Dist rict /se ct or Sch ool/ ot h e r Da t e 36 Teachers 1 2 Kayonza/ Rwinkwavu EP Nkondo I 11 Nov 2021 37 School Feeding Com m ittee 0 1 Kayonza/ Rwinkwavu EP Nkondo I 11 Nov 2021 38 Cooks 2 0 Kayonza/ Rwinkwavu EP Nkondo I 11 Nov 2021 39 P5 students 4 3 Kayonza/ Rwinkwavu EP Nkondo I 11 Nov 2021 40 Teachers 4 2 Burera/ Ruhunde EP Gitovu 12 Nov 2021 41 School Feeding Com m ittee 3 1 Burera/ Ruhunde EP Gitovu 12 Nov 2021 42 Cooks 2 1 Burera/ Ruhunde EP Gitovu 12 Nov 2021 43 P5 students 4 4 Burera/ Ruhunde EP Gitovu 12 Nov 2021 44 Teachers 4 2 Burera/ Kivuye EP Murwa 16 Nov 2021 45 School feeding com m ittee and school tender com m ittee 4 1 Burera/ Kivuye EP Murwa 16 Nov 2021 46 Cooks 2 0 Burera/ Kivuye EP Murwa 16 Nov 2021 47 P5 students 4 4 Burera/ Kivuye EP Murwa 16 Nov 2021 48 Farm er cooperative 6 2 Burera COVMB 17 Nov 2021 TOTAL 148 121 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 129 An n e x 15: Ba s e lin e a n d e va lu a t io n t im e lin e 281.Table 34 presents the tim eline for the baseline study. Table 35 presents the high-level tim eline for the baseline study and m idterm and endline evaluations. Ta b le 34: Ba se lin e st u d y t im e lin e St e p s By whom Da t e De script ion of de live ra ble In ce p t ion Launch call EM, ET Aug 11 2021 Inception report should describe: - Understanding of the project based on project docum ents and literature review - Finalized m ethodology including detailed sam pling plan. Evaluation questions and field procedures - Quality assurance plan - Com m unication protocol - Tim eline (activities, re sponsible party, outputs, and tim ing) - Data collection tools (electronic copies of all clean and final English-version data collection tools and analysis plan) Undertake desk review of docum ents ET Aug Rem ote inception m ission ET Aug-Sep TANGO SUBMITS draft inception report (IR) (includes draft evaluation m atrix, qualitative & quantitative data collection tools and analysis plan) ET Sep 10 EM to review draft IR; joint review session with TANGO team leader to share/incorporate feedback EM, TL Sep 13 TANGO SUBMITS revised IR ET Sep 17 EM to review draft IR and subm it to DEQAS for review (if com pliant with corporate standards) and ERG m em bers for review in parallel EM Sep 20 EM to share DEQAS feedback and com piled ERG m em bers’ feedback with TANGO EM Sep 29 Joint review of DEQAS + ERG m em bers feedback during weekly check-in (1h30 or 2h) EM, TL Sep 30 TANGO SUBMITS revised draft incorporating DEQAS + ERG Mem bers feedback ET Oct 8 WFP to subm it NISR protocol prepared by TANGO Oct 8 EM review; joint review session of revised draft Oct 12 TANGO SUBMITS revised draft IR incorporating DEQAS + ERG Mem bers feedback (includes final evaluation m atrix, qualitative & quantitative data collection tools and analysis plan) EM Oct 15 WFP approval of final IR EM Oct 19 Da t a colle ct ion Data collection/ fieldwork kick-off assum ing NISR approval in place ET Oct 25 – Nov 12 TANGO to provide end of data collection debrief presentation to WFP CO/RBN (in lieu of weekly m eeting) ET Nov 25 Presentation should include an abbreviated list of evaluation findings that can be presented to relevant internal and external stakeholders 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 130 Re p ort in g a n d a n a lysis TANGO SUBMITS draft baseline report ET Dec13 Report should address all the evaluation objectives and questions listed in the scope of work. It should include the following sections: - Acknowledgem ents - List of acronym s and abbreviations - Table of Contents - Executive Sum m ary (2 pp) - Background (programme description and purpose of study) - Methodology and Im plem entation - Methodology Lim itations (strengths and weaknesses) - Results and Findings (in accordance with the objectives) - Conclusions, Lessons Learned and good practices - Recom m endations - Annex: Table of key programme indicators with baseline values - Annex: Scope of Work for the evaluation - Annex: Inception Report for the evaluation - Annex: Survey Instrum ents: questionnaire(s), survey(s), interview protocol(s), focus group discussion protocol(s) as relevant EM to review the draft baseline report EM Dec 13-14 Joint review session on prelim inary feedback from the EM EM, TL Dec 15 TANGO SUBMITS revised draft baseline report ET Dec 17 EM to review the draft BR and subm it to DEQAS for review (if com pliant with corporate standards) and ERG m em bers for review in parallel EM Dec 17 EM to share DEQAS feedback with TANGO EM Jan 11 Joint review of DEQAS EM, TL Jan13 EM to share ERG m em bers’ feedback with TANGO EM Jan 19 Joint review of ERG m em bers’ feedback EM, TL Jan 20 TANGO to share a revised draft incorporating DEQAS + ERG Mem bers feedback ET Jan 27 WFP to subm it final baseline report to USDA EM Jan 31 USDA to send feedback to TANGO EM Feb 21 TANGO SUBMITS datasets and related docum ents ET Feb 28 Datasets and related docum ents should include the following - Raw and clean datasets organized in SPSS with its analysis syntaxes; - Interview transcript and focus group discussion notes etc. - All m aterials above provided in electronic versions. USDA clearance EM Mar 11 TANGO SUBMITS 2-3-page sum m ary of the evaluation param eters and findings for a broader audience ET Mar 18 The brief should describe in language easy-to-understand by non-evaluators and with appropriate graphics and tables sections as follows: - Evaluation design; - Key findings; - Lesson learnt and case studies representative of the intervention; and - Other relevant considerations. Disse m in a t ion a n d follow u p Prepare Managem ent Response (MR) CO TBD 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 131 Share final Baseline Report and m anagem ent response with OEV for publication EM TBD Dissem inate and use Baseline Report results EM, CO TBD ET = Evaluation Team ; EM = Evaluation Manager; blue = deliverable 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 132 Ta b le 35: Eva lu a t ion t im e lin e 2020-2025 Da t e Phases PREPARATION PHASE FOR OVERALL EVALUATION From Dec--July 2021 Assign roles/responsibilities (WFP), Establish Evaluation Com m ittee and Evaluation Reference Group Develop Term s of Reference (TORs) and budget (WFP) Procure independent evaluation firm (WFP) INCEPTION PHASE FOR OVERALL EVALUATION From August-Septem ber2021 Desk review of key project docum ents (evaluation team ) Inception m ission (evaluation team and WFP) Prepare Inception Report including quantitative and qualitative data collection tools (evaluation team ) BASELINE STUDY From October-Decem ber 2021 Preparation of field visits (evaluation team and WFP) Data collection (evaluation team ) Data analysis (evaluation team ) Prepare baseline study report (evaluation team with inputs from ERG) Share final baseline study findings with ERG including USDA (evaluation team ) Request Com m itm ent Letter m odifications, as necessary (WFP) MID-TERM EVALUATION From July 2023- October 2023 Inception: Update to original Inception Report as required, review of desk docum ents (evaluation team ) Preparation of field visits (evaluation team and WFP) Data collection (evaluation team ) Data analysis (evaluation team ) Draft and finalize Mid-term Evaluation Report (evaluation team with inputs from ERG through exit m ission debriefing and com m enting on draft evaluation report) Dissem inate final evaluation finding to ERG m em bers including USDA through workshop and/or other channels (WFP) Prepare Managem ent Response (WFP) FINAL EVALUATION From Septem ber 2025 – Decem ber 2025 Inception: Update to original Inception Report as required, review of desk docum ents (evaluation team ) Preparation of field visits (evaluation team and WFP) Data collection (evaluation team ) Data analysis (evaluation team ) Draft and finalize final Evaluation Report (evaluation team with inputs from ERG through exit m ission debriefing and com m enting on draft evaluation report) 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 133 Da t e Phases Dissem inate final evaluation finding to ERG m em bers including USDA through workshop and/or other channels (WFP) Prepare Managem ent Response (WFP) 282. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 134 An n e x 16: Su m m a ry t e rm s of reference See next page 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 135 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 136 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 137 An n e x 17: Co n flict o f in t e r e s t fo r m s 283.TANGO has subm itted the evaluation team ’s conflict of interest form s to the WFP evaluation m anager. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 138 An n e x 18: Ge n d e r As s e s s m e n t Br ie f: Home Grown Sch ool Fe e din g Programme Com m issioning Office: WFP Rwanda De ce m be r 2021 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 139 In t rod u ct ion 284.WFP’s Hom e -Grown School Feeding (HGSF) Program m e aim s to achieve two strategic goals: 1) im proved literacy of school-age children, and 2) increased use of health and dietary practices. In light of the Governm ent of Rwanda’s (GoR) heightened em phasis on gender equality in school participation, perform ance and outcom es, WFP Rwanda com m issioned an in-depth gender assessment to better understand the nuanced gender context within which HGSF program m e activities unfold, and how these gender dynam ics m ay be influencing or hindering the outcom es of the program m e. Understanding this context is particularly im portant in term s of the sustainability of program m e outcom es over tim e and within the context of WFP’s com m itm ent to a ‘do no harm ’ approach in all of their work.1 WFP’s Gender Policy a nd Gender Tra nsform a t ive Approa ches 285.The pursuit of gender equality and wom en’s em powerm ent is central to WFP’s m ission of saving lives and changing lives and supporting countries in their quest to achieve the Sustainable Developm ent Goals. WFP’s 2009 gender policy introduced a shift from a wom en-centered approach to a broader analysis of the challenges and opportunities in the lives of the wom en, m en, girls and boys whom WFP assists, focusing on gender norm s and power dynam ics as a broader social system . The policy em phasized m en’s roles in creating change and highlighted the im portance of understanding gender relations. WFP’s gender policy (2015-2020) em phasizes the need for a gender transform ative approach and the forthcom ing 2022-2026 policy further highlights the im portance of this approach in WFP’s work, reinforcing the critical need to address root causes of gende r discrim ination and inequality.92 The goal of the updated gender policy is to ensure that WFP optim izes its presence, role and capabilities, in partnership, to advance gender equality and em power wom en. WFP's ability to achieve its strategic objectives depends on it delivering food and nutrition assistance in a m anner that addresses the differing needs, priorities and experiences of the wom en, m en, girls and boys with whom it works 93. Gender a nd School Feeding Around t he World 286.School feeding program m es play a critical role in global developm ent, including the realization of the Sustainable Developm ent Goals (SDGs).94 School feeding responds directly to the SDGs related to food security, education and gender equality (SDG 2, 4 and 5).95 School feeding facilitates education and particularly for girls, it leads to im proved food security, health and nutrition, the effects of which all contribute to ending hunger. School feeding helps close the gender gap in schools and helps to em power wom en by increasing their probability of em ploym ent. When girls are educated, they are m ore likely to have fewer and healthier children and to head fam ilies that are food secure. Ba sic Fa ct s a bout School Mea ls Worldwide 1. Nearly every country in the world has som e form of school m eals program m e in place. 2. School m eals program m es are the m ost com m on social safety net in the world. 3. An estim ated 368 m illion children receive a m eal at school every day, both in developing countries and in affluent countries. 4. The global investm ent in school m eals is about USD75 billion a year. 5. The Midday Meal Schem e in India is the largest school m eals program m e in the world, as it feeds 105 m illion children every day. Ba sic Fa ct s a bout WFP School Mea ls Progra m m es 1. Since the establishm ent of WFP in 1961, school m eals have been part of WFP’s m ission. 2. In 2015, WFP provided school m eals to 17.4 m illion children in 62 countries, spending USD.321 m illion. In the m ajority of these countries, WFP also provided technical assistance. 92 Additional detail on WFP’s forthcom ing Gender Policy 2022-2026 was provided by the MEL team within WFP Rwanda for this assessm ent. 93 WFP Gender Policy 2022 94 How School Meals Contribute to the Sustainable Development Goals: A Collection of Evidence; WFP, 2017. 95 Sustainable Development Goals; United Nations, 2015. Feed Minds, Change Lives: School Feeding, the Millennium Development Goals and Girls’ Empowerment; WFP, 2014. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 140 3. In 2015, WFP provided solely technical assistance on school m eals program m es to 10 governm ents, indirectly benefiting an estim ated 10 m illion children. 4. In 37 countries WFP provided technical assistance on hom e -grown school m eals. School Mea ls’ Cont ribut ion t o t he Sust a ina ble Developm ent Goa ls 287.School m eals contribute to the following specific SDGs: 1. Con t rib u t in g t o ze ro h u n ge r: When the rations are appropriately designed, school m eals can im prove the nutrition status of pre -school children, prim ary school children and adolescents, by addressing m acronutrient and m icronutrient deficiencies. This leads to enhanced nutrition and health, decreased m orbidity, and increased learning capacities. 2. Con t rib u t in g t o e d u ca t ion : When a school m eals program m e is part of a package of investm ents in education, it can help m axim ize the return of these investm ents, because school m eals facilitate access to school, increase enrolm ent and attendance rates and im prove the nutritional status, health and cognitive developm ent of children. 3. Con t rib u t in g t o ge n d e r e q u a lit y: Girls struggle m ore than boys for access to education; one in every ten girls in the world is out of school, while with boys this figure is one in twelve. Wom en and girls are also m ore exposed to hunger and m alnutrition than boys; they represent 60 percent of all undernourished people in the world. And when adolescent girls are out of school, they are m ore vulnerable to forced m arriage, early pregnancy, violence and even hum an trafficking. Whe n adequately designed, school m eals program m es can narrow these gender gaps and help break the vicious cycle of discrim ination against girls. 4. Con t rib u t in g t o t h e e con om y: When well-designed, school m eals program m es have direct benefits for children: they im prove their nutrition status, health, and level of education. These direct outcom es further contribute to wider processes such as the reduction of poverty and inequality and econom ic growth. HGSF Pr o gr a m m e Ove r vie w 288.To im prove health, nutrition, and education outcom es, the GoR and WFP have collaborated on im plem entation of school feeding program m e since 2002. WFP Rwanda has been im plem enting the HGSF program m e funded by the United States Departm ent of Agriculture (USDA) McGovern-Dole International Food for Education and Child Nutrition Program (MGD) since 2015. WFP Rwanda is currently im plem enting phase two of the 5-year HGSF program m e in Rwanda from 2020 to 2025. The HGSF program m e currently provides school health and nutrition interventions to approxim ately 79,000 prim ary students (49% girls / 51% boys) in 108 schools across four districts. In 2022, the program m e will expand to 28 new schools in three additional districts and will include pre -prim ary and prim ary students. Gender Aspect s of t he Current Programme 289.The current program m e provides equal access to direct activities and benefits to girls, and the previous 5- year phase supported and influenced the governm ent’s policies on gender equality, specifically in the recognition of the value of sanitation room s for girls and the work to develop m eals that m eet the nutritional needs of adolescent girls. In addition, the Gender and Age m arker tool (GAM) was used for the program m e when the Country Strategic Program m e was first developed, achieving a score of 3 out of 4. This process includes assessing if data was disaggregated by sex and age and if gender analysis inform ed the form ulation and revision of policies and program m es. This introduced a three -pronged approach where gender is em bedded, ensuring that girls and boys benefit equitably from the HGSF program m es, and the presence of gender transform ative practices that encourage and enable wom en to be part of decision-m aking processes in school m anagem ent and HGSF com m ittees. 290.Although the program m e seeks to align with WFP’s Gender Policy, the original program m e design did not include a targeted approach to address existing gender inequalities and the need for wom en’s inclusion and em powerm ent. The inclusion of a gender equality and wom en’s em powerm ent (GEWE) approach in the programme has been discussed previously but has not yet been im plem ented, thus the following gender assessment conducted at baseline is intended to help identify a way forward. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 141 Releva nt Findings from t he 2021 HGSF Progra m m e Endline Eva lua t ion 291.Som e of the key findings from the 2021 HGSF program m e endline evaluation that are m ost relevant to issues of gender equality and m ainstream ing, and also highlight areas where the program m e can m ake im provem ents in this regard, are highlighted be low. Impact and Sustainability 292.The program m e’s influence on national approaches to gender equality is reflected in the Governm ent’s recognition of the value of girls’ sanitation room s and the work to develop m eals that m eet adolescent girls’ nutritional ne eds. There is scope for greater gender sensitivity in the program m e, and potentially for the scaled-up national program m e. Gender 293.While WFP has a strong institutional focus on gender equality, the McGovern-Dole results fram ework does not specifically mention gender equality as an objective. The project proposal does not reference a gender analysis. The CO’s Country Gender Action Plan 2016-2020 refers to a gender baseline study done in 2015- 2016 but the plan notes only lim ited challenges to the HGSF, in terms of the number of fem ale cooks and fem ale teachers. Nutrition 294.WFP’s work on nutrient gap analysis for the national school feeding program m e has expanded thinking on school nutrition and has increased gender sensitivity toward the nutritional needs of adolescent girls, who m ay still be m atriculating in prim ary school. WFP has worked with MINEDUC on constructing local school m eal m enus to address the specific nutritional needs of adolescent girls (e.g., iron-rich foods). Operational Recommendation 6: Implement appropriate gender analysis and approaches for Phase II 295.While the McGovern-Dole Program m e design aligns with relevant gender policies and fram eworks, efforts toward gender-equitable outcom es and gender-appropriate approaches would benefit from a broader lens that m onitors how interventions influence GEWE. The absence of a com prehensive gender analysis, and the m issed opportunity to im plem ent relevant gender-sensitive approaches in interventions, are significant gaps. The program m e should m ake learning on gender an explicit focus for Phase II. WFP should conduct a gender analysis at baseline and consider a gender audit at m idterm and/or at endline. Th e Ge n d e r Co n t e x t o f Rw a n d a Gender Norm s a nd Persist ent Inequa lit y 296.In Rwanda, wom en’s em powerm ent and gender equality have seen im provem ent in recent years across som e key m etrics (political representation, legal reform s, girls’ education, m aternal m ortality, for exam ple). However, gender norm s and the discrim inatory beliefs and attitudes that drive the exclusion of wom en and girls rem ain intact in m any arenas of life are often unaddressed in policy and program s. Patriarchal gender norm s continue to prim arily prom ote m en as leaders, providers and decision m akers, and wom en as wives and m others. Unpaid care work continues to severely im pact wom en’s and girls’ freedom , m obility, econom ic opportunities and quality of life. Men rem ain prim ary decision m akers around how m oney is spent and largely control household incom e autonom ously. Girls as well as boys are prepared from a young age to fulfill these gender roles as adults. 297.Men continue to outnum ber and outrank wom en at m ost levels of leadership within key governm ent roles. 96 Further, violence against wom en and girls has severe negative consequences for their agency, health and well-being and rem ains high in Rwandan society, with 24 percent of adolescent girls aged 15-19 and 28 percent of young wom en aged 20-24 having experienced violence at least once since the age of 15.97 The gendered norm ative attitudes that support and perpetuate violence against wom en and girls in Rwanda rem ain high: 45 percent of girls aged 15-19 and 42 percent of young wom en aged 20-24 believe that a 96 Permanent secretaries of ministries (30 % women); Heads of public institutions (20 % women); Director Generals (DGs) of ministries (34 % women); DG equivalents in ministries (25 % women); Directors of units (24 % women) 97 DHS 2014-2015; NISR, 2015. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 142 husband is justified in beating his wife for at lea st one or m ore of six com m on scenarios in the Rwandan context, including burning food, disagreeing with him or refusing to have sex with him (2014/15 DHS survey). Gender, Nut rit ion a nd Food Securit y 298.Gender-based poverty is a characteristic of poor rural households in Rwanda. Fem ale -headed households in Rwanda, com prising 23 percent of all households on average, are m ore likely to be food insecure than those headed by m en, com prising of 17 percent. Factors that increase the risk of stunting in children include m others who are stunted, have low levels of education, and do not receive antenatal care. Anem ia affects 19 percent of wom en. The percentage of stunted children under the age of 5 has decreased from 38 percent in 2014/15 to 33 percent in 2019/20 98. Gender a nd Educa t ion 299.Achieving gender equality in education is a national priority for Rwanda. In 2019, the total net enrolm ent rate for girls was 98.6 percent and 98.4 percent for boys in prim ary education. At secondary level, the rate was 26.7 percent for girls and 22.1 percent for boys. The num ber of fem ale students is often higher in pre -prim ary, prim ary and lower secondary levels but shifts at higher levels, with m ale students predom inating at tertiary level (60.6% m ales, 44.6% fem ales).99 300.Differences in the dropout rates of girls and boys are m inim al, however data indicates that girls are m uch m ore likely to be out-of-school from age 16 onwards. Girls are also m ore likely to dropout when there are m ore siblings in the household and are also m ore vulnerable to shocks in the household, for exam ple the birth of a new child or the death of the m other. This is related to the expectation for girls to m anage other children and provide dom estic labour for unpaid care work in the hom e. The Rwanda Education Sector Strategic Plan has a dedicated budget line item to address education barriers for girls, including the provision of gender-sensitive water and sanitation facilities. Policy & St ra t egy Environm ent in Rwa nda 301.Rwanda has a num ber of national policies and strategies that support gender equality and affirm the equal rights of wom en and girls, both as a targeted priority as well as a cross-cutting them e. These include: • Vision 2050 • National Strategy for Transform ation (NST1) • Education Policy 2003 • Education Sector Strategic Plan 2018-2024 • Girls Education Policy 2008 • National Com prehensive School Feeding Policy 2019 • National Gender Policy 2021 As s e s s m e n t Me t h o d o lo gy Scope of t he St udy 302.This gender assessm ent has been com m issioned by WFP Rwanda as part of the Baseline Evaluation of the USDA-funded McGovern-Dole International Food for Education and Child Nutrition Grant for WFP’s HGSF Program m e in Rwanda (2020-2025). This assessm ent contributes form ative research with the purpose of gathering the necessary data to render the programme gender-transform ative in all activities and outcom es. Resea rch Object ives 303.The m ain objective of the gender assessm ent is to assess how gender dynam ics in the im m ediate environm ent (hom e, school and com m unity) are potentially influencing and/or hindering the im pact of the HGSF program m e, and what opportunities the program m e has to prom ote gender equality and wom en’s empowerment m ore holistically throughout its activities, with an em phasis on adopting a gender transform ative approach. 98 DHS 2014/15 – 2019/20; NISR Rwanda. 99 Education Statisitcal Yearbook; MINEDUC, 2019. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 143 304.Specific objectives include: • To identify and understand how the influence of household gender dynam ics (productive/reproductive roles, financial management and decision m aking, agency, division of labour/unpaid care work, gendered approaches to parenting) affects wom en’s and m en’s positive participation in their role as parents and in schools in different ways, as well as the influence it has on the attitudes of children. • To identify and understand the way gender norm s and power dynam ics in the hom e, com m unity and school affect wom en’s and girls’ access, participation and benefits from the school feeding programme and educational/em ploym ent outcom es. • To inform a gender transform ative approach that intentionally incorporates considerations around gender norms, responsibilities, and power relations into project im plem entation, m onitoring andevaluation. Met hodology Study design & Data collection methods 305.This gender assessm ent takes the form of a participatory, qualitative research exercise to identify the gender norm s, roles and responsibilities within the hom e and com m unity, including access to and control over relevant resources, structures of decision-m aking and participation between men and women (parents and teachers), as well as gender-related vulnerabilities and opportunities for girls and wom en in select aspects of the HGSF program m e including paid work opportunities, com m unity leadership roles, etc. The gender assessm ent also explores how gender norm s and beliefs advance or constrain gender equality, wom en's em powerm ent, positive parenting, and inclusive learning practices. 306.Qualitative and quantitative data for this gender assessm ent has been collected through both secondary re search / desk review and prim ary research / field data collection in four HGSF program m e schools. Data collection involved a m ix of focus groups discussions (FGDs), household interviews (HIs) and key inform ant interviews (KIIs). FGD participants included parents, teachers, com m ittee m em bers, cooks, boys and girls. Ke y Fin d in gs Tying It All Toget her: Revisit ing t he Resea rch Quest ions 307.The findings of this gender assessm ent are rich and nuanced, providing a com prehensive overview of the gender environm ent surrounding the HGSF program m e and its im pact, both direct and indirect, on the success and sustainability of the program m e, as well as the education sector as a whole, in both the short and long term . The below sum m ary highlights the guiding research questions and critical content of this report. 308.How a re ge n d e r d yn a m ics in a n d a rou n d t h e HGSF p rogra m m e p ot e n t ia lly h in d e rin g the im pa ct and/or edu ca t ion a l outcom es of the program m e (i.e . gender dyn a m ics rela t ed t o m obilit y, le a de rsh ip/a ge n cy, pe rm ission , power dyn a m ics, pa ren t in g/role m odel beh a vior for girls and boys, fin a n cia l de cision -m a kin g, in flu e n cin g parent con t ribu t ions, a du lt a u t h orit y figu re s (p a re n t s, deputy head t e a ch e rs, cooks, com m it t e e s)? 309.The gender dynam ics in and around the HGSF program m e are in som e ways hindering the positive im pact of the program m e across num erous areas. Gender norm s and discrim inatory attitudes around girls’ education that are left unaddressed by the program m e, and the education sector m ore broadly, are continuing to perpetuate boy preference and the belief that education for girls is not particularly valuable or necessary, despite considerable efforts m ade by the governm ent to prom ote girls’ education. Gender dynam ics in the hom e that reinforce a norm ative belief in wom en’s and girls’ inferiority and their prim ary role of servitude in life continue to create im m ense barriers for them to be able to fully participate in and benefit from different aspects of the HGSF program m e on par with m en and boys. Unequal power dynam ics between spouses in the hom e also negatively affect parents’ ability to ensure contributions are paid in full and on tim e, and without creating an additional burden on wom en to provide contributions in-kind through m anual labour. Gender dynam ics in the school setting are further role m odeling unequal behaviour for boys and girls and reinforcing a m yriad of gender stereotypes, including norm alizing gender-based violence and the exclusion of wom en and girls. In addition, norm s around gender, power and food in the hom e m ay also 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 144 be hindering the program m e’s objectives and positive m essaging in relation to nutrition and food security and ensuring that wom en and girls are able to claim their equal rights to both. 310.How doe s t h e gen de re d en viron m e n t in flu en ce st u de n t a t t it u des a n d pe rce pt ion s of e du ca t ion a s well a s access t o e d u ca t ion , girls’ a n d w om e n ’s se n se of a ge n cy a n d d e cision -m a kin g p ow e r, a n d girls’ a n d w om e n ’s re a l a cce ss t o u n iq u e a n d va lu a b le le a d e rsh ip or e m p loym e n t opport u n it ie s wit h in t h e progra m m e/sch ool/com m u n it y? 311.The findings from this assessm ent illustrate how the gendered environm ent is influencing students’ attitudes and perceptions around education, as well as their real access to education. Gender norms promote education for boys and dom estic work for girls. Both boys and girls are learning from a young age the distinct gender roles they are expected to fulfill in their hom es and com m unities, roles which uniquely privilege boys with an environm ent of encouragem ent, support, freedom , m obility and independence, and uniquely disadvantages girls with an environm ent of discouragem ent, a heavy daily burden of dom estic work, and the expectation that girls will prim arily becom e wives and m others in som eone else’s household in the future. The significant burden of dom estic work is a key driver in girls’ exclusion from education, contributing to them m issing classes, being late for school, dropping out of school entirely and struggling to be able to revise their studies or com plete hom ework. The widespread attitude that education is m ost im portant for boys because boys will be future breadwinners and are ‘less vulnerable’ than girls, while girls are expected to becom e wives and m others and rem ain in the dom estic sphere for the m ajority of their lives, is entrenched at all levels and presents distinct challenges for the overarching aim s of the program m e and the education sector as a whole. This attitude is reinforced by the norm s, behaviours and beliefs that are perpetuated by parents, siblings, teachers and other com m unity m em bers. 312.Girls’ and wom en’s sense of agency and decision-m aking power is profoundly shaped by the environm ent in the hom e, school and com m unity, reinforcing ideas of their perceived low status in society, denying them the ability to m ake decisions about their lives, control household incom e, access paid em ploym ent, or advocate for them selves and their rights (including their right to adequate and nutritious food, as well as their right to education). Their access to leadership or em ploym ent opportunities is curtailed by gender dynam ics within the hom e and school setting, where m en and boys dom inate in term s of key leadership roles, influence, paid roles within the program m e and decision-m aking powe r. The distinct gender expectations for wom en and girls to spend extensive hours on childcare and dom estic work serve to prevent them from having the tim e or energy to take on such opportunities. Although som e wom en m entioned they could m anage both, promoting wom en’s paid work outside of the hom e without sim ultaneously addressing the need for redistribution of unpaid care work within the hom e, creates a double burden of work for wom en. In addition, the need for girls and wom en to obtain form al perm ission from m en to leave the hom e or spend tim e on tasks outside of unpaid care work is another hurdle for them to be able to access and benefit from the HGSF program m e to the sam e degree as boys, across all program m e activities. 313.Wh a t op p ort u n it ie s d oe s t h e HGSF p rogra m m e h a ve t o p rom ot e ge n d e r e q u a lit y a n d w om e n ’s e m powe rm e n t t h rou gh it s a ct ivit ie s? Wh ich fa ct ors or con dit ion s cou ld be t t e r fa cilit a t e t h e se op p ort u n it ie s? Wh a t com b in a t ion of in t e rve n t ion s (if a n y) a p pe a r t o b e m ore su cce ssfu l (for e xa m p le : a n en a blin g e n viron m e n t a n d st ron g SBCC m e ssa gin g)? 314.There are som e notable positive im pacts of the program that should be continued, and further strengthened by adopting a gender transform ative approach. School m eals have helped to decrease absenteeism and lateness for girls; girls’ absenteeism due to illness decreased from 8 percent at baseline in 2016, to less than 2 percent in 2021. Attendance for girls also increased from 91.7 percent at baseline in 2016, to 96 percent in 2021. Girl participants in this study indicated that the sanitary room s for girls in schools have allowed them to access the m enstrual hygiene products and support they need during m enstruation, helping to reduce absenteeism in school and im proving their overall com fort level. In addition, school m eals provided in the afternoon help girls to rem ain in school for afternoon classes, which is im portant for their consistent attendance and overall school perform ance. Separate latrines for boys and girls reportedly also help girls to have m ore privacy at school. In term s of general perceptions around the benefits of the program m e, all participants reported that the program m e has been helpful in reducing school drop outs and absenteeism , as well as im proving school perform ance and students’ ability to focus in class without feeling hungry. 315.The HGSF program m e also has num erous opportunities to prom ote gender equality and the em powerm ent of wom en and girls’ by m ainstream ing gender considerations m ore intentionally and effectively across its various intervention areas. Addressing gender norm s and power dynam ics at the level of prim ary school is 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 145 an advantage for the HGSF program m e and the education sector as a whole. Engaging children at a young age helps to prevent the replication of these inequalities and barriers as girls and boys grow up. Taking a transform ative approach to addressing these deeply im bedded gender barriers also offers significantly better ‘value for m oney’ from a donor perspective; if the root causes of such issues are not addressed, then the sam e problem s will appear again and again over tim e, chronically hindering the im pact and success of the program m e, slowing down progress and requiring additional investm ents to be m ade year after year with lim ited gains. In addition, by not directly acknowledging and tackling the known gender barriers that are creating harm and exclusion for wom en and girls, the HGSF program m e can unintentionally norm alize the status quo and reinforce discrim inatory gender norm s and power dynam ics in their current form . Re co m m e n d a t io n s 316.Based on the findings from this report, a com prehensive package of interventions are recom m ended in line with established best practices in gender transform ative m ainstream ing within program m atic work and further inform ed by first-hand knowledge and expertise on ‘what works’ in the Rwandan context. These transform ative interventions can produce significant positive results in term s of gender equality in the HGSF program m e and across the education sector, and m ake significant progress in breaking down system ic gender barriers related to education, food security and nutrition. Key recom m endations include:  In ve st in b a se lin e ge n d e r a n d GBV e d u ca t ion for e ve ryon e in volve d (partners, schools, teachers, parents, students) to im prove basic understanding of key concepts, em phasize the im portance of gender equality and the prevention of discrim ination and violence, and ensure that the root causes of wom en’s and girls’ exclusion are addressed, with an em phasis on wom en’s and girls’ rights and unpaid care work. Groups should be engaged separately based on age and sex and with sam e -sex facilitators (i.e. girls should be engaged in separate groups from boys, adult wom en should be engaged in separate groups from m en), in line with established best practice in the sector. This approach is highly valuable for addressing the barriers raised in this report, and can be conducted with even a sm all num ber of schools to start out, scaling over tim e. (WFP)  En ga ge p a re n t s in t a rge t e d d ia logu e a n d le a rn in g se ssion s around gender roles and power dynam ics in the hom e, including decision m aking and agency, control over financial resources, power sharing, unpaid care work, equal parenting and feeding practices, and role m odeling gender equality for children. These activities can be integrated across existing program m e touchpoints via program m e trainings or m eetings facilitated by WFP, WV and GHI. Sm all groups of parents can be engaged at first depending on availability of resources for these types of interventions, and these groups can then becom e school / com m unity cham pion groups that continue to influence and educate other parents and adults. This approach can also be applied to engaging students and teachers, and is a less resource intensive way of fostering gender transform ative outcom es that are com m unity-owned. (WFP, World Vision (WVI), Gardens for Health International (GHI), Education Sector)  En su re m a n d a t ory e q u a l re p re se n t a t ion on com m it t e e s a s w e ll a s for p a id role s; this could m ean establishing equal representa tion quotas beyond the m inim um of 30 percent, to a fully equal 50/50 split that would help to norm alize wom en’s and girls’ fully equal participation beyond the 30 percent standard, which often tends to be perceived and acted upon in a tokenistic way. (WFP/Education Sector)  In -de pt h e n ga ge m en t of ke y le a de rsh ip figu res wit h in t h e progra m m e on t h e ge n de r issu es a n d ba rrie rs h igh ligh t e d wit h in t h is re port ; this can include special sessions provided to program m e partners, teachers, local leaders, sector education officers, etc. to ensure m eaningful capacity is built around core gender concepts, gender transform ative approaches and tools/m ethods, and that the program m e can be effectively im plem ented and evaluated in a gender transform ative way. (WFP, WVI, GHI, Education Sector)  Ad voca t e for t h e Gove rn m e n t t o a lloca t e a la rge r b u d ge t t o t h e p rogra m m e ove ra ll t o m a ke su re girls a n d w om e n d on ’t ge t le ft b e h in d ; the findings of this assessm ent can provide an additional opportunity to further highlight persistent gender barriers that are hindering the program m e and encourage the GoR to put m ore em phasis on addressing these issues in a transform ative m anner in 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 146 future years of the program m e, including allocating sufficient resources for this purpose. (WFP/Education Sector/National School Feeding Steering Com m ittee)  Le a rn from ot h e r p rogra m m e a sp e ct s a n d d e ve lop b e sp oke ca m p a ign s a rou n d sp e cific ge n d e r issu e s a n d ide n t ified ba rrie rs (sim ilar to the approach taken for previous challenges related to WASH as well as parent contributions, etc.); this can m ean targeting unpaid care work as a key barrier, for exam ple, and engaging com m unities in education/sensitization around this issue, as well as developing and dissem inating prom otional/behaviour change m aterials to target problem atic behaviours/attitudes while also helping to norm alize conversations about gender equality, power dynam ics and girls’ and wom en’s rights. (WFP/Education Sector)  Est a b lish a ge n d e r foca l p e rson a t e a ch sch ool; the focal person should have their capacity built in advance so they are fully equipped to be able to recognize and understand gender issues in the school context, effectively address them and prom ote a shift in attitudes and m indsets around gender and the rights of wom en and girls in the school environm ent and wider com m unity. (WFP/Education Sector)  In t e gra t ion of h a rm on ize d , con sist e n t SBCC m e ssa gin g a rou n d ge n d e r in t o e xist in g a ct ivit ie s a n d ca m pa igns within schools and across program m e activities; SBCC cam paigns targeting som e of the key barriers highlighted here would help to shine a light on highly problem atic gender issues that have becom e norm alized over tim e (girls’ excessive burden of dom estic work, for exam ple), but which are hindering the positive im pact of the program m e as well as violating girls’ rights m ore broadly. SBCC efforts should also include thorough gender m ainstream ing across existing SBCC m aterials and ensure no m aterials prom ote gender stereotypes or reinforce harm ful norm s and power dynam ics. (WFP, WVI, GHI, Education Sector)  En su re p a rt n e rs e n ga ge st u d e n t clu b s on ge n d e r e q u a lit y; this could m ean developing new clubs that focus on gender equality, girls’ rights and violence awareness and prevention, and/or m ainstream ing these them atics and discussions into existing student clubs. Learnings from successful clubs (literacy clubs, nutrition clubs, etc.) im plem ented so far can also be applied to this approach where appropriate. (WVI, GHI)  En su re a ge n d e r t ra n sform a t ive le n s is a p p lie d t o t h e feedba ck loop/form a l com pla in t m e ch a n ism cu rre n t ly be in g de velope d , so that individuals within the school and/or com m unity can raise gender concerns / issues where they arise and program m e partners can intervene in a tim ely m anner to rectify such issues, as well as learn from the process. This could include everything from an anonym ous ‘suggestion box’ approach at school level to im plem enting quarterly gender m onitoring site visits. (WFP)  En ga ge fa t h e rs d ire ct ly, a n d t a rge t m e n a n d b oys for e d u ca t ion a rou n d ge n d e r n orms and equ a lit y; this engagem ent should be bespoke for a m ale audience and tailored to specific concerns related to toxic m asculinity and harm ful power dynam ics that are driving the exclusion and violence that girls and wom en are currently facing. (WFP, WVI, GHI, Education Sector)  Est a b lish a sa fe a n d con fid e n t ia l GBV / ge n d e r d iscrim in a t ion re p ort in g m e ch a n ism within schools with a specific safeguarding focus, to ensure girls are able to raise concerns and access tim ely support, and to ensure accountability to program m e beneficiaries and WFP’s com m itm ent to m itigate risks of GBV. This could include equipping school leadership or an appointed school-based gender focal person with additional knowledge and understanding around GBV as well as practical skills and inform ation for receiving reports and safely referring children to existing services. (WFP/Education Sector)  In t e gra t e ca p a cit y b u ild in g a n d le a rn in g op p ort u n it ie s a rou n d ke y ge n d e r con ce p t s a n d ge n d e r t ra n sform a t ive a pproa ch e s in t o qu a rt e rly progra m m e workshops for partners; in this space, partners can learn m ore about gender transform ative work, present on their progress on gender m ainstream ing, access additional technical support, learn from each other, establish what works/best practice, continue to build their individual and collective capacity, and actively strategize to solve problem s and continuously im prove the program m e. (WFP) 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 147  Est a b lish ge n d e r d a t a re p ort in g m e ch a n ism s t h a t go b e yon d p u re ly q u a n t it a t ive m e t rics a n d a re a ccou n t a ble t o pa rt n e rs a nd t h e com m u n it y; gender inequality, exclusion, violence and discrim ination are driven by social system s of power and privilege and cannot be succinctly or accurately m easured by looking at quantitative representation of m en/wom en and boys/girls alone. Gender data collection should always assess the qualitative experiences of boys and girls, wom en and m en, to be able to effectively evaluate where progress is really being m ade in a m eaningful way; this is even m ore crucial when taking a gender transform a tive approach that is attem pting to address root causes of key gender barriers. In addition, data collection should be accountable to program m e partners as well as partner com m unities within the program m e, to ensure everyone understands why data is being collected and how it will im prove the program m e, and to m ake sure that partners and com m unities have access to the key findings and insights of this data collection. (WFP) 317.Overall, WFP and the com m unities it serves stand to benefit significantly through the im plem entation of gender transform ative interventions that address the key gender barriers articulated in this report, in the interest of both short term and long-term im pact. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 148 An n e x 19: Observa t ions on p rogramme t argets 318.This annex provides the evaluation team ’s observations and suggestions for adjustm ents to program m e targets. The analysis includes only the indicators for which the evaluation team is responsible to collect data (these are specified in Annex 9), and therefore indicators for which we have measured baseline values (indicators whose baseline value is zero are not discussed). Where possible, the evaluation team has reflected back on Phase I perform ance indicators. Act ivit ie s Pe rform a n ce In d ica t or Ph a se I MT Ph a se I EL Ph a se II BL-G1 Ph a se II BL-G2 Ph a se II BL-CN FY21 FY22 FY23 FY24 FY25 Com m e n t s 4 Support Im proved Literacy Percent of stude nts who, by the end of two grades of prim ary schooling, dem onstrate that they can read and understand the m eaning of grade level text 59.0 77.7 60.0 59.9 68.7 59% 59% 64% 64% 69% ET observation: Targets should be set separately for Group 1 and 2 schools. Even though baseline values are sim ilar, rate of change m ay be different. WFP can look to Phase I progre ss to asse ss the effectiveness of approaches as part of the Phase II target-setting process. 1.1 Provide Nutritious School Meals Average student attendance rate in USDA supported classroom s/schools 90.9 56.9 65.5 98% 98.5% 99% 99% 99% ET observation: No baseline value for this in Phase I. Again, targets should be set separately for Group 1 and 2 schools. Targets seem on the high side but m ay be within reach if there are no external shocks like COVID, based on Phase I results. 4.3 Support Teachers' professional developm ent Num ber of school adm inistrators and officials in target schools who dem onstrate use of new techniques or tools as a result of USDA assistance 71.4 0.0 0.0 329 329 329 95 95 ET observation: WFP should convert the num bers into percentages to asse ss target feasibility based on Phase I results. Set separate targets for Group 1 and 2 schools, with clear docum entation on assum ptions behind target-setting and the actual calculations. 4.3 Support Teachers' professional developm ent Num ber of school adm inistrators and officials trained or certified as a result of USDA assistance 93 104 103 0 0 365 365 365 105 105 ET observation: WFP should convert all num bers into percentages for basic com parison to assess target feasibility based on Phase I results (noting that they are different studies with different population and sam ple sizes). Set separate targets for Group 1 and 2 schools, with clear docum entation on assum ptions 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 149 Act ivit ie s Pe rform a n ce In d ica t or Ph a se I MT Ph a se I EL Ph a se II BL-G1 Ph a se II BL-G2 Ph a se II BL-CN FY21 FY22 FY23 FY24 FY25 Com m e n t s behind target-setting and the actual calculations. 1.1 Provide Nutritious School Meals Num ber of students enrolled in school receiving USDA assistance 9,976 18,672 15,131 7,199 5,013 88,397 117,095 117,095 28,698 28,698 ET note: This is just for the 41 schools that we interviewed. We do not know the attendance of all 145 schools but understand WFP is also collecting that data and can verify for the panel schools based on our surveys. ET observation: WFP should convert the num bers into percentages to asse ss target feasibility based on Phase I results (noting that they are different studies with different population and sam ple sizes). Set separate targets for Group 1 and 2 schools, with cle ar docum entation on assum ptions behind target-setting and the actual calculations. 1.1 Provide Nutritious School Meals Num ber of daily school m eals (breakfast, snack, lunch) provided to school￾age children as a result of USDA assistance 16,624 14,418 1,735 2,753 16,268,676 22,833,525 22,833,525 5,596,110 5,596,110 ET note: Numbers are only for the 41 schools we interviewed. ET observation: No com m ent here. This is an activity indicator and should already have a calculation behind it. 1.1 Provide Nutritious School Meals Number of school-age children receiving daily school m eals (breakfast, snack, lunch) as a result of USDA assistance 16,624 14,418 1,735 2,753 88,397 117,095 117,095 28,698 28,698 ET note: Numbers are only for the 41 schools we interviewed. ET observation: No com m ent here. This is an output indicator and should already have a calculation behind it. There should be a correlation between num ber of school m eals given and num ber of children receiving these m eals, especially since 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 150 Act ivit ie s Pe rform a n ce In d ica t or Ph a se I MT Ph a se I EL Ph a se II BL-G1 Ph a se II BL-G2 Ph a se II BL-CN FY21 FY22 FY23 FY24 FY25 Com m e n t s it is usually one m eal/day per child. That correlation is not clear at first sight, but WFP should have its calculations for this. 5.7.1 Build Capacity of cooks and storekeepers Num ber of individuals who dem onstrate use of new safe food preparation and storage practice s as a result of USDA assistance 85.0 40.0 4.7 0 0 289 365 365 76 76 ET note: This is just for the 41 schools in our panel. ET observation: WFP should convert the num bers into percentages to asse ss target feasibility based on Phase I results (noting that they are different studies with different population and sam ple sizes). Set separate targets for Group 1 and 2 schools, with cle ar docum entation on assum ptions behind target setting and the actual calculations. In this case Phase I and 2 targets m ay be close r together given the high turnover of cooks, but there m ay be residual institutional m em ory and they already have better infrastructure so pace of change m ay still be higher. 2.2 Construction of water system s Num ber of schools using an im proved water source 99 107 107 28 9 108 128 135 28 28 ET note: This is just for the 41 schools in our panel. ET observation: WFP should convert the num bers into percentages to asse ss target feasibility based on Phase I results (noting that they are different studies with different population and sam ple sizes). Set separate targets for Group 1 and 2 schools, with cle ar docum entation on assum ptions behind target setting and the actual calculations. 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 151 Act ivit ie s Pe rform a n ce In d ica t or Ph a se I MT Ph a se I EL Ph a se II BL-G1 Ph a se II BL-G2 Ph a se II BL-CN FY21 FY22 FY23 FY24 FY25 Com m e n t s Num ber of schools reached with LRP activities as a result of USDA assistance 108 108 107 28 0 108 135 135 28 28 ET observation: Seem s to be a straightforward output count taking into account the Group 1 phase -out. That would give targets as added in the colum ns [non-cum ulative] 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 152 Acr o n ym s ACR Annual Country Report AU African Union CBEHPP Com m unity‐based Environm ental Health Prom otion Program m e CFSVA Com prehensive Food Security and Vulnerability Analysis CHAI Clinton Health Access Initiative CHW Com m unity Health Worker CO Country Office CSRFSN Country Strategic Review of Food Security and Nutrition DDP District Developm ent Plan DEO District Education Officer DEQAS Decentralized Evaluation Quality Assurance System DFID Departm ent for International Developm ent EB Executive Board EC Evaluation Com m ittee EGRA Early Grade Reading Assessm ent ERG Evaluation Reference Group ESSP Education Sector Strategic Plan EQAS Evaluation Quality Assurance System ET Evaluation Team FCDO Foreign, Com m onwealth and Developm ent Office FY Fiscal Year GDP Gross Dom estic Product GEWE Gender Equity and Wom en’s Em powerm ent GHI Gardens for Health International GNI Gross national incom e HCI Hum an Capital Index HGSF Hom e Grown School Feeding LARS Learning Achievem ent in Rwandan Schools MDG Millennium Developm ent Goal MGD McGovern-Dole MHM Menstrual hygiene m anagem ent MINAGRI Ministry of Agriculture and Anim al Resources MINALOC Ministry of Local Affairs MINEDUC Ministry of Education MINESANTE Ministry of Health NCDA National Child Developm ent Agency NISR National Institute of Statistics of Rwanda NSF National School Feeding NSF National School Feeding Program m e NSFSC National School Feeding Steering Com m ittee NSFP National School Feeding Policy ODA Official developm ent assistance OECD-DAC Organisation for Econom ic Co-operation and Developm ent - Developm ent Assistance Com m ittee OEV (WFP) Office of Evaluation PMP Perform ance Monitoring Plan RB Regional Bureau RBC Rwanda Biom edical Centre RBN (WFP) Regional Bureau Nairobi 2 Ju n e 2022| DE/ RWCO/ 2021/ 025 153 REB Rwanda Education Board SDMS School Data Managem ent System SEO Sector Education Officer SGAC School General Assem bly Com m ittee SGBV Sexual and Gender-based Violence SMC School Managem ent Com m ittee TOR Term s of Reference TWG Technical Working Group USDA United States Departm ent of Agriculture VIP Ventilated im proved pit WASH Water, Sanitation and Hygiene WFP World Food Program m e WVI World Vision International UN CCA United Nations Com m on Country Analysis UNCT United Nations Country Team UNDAP United Nations Developm ent Assistance Program m e UNHDI United Nations Hum an Developm ent Index USAID United States Agency for International Developm ent USDA United States Departm ent of Agriculture VNR Voluntary National Review WASAC Water and Sanitation Corporation Wo r ld Fo o d Pr o gr a m m e Rw a n d a h t t p s:/ / ww w .w fp .org/ cou n t rie s/ rw a nda Wo r ld Fo o d Program m e Via Cesare Giulio Viola 68/70 00148 Rom e, تtaly T +39 06 65131 w fp.org