This publication was produced at the request of the United States Department of Agriculture. It was 
prepared independently by International Solutions Group.
Foreign Agricultural 
Service, United States 
Department of Agriculture
Rwanda McGovern-Dole
Project
Midterm Evaluation
July 2019
DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the 
United States Department of Agriculture or the United States Government.
WFP's USDA McGovern-Dole International Food for Education 
and Child Nutrition Program's Support in Rwanda 2016-2020 
Evaluation Report: Midterm Evaluation
The United States Department of Agriculture (USDA) McGovern-Dole International Food for 
Education and Child Nutrition Program in Rwanda is implemented by the World Food 
Programme (WFP), originally awarded in Fiscal Year 2015 to 2020. The McGovern-Dole Project, 
implemented in 104 schools, provides daily hot meals to 40,000 primary school children in 
Nyamagabe and Nyaruguru districts, Southern Province, and 43,000 in Karongi and Rutsiro 
districts, Western Province. 
Agreement Number: FFE-696-2015/007-00
Funding Year: Fiscal Year 2015
Project Duration: 2016-2020
Implemented by: World Food Programme
Evaluation Authored by: 
 
Jeanne Downen
Bruce Ravesloot
Justin Tuyiringire
Daria Muteteri
Juventine Mujawase
Monica Mueller
Lloyd Banwart
 
Prepared 
14 July 2019
WFP Rwanda
Evaluation Manager: Daniel Svanlund
Prepared by
Jeanne Downen, Team Leader
Bruce Ravesloot, Senior Evaluator
Justin Tuyiringire, Daria Muteteri and Juventine Mujawase, National Evaluators
Monica Mueller, Research Analyst/ Quality Assurance Manager
Lloyd Banwart, Data Analyst
Decentralized Evaluation
WFP's USDA McGovern-Dole International Food for Education and 
Child Nutrition Program's Support in Rwanda 2016-2020
Evaluation Report: Midterm Evaluation
DECENTRALIZED EVALUATION FOR EVIDENCE-BASED DECISION MAKING
WFP Rwanda USDA McGovern-Dole Midterm Evaluation Report i 
|Page
Acknowledgements 
The Evaluation Team would like to acknowledge the valuable assistance received from 
the colleagues at the World Food Programme in Rwanda and thank them for their time 
and thoughtful inputs. We thank Daniel Svanlund, Monitoring and Evaluation Officer, 
for his support and dedication as evaluation manager, as well as Amy Blauman and 
MarieClaire Gatera who served as focal points from inception through the fieldwork 
and reporting phases. We also thank Edith Heines, WFP Representative and Country 
Director, and all the management staff who gave valuable time and feedback. All 
extended their support and hospitality to the team and assisted us with our many 
requests. In addition, we extend our appreciation to the Government of Rwanda 
officials, United Nations partners, WFP cooperating partners and the donors, who all 
provided valuable input. Dez Byamukama, World Vision Deputy Chief of Party, 
provided crucial leadership to the work of the World Vision literacy team. Mary 
Kulabako and the literacy team at World Vision Rwanda were generous with their time 
and expertise, contributing to the development of the literacy assessment tool, training 
the field team on its application, and adding insight to assessment results. We wish to 
acknowledge the generosity and hospitality of the government staff who took their 
time to explain how they carry out their jobs and to share their perspectives on this 
programme. Finally, we thank the communities and households who shared their 
views and experiences with us. It is our sincere hope that the people with whom WFP 
and the government work will benefit from this evaluation and experience positive 
gains in their lives as a result of implementing its recommendations.
Disclaimer
The opinions expressed in this report are those of the Evaluation Team, and do not
necessarily reflect those of the World Food Programme. Responsibility for the
opinions expressed in this report rests solely with the authors. Publication of this
document does not imply endorsement by WFP of the opinions expressed.
The designation employed and the presentation of material in maps do no imply the
expression of any opinion whatsoever on the part of WFP concerning the legal or
constitutional status of any country, territory or sea area, or concerning the
delimitation of frontiers.
WFP Rwanda USDA McGovern-Dole Midterm Evaluation Report ii
|Page
Table of Contents
Executive Summary ................................................................................ v
1. Introduction........................................................................................ 1
1.1 Overview of the Evaluation Subject....................................................................... 1
1.2 Context................................................................................................................... 5
1.3 Evaluation Methodology and Limitations ............................................................ 7
2. Evaluation Findings .......................................................................... 12
2.1 Relevance of the Operation................................................................................. 12
2.2 Results of the Operation ..................................................................................... 19
2.3 Factors Affecting the Results ..............................................................................40
3. Conclusions and Recommendations..................................................44
3.1 Overall Assessment/Conclusions........................................................................44
3.2 Recommendations ..............................................................................................48
Annexes ................................................................................................ 51
Annex 1: Terms of Reference .................................................................................... 51
Annex 2: McGovern-Dole Program Objectives and Activities ................................. 76
Annex 3: Map............................................................................................................ 77
Annex 4: Stakeholder Analysis .................................................................................78
Annex 5: Additional Relevant Interventions (non-USDA-funded) ........................ 80
Annex 6: Results Framework and Foundational Results......................................... 81
Annex 7: Government policies and priorities related to food security, social 
protection and education..........................................................................................85
Annex 8: International Assistance to Rwanda.........................................................87
Annex 9: Evaluation Matrix..................................................................................... 88
Annex 10: List of Persons Interviewed and Focus Groups ......................................96
Annex 11: Topical Outlines .....................................................................................100
Annex 12: Field Schedule........................................................................................ 109
Annex 13: Methodology ........................................................................................... 111
Annex 14: Supplementary Tables ........................................................................... 123
Annex 15: Bibliography........................................................................................... 130
List of Acronyms ................................................................................. 133
WFP Rwanda USDA McGovern-Dole Midterm Evaluation Report iii
|Page
Table of Tables
Table 1: McGovern-Dole Program overview.................................................................. 2
Table 2: Criteria and evaluation questions .................................................................... 8
Table 3: McGovern-Dole Program beneficiary output indicatorsa ..............................20
Table 4: Attendance, actual vs planned, FY 2018........................................................20
Table 5: USDA outcome indicators..............................................................................23
Table 6: Reading, listening and comprehension of short story at midterm ...............25
Table 7: Words correct per minute (detailed), third grade .........................................26
Table 8: Reading comprehension, third grade (all students to read and/or listen)...26
Table 9: Reading comprehension, third grade (only students to NOT complete 
reading; passage read to them)....................................................................................27
Table 10: Reading comprehension, third grade (only students to complete reading 
passage)........................................................................................................................27
Table 11: Reading performance for third-grade students at midterm ........................28
Table 12: Reading outside of school (third grade) at midterm....................................30
Table 13: Student health and hygiene knowledge, baseline and midterm..................32
Table 14: School water source and availability at midterm.........................................33
Table 15: Sources of food for school meals, at midterm..............................................34
Table 16: School PTA engagement at midterm............................................................36
Table 17: External support to schools at midterm.......................................................38
Table 18: McGovern-Dole Program objectives and activities .....................................76
Table 19: Stakeholder analysis and mapping ..............................................................78
Table 20: Government policies and priorities related to food security, social 
protection and education .............................................................................................85
Table 21: Overview of main data collection tools ....................................................... 111
Table 22: Schools visited and number of third-grade respondents at midterm........ 114
Table 23: Head teacher survey respondents............................................................... 115
Table 24: Anticipated methodological limitations and mitigation strategies; 
comments post-evaluation.......................................................................................... 117
Table 25: Re-estimated values for reading comprehension...................................... 120
Table 26: Re-estimated data for EGRA results.......................................................... 120
Table 27: Ethical issues and safeguards implemented............................................... 121
Table 28: Student reading and comprehension (third-grade), baseline and midterm
.................................................................................................................................... 123
Table 29: Words correct per minute (detailed), third-grade, by district .................. 123
Table 30: Reading and comprehension at midterm, by district ............................... 124
Table 31: Students' home support and time allocation at midterm.......................... 124
Table 32: Student attentiveness, by grade, baseline and midterm ........................... 125
Table 33: School administrator use and application of teaching techniques at 
midterm...................................................................................................................... 125
Table 34: Health and hygiene practices at midterm, by district ............................... 126
Table 35: Health and hygiene practice utilisation/application at midterm.............. 126
Table 36: Parents' knowledge of education benefit and storekeepers' knowledge of 
good practices, baseline and midterm ....................................................................... 127
Table 37: School meals distributed, actual versus planned, FY 2018 ....................... 128
Table 38: Commodity distributions, expected v. actual ............................................ 129
WFP Rwanda USDA McGovern-Dole Midterm Evaluation Report iv
|Page
Table of Figures
Figure 1: McGovern-Dole Program coverage in Rwanda............................................. 77
WFP Rwanda USDA McGovern-Dole Midterm Evaluation Report v 
|Page
Executive Summary 
1. Overview. This is an independent midterm evaluation of World Food Programme 
(WFP) Rwanda’s Home-Grown School Feeding (HGSF) Program 2016-2020, which is 
funded primarily by the United States Department of Agriculture (USDA) McGovern￾Dole International Food for Education and Child Nutrition Program.1 The evaluation 
was commissioned by WFP Rwanda and conducted by TANGO International. With the 
dual objectives of accountability and learning, the evaluation aims to (1) assess the 
program against OECD-DAC criteria: relevance, effectiveness, efficiency, impact, and 
sustainability, (2) collect performance indicator data for strategic objectives and 
higher-level results, (3) assess whether the program is on track to meet targets, and 
(4) review the results framework and theory of change. This report provides evidence￾based findings to inform operational and strategic decision-making and identify mid￾course corrections for action by WFP Rwanda and its partners. 
2. The evaluation also reports and examines indicators required by McGovern Dole. The 
main evaluation questions, as indicated in the Terms of Reference, are: 
• Have literacy rates of school-age children improved over the duration of the 
program? 
• Has the use of health and dietary practices increased? 
• What is the level of community-level involvement and participation in decision￾making in school governance mechanisms? 
• What are the key institutions and governance structures required to effectively 
deliver, implement, and sustain school meal interventions? 
3. The Evaluation Team (ET) has also paid special attention to the WFP Country Office’s 
interests in gaining insight into the status of capacity development efforts with the 
national government and how it can work more strategically in this respect.
4. The evaluation covers activities from program start in January 2016 through June 
2018, and spans all four districts of implementation. 
5. The intended primary users of the evaluation are the WFP Rwanda Country Office
(CO), implementing partners World Vision, Gardens for Health International (GHI), 
and Rwanda Biomedical Centre, to understand program performance to date and 
obtain insights to inform adjustments; the Rwanda Ministry of Education
(MINEDUC), and the Ministry of Agriculture and Animal Resources (MINAGRI) to 
check program alignment with government priorities, especially around transition and 
sustainability; donors USDA and MasterCard, to learn about program results and 
inform future investments in Rwanda and beyond; and WFP Regional Bureau Nairobi, 
headquarters, Office of Evaluation, and Executive Board, for wider organisational 
learning and accountability. stopped
6. The evaluation used a mixed-methods approach incorporating primary and secondary 
data at national, subnational and school levels. This included a desk review, head 
teacher survey, Early Grade Reading Assessment (EGRA) survey, qualitative fieldwork 
with key informants and focus groups, direct observation of program activities, and 
examination and triangulation of quantitative data from WFP and partner monitoring 
reports and databases. Quantitative EGRA data were statistically powered to provide 
accurate point estimates of student literacy and water, sanitation and hygiene (WASH) 
data points. Limitations included challenges with baseline and midterm data
1 The program in Rwanda is hereafter referred to as the “McGovern-Dole Program.”
WFP Rwanda USDA McGovern-Dole Midterm Evaluation Report vi
|Page
comparability, the absence of established benchmarks for reading and comprehension 
skills, and the inability to conduct a systematic parent survey (which was also a 
limitation at baseline). Measures were taken to mitigate these limitations as far as 
possible, including re-estimation of midterm values based on EGRA baseline data, 
triangulation using qualitative data, reference to provisional literacy standards from 
similar studies, and incorporation of questions on parent indicators in interviews with 
school personnel and with parents interviewed with respect to their other roles in the 
program (e.g., cooks, farmer cooperative members). 
7. Country Context. The McGovern-Dole Program targets districts in the south and 
west of Rwanda with some of the highest rates of poverty and extreme poverty in the 
country (based on 2014 figures). 2 These areas are also characterized by severe food 
insecurity and high stunting rates. Primary school enrolment is high, though the 
quality of education in rural areas is generally poor and mean years of schooling is 3.8, 
which is categorized on the SDG dashboard as “stagnating.”3 Women’s empowerment 
and gender equality have improved in recent years, including high rates of girls’ 
primary school enrolment.
8. Program Overview. The McGovern-Dole Program, implemented in 104 schools, 
provides daily hot meals to 40,000 primary school children in Nyamagabe and 
Nyaruguru districts, Southern Province, and 43,000 in Karongi and Rutsiro districts, 
Western Province. The meals are intended to fulfil a significant portion of daily 
nutritional requirements, reduce micronutrient deficiencies, and improve iron uptake 
when combined with de-worming medications. The food commodities are sourced as 
in-kind food commodity transfers from the United States and local and regional 
purchase. The aims of the WFP-implemented McGovern-Dole Program are to support 
the government in strengthening its national school feeding program and build 
government capacity to incorporate some or all of the McGovern-Dole Program
activities into its national program by 2020. The program is funded by USDA 
McGovern Dole for US$25 million over the life of the program (2016-2020), with 
additional funding from MasterCard, Feed and Caterpillar foundations.
9. Relevance and Coherence. The McGovern-Dole Program design is aligned with 
national policies and direction, WFP corporate strategic objectives, and United￾Nations-wide, system-wide commitments and other ongoing WFP operations, 
specifically, SDG 2 Zero Hunger; SDG 4 Quality Education; SDG 17 Partnerships for 
Goals; and the Government of Rwanda’s Vision 2020; School Health and Nutrition 
Policy; Social Protection Sector Strategy, Education Sector Strategy, among others. 
The program supports WFP’s Gender Policy but the original design does not include a 
specific approach to address gender equality and women’s empowerment; the 
inclusion of a GEEW approach in the program has been discussed but has not yet been 
implemented. 
10. Results of the Operation. Progress on program targets for component 
interventions is on track. 
11. Literacy and Teaching Methods. World Vision utilizes the Literacy Boost 
instruction methodology. At midterm, about 60 percent (against the 69 percent 
endline goal) of the students demonstrate the reading and comprehension 
competencies tested by the EGRA, and the percentage of students who are able to read 
and comprehend grade level text has increased to over half of the endline target. The 
2 NISR. 2015. Rwanda Poverty Profile Report 2013/14. 3 SDG Center for Africa and Sustainable Development Solutions Network. 2018. 
WFP Rwanda USDA McGovern-Dole Midterm Evaluation Report vii
|Page
main contributing factors to reading improvements noted by students, teachers and 
administrators were the print-rich classroom environments and stronger teacher 
expertise to work with both slow and fast learners using a wide range of teaching aids 
promoted by the program. Indicators for administrators, officials, teachers, and 
educators (in all 104 schools) have been exceeded by over double the target number of 
individuals demonstrating new tools and techniques at midterm. Similarly, all parents 
who had children enrolled in community reading clubs were trained on the importance 
of literacy, exceeding the target number by 4,710. 
12. Health and Dietary Practices. At midterm, students have shown minimal 
progress on their ability to name three target health and hygiene practices. Drinking 
water, latrine facilities, and school gardens were present at all schools, though the 
quality and usage of the water and garden components was mixed. Rainwater 
harvesting tanks have been installed to improve water access, though a reliable year￾round availability of water remains a challenge for many schools. The latrines installed 
by World Vision and approved by the government were of better quality than those 
built prior to the program, and include gender-specific facilities and separate toilets 
for disabled students. 
13. Community Participation and School Governance. Parent appreciation of the 
value of education is estimated to have increased. This is important as under the 
program model, communities are expected to contribute cash, fuel, labour, and food. 
The ET observed that while this contribution model was operating, it is facing 
challenges as only 40 – 50 percent of the requested cash contribution is being met. 
Some of the reasons for the shortfall cited by school heads include: many parents see 
school feeding as the responsibility of the school or of WFP, some distrust the school’s 
management of the money, and some households find it difficult to contribute when 
they have several children in school.
14. Capacity of Local Institutions. In terms of implementation of the McGovern￾Dole Program, there is strong support from program partners and others for the 
components of its integrated approach to HGSF. Current local institutional capacity to 
oversee and implement is generally good. There is healthy support in program schools 
for core elements of an enabling learning environment. This support comes 
predominantly from WFP and its partners, including World Vision, Government of 
Rwanda, MINEDUC, UNICEF, GHI, and MINISANTE. At national level, the policy 
framework for school feeding is in process, and greater capacity to manage the planned 
expansion of the national school feeding program can be strengthened though
government capacity to finance such a program is unclear. 
15. Factors Affecting the Results. Among the internal factors contributing to the 
success of the McGovern-Dole Program is WFP’s selection of partners with strong 
technical and implementation capacity. All partners reported that they were satisfied 
with WFP management of the program. There is regular interaction between WFP, 
partners and schools. Local coordination with the District Education Officers is good, 
and coordination and communication between WFP and other stakeholders at the 
district and local level is strong. Staffing levels are sufficient at operational level but 
insufficient at national level. The government and WFP are in the process of defining 
local procurement options for school feeding, which is key to scaling up the national 
school feeding program and informing McGovern-Dole support to this process. A local 
procurement strategy is being examined by a multi-sectoral task force consisting of 
MINAGRI, MINEDUC, MINECOM and WFP. Among the external factors contributing 
to McGovern-Dole Program performance is that schools are generally well organized 
WFP Rwanda USDA McGovern-Dole Midterm Evaluation Report viii
|Page
with regular support from district officials. A good enabling policy environment exists 
as school feeding is endorsed is the strategic plans of MINEDUC and MINAGRI, and 
MINISANTE’s strategic plan supports school activities on sanitation, hygiene, and 
nutrition education. However, at the time of the MTE, the government had yet to 
approve the school feeding policy and strategic plan, which have been pending for two 
years. The ET was told by MINEDUC that a new policy to scale up school feeding was 
pending approval by the Cabinet. Government capacity at the subnational level to 
manage and monitor the McGovern-Dole Program is strong, with good systems and 
communications between district education officials and schools. However, the 
capacity of the government at national level to manage both the McGovern-Dole 
Program and the national school feeding program is weak, and there are no 
operational monitoring and evaluation and communication systems between schools, 
districts and the national level.
16. Relevance, Effectiveness, Efficiency, Impact, Sustainability. The McGovern￾Dole Program is highly relevant. It is consistent with and aligned with government and 
WFP policies and priorities. It is targeted to the most food insecure districts and at 
midpoint is demonstrating effectiveness. The program design assumes effective and 
efficient coordination between government levels, which interviews show is not always 
present. The program design assumes that parent contributions will play a significant 
part in supporting the costs of this school feeding model. During the qualitative survey, 
school heads reported that they are struggling to raise funds from parents, and that 
parents are not contributing to the extent expected by the program. The majority of 
schools have not faced food delivery or food shortage issues. Stock management and 
storage activities in schools were in line with WFP standard operating procedures, and 
WFP field monitors successfully managed food quality and minimized waste. WFP’s 
assistance to farmer cooperatives to develop improved agricultural and business 
practices is an important contribution to the sustainability of the McGovern-Dole 
Program. Sustainability will also depend on government readiness and capacity to 
scale up the existing national program for secondary schools to include primary 
schools by 2024. The incorporation of the McGovern-Dole Program components into 
a scale-up of the national school feeding program presents a budgetary challenge for 
the government. It may entail WFP support for school feeding in the most food 
insecure areas for five to seven more years before the government has the funding and 
capacity to incorporate some or all of the activities that are currently supported by 
USDA.
17. Conclusion. The McGovern-Dole Program has made a good start in establishing 
consistent, integrated, and collaborative operations and in successfully implementing 
and monitoring a multi-faceted program. At midpoint, WFP and the government are 
building the systems and linkages needed for a successful and sustainable program. The 
McGovern-Dole Program’s integrated activities are appropriate to local needs. Progress 
on program targets for component interventions is on track. It will take another five￾year phase to establish the systems, capacities and resources necessary for government 
to integrate some or all of the McGovern-Dole Program activities into the national HGSF 
program. The USDA model is different from the government school feeding program
and it has yet to be determined what components of the McGovern-Dole Program will 
eventually transition to government. Evidence from the program can provide useful 
lessons and insights as the government scales up its national school feeding program. 
The program now needs to move forward to develop a sustainability strategy for the 
selected activities with the government, financed and fully managed by the government. 
WFP Rwanda USDA McGovern-Dole Midterm Evaluation Report ix
|Page
This requires, in addition to a strategy, a road map that lays out specific responsibilities 
and targets for stakeholders. 
Operational Recommendations (2019 – 2020)
18. Recommendation 1: Strengthen WFP management, role clarity and staff 
capacity for functions related to the McGovern-Dole program. This includes 
strengthening current management oversight and senior manager engagement in 
advocacy on relevant McGovern-Dole Program elements to support the government’s 
school feeding expansion.
• High priority; short term (2019 first quarter); CO responsible
19. Recommendation 2: Organize reflection meetings to inform knowledge 
management, advocacy, and strategic thinking. The meetings should focus on 
how the McGovern-Dole Program can best contribute to the expansion of the national 
school feeding program. 
• High priority; short term (begin first quarter 2019); CO responsible
20. Recommendation 3: Contribute to the development of a school kitchen 
model that integrates primary and secondary school kitchen infrastructure 
and can still be supported by parent and community contributions. 
• Medium priority; medium term (by end 2020); CO responsible
Strategic Recommendations (2019 and beyond)
21. Recommendation 4: Initiate a structured transition or continuation 
planning process with the Government. Based on the results of this process, 
WFP can initiate a transition of relevant McGovern-Dole Program elements into the 
national program at the end of the current McGovern-Dole Program in 2020, and/or 
engage with Government to design a Phase 2 to the current McGovern-Dole Program 
that provides tailored support to the school feeding expansion process. 
• High priority; short term to end of program (2019 first quarter); CO responsible
WFP Rwanda McGovern-Dole Midterm Evaluation Report 1 
|Page
1. Introduction 
1. This is the report of an independent midterm evaluation of World Food Programme 
(WFP) Rwanda’s Home-Grown School Feeding (HGSF) Program 2016-2020, which is 
funded primarily by the United States Department of Agriculture (USDA) McGovern￾Dole International Food for Education and Child Nutrition Program. 4,5 The evaluation 
was commissioned by WFP Rwanda to provide evidence and an independent assessment 
of the program’s performance. It has the dual objectives of accountability – to assess and 
report on program performance and results, and learning – to determine why certain 
results occurred, and to draw lessons and derive good practices for learning. The 
evaluation was conducted by TANGO International. 
2. The MTE aims to (1) review program relevance, effectiveness, efficiency, impact, and 
sustainability, (2) collect performance indicator data for strategic objectives and higher￾level results, (3) assess whether the program is on track to meet targets, and (4) review 
the results framework and theory of change. It will provide evidence-based findings to 
inform operational and strategic decision-making, and identify any mid-course 
corrections for action by WFP Rwanda and its partners. Findings will be disseminated 
and lessons incorporated into relevant information-sharing systems. 
3. The stakeholders and intended users of the MTE are as follows:
• WFP Rwanda Country Office (CO), and its implementing partners World Vision, 
Gardens for Health International (GHI), and Rwanda Biomedical Centre – to 
adjust course as necessary for the remainder of the program term;
• Ministry of Education (MINEDUC) and Ministry of Agriculture and Animal 
Resources (MINAGRI) – to learn whether the program is performing well and is 
aligned with their priorities, particularly in terms of capacity development, 
handover and sustainability.
• United States Department of Agriculture (USDA) (main donor) – to learn whether 
the program is performing well. USDA may use findings and lessons to inform 
McGovern-Dole program funding, design, and implementation decisions.
• MasterCard (donor) – may use the findings to inform its decision on the best 
models of school feeding to help target its funding.
• WFP Regional Bureau (RB) Nairobi – to provide strategic guidance, program
support, and oversight.
• WFP headquarters (HQ) and the Office of Evaluation (OEV) – may use the 
evaluation for wider organisational learning and accountability.
• WFP OEV may use the evaluation findings to feed into evaluation syntheses and 
for annual reporting to the Executive Board.
4. The MTE is timed midway through the program, from its start in January 2016 through 
June 2018, and covers all four districts where the program is implemented.
1.1 Overview of the Evaluation Subject
5. WFP Rwanda has been implementing the HGSF program funded by the United States 
Department of Agriculture McGovern Dole (hereafter referred to as “McGovern-Dole 
4 USDA McGovern-Dole Grant FFE-696-2015/007-00
5 The program in Rwanda is hereafter referred to as the “McGovern-Dole Program.”
WFP Rwanda McGovern-Dole Midterm Evaluation Report 2 
|Page
Program) since October 2015. WFP provides a midday meal to 40,000 children in grades 
1 to 6 in Nyamagabe and Nyaruguru districts in Southern Province consisting of 120 
grams (g) of maize, 30g beans, 15g vitamin A fortified vegetable oil, and 3g iodised salt. 
In Karongi and Rutsiro districts in Western Province, a different meal consisting of 120g 
of Supercereal and 15g of sugar is provided to 43,000 primary school students. The food 
commodities selected for the program are intended to fulfil a significant portion of each 
student’s daily nutritional requirements, reduce micronutrient deficiencies, and improve 
iron uptake when combined with deworming medications, as there is a high prevalence 
of anaemia among school children in Rwanda. 
6. HGSF is a modality for school feeding used by WFP that provides food produced and 
purchased within a country to the extent possible. The McGovern-Dole International 
Food for Education and Child Nutrition Program (McGovern-Dole) is a USDA program
administered by the Foreign Agricultural Service (FAS) that delivers international school 
feeding using donated in-kind commodities, usually supplied by the United States, and 
also delivers financial and technical assistance. This McGovern-Dole Program in Rwanda, 
implemented by WFP, uses both modalities for procurement to deliver school meals. 
USDA awarded a Local and Regional Procurement (LRP) agreement to WFP in FY16 
meant to complement the McGovern-Dole Program; the latter LRP project is not the 
subject of this evaluation, though local and regional procurement approaches, more 
broadly speaking, are discussed.
7. WFP is implementing the McGovern-Dole Program with the aims of supporting the 
government in developing a national school feeding program and building government 
capacity for complete handover of activities by 2020. Table 1 provides a program 
overview. Annex 2 lists the activities for each objective. 
Table 1: McGovern-Dole Program overview
Operation: McGovern-Dole 
Program overview Approval date: Proposal submitted FY 2015
Duration: 1 October 2015 – 30 
September 2020
Budget: USDA McGovern Dole: US$25 million over five years 
(US$5,281,900 in FY16 and $4,929,525 in each subsequent 
year through FY20);6 additional funding from MasterCard, 
Feed and Caterpillar Foundation 
Geographic areas: 104 schools (target: 83,106 primary school students) in Nyaruguru and 
Nyamagabe districts (daily hot meals) and Rutsiro and Karongi districts (daily porridge meals) 7 (map, 
Annex 3)
Thematic areas: School feeding & education, WASH, health & dietary practices, national capacity 
building
Transfer modality: In-kind food transfers: US food commodities and local and regional food 
purchase 
6 The programme received from MasterCard US$2 million for Years 1 and 2 and an additional US$1.25 million for 
Year 3, and small donations from Feed and Caterpillar Foundation in Year 1 totalling <US$200,000. Budget figures 
reported for USDA, MasterCard, Feed and Caterpillar Foundation are from email correspondence with the CO (8 Aug 
2018). We note minor discrepancies between those figures and budget documentation. The WFP Rwanda FY 2015 
McGovern-Dole proposal specifies a life-of-project budget of US$26,931,200. The WFP Programme Cost Sheet for
HGSF for FY 2015 indicates a total programme cost of US$22,576,643; the Programme Cost Sheet for Year 2 
indicates a total programme cost of US$38,519,927.
7 These are two food baskets: the hot meal uses maize, beans, salt and fortified oil; the porridge meal uses Supercereal 
(CSB+) and sugar.) Some schools occasionally provide locally grown vegetables to enrich the meals.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 3 
|Page
Key partners
Government of Rwanda: 
-- MINEDUC; MINAGRI; MINALOC; Ministry of Health (MINISANTE); Rwanda Biomedical Centre 
(RBC) (deworming)
United Nations: 
--UNICEF coordination: educational standards and national guidelines
--UNDAP coordination: increasing access to quality education, health, nutrition, WASH
NGOs: 
--World Vision: sub-recipient on literacy, health, WASH
--Adventist Development and Relief Association (ADRA): building kitchens, storerooms and fuel￾efficient stoves in the supported schools (ADRA was only a partner during Phase I of kitchen 
construction in schools in the Western Province) 
Gardens for Health International (GHI): school gardens
--WFP-supported smallholder farmer cooperatives trained to increase their marketable surplus and 
link to McGovern-Dole Program
8. Partners. Various government ministries play significant roles in the design and 
implementation of the HGSF. Most fundamentally, MINEDUC leads the education sector 
including policy formulation, planning, coordination, regulation, monitoring and 
evaluation.8 It works closely with the semi-autonomous Rwanda Education Board (REB), 
which oversees the coordination and implementation of pre-primary, primary and 
secondary education activities (see discussion at para. 60). The Ministry of Local 
Government (MINALOC) oversees schools and teachers and also has an oversight role in 
the government social protection and poverty reduction initiative, the Vision 2020 
Umurenge Programme (VUP), which targets schools in the poorest areas (details in Table 
20). In addition, the McGovern-Dole Program works with the Ministry of Health 
(MINISANTE), which has complementary programs to reduce primary dropout rates of 
girls and promote community‐based environmental health, which is central to water, 
sanitation and hygiene (WASH) activities. The program partners with MINAGRI on its 
work with 24 smallholder agricultural cooperatives,9 where farmers are given training in 
developing business plans and coached on cooperative governance, financial 
management and access to markets. 
9. WFP works with non-governmental organisations (NGO) for specialised program 
components. World Vision Rwanda, the largest NGO in Rwanda, focuses on literacy, 
health and WASH. Along with MINEDUC and Save the Children, World Vision piloted 
the Literacy Boost literacy instruction methodology (see para. 79) and is globally 
recognised for its education programming. Other implementing partners are GHI for 
school gardens, and Rwanda Biomedical Centre, which distributes deworming medicine. 
10. The program’s main United Nations agency partner is UNICEF, whose Child‐Friendly 
School standards Rwanda has adopted as quality guidelines for school infrastructure and 
software inputs; UNICEF also supported the Learning Achievement in Rwandan Schools 
Assessment to improve the quality of education and measure learning outcomes in literacy 
and numeracy.10 The McGovern-Dole Program is also considered in the joint UN 
8 Evaluation ToR.
9 WFP. 2018b. McGovern-Dole Programme Semi-Annual Report Narrative, October 1, 2017 – March 31, 2018. 10 Evaluation ToR. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 4 
|Page
Development Assistance Programme (UNDAP) 2018‐2023, which focuses on increased 
and equitable access to quality education, health, nutrition and WASH services. 
11. Annex 5 contains information on other (non-USDA-funded) relevant programs 
implemented by these partners and others. 
12. Changes in program design. WFP’s implementation of the program is consistent with 
the original program design. The ET notes that the program had initially planned to install 
lower-cost kitchen facilities to allow for the construction of a greater number of kitchens. 
However, the design did not consider the need for maintenance and training on 
maintenance. When quality problems arose in the lower-cost kitchens, the program opted 
for a revised design at higher cost. See also, relevant discussions at para. 98 and para. 117. 
13. Relevant issues from past evaluations. The ET had access to two evaluations, the 
2013-2016 Common Country Programme (CCP) evaluation11 and the McGovern-Dole 
Program baseline.12 Though the CCP evaluation was carried out while the McGovern-Dole 
Program was relatively new, two of the recommendations are relevant to this evaluation. 
First, it recommended that the CO draft a capacity development strategy for portfolio 
activities to guide implementation (Recommendation 1) and feed into the next Country 
Strategic Plan (CSP). While the CCP recommendation does not specifically address the 
McGovern-Dole Program, the ET did identify a need for WFP to undertake capacity 
development of government partners as part of a longer-term strategy for sustainability. 
Recommendation 6 in the CCP was to carry out a systematic analysis of key areas of 
learning to inform choices around approach and strategies for the next CSP. The current 
evaluations makes a similar point, but specific to the McGovern-Dole Program, 
recommending that the CO assess strengths and shortcomings in the approach and 
prioritise what is working well. In terms of operational issues, the baseline concluded that 
most of the targeted schools have access to an improved water source, but that the water 
supply was unreliable. While provision of water has expanded under the McGovern-Dole 
Program, the same finding applies in the MTE (see para. 51 and para. 126). The ET notes 
that WFP and MINEDUC have advocated for an improved water supply to primary schools 
and that the Ministry of Infrastructure (MININFRA) has committed to prioritizing 
connecting McGovern-Dole Program schools to the main water grid.13 
14. Gender. The Education Sector Strategic Plan 2013-2018 emphasises access to learning 
for disadvantaged students such as girls, the poor and disabled.14 Primary school 
enrolment is high, and gender equity has been largely achieved in primary and secondary 
schools. In 2016, the net enrolment rate for girls was 98.0 percent and 97.3 percent for 
boys. Girls comprised 50.1 percent of enrolled primary school students and boys 
accounted for 49.9 percent in 2016. The primary school completion rate in 2016 was 
higher for girls than boys, at 71.1 percent and 59.3 percent, respectively. The number of 
female students is higher in pre-primary, primary and secondary levels but shifts at 
higher levels, with male students predominating at tertiary level (60.6 percent males, 44.6 
11 WFP. 2017c. Operation Evaluation. Rwanda Common Country Programme 200539, Mid-Term Evaluation (2013-
2016). Evaluation Report. Prepared by IRAM. 12 Ipsos, Inc. 2016. Baseline Study: Home Grown School Feeding Program 2016-2020. July. Report authors P. Mukiri 
and A. Kaburu.
13 WFP. 2018b.
14 Republic of Rwanda, Ministry of Education. 2013. Education Sector Strategic Plan, 2013/14 – 2017/18
WFP Rwanda McGovern-Dole Midterm Evaluation Report 5 
|Page
percent females).15 The education sector plan also states that there is a dedicated budget 
line item to address education barriers for girls, including the provision of gender￾sensitive water and sanitation facilities. 
15. The McGovern-Dole Program is guided by the WFP 2009 Gender Policy (later subsumed 
by the WFP 2015-20 Gender Policy).16 The program promotes the participation of girls 
and indicators data are disaggregated by gender. However, the program proposal does 
not include a specific approach to Gender Equality and Women’s Empowerment (GEEW) 
or sexual and gender-based violence as noted in the 2009 policy; including a GEEW 
approach in the program was discussed in 2017 but has not yet been implemented. World 
Vision is working with district governments to improve latrine coverage in primary 
schools, which supports increased access to clean water, improved sanitation, and 
improved hygiene of all school‐aged children with an emphasis on hygiene issues affecting 
girls (MGD 2.4).17 
1.2 Context 
Poverty and food security in relation to the subject of the evaluation
16. Rwanda has made substantial progress toward development goals and reducing poverty 
since the 1994 genocide. It was ranked 159th out of 188 countries in the 2015 Human 
Development Index with a value of 0.498, placing it in the low human development 
category.18 Poverty decreased between 2011 and 2014 from almost 45 percent to just over 
39 percent;19 extreme poverty declined from 24.1 to 16.3 percent.20 In 2014, districts in 
the south and west had the highest rates of poverty and extreme poverty. Districts 
targeted in this program—Karongi, Rutsiro, Nyaruguru and Nyamagabe—had some of the 
highest poverty levels. From 2011 – 2014, poverty declined in three of the districts but 
increased in Rutsiro from 46.8 to 51 percent; extreme poverty decreased in all four 
districts with the largest decrease in Nyamagabe, from 34.6 percent to 13.3 percent.
17. The government has set the goal of moving from low-income to lower-middle-income 
status by 2020 and to upper-income status by 2050. Through its Social Protection Sector 
Strategy (2018-2024), the government has committed to providing a life-cycle approach 
to social protection systems. Income inequality is decreasing as the economy grows, which 
strengthens the possibility that the government will achieve its goals and self-finance 
social protection, including access to education and food security safety nets for all. 
Rwanda hosts over 150,000 refugees from the Democratic Republic and Congo and 
Burundi, who either reside in camps or in urban areas, primarily Kigali;21 the government 
committed in 2016 to integrate refugees into national health insurance, education, and 
documentation systems.22 Good governance and strong institutions are important 
15 Republic of Rwanda, Ministry of Education. 2016 Education Statistical Yearbook.
16 Evaluation TOR.
17 WFP Rwanda FY 2015 McGovern-Dole proposal.
18 UNDP. 2016. Human Development Report 2016: Human Development for Everyone. 
19 National Institute of Statistics Rwanda (NISR). 2015. Rwanda Poverty Profile Report 2013/14 – Results of 
Integrated Household Living Conditions Survey (EICV4). 20 NISR. 2015. Rwanda Poverty Profile Report 2013/14. 21 UNHCR. 2018. Operational Update: Rwanda, August 2018. 22 UNHCR. ND. Operations Plan: Rwanda 2018. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 6 
|Page
prerequisites for government leadership in the area of school feeding, and this can guide 
future WFP country strategy toward stronger capacity development.
18. The 2015 Comprehensive Food Security and Vulnerability Analysis (CFSVA) found that 
20 percent of households were food insecure (473,847 households).23 Among those, 13 
percent or 63,696 of households were severely food insecure. The highest incidence of 
food insecurity was in western and northern Rwanda, and stunting rates were highest in 
the West (46 percent). The four districts targeted by the McGovern-Dole Program are 
among those with the highest percentages of food insecure households: Rutsiro (57 
percent), Nyamagabe (42 percent), Nyaruguru (37 percent) and Karongi (35 percent).
Key data and trends related to relevant SDGs
19. SDG 2 – Zero hunger. Rwanda is maintaining progress toward SDG targets for wasting 
(3.0 percent) and obesity (5.4 percent).24 Undernourishment and stunting rates are 
severe, at 41.1 percent and 44.3 percent, respectively, and the SDG dashboard shows 
stunting as “moderately increasing.” Between 2012 and 2014, annual crop production 
increased by 5.7 percent, higher than the population growth rate (2.5 percent).25 
20.SDG 4 – Quality education. Net primary enrolment in Rwanda is 95.9 percent.26Mean 
years of schooling is 3.8, an indicator value characterized in the SDG dashboard as 
“stagnating” and indicating major challenges. The literacy rate for 15-24 year olds, both 
sexes, is 82.3 percent. 
21. SDG 17 – Partnerships for the goals. Rwanda has made progress or achieved two of 
three applicable indicators for SDG 17: Government Health and Education spending (12.7 
percent of GDP) and the Tax Haven Score of zero, which is the best possible.27 Tax 
revenue, at 19.3 percent of GDP, is moderately increasing.28
Gender and health dimensions of the context and the evaluation subject
22.Gender-based poverty is a characteristic of poor rural households in Rwanda. Female￾headed households in Rwanda, comprising 27 percent of households, are more likely to 
be food insecure than those headed by men. 29 In two of the districts targeted by the 
McGovern-Dole Program, Nyamagabe and Nyaruguru, 29 percent of households are 
female-headed. Women tend to engage in low-paid or unpaid work such as subsistence 
agriculture or household labour, and it is common for female household heads to be 
widowed and less educated than men. Factors that increase the risk of stunting in children 
include mothers who are stunted, have low levels of education, and do not receive 
antenatal care. Fortunately, as of 2015, only three percent of women were stunted, and 96 
percent of pregnant women received antenatal care. Anaemia affects 19 percent of 
women. Moreover, 18 percent of women of reproductive age do not have any education – 
23 MINAGRI, NISR, WFP. 2015. Rwanda Comprehensive Food Security and Vulnerability Analysis. March 2016.
24 SDG Center for Africa and Sustainable Development Solutions Network. 2018. Africa SDG Index and Dashboards 
Report. July.
25 NISR. 2015. Rwanda Poverty Profile Report 2013/14--Results of Integrated Household Living Conditions Survey 
(EICV4).
26 SDG Center for Africa and Sustainable Development Solutions Network. 2018. 27 Sachs, J., et al. 2018. SDG Index and Dashboards Report 2018.
28 SDG Center for Africa and Sustainable Development Solutions Network. 2018.
29 MINAGRI, NISR, WFP. 2015.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 7 
|Page
a characteristic that is likely to change given the high rates of primary and secondary 
enrolment and completion for girls. 
23.Nationally, stunting is steadily decreasing; it dropped from 43 percent in 2012 to 37 
percent in 2015. 30 Diets tend to be low in nutrients and protein, and only 15 percent of 
children aged 6 to 23 months have a minimum acceptable diet. Stunting is more prevalent 
among boys under age five (41 percent) than girls (33 percent). Poor access to safe water, 
sanitation, and health services underlay the high level of malnutrition.
24.A reliable supply of potable water is essential to good health and normal growth among 
children, who are especially vulnerable to illness from unsafe water and poor sanitation. 
Nationally, 25 percent of Rwandans do not have access to safe drinking water for their 
households, especially in rural areas. As of 2016, only 28.8 percent of all primary schools 
in Western province had access to tap water and 40.7 percent have access to seasonal rain 
water harvesting, which poses challenges during the dry season. In Southern province, 
30.8 percent of all primary schools had access to tap water and 45.9 percent have access 
to rain water harvesting systems. The Education Sector Strategic Plan 2013-2018 includes 
school health initiatives to promote good hygiene, to strengthen school nutrition 
programs to address poverty-driven hunger, and prioritises providing schools with access 
to electricity and water. 
25.At national levels, women’s empowerment and gender equality have improved in recent 
years, as evidenced by the high level of representation by women in Parliament, the high 
rate of girls’ enrolment in primary school, enactment of gender-sensitive policies, and 
reductions in maternal mortality rates. However, low levels of women’s representation in 
local government bring into question whether gender equality programming by 
development actors is fully supported.31 Rwanda's Gender Inequality Index (GII)32 value 
of 0.383 ranks 84th out of 159 countries. 
Government policies and priorities related to food security
26.The McGovern-Dole Program in Rwanda is consistent with relevant national government 
frameworks such as Vision 2020, which describes the country’s development vision for 
modernizing agriculture and for becoming a middle-income country, and the Multi￾Sectoral Food and Nutrition Policy and Strategic Plan. See related discussions in 
paragraphs 8 and 58, and Annex 7, Table 20 for a detailed description of these and other 
relevant government food security policies.
Other international assistance in Rwanda
27. See Annex 8 for an overview of international assistance to Rwanda as relevant to the 
evaluation subject.
1.3 Evaluation Methodology and Limitations
28.The main evaluation questions, as indicated in the Terms of Reference, are: 
30 WFP 2017e. Country Programme Standard Project Report 2017.
31 Evaluation TOR.
32 The GII captures inequality in reproductive health, education, political representation and economic activity, with 
scores from zero to 1, where 1 indicates higher inequality. UNDP. 2016. Human Development Report 2016: Human 
Development for Everyone. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 8 
|Page
• Have literacy rates of school-age children improved over the duration of the 
program? If so, how and why? For example, are students able to read grade-level 
text? Are teachers demonstrating new methods of teaching? 
• Has the use of health and dietary practices increased? If so, how? Has illness￾related absence decreased? Are students washing their hands? Are schools and 
school kitchens clean? How are school gardens being used? 
• What is the level of community-level involvement and participation in decision￾making in school governance mechanisms (PTAs and School Management 
Committees (SMCs))? Particularly, what is the level of involvement and 
participation of women? Also, what is the level and sustained continuity of 
community contributions in cash and in kind?
• What are the key institutions (i.e., international, national, provincial/district and 
local) and governance structures required to effectively deliver, implement, and 
sustain school meal interventions? What relationship structures among these 
institutions yield the most successful and effective school meal programs? Is WFP’s 
capacity support to smallholder farmers and key line ministries 
appropriate/sufficient to effectively facilitate national ownership? Has the 
provided capacity support increased the government’s capacity to own and sustain 
a national school meals program? 
29.The MTE was designed to address these questions and assess the McGovern-Dole 
Program against OECD-DAC criteria: relevance, effectiveness, efficiency, impact, and 
sustainability. As appropriate to an MTE, the ET has given roughly equal weight to all 
DAC criteria, though with the importance of sustaining impacts and an eye toward 
supporting government expansion of the school feeding program, we have viewed all 
criteria through the lens of sustainability. Table 2 presents the key evaluation criteria and 
corresponding questions:
Table 2: Criteria and evaluation questions
Criteria Evaluation Questions
Relevance To what extent is the program in line with the needs of beneficiaries (boys and girls) 
and partners, including government? 
To what extent is the activity aligned with WFP, partner, UN agency, and donor policies 
and priorities?
To what extent is the intervention based on a sound gender analysis? To what extent is 
the design and implementation of the intervention gender-sensitive?
Effectiveness To what extent are the outcomes or objectives of the intervention likely to be achieved?
What are the major factors influencing progress in achievement or non-achievement of 
the outcomes/objectives of the intervention?
To what extent does the intervention deliver results for boys and girls? 
Efficiency Is the program implemented in a timely way? Are the activities cost-efficient? Is the 
program implemented in the most efficient way compared to alternatives? Were the 
program strategies efficient in terms of financial and human resource inputs as 
compared to outputs?
Does the monitoring system efficiently meet the needs and requirements of the 
program? 
Impact What are the medium-term effects on beneficiaries’ lives?
What are the gender-specific medium term impacts? Did the intervention influence the 
gender context?
WFP Rwanda McGovern-Dole Midterm Evaluation Report 9 
|Page
Sustainability To what extent is the government taking ownership of the program (e.g. demonstrated 
commitment and contributions)?
What is the demonstrated capacity at central and sub-national levels to manage the 
program? 
Are local communities (PTAs, farmers’ groups, etc.) fully involved in and contributing 
toward school feeding and education activities?
Has the policy framework supporting the HGSF been strengthened within the program 
period?
What are the major factors influencing the achievement or non-achievement of 
sustainability of the program?
30.The ET further developed the evaluation questions and sub-questions in an evaluation 
matrix (Annex 9), which tailors the OECD-DAC criteria to the country context and 
operating environment. The evaluation questions and matrix were reviewed and validated 
by WFP in the inception phase and finalized before fieldwork. To orient the reader to the 
evaluation methodology, the report outline follows the standard WFP template that 
logically sequences the DAC criteria along expected pathways of change, starting with 
design decisions and ending with an assessment of overall progress against expected 
impact and sustainability. The lines of inquiry further focus on “unpacking” factors that 
affect performance against program indicators. This provides a practical approach for the 
reader to understand the challenges and opportunities of a complex program at its 
midpoint, and the recommendations going forward.
31. The mixed-methods evaluation used secondary and primary data to assess program 
performance against targets and factors affecting performance. The approach included a 
desk review;33 a head teacher survey; a student Early Grade Reading Assessment (EGRA) 
survey; qualitative fieldwork including semi-structured interviews with key informants
and focus groups (see list in Annex 10 and topical outlines in Annex 11) and observation 
of program activities at schools;34 and examination and triangulation of quantitative data 
from WFP and partner monitoring reports and databases. A GEEW approach was 
integrated at all stages, as noted in the methodology discussion.
32.Quantitative EGRA data were statistically powered to provide accurate point estimates of 
student literacy and WASH data points. The confidence interval for all EGRA indicators 
is presented for the midterm analysis and the “re-estimated” baseline EGRA indicators 
(see para. 38). The confidence interval reflects the range that there is 95 percent 
confidence the actual population value will fall. The confidence interval across the EGRA 
indicators is relatively small. In other words, there is not a lot of variance across the third￾grade student population(s), and the reader can ascertain that there is a high probability 
that the actual population values fall within these intervals. The sample of school children 
for the EGRA survey was stratified by gender, allowing statistically valid comparisons of 
results for boys and girls.
33.The EGRA survey team visited 20 schools and administered the EGRA to 220 boys and 
221 girls (Annex 13, Table 22), ensuring equal gender representation in the EGRA. 
Gender-disaggregated data for the EGRA and other key indicators are presented in 
33 The only past evaluation/review of the current project is per its inclusion in the WFP Operation Evaluation: 
Rwanda, Common Country Programme, 200539, Mid-term Evaluation (2013-2016). 34 Direct observation will be employed where possible without disruption to normal activities of classrooms, cooking, 
distribution of meals, etc.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 10 
|Page
Section 2.2 and Annex 14. Replicating the baseline approach, the EGRA survey team also 
administered the head teacher survey to the relevant staff in these 20 schools.35 Primary 
quantitative data from the EGRA and head teacher survey were triangulated with 
quarterly and annual program data and with qualitative results. In all, the quantitative 
and qualitative work encompassed 96 interviews (55 with men and 41 with women) and 
eight focus groups (with 24 male participants and 21 female participants). The ET ensured 
that it interviewed males and females at each school, separately where possible, to obtain 
the viewpoints of both men and women. During interviews with the farmer cooperatives, 
the ET ensured that women as well as men were given the opportunity to discuss the 
program, and that women were full participants in the discussion. The final fieldwork 
schedule is at Annex 12.
34.The ET collected data at national, subnational and school levels. National and subnational 
lines of qualitative inquiry focused on the enabling environment and the potential for 
continuation and eventual handover of HGSF activities. School-level data collection 
focused on assessing and validating progress toward targets and quality of activities, 
outputs and outcomes. The ET was comprised of one female team and one male team, 
which facilitated the gender-sensitive collection of qualitative data.
35. See Annex 13 for a fully detailed description of methodology, sampling and limitations.
36.WFP's decentralised evaluations must conform to WFP and UNEG ethical standards and 
norms. The evaluators are responsible for safeguarding and ensuring ethics at all stages 
of the evaluation cycle. This includes, but is not limited to, ensuring informed consent, 
protecting privacy, confidentiality and anonymity of participants, ensuring cultural 
sensitivity, respecting the autonomy of participants, ensuring fair recruitment of 
participants (including women and socially excluded groups) and ensuring that the 
evaluation results in no harm to participants or their communities. 
37. Annex 13 describes the ethical issues considered for the design, data collection, data 
analysis, reporting and dissemination of the evaluation, and the safeguards and measures 
to manage these issues. The ET followed the safeguards described in Annex 13, Table 27
and no ethical issues arose during the survey or any other component of the evaluation 
process. While no institutional clearances were required, the survey team was introduced 
to schools with advance communication from the CO and a formal letter explaining the 
purpose of their visit.
38.The ET encountered the following methodological limitations: 
39.Limitation 1: The ET requested WFP to share the baseline data in order to undertake 
unpaired longitudinal statistical tests of EGRA indicators. For this report, TANGO has 
provided “re-estimated” values based on the baseline data shared by WFP (see Annex 13, 
Table 25 and Table 26). The presence of non-response data within the reading 
comprehension questions in Section 5 in the baseline reduces the comparability the 
indicator Percent of students who, by the end of two grades of primary schooling, 
demonstrate that they can read and understand the meaning of grade level text from 
35 In most cases the team interviewed the head teacher, whose position is typically equivalent to that of school 
administrator. In the few cases where the head teacher was not available or this specific position did not apply, the 
team interviewed a staff member in a comparable role with adequate institutional knowledge to respond to the survey 
questions. The number and type of head teacher survey respondents are detailed in Annex 13, Table 23, 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 11 
|Page
baseline to endline. For comparability purposes, TANGO based the “comprehension” 
portion of this indicator on the correct response to comprehension questions 1 and 2 (out 
of 5 questions). This presents a substantial limitation to the comparability of baseline data 
with the midterm data; however the ET feels this assumption is conservative (i.e., the 
baseline value may be higher than the “re-estimated” point estimate, but it is unlikely to 
be lower). 
40.Limitation 2: TANGO tried to replicate the baseline findings for the EGRA data and was 
unable to, as discussed above and in additional detail in Annex 13-H. Because of this, the 
ET advises that as preparation for the endline, a more thorough review of baseline data 
should be carried out than is typically done for a midterm or endline evaluation, focusing 
on required indicators. The endline evaluation should include retroactive questions and 
questions focused on the perception of change over the past five years to mitigate 
potentially unreliable baseline data. The existing data should also be assessed to determine 
readiness for required assessment of GEEW. Regardless of the quality of the baseline data, 
the methodology applied at the MTE is valid and appropriate. 
41. Limitation 3: The content and comprehension questions on the EGRA tool were 
modified with World Vision input (see para. 262); as a result they differ from the baseline 
tool. This was not a limitation per se, but we note it for the sake of transparency. The 
adjustment is consistent with the process followed at baseline, when education partners 
in Rwanda recommended “that the tool go through an adaptation process to ensure that 
the students have no prior exposure to the content.”36 This adjustment guards against 
students preparing for specific content and thus skewing the results. The results are 
considered comparable because the midterm assessment was designed to align with the 
third-grade reading level, also used at baseline. 
42.Limitation 4: Because the head teacher survey is perception-based, findings reflect the 
extent to which head teachers are aware of those activities, which may vary based on 
program application across sampled schools. Findings from perception-based survey 
questions do not definitively suggest that the program is achieving (or not achieving) 
expectations relating to program activity coverage or quality. 
43.Limitation 5: Some indicator targets are expressed as counts, rather than percentages, 
e.g., number of head teachers. Midterm values for these indicators were computed by 
extrapolating from the sample data. This is deemed an acceptable practice because the 
sample was representative. 
44.Limitation 6: The program, and the country of Rwanda, do not have performance 
benchmarks for reading and comprehension. The World Vision literacy team noted that 
Rwanda is still in the process of establishing national standards, but shared a 2016 literacy 
assessment report supplied by MINEDUC that has some benchmarks that the ET used to 
put the EGRA results in context.37
45.Limitation 7: The monitoring data, such as in the semi-annual reports, contain errors 
in addition (e.g., some male and female targets do not sum to the overall target). The Excel 
reporting template does not make use of formulas, making it difficult to determine how 
cumulative figures are calculated and whether the calculations are correct, and not all 
36 Ipsos, Inc. 2016. Page 16.
37 EDC. 2016. Early-Grade Literacy in Rwanda: Taking Stock in 2016. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 12 
|Page
quarters are individually reported (e.g., Oct-Dec and Jan-Mar quarters are presented as 
a sum). Data accuracy and cross-checking would be improved by using embedded Excel 
formulas to compute sums and thus ensure mathematical accuracy, and showing quarter￾by-quarter accounting of data for each indicator (rather than grouping quarters). We have 
addressed these limitations by cross-checking data across monitoring documents, 
triangulating results with those of the midterm survey conducted by the ET, and 
consultation with the CO on specific data questions. 
46.Limitation 8: WFP indicators are tracked October – March and April – September 
based on the US fiscal year. However, the Rwandan academic year starts in January. 
Therefore, for student enrolment numbers, the October – March period totals are not an 
accurate reflection of the reality: for October, November and December, the totals still 
include graduating sixth-grade students, while January, February and March no longer 
include the graduating class but do include incoming first-grade students. The CO 
understands that this was discussed with USDA at the beginning of the program and that 
USDA directed WFP to continue reporting based on the US fiscal year; WFP notes that 
the April – September numbers are a more accurate reflection of each academic year 
versus the previous period.38 The ability to accurately calculate achievement with respect 
to number of meals distributed is also affected by this non-standard reporting practice; 
see Annex 14, Table 37. 
47. Limitation 9: The baseline did not establish a value for the McGovern-Dole indicator, 
Percent of parents in target communities who can name at least three benefits of primary 
education. The report noted that parent data could not be collected via a household survey, 
which would require approval by the National Institute of Statistics in Rwanda (typically a 
three-month process). A parent survey had been planned for 2016 but was not carried out. 
The MTE faced the same limitation to conducting a parent survey but added a question on 
this indicator in the head teacher survey to report progress based on head teachers’ 
estimates. The team also collected qualitative information on this topic via interviews with 
PTA members and took the opportunity to ask cooks and members of farmer coops about 
their children's participation in reading groups. 
48.Limitation 10: Staff turnover resulted in a lack of historical knowledge (see also para. 
122). This limitation was mitigated by interviewing former staff.
2. Evaluation Findings 
49.The evaluation findings for the McGovern-Dole Program in Rwanda and the evidence to 
substantiate them are presented below, structured as responses to each evaluation 
question.
2.1 Relevance of the Operation
Results framework/ theory of change
50.The results framework and foundational results are presented in Annex 6. The program 
design follows the McGovern-Dole Results Framework. Activities have been planned, 
sequenced and implemented to ensure achievement of Strategic Objective 1, Improved 
Literacy of School-Age Children. Activities have also been aligned with the objectives of 
SO2, Increased Use of Health and Dietary Practices, to ensure that the broad range of 
38 Email communication with CO, 29 Nov 2018.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 13 
|Page
interventions that support improved student attendance (MGD 1.3) are achieved. Both 
show a logical causal chain. The foundational results incorporate the actions that the 
McGovern-Dole initiative considers critical to the development of a sustainable, 
government-financed and -managed HGSF program for primary schools. This includes 
increased capacity of government institutions, which has been built in terms of 
implementing a WFP-assisted program. More time and focus on capacity strengthening 
are required to build an improved policy and regulatory framework, parts of which are in 
place. Similarly while community groups and local organisations are engaged with the 
program, there is a need to foster their increased engagement. An expanded school 
feeding program will require an increased financial commitment by government. The 
WFP Rwanda FY 2015 McGovern-Dole proposal (p.15) recognises that the McGovern￾Dole Program for primary schools will not be incorporated into the government’s national 
school feeding program by 2020 due to financial, institutional, and agricultural 
challenges. The ET finds this assumption realistic, based on the time required to 
transition school feeding in other countries. In addition, the CO will need to revisit 
expectations and timing for the envisaged transition based on a redesign of the program
that includes the shift from the development of a national program. 
51. Several implicit assumptions underlying the program theory of change do not hold true. 
First, water is a prerequisite for the WASH interventions, but in reality, water resources 
at many schools are unreliable (see discussion at para. 126). In fact, the ET learned that 
communities use washing practices that do not require water. Second, the program design 
depends on consistent and adequate parent contributions to maintain school meal 
activities, such as helping fund cooks’ salaries and providing fuel wood. The program 
proposal notes the limited resources of the poorest families and communities as a 
constraint to local support, and the findings of this evaluation indicate that relying on 
parent contributions is not a valid assumption, given variable family resources and 
different perceptions on who should bear the responsibility to support school-based 
services (see in-depth discussions in para. 107, para. 108 and para. 131). Further 
operational research is recommended to assess the feasibility of community/parent 
contributions and identify a model that is sustainable.
52.Another design assumption was made regarding the local purchase model to sustain 
school feeding over the longer term. The program proposal defined the model in terms of 
proximity of farmers and farmer cooperatives, which initially appeared reasonable given 
the Rwandan context – a small country with extensive regional trade relationships. 
Program staff and government representatives interviewed by the ET stated that linking 
smallholder farmers with the structured demand created by school feeding was assumed 
to be a preferred local purchase model, whereby local cooperatives would be selected for 
capacity-strengthening support based on their proximity to program schools.39 
53. However, this assumption was made with limited analysis or stakeholder consultation. 
After the program started, it became clear to WFP, mainly through consultation with 
government, that a local purchase model in Rwanda could take many forms. At program 
midpoint, WFP management decided to explore a more comprehensive range of options 
for local purchase by initiating a sector study that included review of national and regional 
purchasing models.Interviews with WFP staff indicated that the study results will be used 
39 Also indicated in WFP Rwanda FY 2015 McGovern-Dole proposal, Section 4.3 Capacity Building.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 14 
|Page
to inform further consultation with the government on the optimal local purchase model. 
No decision has yet been taken on this question, and until the study results are available, 
the ET does not recommend a particular model. While we view the local purchase model 
as a good initiative, it is not yet thoroughly tested in terms of meeting the demands of a 
HGSF activity; we agree that the Rwanda context would indeed allow a range of options 
and that the study scope is appropriate. 
Relevance to needs
54.WFP’s targeting of the most food insecure districts is appropriate to geographic needs and 
to the needs of primary schools in some of the poorest areas in those districts. The schools 
were selected by District Education Officers (DEOs), local mayors, and World Vision, and 
verified by WFP, MINEDUC, and World Vision. Teachers, DEOs, and WFP staff cited 
school meals as having increased regular attendance and reduced dropouts. Retention 
rates are high for both girls and boys and well above WFP targets. The program does not 
distinguish between girls and boys in terms of the school meal contents or portion, which 
the ET views as a positive attribute as it reflects gender equity in food allocation. It also 
does not distinguish between men and women in terms of targeting agricultural 
cooperatives; the ET notes that Rwanda’s National Cooperative Policy promotes 
membership for women and gender equality in cooperatives. 40
55. Synergies with partners have been highly relevant to needs. Building reading skills and 
encouraging reading outside of school are major challenges to education; through the 
partnership with World Vision, literacy groups (also called reading groups or reading 
clubs) are functioning and students are attending. These groups add value because many 
parents are non-literate and the opportunity for boys and girls to obtain additional help 
in reading outside of school is relevant to reinforcing the value of education. School 
infrastructure, particularly substandard kitchens, inadequate latrines, and poor water 
storage, has been a challenge in these districts. This infrastructure is being upgraded 
through the construction of new kitchens, separate latrines for girls and disabled 
students, and water storage tanks in many schools. Separate latrines benefit girls in 
particular, allowing them to address their hygiene needs and reducing absenteeism. 
World Vision is responsible for the WASH component; handwashing stations have been 
established outside of classrooms and schools are supplied with water purification tablets 
for drinking water. A partnership with ADRA to build kitchens was dropped due to 
problems with the quality of the construction. WFP is in the process of coordinating 
construction with private construction firms and is directly managing the completion of 
remaining kitchens. School gardens are supported by GHI and are intended to provide 
practical nutrition education. FAO was part of the initial design but did not participate
due to budgetary limitations. This delayed the start of school garden activities until 
recently. Gardens were present at all schools visited by the ET. 
56.The McGovern-Dole Program is relevant to local needs, where rural schools in food 
insecure areas lack basic infrastructure, including WASH facilities, and students have 
limited access to reading material and few resources for reading outside of school. During 
interviews with MINEDUC and district officials, respondents stated that there was a 
40 Republic of Rwanda, Ministry of Education. 2014. National School Health Policy, Kigali 2014. Republic of Rwanda. 
National Cooperative Policy in Rwanda Toward Private Cooperative Enterprises and Business Entities for Socio￾Economic Transformation. Revised version [1]. Kigali, January 15, 2018.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 15 
|Page
noticeable improvement in test scores in schools supported by the McGovern-Dole 
Program, which was credited to the school meals and the literacy activities;41 MINEDUC 
sees the McGovern-Dole Program as very important to the retention and health of 
primary school students. The mobilization of community support for school feeding is 
relevant to create greater ownership and interest among parents in the components of the 
McGovern-Dole Program and their importance to a quality education. There is a question 
of whether the design is appropriate to local capacity. For example, each school must pay 
cooks, purchase firewood, buy soap and utensils, and pay for water. Schools depend in 
part on contributions from the community to meet these costs, and parents may donate 
firewood or vegetables if unable to provide cash. Most of the schools visited by the ET 
were struggling to cover these expenses as community contributions fall short of targets 
by more than half. 
Coherence with national policies
57. WFP Rwanda’s McGovern-Dole Program is aligned with the government’s policy direction 
and intention to expand school feeding. The WFP Rwanda FY 2015 McGovern-Dole
proposal states that the program was designed to support the government’s Home‐Grown 
School Feeding program (also abbreviated here as HGSF), and that the McGovern-Dole 
Program would contribute to the continued development and sustainability of the national 
HGSF program. The government’s HGSF policy endorses a school feeding program based 
on local purchase of commodities with a view to eventual nationwide implementation 
without external support. The government operates two other school feeding programs, the 
One Cup of Milk per Child program and a midday meal program for full-day secondary 
school students, while the McGovern-Dole Program targets primary schools in the most 
vulnerable districts. The Education Sector Strategic Plan 2013-2018 and the National 
School Health Policy 2013 mention the government’s intention to create a national HGSF 
program. Interviews confirm the view among senior government officials that the 
McGovern-Dole Program is an important catalyst for further developing the national 
program. The focus on primary school aligns closely with the government initiative to 
expand its secondary-school-level feeding program into primary schools, starting with 
primary grade 6 and progressively expanding to one additional grade level each year to 
include all primary school students by 2024.
58.Vision 2020 is Rwanda’s development plan to reach lower-middle-income status and to 
reduce poverty and aid dependency. Rwanda met most MDGs by 2015 and continues to 
make progress on development indicators.42 Economic growth in recent years has been 
possible due to the country’s political stability. This is particularly relevant to this 
evaluation as the McGovern-Dole Program aims to contribute to national development 
goals on education and food security, and is working toward national capacity building 
and a program transition. Thus there is a strong coherence with government policies and 
plans, including increasing the literacy rate to 100 percent by 2020.
59.Interviews show that the program was designed to align with government-led initiatives 
to expand access to safe water and sanitation services, including with partners such as 
UNICEF and WaterAid. Program resources are insufficient to provide water access to all 
41 National literacy testing is done in grades 6, 9 and 12, so comparable test score data are not available to corroborate 
these statements. 
42 World Bank Group. 2018. Rwanda Overview. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 16 
|Page
schools that need it. Schools that receive direct support are selected in coordination with 
MINEDUC and MINALOC at district level, taking into account geographic focus of other 
water access initiatives that can potentially support the other schools. 
60.A recent study by the Education Development Center states that one of the challenges to 
improving literacy instruction is that the two main bodies responsible for education, 
MINEDUC/REB and MINALOC, are not institutionally connected. MINEDUC/REB is 
responsible for improving the quality of instruction in schools, while MINALOC manages 
schools and teachers. The REB functions at the national level and is responsible for the 
delivery of materials, training, assessment, and other resources. MINALOC operates 
through local directors of education, sector education officers, head teachers, and 
teachers. The report states that literacy instruction may not be a priority for mayors 
amidst more urgent demands and limited funding. It notes that the government is 
addressing these issues but that “more needs to be done in cooperation between 
MINEDUC and MINALOC at each level of the governance system.”43
Coherence with WFP strategies, policies and normative guidance
61. The McGovern-Dole Program aligns with WFP corporate strategic objective (SO) 1: Support 
countries to achieve zero hunger. Despite Rwanda’s economic progress, one-fifth of 
households are food insecure in the four targeted districts. Since Rwanda is a soon-to-be 
middle-income country, the McGovern-Dole Program aligns with WFP’s strategic concern 
about assisting vulnerable populations within middle-income countries. It further aligns 
with the Education Sector Strategic Plan (2013-2018), School Health and Nutrition Policy 
2014, National Strategy for Transformation 2017-2024, Strategic Plan for Agriculture 
Transformation 4 (2018-2024). 
62.WFP’s support to government in building capacity for the national school feeding 
program is highly coherent with WFP’s shift to an upstream role and its global strategic 
focus on helping countries to strengthen national policies and systems.44 It is aligned with 
WFP’s School Feeding Policy (2013) focus on helping to establish and maintain 
government-led school feeding programs through technical support and capacity 
development, and to develop links with smallholder agricultural producers to supply 
schools, support livelihoods, and strengthen market linkages. This approach is relevant 
to Rwanda’s policy direction and is a good opportunity for WFP to support the 
government’s process. As a first step in this process, WFP provided technical assistance 
to MINEDUC to develop the draft national school feeding policy and strategy.45 
63.WFP’s approach of supporting embedded staff in MINEDUC and MINAGRI to coordinate 
USDA partners and track implementation aligns with corporate SO 5 to partner for SDG 
results and is a useful way to foster collaboration, ensure alignment of policies and 
strategies, and build government capacity for school feeding. The selection of technically 
skilled partners to implement the literacy, WASH, and nutrition components is aligned 
with SO 4 to partner to support implementation of SDG results. The McGovern-Dole 
Program includes a component to develop the capacity of smallholder cooperatives to 
eventually supply commodities for school feeding. This is aligned with SO 3, to achieve food 
43 EDC, 2016.
44 WFP 2017a. WFP Strategic Plan (2017-2021).
45 WFP. 2016b. Food and Safety Net Assistance to Refugee Camp Residents and Returning Rwandan Refugees. 
Standard Project Report 2016.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 17 
|Page
security. The ET notes that the USDA requirement to use imported food is not aligned with 
WFP’s policy supporting local procurement. The ET notes that this is a temporary 
misalignment, and that the McGovern-Dole Program is developing the capacity of local 
smallholder cooperatives to eventually support local production of vegetable oil and CSB+. 
WFP is also investing in comprehensive research to provide relevant options for HGSF 
commodities. 
64.Gender. The McGovern-Dole Program is guided by the WFP 2015-20 Gender Policy;46 it 
promotes the participation of girls and indicator data are disaggregated by gender. The 
program provides for the construction of gender-sensitive toilet facilities for girls (as well 
separate, non-gendered toilets for disabled students) but does not include a specific 
approach to address gender equality and women’s empowerment or sexual and gender￾based violence. The ET noted that the majority of cooks at the schools visited were male, 
while cleaners were female. The reason given is that men have the physical strength 
required to handle the cooking. This point was also noted by in the 2016 CP evaluation, 
with the suggestion that strategies for ensuring a better gender balance could be 
employed.47 The proportion of women in leadership positions in program management 
committees is slowly increasing (37 percent) but is below the target of 50 percent. WFP is 
working with government, partners, and stakeholders to identify ways to increase 
women’s representation in program management at the local level.48 
65.Effective partnerships. The main coordination mechanisms between program 
partners are Semi-Annual Steering Committee meetings led by MINEDUC; Quarterly 
Coordination meetings, led by MINEDUC and WFP, which focus on operational issues; 
Quarterly Technical meetings with all stakeholders at district level; and regular 
monitoring meetings, also at district level. Interviews with partner and government staff 
showed that MINEDUC, which is the main education counterpart for the McGovern-Dole 
Program, has varying degrees of coordination with MINALOC at local level, though 
coordination is quite close on school infrastructure and water access. MINALOC is less 
involved on issues related to education quality and student performance. This has 
resulted in a lower level of understanding among MINALOC senior officials such as the 
mayor and vice-mayor. Interviews showed that the role of the mayor, in particular, is 
crucial in bridging the divide between national and subnational (district) governments 
with the aim of institutionalizing HGSF results. 
66.Interviews with government representatives indicate that REB is not closely engaged in 
the McGovern-Dole Program at either national or subnational level, which appears to 
contrast with other literacy programs that do have formal partnership with REB. REB 
does participate in the HGSF Steering Committee meetings but to date, this is the extent 
of their engagement in the program. The formal program partnership with MINEDUC 
does not automatically translate into coordination with REB, as REB has its own mandate 
within the larger MINEDUC organization. Feedback further indicates that this could 
present a challenge for scaling up Literacy Boost activities, given the leading role REB 
plays in curriculum development. At the national level, efforts to establish closer 
coordination with REB are underway through the steering committee meetings.
46 Initially the WFP 2009 policy, later subsumed by the WFP 2015-20 Gender Policy. Evaluation TOR.
47 WFP 2016b.
48 WFP Rwanda 2017e.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 18 
|Page
67.World Vision works with district governments to improve latrine coverage in primary 
schools. It has constructed or rehabilitated 39 ventilated improved pit latrine blocks; this 
number was determined by a needs assessment and will ensure that by the end of the 
program, all 104 schools have new or improved latrines (others will be constructed 
through other initiatives). World Vision is also building or enhancing water collection 
systems; it has fully achieved the FY 2018 target of installing 15 water tanks).49 This 
supports increased access to clean water and improved sanitation and improved hygiene 
of school‐age children with an emphasis on hygiene issues affecting girls (MGD 2.4).50
World Vision will also initiate new menstrual hygiene management systems in schools. 
PTAs will be trained on the importance of menstrual hygiene education. While these 
measures are very important to encouraging regular attendance by girls, there is 
otherwise limited explicit consideration of gender in the program. 
Relevance to USDA policy 
68.The program seeks to achieve MGD SO1: improved literacy of school-age children, and 
MGD SO2: increased use of health and dietary practices in the targeted areas. The 
program aims to achieve McGovern-Dole Foundational Results51 in several important ways. 
For example, the program supports the foundational result of increased capacity of 
government institutions by strengthening the government national school feeding program
and by building capacity at national, district and school levels to eventually integrate the 
McGovern-Dole Program, as well as by strengthening literacy and good health and hygiene 
practices among students and school personnel. It supports the foundational results of
increased engagement of local organizations and community groups by building the 
capacity of local agricultural cooperatives to eventually supply food to school feeding 
programs. 
69.The McGovern-Dole Program is also consistent with the United States FAS Strategic Plan, 
specifically Goal 4, Objective 4.1: Implement non trade-focused congressionally mandated 
programs.52
70.The program contributes to FAS’ sustainability goal by strengthening government capacity 
to implement a national school feeding program and to assume full management of the 
school meals activity implemented under the McGovern-Dole Program, although on a more 
limited scale. By helping local agricultural cooperatives build their capacity to supply food 
for school meals, the McGovern-Dole Program is helping to ensure a sustainable source of 
supply for school feeding programs that will also help communities to develop new markets 
and greater resilience.53, 54 
49 WFP. 2018b.
50 WFP Rwanda FY 2015 McGovern-Dole proposal. 51 The Foundational Results include increasing the capacity of government institutions (MGD 1.4.1/2.7.1), improving 
the policy and regulatory framework (MGD 1.4.2/2.7.2), increasing government support (MGD 1.4.3/2.7.3) and 
increasing the engagement of local organizations and community groups (MGD 1.4.4/2.7.4). WFP Rwanda. 2015. 
Results Framework. PowerPoint. (“Foundational Results” diagram).
52 United States Department of Agriculture, Foreign Agricultural Service. NDa. Foreign Agricultural Service Strategic 
Plan (2019-2022). 
53 United States Department of Agriculture, Foreign Agricultural Service. NDb. FY 2019 Food Assistance Proposal 
Guidance and Request for Applications.
54 See also the discussion on impact and sustainability in Section 3.1
WFP Rwanda McGovern-Dole Midterm Evaluation Report 19 
|Page
Coherence with relevant UN-wide system-wide commitments 
71. United Nations Agencies. Rwanda is one of eight pilot countries in which United 
Nations agencies coordinate closely through One UN. The agencies coordinate their 
activities for economic and social transformation and transformational governance 
through the United Nations Development Assistance Programme (UNDAP).55 As of April 
2018, the 2018-2023 UNDAP was still being finalised. United Nations inter-agency 
collaboration includes WFP and UNHCR’s joint work on refugee issues and WFP, 
UNICEF and UNESCO collaboration on education activities such as creating Child‐
Friendly School standards and assessing literacy and numeracy. WFP and FAO have 
collaborated on Purchase for Progress (P4P) agricultural support activities to support 
smallholder farmers to access markets, though FAO does not collaborate on school 
gardens as in other countries. Since 2011, the government has integrated lessons learned 
from that program to create Common P4P (CP4P), which purchases food from 
smallholder cooperatives for the National Strategic Reserve.56 
Synergy with other WFP ongoing operations
72. WFP Rwanda also implements Protracted Relief and Recovery Operation (PRRO), Food 
and Nutrition Assistance to Refugees and Returnees (2016-2018) (budget: US$119.4 
million), and a Country Programme: Enhancing National Capacity to Develop, Design 
and Manage Nationally Owned Hunger Solutions in Rwanda (2013-2018) (budget 
US$51.8 million).57 WFP promotes internal program synergies by training smallholder 
cooperatives to increase their production for market under the framework of the USDA￾funded LRP project.58 
Key Findings and Conclusions – Relevance
• The McGovern-Dole Program is aligned with government, WFP and UN policies and 
priorities. 
• WFP’s support to government in strengthening capacity for the national school 
feeding program is highly coherent with WFP’s shift to an upstream role. 
• The program’s integrated activities are appropriate to local needs.
• Progress on program targets for component interventions is on track. 
• A challenge to improving literacy instruction is that the two main government 
bodies responsible for education are not institutionally connected.
2.2 Results of the Operation
Outputs 
73. WFP planned to reach 83,106 students annually in grades 1-6.59 At midterm, consistent 
with previous quarters, performance for this output exceeds or is very close to targets for 
boys, girls, and overall. As shown in Table 3, the program reached 99 percent of planned 
student beneficiaries in 2016, 103 percent in 2017, and 101 percent in 2018. The 2018 
55 United Nations Rwanda. 2018. UNDAP 2018-2023 for Rwanda. Signed 31 July 2018.
56 WFP. 2014. Purchase for Progress - P4P Rwanda.
57 WFP. 2018a. WFP Rwanda Country Brief. March 2018.
58 World Food Programme. 2018c. Country Programme Standard Project Report 2018 (Rwanda).
59 The official duration and ages for primary school in Rwanda are 6 years, ages 7-12. Education and Policy Data 
Center, FHI 360. 2014. Rwanda Education Profile. 2014 Update.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 20
|Page
percentage is based on the two quarters spanning 1 April – 30 September, which the CO 
indicates is the most accurate reflection of beneficiary numbers (see Limitations, para 38, 
for a discussion of measurement challenges with reporting based on FY quarters that do 
not correspond with the school year). 
Table 3: McGovern-Dole Program beneficiary output indicatorsa 
Indicator
2016 
Actual/ 
Planned
% 
Achieved
2017 
Actual/ 
Planned
% 
Achieved
Actual 
1 Oct 
2017 – 
31 
Mar 
2018b
Actual
1 Apr 
– 30 
Sep 
2018 
2018 
Actual/ 
Planned
% 
Achieved
Total 
student 
beneficiaries
82,360/ 
83,106 99% 85,513/ 
83,106 103% 84,992 83,590 83,590/ 
83,106 101%
Total male 
beneficiaries
41,703/ 
39,890 105% 43,612/ 
39,890 109% 43,296 42,712 42,712/ 
39,890 107%
Total female 
beneficiaries
40,657/
43,215 94% 41,901/ 
43,215 97% 41,696 40,878 40,878/ 
43,215 95% 
Source: SPR 2016, SPR 2017, semi-annual reports April and October FY 2018. a While the overall annual target is stated as 83,106, the sum of male and female beneficiary targets is 83,105, a 
difference of one beneficiary. The CO confirmed (email 27 Dec 2018) that these are the targets stated in the signed 
agreement. We have computed the percentage achievement using the stated targets but acknowledge that these 
contain this minor error.
b Values given for reference only. Per the CO, the 1 Apr – 30 Sep period is the more accurate figure to use in 
assessing performance against target. We have therefore used this figure to compute percent achievement.
74.In the 1 April – 30 September period in 2018, targets for providing school meals in FY 
2018 have been nearly met for girls (95 percent), and exceeded for boys (107 percent) 
(Annex 14, Table 37). There is substantial underachievement for the number of new 
students receiving meals (6 percent of target), even while the overall combined target for 
new and continuing students is exceeded (101 percent). 
75. Table 4 reports school attendance. In the 1 April – 30 September period in 2018, 
attendance targets for FY 2018 have been largely met for girls (91 percent), and exceeded 
for boys (101 percent).
Table 4: Attendance, actual vs planned, FY 2018 
Indicator
1 Oct 2017 –
31 Mar 2018 
Actuala
Actual
1 Apr – 30 
Sep 2018
FY 2018 
Planned
% Achieved
(actual/ 
planned)
Number of students regularly 
(80%) attending USDA 
supported classrooms/schools 
(overall)
81,707 79,931 83,106b 96% 
Number of students regularly 
(80%) attending USDA 
supported classrooms/schools 
(females)
41,455 39,473 43,215 91% 
Number of students regularly 
(80%) attending USDA 
supported classrooms/schools 
(males)
40,252 40,458 39,890 101%
Source: Semi-annual reports for April and October FY 2018. a Values given for reference only. Per the CO, the 1 Apr – 30 Sep period is the more accurate figure to use in 
assessing performance against target. We have therefore used this figure to compute percent achievement. This is 
the target listed in the semi-annual reports and the signed agreement.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 21
|Page
76. The distribution of literacy starter kits was completed in Year 2; this is a change from the 
original program design, which called for a progressive distribution of starter kits each year, 
reaching all 104 schools by the end of the program. World Vision revised this activity to 
distribute all kits during the first half of the program so that schools could benefit sooner.60
At midterm, administrators and officials in all 104 schools have received training in new 
tools and techniques; about double the target number are demonstrating the use of new 
tools and techniques (181 against a FY 2018 target of 93).61 The corresponding indicator for 
teachers and educators was also exceeded, with 949 individuals demonstrating new tools 
and techniques against a FY 2018 target of 272. Because the program decided to train all 
parents who had children enrolled in community reading clubs, the target for training 
parents on the importance of literacy was also exceeded: 6,166 parents were trained, against 
a FY 2018 target of 1,456.
77. Over the program cycle, WFP plans to use 4,657 MT of SuperCereal (CSB+) and 540 MT 
of vegetable oil. 62, 63 Annex 14, Table 38 shows the commodities distributed from FY 2016 
Q1 through FY 2018 Q3. The life-of-program vegetable oil target has been exceeded (101 
percent); the life-of-program CSB+ distribution has met 71 percent of target. Deliveries 
to schools are made every three months (one time per term) to minimize delays in the 
rainy season. The FY 2018 Q3 report notes delivery delays for both commodities owing to 
challenges such as limited access to schools during the rainy season and is working with 
the government to resolve this. The other main logistics challenge is poor rural road 
infrastructure, which is a disincentive to private transport companies to pursue delivery 
contracts. The majority of schools visited by the ET reported no issues with delivery or 
with running out of food. WFP is able to purchase sufficient amounts of maize and beans 
at national level to supply the McGovern-Dole Program, with funding provided by 
MasterCard. Overall, stock management and storage activities at schools were in line with 
WFP standard operating procedures, as promoted through storekeeper trainings and 
regular refreshers. The ET also observed WFP field monitors managing food quality by 
transferring commodities among schools to ensure that food with best-before use dates 
were evenly distributed, thus preventing waste due to expired commodities. 
60 Email communication with CO, 28 Nov 2018.
61 Source: Rwanda(MGD15)_Semi Annual Report April 1 2018_ September 30 2018 final. Officials are reported to 
include head teachers, sector and district education officers, SBMs, DoS. (The latter two acronyms are not spelled out 
in the report and neither the ET nor the CO could not identify what positions they refer to.)
62 WFP Rwanda FY 2015 McGovern-Dole proposal. Note that reporting is in net metric tons (NMT). One NMT is 
equal to 2,000 pounds (0.907 MT or 907.18 kg). 
63 Maize, beans, salt and sugar are non-USDA commodities and are purchased with other funds.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 22
|Page
Outcomes 
78.Table 5 shows the latest data available for the USDA outcome indicators and additional 
requested midterm indicators. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 23
|Page
Table 5: USDA outcome indicators
Outcome Indicator1 End 
Target1
Baseline
(Jun 2016)1, 
2
2016
(Dec) 1
2017
(Dec) 1
FY 2018
(Q1-Q2) 3
Midterm 
(Oct 
2018)4
Outcome: Increased equitable access to and utilization of education
Student-level
Retention rate of girls in WFP￾assisted primary schools >85% 98.9% 95.6% 99.0% No data No data
Retention rate of boys in WFP￾assisted primary schools >85% 98.4% 95.7% 95.0% No data No data
% of students who, by the end of two 
grades of primary, demonstrate that 
they can read and understand the 
meaning of grade level text
69% 41.5%10
No data:
Follow up at midterm and 
endline 
59.0%11
% of female students who 
demonstrate the above 77% 43.3%10 62.9%11
% of male students who demonstrate 
the above 62% 39.8%10 56.8%11
School-level
% of students in classrooms identified 
as attentive by their teachers 80% 60.0%
No data:
Follow up at midterm and 
endline 
57.46
% of parents in target communities 
who can name at least three benefits 
of primary education 90% No data
69.86
# of schools using an improved water 
source 104 62 996
Outcome: Ownership and capacity strengthened to reduce undernutrition and increase access to 
education at regional, national and community levels
# of school administrators and 
officials in target schools who 
demonstrate use of new techniques or 
tools 
93 0 No data 18 93 1817
# of teachers/ educators/ teaching 
assistants in target schools who 
demonstrate use of new and quality 
teaching techniques or tools
984 0 No data 28 326 9497
# of educational policies, regulations 
and/or administrative procedures in 
each stage of development5
1 0 No data 0 
0 
(draft 
policy in 
Stage 2)
08 
# of child health and nutrition 
policies, regulations, or 
administrative procedures in each 
stage of development5
1 0 No data 0 
0 
(activity 
has not 
started)
09 
Colour code key: End target achieved End target not achieved No data
WFP Rwanda McGovern-Dole Midterm Evaluation Report 24
|Page
1 As reported by WFP in the SPR corresponding to the year indicated in the column heading. Targets reflect revised targets 
given in USDA indicators - revised targets – 070817.
2 With the exception of the base values for the retention rate indicators, which are drawn from a WFP survey of October 2013.
3 Source: Semi-annual Apr 2018 data.
4 Source: Midterm survey unless otherwise noted.
5 Stage 1: Analysed Stage 2: Drafted and presented for public/stakeholder consultation Stage 3: Presented for 
legislation/decree Stage 4: Passed/Approved Stage 5: Passed for which implementation has begun
6 Value extrapolated from sample data
7 Source: Rwanda(MGD15)_Semi Annual Report April 1 2018_ September 30 2018 final 
8 This activity has not yet begun. Source: Rwanda(MGD15)_Semi Annual Report April 1 2018_ September 30 2018 final
9 School feeding policy is still in draft form; validation by the cabinet is pending. Source: Rwanda(MGD15)_Semi Annual 
Report April 1 2018_ September 30 2018 final
10 Re-estimated from baseline data. See Annex 13 for explanation.
11 These are point estimates. For this data, differences between estimated baseline data and midterm data are statistically 
significant at the 1 percent (***) level.
Evaluation Question 1: Literacy and Teaching Methods64
79.World Vision implements the program’s literacy component. It applies the Literacy Boost 
instruction methodology, which focuses on improving children’s reading abilities. USDA 
support has enabled the scale-up of World Vision’s existing program.65 World Vision 
employs a Literacy Specialist, based in Kigali, who oversees an Education Manager and 
two Literacy Boost coordinators in each district who lead field-level activities, which 
include teacher training, reading clubs, fostering print-rich classrooms, and promoting 
reading in small groups (in school) and clubs (outside school). World Vision trains 
teachers and head teachers on literacy instruction and on improved teaching techniques, 
with annual refresher training. The methodology used by World Vision follows the 
methods used by MINEDUC for teacher training and supervision, making it feasible for 
MINEDUC to eventually absorb the activity if it chooses.
80.As shown in Table 5 above,66 students’ reading and comprehension in Kinyarwanda show 
that progress is on-track relative to target. At midterm, 59 percent of the students 
demonstrate the reading and comprehension competencies tested by the EGRA, an 
increase of 17.5 percentage points over baseline; there are no statistically significant 
differences between boys and girls. Midway through the program, this indicates good 
progress toward achieving program’s 2020 target of 69 percent of students able to read 
and comprehend grade-level text. District-wise, performance is best in Nyaruguru (58.2 
percent demonstrate the required competencies) and lowest in Karongi (41.8 percent) 
(see Annex 14, Table 30). 
81. Table 6 presents the results of testing students’ oral reading fluency and their 
comprehension of a short story that they read aloud if able, or that was read to them if 
they were not able to read it themselves.67The first measure, percent of students to read 
short story, refers to students who complete the reading of a short story aloud, regardless 
of the number of reading errors. No time restrictions were placed on this exercise; the 
measure is based on the number of students who complete reading the story. However, 
64 Have literacy rates of school-age children improved over the duration of the programme? If so, how and why? For 
example, are students able to read grade-level text? Are teachers demonstrating new methods of teaching?
65 WFP Rwanda FY 2015 McGovern-Dole proposal. 66 Sample size 441 (220 boys, 221 girls). See Annex 13, Table 22. 67 The project does not specify targets for this indicator.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 25
|Page
the assessors were instructed to adjust the exercise to reading to the student if the student 
was not able to correctly read aloud at least three of the first five words of the story. The 
results show that almost two-thirds of students (64.6 percent) were able to read the 
complete short story aloud; a little more than one-third could not read three of the initial 
words in the passage. There was a small but statistically significant difference in girls’ 
performance on this task, with 68.8 percent of girls able to read the story aloud, versus 
60.5 percent of boys.
Table 6: Reading, listening and comprehension of short story at midterm
Total Male Female
Percent of students to read short story1 64.6 60.5* 68.8
Percent of students to fully comprehend the short story (regardless if 
they read or listened)
82.8 82.4 83.2
n 441 220 221
Source: Midterm EGRA survey
Difference between male and female students at midterm is statistically significant at 10% (*), 5% (**) or 1% (***) 
level
1 No time restrictions were placed on the students for this exercise, however the 60-second point is tracked in the 
data. This measure includes students who complete the story (regardless of time taken) with minimal errors.
82.A recent report on literacy in Rwanda notes, “The design and focus of EGRA and other 
tests of fluency and comprehension are based on the theory that a minimum level of 
reading speed is essential to comprehension. The body of research most often used to 
support EGRA and similar tests holds that to understand a simple passage, given the 
capacity of short-term memory, students should read a minimum of 45–60 words per 
minute.’ ”68 The report also states that in 2o12, a Rwanda National Standards Committee 
defined third-grade Kinyarwanda reading fluency as 33-47 words correct per minute 
(WCPM).69 As shown in Table 7, approximately one-third of the midterm sample (16.1 
percent + 18.8 percent) is reading at or above this WCPM range.
83.Notably, almost one in five students (18.4 percent) could not read any words at all. As 
shown in Annex 14, Table 29, the inability to read is most prevalent in Rutsiro (24.5 
percent), followed by Nyamagabe (20.9 percent), Karongi (14.9 percent) and Nyaruguru 
(13.6 percent). Karongi has the highest percentage in the highest WCPM range (41-56 
WCPM; 30.9 percent), and leads the other districts in the proportion of students reading at 
31 or more WCPM. Head teachers were given immediate feedback on EGRA results by the 
quantitative team. In at least one instance, teachers in a poor-performing school told the 
qualitative ET that school officials had met to discuss why reading scores were low. 
68 EDC 2016, citing Abadzi, H. 2011. Reading fluency measurements in EFA FTI partner countries: Outcomes and 
improvement prospects. Washington, DC: Global Partnership for Education Series on Learning, #1. Page 4.
69 EDC 2016, citing Clark-Chiarelli. 2012. Proposed National Reading Standards, Kinyarwanda and English, P3 & P5. 
The proposed standard is lower than WCPM in English because Kinyarwanda, as an agglutinating language, has 
longer, more multi-syllabic words.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 26
|Page
Table 7: Words correct per minute (detailed), third grade
% of students in WCPM range
WCPM Overall Males Females
0 18.4 25.0*** 11.8
1 to 15 14.3 8.6 19.9
16 to 30 32.4 25.5** 39.3
31 to 40 16.1 16.8 15.4
41 to 56 18.8 24.1** 13.6
n 441 220 221
Sources: Midterm EGRA survey; EDC 2011.
Difference between male and female students at midterm is statistically significant at 10% (*), 5% (**) or 1% 
(***) level
84.Table 7 also shows notable statistical differences in boys’ and girls’ performance at certain 
ranges of WCPM, with a quarter of the male sample reading no words at all compared to 
11.8 percent of females. Girls also outperform boys at the 16-30 WCPM range, but this 
trend reverses at the highest WCPM range, with nearly a quarter of boys reading 41-56 
WCPM but just 13.6 percent of girls. 
85.In the midterm EGRA, after reading the story (or, for those with low reading performance, 
having listened to it), students are asked five questions to gauge their understanding of 
the passage. These are “locator” or recall questions, which do not require inference or 
interpretation. In 2012, the National Standards Committee proposed comprehension 
standards as follows: 0-44 percent correct answers do not meet standards, 46 [sic] -69 
percent correct meets standards, and 70+ percent exceeds standards.70The program has 
not set targets for this task. The following tables present the findings both as number of 
correct answers and as percentages to enable comparison of the results against the 2012 
proposed national standards. Table 8 shows how the groups of students who read and 
who listened performed on the comprehension task in terms of number of questions 
answered correctly. Summing the percentages of students in each rating, for the sample 
as a whole, 17.3 percent rate below standard on comprehension, and 82.8 percent meet 
or exceed standard. 
Table 8: Reading comprehension, third grade (all students to read and/or listen)
Total correct answers to 
comprehension questions
Rating (aligned to 
2012 NSC proposal)1
% of students 
to achieve Total %
0 (0% of total questions) Does not meet standard
(0-44% correct) 4.3
1 (20%) 17.3
Does not meet standard
(0-44% correct) 3.2
2 (40%) Does not meet standard
(0-44% correct) 9.8
3 (60%) Meets standard
(45-69% correct) 27.7 27.7
4 (80%) Exceeds standard
(70+% correct)
37.6 55.1 5 (100%) 17.5
n= 441
Source: Midterm EGRA survey
1 NSC = National Standards Committee
70 EDC. 2016. Early-Grade Literacy in Rwanda: Taking Stock in 2016.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 27
|Page
86.Table 9 isolates the results to students with low oral fluency, who listened to the story as 
read by the assessor, and Table 10 reports findings for the students who could complete 
the reading on their own. The findings show that students who could read aloud have 
much better comprehension ratings: 92.6 percent (Table 10) meet or exceed standard 
compared to 64.8 percent (Table 9) of those with lower oral fluency. The difference is even 
more marked when comparing the percentages in the exceeds category: 65.6 percent of 
students who read on their own answered 4-5 comprehension questions correctly (Table 
10) compared to 35.9 percent of those who listened to the story (Table 9). The finding that 
students perform better on comprehension when they read a passage on their own rather 
than hear it read to them, underlines the importance of students having physical reading 
materials.
Table 9: Reading comprehension, third grade (only students to NOT complete reading; 
passage read to them)
Total correct answers to 
comprehension questions
Rating (aligned to 
2012 NSC proposal)1
% of students 
to achieve
Total 
%
0 (0% of total questions) Does not meet standard
(0-44% correct) 12.2
1 (20%) 35.3
Does not meet standard
(0-44% correct) 7.1
2 (40%) Does not meet standard
(0-44% correct) 16.0
3 (60%) Meets standard
(45-69% correct) 28.9 28.9
4 (80%) Exceeds standard
(70+% correct)
24.4 35.9 5 (100%) 11.5
n= 156
Source: Midterm EGRA survey
1 NSC = National Standards Committee
Table 10: Reading comprehension, third grade (only students to complete reading passage)
Total correct answers to 
comprehension questions
Rating (aligned to
2012 NSC proposal)1
% of students 
to achieve
Total 
%
0 (0% of total questions) Does not meet standard
(0-44% correct) 0.0
1 (20%) 7.4
Does not meet standard
(0-44% correct) 1.1
2 (40%) Does not meet standard
(0-44% correct) 6.3
3 (60%) Meets standard
(45-69% correct) 27.0 27.0
4 (80%) Exceeds standard
(70+% correct)
44.9
65.6 5 (100%) 20.7
n= 285
Source: Midterm EGRA survey; EDC 2016.
1 NSC = National Standards Committee
*Third-grade n not reported. The FARS total sample was 1799 students in grades 1, 2 and 3.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 28 |Page 
87.Table 11 shows results for specific subtask measures of reading performance.71 The program does not set targets for these 
subtasks, nor do we find established benchmarks or comparable studies for Rwanda. Rwandan students tested read 49 
letter-sounds per minute.72 
88.The main contributing factors to reading improvements noted in interviews and focus groups with students, teachers and 
administrators were the print-rich classroom environments and stronger teacher expertise to work with both slow and fast 
learners using a wide range of teaching aids promoted by the program. As part of their regular monitoring both WFP and 
World Vision staff observe teachers in the classroom to ensure they are using the new techniques promoted by the program, 
and provide feedback to the DEO on underperforming schools. 
Table 11: Reading performance for third-grade students at midterm
Indicator
Baseline Midterm
Reported
Re￾estimated a
Confidence
Lower 
Bound
Interval
Upper 
Bound
# of 
Tasks
Mean 
per 
Minute
Confidence
Lower 
Bound
Interval
Upper 
Bound
# of 
Tasks
Reading letter-sounds 11.0 16.4 15.6 17.3 100 49.0*** 46.6 51.5 95 
Reading syllables 25.0 45.6 42.5 48.7 100 55.8*** 53.3 58.3 100
Familiar words 11.0 21.1 19.5 22.7 50 29.9*** 28.7 31.1 50
Unfamiliar words 7.0 14.8 13.5 16.0 50 33.7*** 32.7 34.6 50
Correct words in text/story 10.0 20.1 18.5 21.8 65 22.8** 21.3 24.4 56
n n/a 402 441
Source: Baseline and midterm EGRA surveys
Differences between re-estimated baseline data and midterm data are statistically significant at the 10 percent (*), 5 percent (**) or 1 percent (***) level.
71 The project does not specify targets for this indicator.
72 We recognize the debate regarding whether it is feasible to compare reading fluency across languages, as outlined for example in Abadzi 2012: Developing Cross￾Language Metrics for Reading Fluency Measurement: Some Issues and Options. Global Partnership for Education Working Paper Series on Learning No. 6.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 29
|Page
89.As shown in Table 12, about 60 percent of students report reading outside of school, a 
practice shown to significantly contribute to improved literacy. Teachers told the ET that 
they have observed students reading more at school since joining the literacy groups. 
Focus groups of third- and fourth-grade students interviewed by the ET reported 
attending literacy groups, though some occasionally missed sessions. Of those who do not 
read outside of school, the main reasons given in the quantitative survey are too much 
work, no materials and no time.73The qualitative research found that common reasons for 
having no time for reading were: students cannot attend reading groups because parents 
require them to do household chores74 (and some parents do not see the point of reading 
groups outside of school); some students are sent to the market, to look for firewood or 
to fetch water; some babysit or do farm work; other children prefer to play instead of 
attend reading groups. Nevertheless, as shown in Annex 14, Table 31, 88 percent of 
students state that they have enough time to study and do homework and that their 
parents help them with their homework and reading (83.2 percent). 
90.Parents met at the schools and in farmer cooperatives by the ET confirmed that they help 
their children in grades 1-3 with reading in Kinyarwanda, though they cannot help older 
children who study in English. Interviews with teachers and administrators indicated that 
the introduction of English and Kinyarwanda reading at the same time was problematic 
for children in grades 1-3 and caused confusion around pronunciation. School staff said
that it would be better to focus on Kinyarwanda first and introduce English when basic 
literacy in Kinyarwanda was achieved, from fourth grade onwards.
91. The ET found that support for the community-based literacy groups is high among 
educators, parents, and government officials. Parents and teachers told the ET that their 
children have improved in reading since joining the groups. Community-based literacy 
groups are operating regularly, though the support and oversight is limited in some areas. 
At each school, World Vision trains one teacher, administrators and three facilitators in 
the approach, provides reading materials, and helps to establish three reading groups 
around each school in the McGovern-Dole Program. Partner staff in Southern Province 
reportedly visit the groups on a regular basis. However, in Western Province, World 
Vision has not adequately addressed transport needs; WFP provided World Vision with 
eight motorcycles for Literacy Boost coordinators, but to date only three of the 
coordinators have obtained driving licenses. The result is that literacy staff share a vehicle 
with other staff and can visit only one school per week out of 50 schools, which is not 
adequate to maintain quality in that district. 
92.Interviews with teachers and administrators showed that the trained teachers attempt to 
meet with the club facilitators on a weekly basis, usually on a Friday, to coordinate school 
and club activities. Curiously, several head teachers and assistant head teachers told the 
ET that they had never visited a reading group. In general, facilitators, teachers and 
administrators indicated that many clubs are too large, or mix students of different grade 
levels, which had a negative effect on the reading activities. This was confirmed by 
students who stated that some club activities were at times unorganised and that most 
activities centred on the facilitator reading to the group. These issues were attributed 
mainly to either too few facilitators for the number of children attending, or the absence 
73 Students were read each of the listed options and asked whether each given option applied to them (yes/no).
74 E.g., cooking, fetching water, washing clothes
WFP Rwanda McGovern-Dole Midterm Evaluation Report 30
|Page
of one or more facilitators at some sessions. Staff interviews showed that 2019 planning 
will explore whether additional facilitators could be trained to enable smaller groups. 
93.While the percentage of students reading outside of schools is an achievement for the
program, the sustainability of community-based literacy groups in the short term is a 
challenge. The program trains teachers, monitors community volunteers who act as 
literacy group facilitators, and supplies reading material for students to take home on a 
rotating basis. This approach is expected to generate community-level demand for greater 
reading material in the future. However, MINEDUC and school budgets for books are 
small (and children are not allowed to take them home from school), and cash support 
from parents buy books face the same challenge as the support for school feeding, so the 
source of future support to literacy groups is unclear. 
Table 12: Reading outside of school (third grade) at midterm
Indicator Point 
Estimate
Percent of students to read outside of school in last week 59.6
Male students 58.6
Female students 60.6
n 441
Reasons for not reading outside of school (multiple response)
Too much work 30.9
No materials 20.8
No time 19.1
Not interested 2.2
No reading area 1.1
No light 0.6
n 178
Source: Midterm EGRA survey
94.The head teacher survey collected teachers’ perspectives on student attentiveness, by 
grade (Annex 14, Table 32).75 Overall about 57 percent of students are considered 
attentive, and attentiveness declines with each higher grade (the highest estimate is for 
first grade, 66.0 percent; the lowest is for sixth grade, 50.9 percent). 
95.As shown in Annex 14, Table 33, the program has exceeded the target for number of head 
teachers/ school officials who received trainings or certifications as a result of USDA 
assistance: 9976 versus a target of 93.
Evaluation Question 2: Health and Dietary Practices77
96.As shown in Table 13, students have shown minimal progress on their ability to name 
three target health and hygiene practices.78 Performance on this indicator is lowest in 
Rutsiro District (40.5 percent) and highest in Nyaruguru (60.9 percent) (Annex 14, Table 
34). While the majority of students interviewed were not able to name the target practices 
(e.g., handwashing with soap after visiting toilet; handwashing before eating), they were 
75 Values estimated as follows: in the head teacher survey, teachers for each grade estimate what percent of their 
students are attentive. This total is weighted using the reported number of students in each classroom.
76 Extrapolated value
77 Has the use of health and dietary practices increased? If so, how? Has illness-related absence decreased? Are 
students washing their hands? Are schools and school kitchens clean? How are school gardens being used?
78 Annex 14, Table 35 shows results for individual practices.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 31
|Page
able to name practices that are part of their daily lives at school, such as keeping hair short 
and washing their uniforms. Among all students, when asked what health and hygiene 
practices they used regularly, 91.4 percent said they practice “other personal hygiene” 
(e.g., bathing) regularly and 42.0 percent said they wash hands with soap after visiting 
the toilet. The lower percentage of those washing their hands after visiting the toilet may 
be due to broken water buckets at some schools, or buckets that lack water and/or soap, 
as observed by the ET. Additional emphasis on this topic is needed to achieve the target 
of 80 percent. It bears noting that there is a teacher in charge of hygiene at each school, 
whose duties include making sure that hand washing taps and soap are available and 
used, that students are organized to clean the latrines on a regular basis, and that students 
transfer hygiene practices to home and talk to parents about what they learned. The ET 
interviewed the teachers in charge of hygiene at each school. Teachers stated that students 
were learning the practices and that personal hygiene of students in particular had 
improved, but noted that schools and homes faced challenges with adequate water supply. 
97. The ET observed that schools and kitchens were generally clean but many handwashing 
stations were empty of water by midday, and that not all students used them. Head 
teachers said that deworming is carried out regularly, and at one school the ET observed 
deworming medicine being administered to students by a community volunteer from the 
government health centre. 
98.The ET observed several ADRA-built kitchens had poor smoke ventilation and breaks in 
the concrete flooring. WFP has taken over construction of new kitchens, using private 
contractors. In its interview with the ET, ADRA staff stated that quality was constrained 
by budget limitations and that kitchens suffered damage in part due to a lack of training 
and awareness on maintenance; WFP notes that the budget for kitchen construction was 
accepted by ADRA, which has considerable experience constructing kitchens. The new 
kitchens are being constructed according to a revised design and cost approximately 30 
percent more to build than the ADRA kitchens. It remains to be seen if the WFP￾constructed kitchens are affected by the same maintenance issues. The ET observed that 
where primary schools and secondary schools are co-located, their respective kitchens 
and stores are separate, including the newly constructed ones. While there may be donor 
constraints and concerns about low quality in most secondary school kitchens in this 
respect, the government is emphasising the integration of school facilities for greater cost 
efficiency and efficiency of use.
99.Drinking water was generally supplied by a combination of water brought from home and 
from local water points. Drinking water stations were often observed empty, with too few 
stations for the number of students. Responsibility to refill these stations resided mainly 
with the students and in some schools with the WASH focal point. Teachers indicated that 
students could ask for water to be refilled but this practice was rarely observed during the 
several hours spent at schools by the ET. The schools receive water purification tablets 
from World Vision for the drinking water stations, and schools are also supposed to 
contribute to the supply of tablets. At least two schools visited had run out of the tablets 
and were waiting for a new supply.
100. The schools visited had either older, government-built latrines, latrines built by 
World Vision under the McGovern-Dole Program and in a few cases, latrines built with 
community support. The USDA latrines were of better quality than the others, in terms of 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 32
|Page
building material (baked bricks instead of mud, quality of the slabs, ventilation, and 
accessibility). Students are required to clean the latrines during the school day. The ET 
also observed that some non-program latrines are not ventilated and emit a very strong 
odour and not all latrines were separated by gender; in some schools, they are divided by 
grade. Administrators cited insufficient latrines as the main reason for this. 
 Table 13: Student health and hygiene knowledge, baseline and midterm
Indicator Point Estimate
Baseline Midterm Target n
 Percent of students who can identify at least 3 
key health and hygiene practices (female) n/a 49.2 80.0 441
 Male students 48.0 48.6 80.0 220
Female students 47.0 49.8 80.0 221
 Source: Midterm EGRA survey
No significant difference between male and female students at midterm is observed at a p<0.05 level or lower.
101. Table 14 shows schools’ water sources and water availability, based on responses 
to the head teacher survey.79 We note that data collection was conducted at the start of the 
rainy season, which may have influenced responses. Nearly all schools (95 percent, 
extrapolated to 99 of the 104 schools in the program) report having access to an improved 
water source – primarily piped water.80 In some cases, this refers to a piped water point 
in the school; in others cases it refers to community water points in proximity of the 
school. Most schools visited by the ET have some type of access to piped water and also 
use rainwater harvesting tanks as improved water sources. However, qualitative findings 
also showed that access to piped water did not mean reliable availability of water. Due to 
a general water shortage, water is distributed across the grid and schools receive water to 
their piped water point on selected days of the week. Most head teachers interviewed 
reported access for three to four days per week when it was their turn to receive water. 
This is consistent with the survey data in Table 14, which show that 75 percent of schools 
reported water available from the primary source for more than four days per week. Head 
teachers stated that during the rainy season, shortages could be met by using the 
rainwater harvesting tanks; during the dry season, schools relied on piped water only, and 
several schools reported water shortages. Several of the most remote schools visited lack 
a piped water source; the main reason for this cited by head teachers is the high cost of 
the infrastructure required to provide piped water to remote locations. 
79 See para. 42 for a description of limitations of the perception-based head teacher survey.
80 This survey question was designed to replicate the baseline question, and as such only allows one response. 
Therefore it is possible that schools use more than one improved water source, at the same or different times of year; 
enumerators directed respondents to answer in terms of access at the time of the interview. Nuances regarding water 
sources were explored in qualitative work and are explained further in this paragraph.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 33
|Page
Table 14: School water source and availability at midterm
% of Schools Target # of Schoolsa 
Schools using an improved water source 95.0 104 99b
Piped water (improved) 80.0 n/a 83
Protected spring (improved) 10.0 n/a 10
Rainwater (improved) 5.0 n/a 5
Unprotected spring (not improved) 5.0 n/a 5
Water is available from primary water source 4+ 
days per school week 75.0 n/a 78
n 20 104
Source: Midterm head teacher survey a Values extrapolated from sample data
b There is a small discrepancy in the midterm value for this indicator as found in the MTE survey data 
(value: 99) and the 2018 semi-annual report (value: 104). The midterm survey finds that 95% of the 
sample has an improved water source, which was determined by enumerator observation, which suggests 
that at least one school was observed to lack an improved water source. In an email communication (28 
Nov 2018), the CO stated that at baseline, 62 of the 104 schools already had an improved water source 
(this is consistent with earlier documentation) and that since the baseline, World Vision has supplied 55 
schools with rainwater harvesting tanks “meaning that all 104 schools now have an ‘improved water 
source.’ ” A possible explanation for this discrepancy is a possible difference in how improved water source 
was defined at baseline and midline and/or interpreted by the respondents. 
102. School gardens are a standard feature of WFP’s school feeding programs, with an 
educational purpose to improve nutrition knowledge and education among families with 
malnourished children. The gardens function as demonstration plots to help students, 
school staff and families make the connection between growing food and good diets, 
develop life skills and increase environmental awareness. In cases where the gardens have 
some production, this produce can be added to school meals. GHI, which joined the 
McGovern-Dole Program in 2017, has so far focused on training school administrators 
and teacher representatives, including on garden strategy, compost making and 
agriculture tools for school demonstration plots in each district. The size of the plots was 
determined in consultation with the schools and depended on school commitment and 
resources. Schools organize student garden clubs to manage the gardens. Labour for 
garden maintenance differs per school and can be based on a combination of parents, 
school staff and students involved in the clubs. Some schools reported that parents also 
provided inputs for the gardens.
103. Qualitative findings indicated several challenges to the adoption of school gardens, 
including land availability at schools, damage by animals, resources to purchase garden 
inputs, and unreliable rainfall. The gardens are mostly cultivated in the rainy season. The 
ET’s visit was after a prolonged dry season and at the very beginning of the rainy season; at 
this time, the majority of school gardens observed were not well maintained and had not 
yet been prepared for cultivation. As a result, the ET was not able to properly assess 
gardening techniques. The qualitative findings show that so far student and parent interest 
in the gardening activities has been low. Another challenge is that non-trained teachers do 
not fully understand and support the garden activities, partly due to their misperception 
that trained teachers receive incentives to help maintain the gardens. GHI plans to evaluate 
the uptake of its interventions in 2019 to inform its activities in years 4 and 5, which will 
focus more strongly on nutrition education.
104. Table 15 presents findings from the head teacher survey regarding sources of food 
for school meals. WFP is working with local agricultural cooperatives to build their 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 34
|Page
capacity to eventually supply food to schools but the cooperatives have not yet started to 
produce food for the McGovern-Dole Program. While the objective of the school gardens 
is not to contribute to the school meals, about half of the sampled schools reported using 
food from school gardens for school meals. Qualitative findings indicate that the meal 
contributions from the gardens are generally very small and seasonal, mainly consisting 
of leafy greens when available. In some schools, qualitative findings showed that garden 
produce was also sold or given away to teachers and students. 
Table 15: Sources of food for school meals, at midterm
% of Schools # of Schools a 
Partnership with local farmer groups 0 0
Source of food for schoolchildren:
NGOs providedb 100 104
School garden 50 52
Local markets 40 42
Parents provided 5 5
n 20 104
Source: Midterm head teacher survey 
a Extrapolated from sample data
b The survey question is Where is the food for the children obtained? This response option is worded 
NGOs provided; there is no response option for WFP. NGOs include WFP, and there are no agencies 
providing food to the schools.
Evaluation Question 3: Community Participation in School Governance Mechanisms81
105. Because a parent survey was not possible,82 the head teacher module of the school 
survey included a question asking head teachers to estimate parents’ appreciation of the 
value of education. The teachers estimated that about 70 percent of parents (target: 90 
percent) were able to name three or more benefits of education (Annex 14, Table 36).83 
106. The head teacher survey also included a question asking head teachers to estimate 
what percent of cooks and storekeepers could identify three or more safe food preparation 
and storage practices. The survey finding for this indicator is 85 percent (target: 95 
percent); WFP staff believes this is an overestimation.84 While it is difficult to validate a 
quantified estimate of cooks’ and storekeepers’ ability to verbalize food storage and safety 
techniques, based on direct observation85 and on interviews with teachers, 
administrators, cooks and storekeepers, the ET observed that the majority of schools were 
practicing safe food preparation and storage practices. Project performance indicators 
show that 459 of 520 cooks and storekeepers (88 percent) were trained on food 
preparation and storage practices as of September 2018.86 The ET acknowledges that 
81 What is the level of community-level involvement and participation in decision-making in school governance 
mechanisms (PTAs and School Management Committees)? Particularly, what is the level of involvement and 
participation of women? What is the level and sustained continuity of community contributions in cash and in kind?
82 A parent survey was not feasible within the evaluation timeframe because it would have required approval by the 
National Institute of Statistics in Rwanda, typically a 3-month process. See paragraphs 47 and 260. 83 No further details are available on what benefits parents named. This question was added to the head teacher 
survey as a means of obtaining a general estimate of the parent populations’ knowledge around this group of concepts 
(not each one individually).
84 Based on CO comment (row 99) on an earlier report draft. Filename Comments on final report from 
EC_21.12.2018
85 The ET used WFP Rwanda training guidelines for food preparation and safe storage as the standard for visual 
assessment. 86 WFP Rwanda McGovern-Dole Semi Annual Report April 1 2018 to September 30 2018 final. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 35
|Page
knowledge does not always translate into consistent practice and some schools have water 
and infrastructure challenges; however as noted the staff at the schools visited by the 
qualitative team were using safe practices such as ensuring food was well cooked, utensils 
were washed after use, and kitchen items were washed after use and stored in a clean area 
of the kitchen. The ET also observed wall posters about safe food handling and storage in 
the kitchens.
107. The WFP Rwanda FY 2015 McGovern-Dole proposal sees engaging local 
communities as a means of engendering support for school feeding and, in line with 
government policy, helping ensure that local communities will by 2018 begin contributing 
to the McGovern-Dole Program through donations of food, fuel, labour and where they 
are able, cash donations.87 The proposal further states a 2020 goal of 75 schools 
contributing up to 30 percent of the school feeding basket through cash and in-kind 
contributions, though it notes that previous efforts to involve communities were not 
successful. The community contribution under the USDA model is a contribution by 
parents of approximately Rwf 200 (US $0.23) per child per month primarily to pay cooks 
and cleaners and to provide firewood to cook school meals. In some schools, students 
bring firewood from home. If parents are unable to provide a cash contribution, they can 
provide equivalent value in-kind materials or, in special cases, they are exempted by the 
head teacher from contributing. In such cases, children are still allowed to participate in 
the school meal program, although some parents questioned the fairness of this. 
Community contributions (often referred to as parent contributions) to school feeding 
and involvement in the design and monitoring of the McGovern-Dole Program are key to 
its transition strategy.88 The approach is used in many WFP school feeding programs in a 
number of countries.
108. At midpoint, the parent contribution model was operating, but facing challenges 
in the schools visited by the ET, despite frequent sensitisation. District education 
authorities, local cell representatives, NGO partners, school head teachers and others 
have been involved in sensitising parents to the importance of supporting the program. 
Less than a year into the planned parent cash contribution phase, the majority of schools
visited by the ET were receiving less than 50 percent of the requested contribution. KIs 
told the ET that many parents see school feeding as the responsibility of the school or 
WFP, distrust the school’s management of the money, or find it difficult to contribute 
when they have several children in school.89 Feedback from school staff, parents and local 
government representatives was consistent across all schools that it was generally the 
poorer households in the community who were unable or unwilling to provide cash 
contributions. The shortfall in cash contributions means that salaries for cooks are often 
delayed, schools have fewer cooks than they need and cooks need to work longer to 
compensate, and schools are forced to tap other school funds. The contribution of 
firewood is similarly low, again forcing schools to find alternate ways to obtain fuel for 
the stoves. Several stakeholders noted similar problems with the parent contribution to 
secondary school meals. While most key stakeholders say that continued sensitisation is 
the solution, the assumption that parent contributions will provide critical financial 
87 WFP Rwanda FY 2015 McGovern-Dole proposal. Page 5.
88 WFP Rwanda FY 2015 McGovern-Dole proposal. 89 Parents with students in secondary school are also expected to provide a cash contribution to support the 
government’s secondary school feeding programme.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 36
|Page
inputs to the McGovern-Dole Program needs to be examined. A question regarding the 
feasibility of the parent contribution was also raised in the 2017 WFP Country Programme 
evaluation.90
109. Table 16 presents descriptive data about Parent Teacher Associations (PTA) and 
their activities, as reported by head teachers. On average, each school has 1.5 PTAs or 
similar groups. A PTA commonly consists of parents, teachers and the school 
administrator.91 PTA meetings are led by the school administrator and in some cases 
community leaders also participate. PTAs monitor attendance and school meals, and 
assist with awareness-raising around health and nutrition, and support the reading clubs 
and school gardens. WFP and World Vision are building the capacity of PTAs to monitor 
school feeding, and the majority of PTAs have received trainings on a variety of topics 
related to health, nutrition, and governance. Only about one-third received training on 
school infrastructure, though discussions with the ET indicate that there is a need to 
provide more training and monitoring on the care and maintenance of the school 
kitchens. World Vision has introduced its social accountability approach, Citizen Voice 
and Action (CVA), to help raise awareness among community members on government 
responsibilities around education, health, WASH and school feeding, and the 
community’s rights to monitor the delivery of quality services. ET interviews with field 
staff indicated that parents of school children are now showing up for meetings and asking 
questions, and have become more engaged in the program since the introduction of the 
CVA approach. 
Table 16: School PTA engagement at midterm
Point Estimate
Mean number of schools with PTA (or similar group) per school 1.5
Mean number of active PTA members 9.6
Mean number of PTA meetings per year 3.8
Percent of PTAs to have available meeting minutes 100.0
Percent of PTAs to receive training 100.0
Percent of PTAs to receive type of training:
Nutrition / school feeding 90.0
School garden 85.0
Improved school management 80.0
School governance 65.0
Health 60.0
School infrastructure 30.0
n 20
Source: Midterm head teacher survey
Evaluation Question 4: Capacity of Local Institutions92
110. There are two topics around the capacity of local institutions: the ability to 
implement the McGovern-Dole Program, and the capacity to eventually incorporate some 
90 WFP. 2017c. Operation Evaluation. Rwanda Common Country Programme 200539, Mid-Term Evaluation (2013-
2016). Evaluation Report. Prepared by IRAM.
91 Typically the school administrator is the head teacher; these terms are used interchangeably.
92 What are the key institutions (i.e., international, national, provincial/district and local) and governance structures 
required to effectively deliver, implement, and sustain school meal interventions? What relationship structures 
among these institutions yield the most successful and effective school meal programmes? Is WFP’s capacity support 
to smallholder farmers and key line ministries appropriate/sufficient to effectively facilitate national ownership? Has 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 37
|Page
or all of the McGovern-Dole Program components into the national school feeding 
program independent of WFP assistance. In terms of McGovern-Dole Program 
implementation, there is strong support from partners and others for the components of 
the integrated approach. 
111. Table 17 presents the types of external support that schools receive, as found in the 
head teacher survey and discussions around this topic.93, 94 The survey findings suggest 
substantial external support for most activities included in the survey question, with 80 
to 100 percent of sample schools (perceived as) receiving given types of support. The 
exception is that only 55 percent of schools are perceived as having external support for 
health education. Regarding this unexpectedly low finding, the qualitative data indicate 
that many head teachers were aware of activities such as sanitation/WASH training, 
deworming activities, nutrition activities, and water purifying tablets; others mentioned 
that most health education is provided to communities, not directly to schools. 
MINISANTE was viewed as the main source of health activity support. Nevertheless, 
given the minimal progress on students’ health and hygiene knowledge (see discussion at 
para. 12) health education may be an area for further investment. 
112. According to discussions with head teachers, about 90 percent of the external 
support comes from WFP. Table 17 lists the organizations named by head teachers as 
providing each type of support. These findings suggest that while there is healthy support 
in program schools for core elements of an enabling learning environment, this support 
comes predominantly from WFP and USDA partners, which is a factor to be considered 
in a sustainability strategy for an integrated program. 
the provided capacity support increased the government’s capacity to own and sustain a national school meals 
programme?
93 The project does not specify targets for this indicator.
94 See para. 42 for a description of limitations of the perception-based head teacher survey.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 38
|Page
Table 17: External support to schools at midterm
% of 
Schools
# of 
Schools a Sources of Support
Receiving external support for: 
School feeding 100 104 WFP
Training of teachers 100 104 WFP, World Vision
Deworming 95 99 Government
Provision of school materials, textbooks, 
books 95 99 MINEDUC,
World Vision
Renovation/construction of infrastructure 
in school, e.g., classes, kitchens, stores 95 99 WFP, MINEDUC
Sanitation (water and toilets) 85 88 Plan International, 
World Vision, UNICEF
School governance 85 88 WFP, MINEDUC
Other nutrition activities 80 83 GHI
Health education 55 57 MINISANTE
n 20 104
Source: Midterm head teacher survey
a Values extrapolated from sample data
113. Current local institutional capacity to oversee and to implement the integrated 
McGovern-Dole Program components is generally good. At the district level, there is 
regularisation of monitoring and oversight activities among MINEDUC district staff 
funded by the McGovern-Dole Program, sector level officials, local mayors, and other 
stakeholders. There is regular coordination of stakeholders with monthly technical 
committee meetings on the McGovern-Dole Program attended by the DEO, local mayor, 
WFP and partners. After realising that program partners were not coordinating 
adequately, in 2017 WFP, WV and GHI field staff began monitoring school participation 
together. Schools found to be struggling with implementation are reported to sector heads 
and the DEO for follow-up. 
114. The staff knowledge and capacity at ministerial level is strong and progress on 
inter-ministerial coordination has been made by establishment of the inter-ministerial 
steering group for USDA, which is supported by the program. However, institutional 
capacity at both national and decentralized level to plan, manage and scale up relevant 
elements of the McGovern-Dole Program is not yet robust. At national level, there are still 
challenges associated with budget availability and role clarity among government 
departments in a scaled-up school feeding program. At the district level, DEOs and school 
administrators are able to monitor and implement the program but lack the capacity to 
take over responsibility for managing and implementing a USDA-style HGSF program 
with multiple components in their districts. 
115. As of 2017, WFP procured all food for the McGovern-Dole Program locally (though 
a temporary quality-related problem with a local supplier required a regional purchase in 
2018). There is presently no mechanism to directly link the farmer cooperatives and the 
schools. WFP’s long-term objective is to create sustainable connections between farmers 
and private sector buyers, or with government procurement services for the McGovern￾Dole Program. WFP is also assisting smallholder agricultural cooperatives in Southern 
Province and Western Province to improve production with the goal of enabling local, 
small-scale producers to supply commodities to the McGovern-Dole Program, and 
eventually to the government’s national HGSF. WFP trains cooperative members in 
improved methods including basic agronomy, post-harvest storage and handling, quality 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 39
|Page
standards, business management and marketing. To date, WFP has found that small 
improvements have a large impact on increasing production, though the farmer
cooperatives in Western Province presently have weaker capacity than those in Southern 
Province. One KI noted that all cooperatives nationally have issues with governance, 
leadership, and management, and these are key areas that WFP could help strengthen. 
Cooperative members interviewed by the ET said they are motivated to participate in the 
program in order to expand their markets and incomes, and because WFP is helping them 
by feeding their children at school. 
116. One of the key questions around the McGovern-Dole Program that WFP and the 
government are working on is a definition of what “home-grown” means in Rwanda and 
whether in the future the government’s national HGSF program will rely on nationally or 
locally-sourced commodities. A government HGSF program that relies on local sources 
for commodities would require that district offices, and particularly DEOs, have expanded 
budgets and additional capacity to undertake local procurement. The considerable 
logistics required for the McGovern-Dole Program are presently managed centrally by 
WFP. One KI states that government does not presently have the capacity, funding, skills 
or systems to undertake these logistics. The skills and infrastructure for this would need 
to be built over the medium term (five to seven years), with training and additional staff 
to take responsibility for procurement (including liaison with local cooperatives), 
logistics, transport, and quality control. The ET views this as feasible, given the necessary 
financial, staff and training resources. 
Unintended Results
117. The McGovern-Dole Program has yielded a few unintended results:
• While the McGovern-Dole Program is designed for primary schools, the program
is serving as a model for all schools who wish to implement school feeding. 
• The original kitchen model had a small modern stove that decreased cooking time. 
The stoves that were installed use larger cooking pots that require longer cooking 
time and more stirring. The unintended effect is that this favours male cooks, who 
tend to have more physical strength; the initial model was easier for women to use.
Key Findings and Conclusions – Results
• Testing of students in the McGovern-Dole Program show a substantial 
improvement in reading scores over students tested before the program. 
• In the McGovern-Dole Program, school feeding is seen by many parents as the 
school’s responsibility, and fewer than 50 percent of parents are contributing to the 
costs of cooks, fuel and water. 
• School gardens are proving problematic due to limited inputs, lack of water and 
socialization among parents.
• Little progress from baseline has been made with respect to uptake of key health 
and hygiene messages
• Time and workload are potentially as, or more, important than availability of 
reading materials in determining if a student reads outside the classroom.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 40
|Page
2.3 Factors Affecting the Results 
Internal Factors
118. Partners’ capacity. Strong technical and implementation capacity has 
contributed to the strength of the program. World Vision has contracted expert literacy 
and education staff to support the program. WFP itself has extensive experience in school 
feeding programs. GHI is a new partner as of 2017, with over a decade of experience of 
providing agricultural solutions to reduce child malnutrition in Rwanda. The partnership 
with ADRA for kitchen construction was the exception, and WFP has replaced ADRA with 
private contractors for kitchen construction. 
119. Program management. All partners reported that they were satisfied with WFP 
management of the program. Local coordination among WFP, NGO partners, and 
McGovern-Dole Program managers in MINEDUC and MINAGRI is good, with 
stakeholders reporting that they are in regular contact through formal and informal 
mechanisms. Coordination and communication between WFP and other stakeholders at 
the district and local level is strong. WFP supports four USDA HGSF District Coordinators 
who have offices in district-level MINEDUC offices to ensure regular interaction. There is 
regular interaction between program partners and schools. Interviews show a strong 
emphasis on school activity monitoring at all field position levels but also indicate that 
there may be duplication in monitoring focus with insufficient sharing of information 
among program partners, and with government. 
120. One area for improvement noted by partners was that WFP should strengthen its 
evidence building, communications and advocacy for the program as a whole. To date, 
WFP visibility and advocacy actions have focused mainly on the school feeding activities, 
which are WFP’s traditional strength. Interviews with program staff and government 
representatives indicate that coordination for activity planning is strong but that internal 
reflection on strategic and technical issues is at times lacking, particularly for the 
education activities. For example, awareness of WFP staff on the results, challenges and 
opportunities of the Literacy Boost activities is not consistent, particularly at the national￾office level. Similarly, interviews show that internal WFP coordination among technical 
functions such as nutrition, logistics, and GEEW is still at a low level, although the ET 
notes that this is improving as a result of recent management changes and better role 
clarity. This has allowed staff to focus on specific McGovern-Dole Program management 
and technical functions, as opposed to being spread too thinly across a range of WFP 
activities.
121. Program staffing capacity. Staffing levels are sufficient at operational level but 
insufficient at national level. At district level, the McGovern-Dole Program benefits from 
enough implementing and technical staff among all partners, including MINEDUC and 
MINAGRI. All partners indicated that budgets allocated to staffing are sufficient to 
complete the program activities. However, at national level, the senior staffing positions 
are not adequately resourced. Interviews indicate a staffing deficit since the start of the 
program and emphasize a current need still exists for improved staffing of knowledge 
management and advocacy functions, stronger support and activity direction for the 
embedded staff at MINEDUC and MINAGRI, and a stronger leadership structure with 
sufficient seniority to (1) manage activities across the WFP internal technical units and 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 41
|Page
the broader partnership, and (2) to engage directly with government stakeholders at 
senior level. 
122. Staff turnover within WFP. Efficiency was hampered by a lack of clarity 
around roles and responsibilities of WFP staff associated with the McGovern-Dole 
Program, in part due to changes in staffing over the past year. This has been a challenge 
in terms of historical knowledge of the approach. At the time of the MTE, there were 
knowledge gaps and reliance on former staff to understand how certain decisions (food 
basket, selection of cooperatives) were made and what intended next steps were (e.g., 
relating to advocacy and government engagement).While turnover was not particularly 
higher for the McGovern-Dole Program than for other WFP programs, the timing of 
turnover of the McGovern-Dole Program Manager coincided with changes in senior CO 
management at a critical time. Interviews with past and present WFP staff indicated that 
many of the ET observations were shared by previous staff, which indicated positive 
intent for adaptive management, but that these individuals had left before a concerted 
effort to institutionalize associated activities into the program could take place. 
123. Procurement model. Information for strategic decisions around local 
purchasing models was not available in the early stages of the program. The resulting lack 
of clarity around the procurement strategy has caused confusion and differing 
expectations around key program interventions such as the parent contribution, linkages 
with local cooperatives and discussion with government on continuation and integration 
of the USDA-supported activities with the government school feeding program. 
124. The ET acknowledges that WFP is currently in the process of supporting 
government to define the local procurement strategy, through a multi-sectoral task force 
consisting of MINAGRI, MINEDUC, MINECOM and WFP. Having this strategy in place 
would enable the McGovern-Dole Program to better align with government needs and 
priorities. However, this research will only be completed in Year 4. This is likely too late 
for the McGovern-Dole Program to realign support for the development of the local 
procurement strategy (and) to inform the Government strategy for scaling up school 
feeding. In particular, the lack of clear strategic direction in the first half of the program 
has limited the role of the staff embedded at national and subnational levels who are well￾positioned – but lack sufficient direction – to engage constructively in government 
planning and budgeting processes. 
125. Literacy initiative planning. The McGovern-Dole Program has not yet 
undertaken a multi-stakeholder initiative to explore the possibility of linking Literacy 
Boost-type activities with similar initiatives undertaken by government in partnership 
with other organizations, and to assess whether there is interest and opportunity for 
specific elements of literacy boost to be integrated into the government education 
activities. There was limited focus on strategic sector engagement in the first half of the 
program. This has restricted the effectiveness of program efforts at subnational and 
national levels to promote awareness of Literacy Boost results and contribute lessons 
from the McGovern-Dole Program experience, both of which the government could draw 
on in its efforts to improve early literacy. This strategic engagement is needed to lay the 
ground for a USDA HGSF continuation strategy and to inform any future role of partners 
in supporting government education planning and budgeting. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 42
|Page
126. WASH infrastructure. The availability of a reliable, year-round water supply is 
critical to achieving the program’s health and hygiene objectives. The ET finds that the 
program is working toward this but is not yet fully on track to ensure reliable water 
availability in all program schools. As discussed in para. 101, several schools visited 
indicated periods without a reliable water supply, especially in the dry season. The ET 
also observed that in the most remote schools, infrastructure for piped water that can 
provide year-round water access is often poorly maintained or located too far from the 
school to allow convenient access. The McGovern-Dole Program baseline also noted this 
challenge, and stated that the program needs to ensure that clean water is available to the 
schools. We reiterate this prerequisite as an internal factor but, as water supply is a joint 
responsibility of WFP and the government, it is a shared constraint. The ET notes that 
WFP and the government have acknowledged the need for better water access in schools, 
and WFP is in the process of addressing this constraint by shifting additional resources 
into linking with external water initiatives. 
External Factors 
127. Support from school system. An external factor contributing to program
performance is that schools are generally well organized with regular support from district 
officials, and most of those visited by the ET have the basic infrastructure needed to carry 
out the program components. Head teachers and teachers are familiar with McGovern￾Dole Program goals and activities, and are well acquainted with having to meet 
performance targets. However, several teachers and administrators reported feeling 
pressure over government performance targets to promote students to the next grade 
before they were ready. This compromised program results in the Literacy Boost activities 
which are focused on grades 1-3 only; those who were graduated into fourth grade before 
they were ready lost the support of in-school literacy boost activities. 
128. Supportive national policies. A supportive policy environment exists and the 
government’s commitment to expand school feeding is reflected in mainstream 
government documents such as the Education Sector Strategic Plan 2013-2018 and the 
National School Health Policy 2013. At the time of the evaluation, the government’s 
school feeding policy and strategic plan had been pending for two years; a new policy to 
scale up school feeding was pending approval by the Cabinet and the government is 
planning to expand its school feeding as part of the phased elimination of the double shift 
in schools between 2018 and 2024. 
129. Government capacity. Government capacity to manage and monitor the 
McGovern-Dole Program at the subnational level is strong, with good systems and 
communications between district education officials and schools. However, there are no 
operational M&E and communication systems between schools, districts and the national 
level, and government capacity to monitor the national program is weak. Limitations to 
government budget and capacity at the national level to scale up the national school 
feeding program over the next few years, including addressing key issues such as water 
access for schools and the possibility of incorporating elements from the McGovern-Dole 
Program, are significant factors affecting McGovern-Dole Program sustainability. 
Interviews with government officials indicated that the planned national expansion of the 
current school feeding program from secondary to primary schools already presents a 
significant budgetary challenge, and that USDA support for school feeding in the most 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 43
|Page
food insecure areas will be necessary for at least five more years before the government 
has the funding and capacity to support it. The CCP recommendation that WFP prioritise 
a capacity development strategy for CCP activities also applies to the McGovern-Dole 
Program, where the ET also finds a need for WFP to undertake greater capacity 
development of government partners as part of a longer-term strategy for program 
transition and sustainability.
130. Communication and coordination. The program faces challenges related to 
a communication and coordination gap between national and subnational levels. The 
program design, particularly the two-pronged strategy to embed McGovern-Dole 
Program staff at the national and district level, presumes effective and efficient 
coordination between these two government levels. MTE interviews show that this is not 
necessarily the case. The main bridge between districts and central government are the 
mayors, who are already overcommitted due to their role in government decentralization 
processes. In addition, mayors are generally not targeted by McGovern-Dole Program 
activities, which instead have greater involvement of the vice mayors as the highest-level 
representation from the district. The positive progress made in schools supported by 
USDA, including potential implications for government planning and budgeting, is highly 
visible at district level but not communicated to the national level. As a result, the clear 
mandate from national government to district officials to more formally engage in 
McGovern-Dole Program continuation and integration activities is lacking. Interviews 
with program staff and government representatives show extensive coordination and 
monitoring efforts of McGovern-Dole Program activities, but more limited purpose in 
working towards government systems strengthening that can sustain McGovern-Dole 
Program activities. 
131. Parent contributions. The program design is based on the assumption that 
parent contributions will play a large part in covering the costs of a government-led school 
feeding model. This stems from the current system where parents contribute funds to the 
secondary school feeding program and the common practice by district officials and 
school administrators to approach parents to request co-financing for most local 
infrastructure projects. As noted, school administrators in all schools reported that they 
are already struggling to raise the funds from parents to cover the salaries of the cooks 
and the cost of firewood and water. Interviews with parents, school staff and district 
officials indicate that most parents should be able to provide these financial contributions 
and even support higher-cost items such as food purchasing in the future. However, 
parents are not contributing to the extent expected by the program. For this to change, 
other community basic needs such as road and water infrastructure will need to be met 
first, and more sensitisation with parents is required to prioritize education expenditure. 
Thus far, the parent contribution model is not providing the expected inputs and it is 
unlikely this will change in the short- to medium term. 
Key Findings and Conclusions – Factors Affecting the Results
• Partner capacity and coordination for activity planning is strong.
• Under-capacity and staff turnover at national level, and limitations to senior staff 
oversight at the national level, are constraints to the program.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 44
|Page
• WFP and government are still in the process of defining the options for the local 
procurement strategy, which is key to scaling up the national school feeding 
program and informing USDA support to this process. 
• Schools are generally well organised, supported by district officials, and have the 
infrastructure to implement the program. 
• Government capacity to manage and monitor the program at the subnational 
level is strong. However, budget and capacity to scale up the national program 
and to integrate elements of the McGovern-Dole Program in the next few years 
are significant factors for sustainability. 
• There is good policy and infrastructure support from the government for the 
McGovern-Dole Program, although there is room for improvement in setting a 
national policy and strategy for scaling up school feeding, and for linking with 
key sectors such as WASH.
• There is a communication and coordination gap between national and 
subnational levels of government in the program. 
3. Conclusions and Recommendations 
3.1 Overall Assessment/Conclusions
132. This section presents the evaluation’s main conclusions, organized by OECD-DAC 
criteria. 
Relevance
133. The McGovern-Dole Program is aligned with the government’s education and 
health policies and priorities. It is relevant to the food security and educational needs of 
targeted schools to provide a healthy, good-quality learning environment for girls, boys, 
students with disabilities, and children from poor families. The program provides the 
same meal to girls and boys, ensuring that girls have equal access to nutritious food and 
supporting gender equality in educational opportunity. The program is also relevant to 
community needs and interests, as attested by teachers with and school officials with 
regard to high enrolment and low dropout rates, and to the interests of smallholder 
agricultural cooperatives to connect to new markets. Partners’ technical and 
implementation capacities are appropriate to achieving program objectives. WFP 
Rwanda’s role as school feeding implementer is shifting to an upstream role of providing 
specialised technical support and guidance to strengthen national capacity, which is 
consistent with WFP’s global strategic direction.
134. The focus on primary schools aligns closely with the government commitment to 
expand its secondary-school-level feeding program into primary schools. The expansion 
in school feeding complements the shift from half-day to full-day classes in primary 
schools. The national school feeding program expansion will start with primary grade 6 
and progressively expand to one additional grade level each year to include all primary 
school students by 2024.
Effectiveness
135. The McGovern-Dole Program represents the first pairing in Rwanda of in-school 
meals and community-based reading groups to boost early-grade literacy. While the MTE 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 45
|Page
did not include a control group for schools, the results at midterm for reading 
improvement in the USDA-supported schools are encouraging. Education officials also 
report increased attendance and reduced dropout. Retention rates for both girls and boys 
are well above target levels. Students’ reading and comprehension in Kinyarwanda show 
that progress is on-track relative to target, and there are no statistically significant 
differences between boys and girls. The parity in reading skills between girls and boys 
indicates that the program is effective in helping ensure equitable access to education for 
girls. By the endline in 2020, the program should have generated evidence that this 
pairing, along with high-quality health, nutrition and WASH interventions, and parent 
and community support, is an effective way to improve early-grade reading skills. 
136. WFP is working closely with partners at the district level and local collaboration is 
strong. For two years, WFP Rwanda has been collaborating at the national level through 
steering committee meetings with MINEDUC and the REB. The key government 
institutions required to effectively implement the program are involved, though the 
program would benefit from greater inclusion and involvement of the REB. For the 
McGovern-Dole Program to be more effective over its remaining life, it is important for 
WFP to advocate with government to bridge the gap between national and decentralized 
government and ensure that district governments are communicating with national 
government stakeholders, a critical vertical linkage that is not present now.
137. There were problems regarding the quality of kitchens constructed by ADRA in 
USDA-supported schools, which ADRA attributed to budget constraints and a lack of 
training and awareness on maintenance – though WFP notes that kitchens constructed 
by ADRA started deteriorating within three months of construction. WFP ended the 
partnership with ADRA and improved the kitchen design; it is now using private 
contractors and a revised, higher-cost design to construct higher-quality kitchens. As 
noted, due to the design of the kitchen stoves, schools mainly hire male cooks while 
women are employed as cleaners, which affects equity in employment for women. 
138. Effectiveness of the school garden activities and training in strengthening nutrition 
awareness under the McGovern-Dole Program is low. This is mainly due to the later start 
of the garden activities, as well as low interest and capacity at school level to maintain the 
gardens and the associated nutrition education activities. In contrast, the inclusion of 
nutrition into the school classroom curricula is more effective.
Efficiency 
139. A cost-benefit analysis conducted in 2017 concluded that every dollar invested in 
school meals in Rwanda can generate a return of US$4.80 and US$5.60 for home-grown 
and in-kind modalities, respectively, over a child’s lifetime.95 The calculation involves 
attributing a dollar value to the following assumed benefits of the intervention: value 
transfer (which comes entirely from the food value transfer), return on investment, 
improved education and increased productivity, healthier life, and gender equality. The 
benefits are monetized based on assumptions about program impacts and indicate good 
value for money. In the study cited, the majority of the benefit (72 percent) is derived from 
95 MasterCard. 2017. The School Feeding Investment Case: Cost-Benefit Analysis in Rwanda. Report prepared by 
MasterCard for World Food Programme. The report uses “in-kind” to refer to the USDA-supported CSB+ and sugar 
package distributed in the Western Province, and “home-grown” to refer to the locally purchased maize meal, sugar, 
vegetable oil and salt package distributed in Southern Province.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 46
|Page
improved education and increased productivity. The latter is measured in terms of 
projected increases in income derived from wages due to better education and cognition. 
The ET agrees that both the in-kind and home-grown modalities offer clear benefits to 
school children, and the CBA provides a valuable tool for advocacy with government about 
the positive return on investment to the country. Interviews with program partners and 
sector stakeholders indicate that overall value for money of the McGovern-Dole Program 
is high considering total budget with results achieved so far. This is the case both in terms 
of changes at school and community level and the potential contributions of the USDA
model – or elements of this model – to the government expansion of the national school 
feeding program. 
140. Qualitative findings and a review of progress reports indicate that the program has 
been implemented in a timely manner, with only some delay in the school garden activity 
(see para. 55). Program management of activity roll-out, and activity and output 
monitoring, has been efficient. Staff efforts translated directly into effective activity roll￾out without unnecessary time spent on iteration, coordination and decision-making 
processes; planning meetings are generally efficient. One hindrance to efficiency, based 
on the qualitative findings, is that multiple program stakeholders are undertaking similar 
activities at school level, such as various forms of reading tests, including the EGRA. 
School staff and students report this is time consuming and that they do not receive clear 
feedback based on these monitoring activities; in their opinion, having duplicative 
reading tests conducted by WFP, World Vision and government staff is not an efficient 
use of school time. There are also some improvements that can be made to the 
management function in the second half of the program, as the program shifts its focus 
to making a more strategic contribution to the government school feeding expansion. This 
is further discussed under Factors Affecting Results. Findings also indicate that the 
selection of the Literacy Boost method – and its adaptation to the Rwanda context– to 
achieve the literacy results was based on global evidence at the time of program
conceptualization and is generally considered an efficient approach. No specific study was 
undertaken by the program to compare Literacy Boost to other literacy models. 
141. There are areas where efficiency gains could be made in terms of program design 
and aspects of implementation, specifically for the gardens and kitchen activities. The 
later start of the school garden activities, combined with the limited interest at school 
level, raise a question whether continuing resource allocation to this activity in the last 
two years will lead to the expected results related to nutrition education. Through the 
experience with the garden activities so far, the McGovern-Dole Program has an 
opportunity to contribute to the sector discussion on whether school gardens are an 
effective and efficient nutrition education intervention, and what the implications are for 
school gardens as an explicit element of school feeding program design. 
142. The McGovern-Dole Program attempted to develop a cost-effective model for 
school kitchens. However, limited resource allocation to construction quality, specifically 
durability of materials, needs to be accompanied by increased efforts in maintenance 
awareness and capacity. In the absence of this, the more cost-effective kitchens will 
damage quickly and rehabilitation costs will negate the initial investment savings, as is 
currently the case in the McGovern-Dole Program. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 47
|Page
Impact and Sustainability 
143. The WFP Rwanda FY 2015 McGovern-Dole proposal envisioned a progressive 
transition to national funding and implementation between 2020 and 2025, given stable 
multi-year funding from the government. While McGovern-Dole Program components are 
functioning well and outputs are on track, WFP needs to put greater focus on developing a 
comprehensive transition process and strategy with the government. WFP notes that the 
government will not assume all McGovern-Dole Program components; rather, the 
program proposal describes the second component of transition to government as 
supporting and encouraging communities to contribute to school feeding, literacy and 
hygiene activities.96 
144. The McGovern-Dole Program’s support to farmer cooperatives to develop 
improved agricultural and business practices is an important contribution to the 
sustainability of school feeding in Western and Southern provinces, as the aim is for the 
cooperatives to eventually supply commodities to the McGovern-Dole Program. The 
McGovern-Dole Program effort to support the government in defining options for a local 
procurement strategy is a critical step for scaling up of the national school feeding 
program and is a key sustainability factor for the McGovern-Dole Program. However, the 
two years left in the program do not allow sufficient time for these efforts to be reflected 
in government policy development, planning and budgeting for school feeding; the ET 
anticipates that a second phase of USDA support will be required to maximize the 
contribution of the McGovern-Dole Program results to an expanded government school 
feeding program. 
145. The community contribution model, an important part of the USDA model and of 
the government model for a sustainable national school feeding program, is not a 
sustainable financing option in its current form. A major challenge is that the wide range of 
parent contributions, and factors that affect parents’ decision-making – especially among 
poorer female-headed households – is not yet fully understood. A more structured 
approach to documenting and validating the factors affecting parent contributions to 
different household and community needs is required. This will enable an informed 
approach to leveraging household and community resources to support sustainability. 
146. At the time of the midterm evaluation, the McGovern-Dole Program had not yet 
initiated a multi-stakeholder dialogue with government and other, similar, literacy 
initiatives to inform the program continuation and the possibility of a government 
integration strategy for literacy activities. This has restricted the effectiveness of 
McGovern-Dole Program efforts at subnational and national levels to promote awareness 
of the Literacy Boost results and their relevance to government literacy investments. 
Government understanding of the Literacy Boost process and literacy results, and 
government support to incorporating relevant elements into education planning and 
budgeting, are critical factors for continuation of this McGovern-Dole Program element, 
to the extent that it can strengthen existing literacy initiatives and meets a government 
need. 
96 WFP Rwanda FY 2015 McGovern-Dole proposal.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 48
|Page
Summary Assessment
147. WFP’s School Feeding policy states that three factors are critical for the transition 
from a WFP-supported to a nationally-owned school feeding program: (1) an appropriate 
policy or legal framework; (2) the institutional capacity to implement a program; and (3) 
the financial capacity to fund it. It further states that the move from low-income to lower￾middle income country status is the strongest indicator of readiness to finance a school 
feeding program.97 The first two factors are in process with the support of WFP. With 
respect to the third factor, the government has stated its intention to support a national 
program, including primary schools, but acknowledges that the financial commitment 
has not yet been completed.
148. The McGovern-Dole Program has made a good start in establishing consistent, 
integrated, and collaborative operations and in successfully implementing and 
monitoring a multi-faceted program. At midpoint, WFP and the government are building 
the systems and linkages needed for a successful and sustainable program. It will take 
another five-year phase to establish the systems, capacities and resources necessary for 
government to integrate some or all of the McGovern-Dole Program activities into the 
national HGSF program. Discussions are currently underway within the WFP/ 
McGovern-Dole partnership to support the government to develop a roadmap that 
clarifies which elements of the McGovern-Dole Program align with government priorities 
and the extent that these can be integrated into government planning.
Key Findings and Conclusions – Overall Assessment
• The McGovern-Dole Program is relevant to, and consistent with, existing 
policies.
• It should generate, by its end, data showing that a model linking in-school meals 
and literacy boost type activities is effective and can provide useful lessons and 
insights to a national school feeding program. 
• The program will need additional time past 2020 to establish the necessary 
systems, staffing, logistics, and financing for a sustainable national program. 
Envisaged transition milestones are unlikely to be reached by 2020.
3.2 Recommendations
149. The following operational and strategic recommendations are based on the 
evaluation findings and conclusions. They are presented by priority within operational 
and strategic categories, with suggested implementation timelines and responsible 
parties. As there are only two years remaining in the program, which contains much 
promise and much yet to do, most of the recommendations are high priority and should 
begin in the short term. Ultimately, WFP and government capacity to undertake the work 
required to move forward and assist government in its scale-up of the national school 
feeding program for primary schools will determine the priority and sequencing of 
actions. 
97 WFP. 2013.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 49
|Page
Operational Recommendations (2019 – 2020)
150. Recommendation 1: Strengthen WFP management, role clarity and 
staff capacity for functions related to the McGovern-Dole Program. (High 
priority; short term [2019 first quarter]; CO responsible.) This includes strengthening 
current management oversight and senior manager engagement in advocacy on relevant 
McGovern-Dole Program elements to support the government’s school feeding 
expansion. The engagement of senior management is also critical to guide transition 
planning, which includes exploring – with government and sector partners – a potential 
Phase 2 of the McGovern-Dole Program; such a phase would focus on providing tailored 
support to the government’s school feeding expansion and literacy investments. Establish 
more regular and structured technical coordination with WFP units like nutrition, 
logistics, M&E and gender, and with the technical experts and units of the McGovern￾Dole Program partners. 
151. Improving the organization of the multiple functions required for the McGovern￾Dole Program can be done by establishing either (1) an overarching school feeding 
subunit, which would provide a high level of structure to clarify roles and responsibilities 
of existing staff, or (1) a less structured working group with strong engagement of senior 
management. The selection and design of either option needs to be further explored by 
WFP, taking into account national portfolio priorities and workload. In any case, it is 
recommended to update TORs for the remaining life of the program, to reflect roles and 
responsibilities of McGovern-Dole Program staff and coordination structures.
152. One recommended step to enhance current McGovern-Dole Program staff capacity 
is to add a knowledge management and advocacy staff position, and to initiate a process 
to integrate this function into district-level TORs as well. This position/function will be 
critical to provide dedicated support to continuing and transitioning McGovern-Dole 
Program activities, including guiding engagement with the government to support to the 
national school feeding program expansion.
153. Recommendation 2: Organize reflection meetings to inform 
knowledge management, advocacy and strategic thinking. (High priority; short 
term [begin first quarter 2019]; CO responsible.) The meetings should focus on how the 
McGovern-Dole Program can best contribute to the expansion of the national school 
feeding program. The reflection meetings should include national and district staff and 
be held on a semi-annual basis. These meetings are not meant to duplicate coordination 
around activity planning, which is already taking place. These are strategic and technical 
reflection meetings to ask: What is the evidence telling us about the strength of our 
approach? What should be considered for inclusion in an expanded national school 
feeding program? Such reflection will contribute to internal assessment of what is 
working well – especially related to evolving government needs – and inform decisions 
around what activities should be prioritized over others. 
154. Recommendation 3: Contribute to the development of a school kitchen 
model that integrates primary and secondary school kitchen infrastructure 
and can still be supported by parent and community contributions. (Medium 
priority; medium term [by end 2020]; CO responsible.) Primary and secondary schools 
typically are on separate campuses, and their kitchen facilities and stores are thus 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 50
|Page
currently separate. This follows the government’s approach, which already emphasizes 
the integration of facilities. In some schools, where the national secondary school feeding 
program and the McGovern-Dole Program in primary schools operate in parallel, good 
school-led practices are already emerging on how to integrate these school feeding 
activities using parent contributions to develop common infrastructure, which takes into 
account the higher-quality construction introduced by USDA. Defining this model will 
involve examining how to ensure quality given limitations to infrastructure budgets, 
specifically by strengthening staff training and kitchen maintenance practices. The model 
would serve as a demonstration for the national government’s integrated school feeding 
in the future. 
Strategic Recommendations (2019 and beyond)
155. Recommendation 4: Initiate a structured transition or continuation 
planning process with the government. (High priority; short term to end of 
program [2019 first quarter]; CO responsible.) Based on the results of this process, WFP 
can initiate a transition of relevant McGovern-Dole Program elements into the national 
program at the end of the current McGovern-Dole Program in 2020, and/or engage with 
the government to design a Phase 2 to the current McGovern-Dole Program that provides 
tailored support to the school feeding expansion process. For both options, it is essential 
that WFP and government co-lead a planning process so that roles and expectations going 
forward are clear to USDA and government partners. This requires developing a 
comprehensive process and strategy with the government that brings all partners together 
to map out their needs, capacities, and potential contributions and responsibilities going 
forward. 
156. One option to support implementation of this recommendation is for WFP to 
advocate the establishment of an intersectoral working group/platform, led by the 
government with backstopping from WFP as necessary, that focuses on the school feeding 
and education nexus. In this way, the proposed planning process could also be linked to 
the preparation for an updated Systems Approach for Better Education Results (SABER) 
activity. A SABER activity would produce comparative data and knowledge on education 
policies and institutions that would provide input to the government on ways to 
systematically strengthen their education system, and is a good practice in school feeding 
transition planning. Such a platform would ensure that national embedded staff play 
pivotal roles in moving the development of the national school feeding program forward. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 51
|Page
Annexes
Annex 1: Terms of Reference 
Terms of Reference
MIDTERM AND ENDLINE EVALUATIONS of
WFP Rwanda’s Home Grown School Feeding Programme 2016-2020 
(USDA McGovern Dole Grant FFE-696-2015/007-00) 
in 2018 and 2020 for WFP Rwanda 
(TOR original table of contents and annexes omitted) 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 52
|Page
Table of Acronyms
CBEHPP Community‐based Environmental Health Promotion Program
CFSVA Comprehensive Food Security and Vulnerability Analysis 
CHAI Clinton Health Access Initiative
CHW Community Health Workers
CO Country Office
CSR Country Strategic Review of Food Security and Nutrition 
DDP District Development Plans
DEO District Education Officers
DEQAS Decentralized Evaluation Quality Assurance System
DFID Department for International Development 
EB Executive Board
EC Evaluation Committee 
ERG Evaluation Reference Group
ESSP Education Sector Strategic Plan
EQAS Evaluation Quality Assurance System
FAS Foreign Agricultural Service
GEWE Gender Equality and Women’s Empowerment
HGSF Home Grown School Feeding
LRP Local and Regional Procurement
MDG Millennium Development Goal(s)
MGD McGovern Dole
MINAGRI Ministry of Agriculture
MINALOC Ministry of Local Affairs
MINEDUC Ministry of Education
MINESANTE Ministry of Health
MININFRA Ministry of Infrastructure
NST National Strategy for Transformation
OEV WFP Office of Evaluation
PTA Parent-Teacher Association
RB(N) Regional Bureau (Nairobi)
REB Rwanda Education Board
SDG Sustainable Development Goal(s)
SEO Sector Education Officers 
SMC School Management Committees
SPR Standard Project Report 
WFP World Food Programme
UN CCA UN Common Country Analysis
UNDAP UN Development Assistance Plan 
UNDSS UN Department of Safety and Security 
UNHDI UN Human Development Index 
USAID United States Agency for International Development
USDA United Stated Department of Agriculture
WASAC Water and Sanitation Corporation
WFP Rwanda McGovern-Dole Midterm Evaluation Report 53
|Page
1. Introduction
157. This Terms of Reference (TOR) is for the midterm and endline 
evaluations of WFP Rwanda’s Home Grown School Feeding (HGSF) 
Programme 2016-2020 (USDA McGovern Dole (MGD) Grant FFE-696-
2015/007-00) in Rwanda. These two activity evaluations are commissioned by 
WFP Rwanda and will take place from June/July to December 2018 and July 
to November 2020, respectively. 
158. The TOR was prepared by WFP Rwanda based upon an initial document 
review and consultation with stakeholders and following a standard template. 
The purpose of the TOR is twofold. Firstly, it provides key information to the 
evaluation team and helps guide them throughout the evaluation process. 
Secondly, it provides key information to stakeholders about the proposed 
evaluations.
159. The HGSF Programme supports around 85,000 primary students 
annually across 104 schools in Rwanda’s poorest and most food insecure 
districts – Nyaruguru and Nyamagabe in the south and Rutsiro and Karongi in 
the west (see map in Annex 3). Children in the south receive a daily hot meal 
whereas students in the western province are provided a porridge meal. As a 
contribution to the project, some schools occasionally provide locally-grown
vegetables to enrich the meals. The programme also undertakes activities to 
improve student literacy outcomes, increased use of health and dietary 
practices, including WASH, setting up school gardens, providing deworming 
medication and supporting the strengthening of government staff capacities. 
2. Reasons for the evaluations
2.1 Rationale
160. The midterm evaluation is being commissioned to provide an evidence￾based, independent assessment of the project’s performance so that WFP 
Rwanda and its project partners, World Vision and Gardens for Health 
International, can adjust course as necessary for the remainder of the project 
term. 
161. The endline evaluation is being commissioned to provide an evidence￾based, independent assessment of the project to evaluate its success, ensure 
accountability, and generate lessons learned. 
2.2 Objectives 
162. Evaluations in WFP serve the dual and mutually reinforcing objectives 
of accountability and learning.
• Accountability – The midterm and endline evaluations will assess and report 
on the performance and results of WFP Rwanda’s Home Grown School Feeding 
(HGSF) Programme 2016-2020.
• Learning – The evaluations will determine the reasons why certain results
occurred, to draw lessons and derive good practices for learning. It will also 
provide evidence-based findings to inform operational and strategic decision￾making. Findings will be actively disseminated and lessons will be incorporated 
into relevant information-sharing systems.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 54
|Page
163. Specifically, the midterm evaluation will i) review the project’s relevance, 
effectiveness, efficiency, impact, and sustainability, ii) collect performance 
indicator data for strategic objectives and higher-level results, iii) assess 
whether the project is on track to meet the results and targets, iv) review the 
results frameworks and theory of change, and v) identify any necessary mid￾course corrections. Hence, more weight is given to learning as can be expected 
for a midterm evaluation. 
164. Specifically, the endline evaluation will i) review the project’s relevance, 
effectiveness and efficiency, impact, and sustainability, ii) collect performance 
indicator data for strategic objectives and higher-level results, iii) assess 
whether the project has succeeded in achieving McGovern Dole’s two strategic 
objectives, iv) investigate the project’s overall impact, and v) identify 
meaningful lessons learned that WFP, USDA, and other stakeholders can apply 
to future programming. Hence, about equal weight is given to learning and 
accountability.
2.3 Stakeholders and users
165. Several stakeholders both inside and outside of WFP have interests in 
the results of the evaluations and some of them will be asked to play a role in 
the evaluation processes. Table 1 below provides a preliminary stakeholder 
analysis, which should be deepened by the evaluation team as part of the 
inception phase. 
166. Accountability to affected populations is tied to WFP’s commitments to 
include beneficiaries as key stakeholders in WFP’s work. As such, WFP is 
committed to ensuring gender equality and women’s empowerment (GEWE) in 
the evaluation process, with participation and consultation in the evaluation by 
women, men, boys and girls from different groups. 
Table 1: Preliminary stakeholders’ analysis
Stakeholders Interest in the evaluations and likely use of evaluation 
reports for this stakeholder
WFP STAKEHOLDERS
Country Office 
(CO) Rwanda
Responsible for the country level planning and operations 
implementation, WFP Rwanda has a direct stake in the evaluations 
and an interest in learning from experience to inform decision￾making. It is also called upon to account internally as well as to its
beneficiaries and partners for performance and results of its 
operation. 
Regional Bureau 
(RB) Nairobi
Responsible for both oversight of COs and technical guidance and 
support, the RB management has an interest in an 
independent/impartial account of the operational performance as 
well as in learning from the evaluation findings to apply this 
learning to other country offices. The Regional Evaluation Officers 
supports CO/RB management to ensure quality, credible and 
useful decentralized evaluations. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 55
|Page
Office of 
Evaluation (OEV)
OEV has a stake in ensuring that decentralized evaluations deliver 
quality, credible and useful evaluations respecting provisions for 
impartiality as well as roles and accountabilities of various 
decentralised evaluation stakeholders as identified in the 
evaluation policy. 
WFP Executive 
Board (EB)
The WFP governing body has an interest in being informed about 
the effectiveness of WFP programmes. These evaluations will not 
be presented to the Board but their findings may feed into thematic 
and/or regional syntheses and corporate learning processes.
OTHER STAKEHOLDERS 
Beneficiaries As the ultimate recipients of food assistance, beneficiaries have a 
stake in WFP determining whether its assistance is appropriate and 
effective. Consequently, students, teachers, and Parent-Teacher 
Associations (PTAs) are considered key stakeholders. The level of 
participation of women and men, boys and girls in the evaluations 
through interviews, surveys and focus-group discussions in the 
evaluations and their perspectives will be sought. Available data 
will be disaggregated by sex and age when relevant. 
Government As WFP is implementing the HGSF Programme to support the 
government in setting up a national school feeding programme, the 
Ministry of Education (MINEDUC), the Ministry of Agriculture 
(MINAGRI), the Ministry of Infrastructure (MININFRA), Ministry 
of Local Affairs (MINALOC) and the Ministry of Gender and Family 
Promotion (MIGEPROF) as well as the four implementing districts 
have a direct interest in knowing whether activities are aligned with 
its priorities, harmonised with the actions of other partners, and 
meet the expected results. Issues related to capacity development, 
handover and sustainability will be of interest. 
Main donor USDA USDA funds WFP’s HGSF Programme through a McGovern Dole 
Grant (FFE-696-2015/007-00) and so has a strong interest in 
knowing whether their funds have been spent efficiently and if 
WFP’s work has been effective and contributed to their own 
strategies and programmes. 
Other donors Additionally, MasterCard funds some of the food commodities used 
in the programme, and contributed to overall implementation of 
complementary activities.
UN Country Team The UNDAP 2018-23 contributes to the national goal of 
“developing Rwandans into a capable and skilled people with 
quality standards of living and a stable and secure society” through 
Outcome 3 which reads as follows: By 2023 people in Rwanda, 
particularly the most vulnerable, enjoy increased and equitable 
access to quality education, health, nutrition and WASH services. 
The UNCT therefore has a shared interest in the evaluation 
findings, particularly UNICEF, UNESCO, WHO, FAO, UNFPA and 
UNHCR whose work in this area is interconnected with that of 
WFP.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 56
|Page
Other partners 
World Vision, 
Gardens for Health 
International, and 
Rwanda Biomedical 
Centre
NGOs are WFP’s partners for the implementation of some activities 
while at the same time having their own interventions. The results 
of the evaluation might affect future implementation modalities, 
strategic orientations and partnerships. World Vision is a sub￾grantee focusing on literacy and health. Gardens for Health 
International and Rwanda Biomedical Centre are key 
implementing partners.
167. The primary users of the midterm evaluation will be:
• WFP Rwanda and its partners World Vision, Gardens for Health International, 
and Rwanda Biomedical Centre to adjust course as necessary for the remainder 
of the project term;
• MINEDUC and MINAGRI to learn whether the programme is performing well 
and is aligned with its priorities, particularly in terms of capacity development, 
handover and sustainability;
• USDA as the primary funder of the HGSF Programme to learn whether the 
programme is performing well. USDA may use findings and lessons learned to 
inform McGovern Dole programme funding, design, and implementation 
decisions;
• MasterCard as a funder of the HGSF programme may use the findings to inform 
its decision on the best models of school feeding as well as to target its funding;
• Given the core functions of the Regional Bureau Nairobi (RBN), it is expected to 
use the evaluation findings to provide strategic guidance, programme support, 
and oversight;
• WFP HQ may use the evaluation for wider organizational learning and 
accountability;
• OEV may use the evaluation findings, as appropriate, to feed into evaluation 
syntheses as well as for annual reporting to the Executive Board.
168. The primary users of the endline evaluation will be:
• WFP Rwanda and its partners World Vision, Gardens for Health International, 
and Rwanda Biomedical Centre to learn from programme implementation;
• MINEDUC and MINAGRI to learn whether the programme performed well and 
aligned with its priorities, particularly in terms of capacity development, 
handover and sustainability;
• USDA as the primary funder of the HGSF Programme to learn whether the 
programme is performing well. USDA may use findings and lessons learned to 
inform McGovern Dole programme funding, design, and implementation 
decisions;
• MasterCard as a funder of the HGSF programme may use the findings to inform 
its decision on the best models of school feeding as well as to target its funding;
169. Given the core functions of the Regional Bureau Nairobi (RBN), it is 
expected to use the evaluation findings to provide strategic guidance, 
programme support, and oversight;
170. WFP HQ may use the evaluation for wider organizational learning and 
accountability;
WFP Rwanda McGovern-Dole Midterm Evaluation Report 57
|Page
171. OEV may use the evaluation findings, as appropriate, to feed into 
evaluation syntheses as well as for annual reporting to the Executive Board.
3. Context and subject of the evaluations
3.1 Context
172. Rwanda is a densely populated, low-income country with a total 
population of about 12 million people, ranked 159 of 188 countries in human 
development (2016 UNHDI). Since the 1994 genocide, the Government of 
Rwanda has consistently and rather successfully pursued development 
objectives. Rwanda made significant progress in implementing the MDGs and 
is well placed to continue within the SDG framework (UNDAP 2013-18). An 
annual GDP growth rate of 7.2 percent since 2010 has been accompanied by 
decreasing income inequality, although still among the highest in Africa (UN 
CCA 2017). 
173. Moreover, 4.8 million or 41 percent of Rwandans are undernourished 
and over one fifth of the population considered food insecure (SDG 2.1.1) 
(CFSVA 2015). Stunting among children under five years has declined but 
remains high at 38 percent, with significant regional variation, peaking at 60 
percent in certain areas in the northwest (SDG 2.2.1). Stunting is more common 
among children with mothers who are young, did not complete secondary 
education, or are stunted themselves and have an unbalanced dietary intake. 9 
percent of children under five are underweight (low weight for age) and 2.2 
percent are acutely malnourished (low weight for height) (CSR). 
174. Micronutrient deficiencies are also a public health concern; 37 percent 
of children under five and 19 percent of women of reproductive age are anemic. 
The most common causes of anemia are lack of iron in the diet, and intestinal 
worms, preventing the absorption of micronutrients and minerals such as iron. 
Worm infections affect 65 percent of the population in Rwanda, and school￾aged children are particularly affected. Main drivers for malnutrition are poor 
access to quality water, health services and sanitation (WASH) as well as poor 
care practices such as not receiving antenatal care, even among those who can 
access a nutritional, balanced diet (UN CCA 2017).
175. At 98 percent, Rwanda’s primary enrolment rate is among the highest in 
sub-Saharan Africa. Girls and boys show very similar numbers, but equitable 
access is an issue among vulnerable populations such as children with 
disabilities. Worryingly, primary school completion has considerably dropped 
from 73 percent in 2012 to 65 percent (boys 59 percent, girls 71 percent) in 
2016, indicating a high dropout rate. The student-to-teacher ratio is high at 
62:1, leaving little time for teachers to interact with students. The extent of 
limited delivery of quality education is evidenced by low competencies in 
literacy and numeracy among primary-school-going children. Overall, less than 
half of students in public schools, especially in rural areas, achieve the required 
literacy and numeracy competency levels to move on to the next grade (UN CCA 
2017). (In Rwanda, 65 percent of adult women are literate whereas the figure is 
76 percent for men.) The average primary school has one toilet for every 75 
students. The national target is 40:1 for boys and 30:1 for girls. MINEDUC 
estimates that menstrual management alone accounts for an average of 50 
days/girl/year in absences. While some steps have been taken to improve the 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 58
|Page
situation, menstrual management continues to negatively affect girls, especially 
girls in the poorest districts (UN CCA 2017). In addition, only 36 percent of 
schools in Rwanda have access to piped tap water (33 percent in the western 
province and 32 percent in the southern province). 
176. The social protection system has evolved in the last decade, moving from 
a host of fragmented, uncoordinated and often underfunded programmes to 
increasingly coordinated, government-owned programmes operating at scale. 
Thus, great strides were made regarding poverty reduction and vulnerability, 
mainly with households being the primary targeting unit (UN CCA 2017). The 
forthcoming Social Protection Sector Strategy 2018-24 is expected to take this 
further by adopting a life-cycle approach and promoting universal access to 
social security and protection for all Rwandans, whether poor or not. Quite a 
radical shift, and appropriate given the country’s vision to reach upper income 
status by 2050.
177. Rwanda has made commendable progress in ensuring gender equality 
however glaring challenges still exist between males and females. While 
Rwanda has the highest percentage of women in parliament in the world (64 
percent) and female representation in high also in other positions of power (41 
percent of the cabinet, 43 percent of Supreme Court Justices), gender balance 
in local government leadership has not yet been fully addressed as most 
positions continue to be dominated by men, raising concerns around effective 
implementation of all gender equality related programmes. Similar 
observations and arguments suggest that there is a critical gap in the quality of 
overall participation and specifically women’s participation in these platforms 
as there is limited evidence to suggest otherwise.
178. The HGSF Programme supports students in Rwanda’s poorest and most 
food insecure districts: Nyaruguru and Nyamagabe in the southern province 
and Rutsiro and Karongi in the western province (see map in Annex 3). 
Nyaruguru and Nyamagabe districts have particularly high numbers of 
households led by women and people with disabilities, compounding 
vulnerability. The 104 programme schools were selected from sectors with the 
highest poverty levels per the government’s household poverty classification 
(UBUDEHE), also considering each school’s capacity and willingness to 
implement activities.98 
179. WFP is implementing the HGSF Programme to support the government 
in setting up a national school feeding programme, with a view to build capacity 
and complete a full handover of activities by 2020. At least since 2017, the 
Minister of Education has stressed the government’s desire to expand the HGSF 
model countrywide, including eventual ownership – an important step towards 
universal and sustainable school feeding in Rwanda (HGSF semi-annual report 
Nov 2017). Since its establishment in 2017, MINEDUC and WFP co-chair the 
HGSF National Steering Committee which is bringing together key 
stakeholders to coordinate the programme and ensure sustainability.
180. The Government of Rwanda’s mid- to long-term outlook is guided by the 
national development plan Vision 2020 and its new iteration Vision 2050 
(forthcoming), which together envision Rwanda transforming from an agrarian 
to a knowledge-based economy, attaining upper middle-income country status 
98 WFP Rwanda FY 2015 McGovern-Dole proposal.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 59
|Page
by 2035 and high-income status by 2050. To help achieve this, the country’s 
key poverty reduction strategy, the National Strategy for Transformation 
(NST), focuses on three pillars, of which the social transformation pillar has a 
priority area to ensure quality of education for all, aiming at building a 
knowledge-based economy (UN CCA 2017). 
181. As the lead of the education sector, MINEDUC heads policy formulation, 
planning, coordination, regulation, monitoring and evaluation of the entire 
education sector (UN CCA 2017). The ministry works closely with the semi￾autonomous Rwanda Education Board (REB) which provides national 
oversight for coordinating and implementing education activities at pre￾primary, primary and secondary level.
182. Under the Local Government Act (2013) District Administrations have 
responsibility for the delivery of education services. The extent to which 
MINEDUC and REB have influence at the district level is determined by the 
level of interest and priority afforded to education by the District Executives. 
District Development Plans (DDPs) determine district priorities and where 
resources are allocated. District Education Officers (DEOs) are employed by the 
Ministry of Local Government (MINALOC) and managed by their District 
Administrations. DEOs are actively involved in the planning, delivery and 
monitoring of education in their districts. Below the DEOs are sector education 
officers (SEOs) responsible for overseeing delivery of education services and 
running of schools.
183. Guided by the Education Sector Strategic Plan (ESSP), now in its fourth 
cycle, Rwanda has invested significant resources towards improving the quality 
and coverage of all levels of education, as well as towards implementing policies 
that aim to achieve universal and equitable access to basic education for all 
Rwandan children. The provision of universal, compulsory and free nine years 
of basic education for children aged seven to 15 years has had a significant 
impact on increasing access, and this is now being expanded to 12 years. The 
curriculum has undergone a major reform, with a new competence-based 
curriculum being phased in from January 2016 (UN CCA 2017). 
184. MINEDUC, Save the Children, and World Vision are implementing 
Literacy Boost, a proven literacy instruction methodology focused on 
improving children’s reading abilities. Literacy Boost is improving the literacy 
of 195,000 children in grades 1‐3 in 280 primary schools in central Rwanda and 
was scaled up using MGD resources. MINEDUC also worked with USAID’s 
Literacy, Language, and Learning (L3) project 2012‐2016 on improving the 
quality of education.99
185. In support of school health, the Ministry of Health (MINESANTE) has a 
‘12+ Programme’, supported by the Nike Foundation and PSI, targeting 114,500 
girls between the ages of 10‐12 years in primary schools across Rwanda, funded 
by DFID, with the objective of reducing drop‐out of girls by creating safe spaces 
in schools for girls. World Vision also has a cost‐sharing partnership with 
Rwanda’s Water and Sanitation Corporation (WASAC) through its Ubuzima 
WASH project partnering with MINESANTE, training Community Health 
Workers (CHWs) and school‐based volunteers through the Community‐based 
99 WFP Rwanda FY 2015 McGovern-Dole proposal.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 60
|Page
Environmental Health Promotion Programme (CBEHPP). CBEHPP is 
MINESANTE’s approved methodology of working with communities to help 
them identify and solve their own health and hygiene issues. World Vision is 
Rwanda’s largest implementer of CBEHPP and outside of the MGD project is 
supporting MINESANTE in training 45,000 CHWs in 15,000 villages to reduce 
hygiene‐related diseases in communities and within schools.100
186. UNICEF has supported modelling and scaling‐up Child‐Friendly School 
standards, which were adopted as the national quality guidelines for school 
infrastructure and software inputs. The Learning Achievement in Rwandan 
Schools (LARS) Assessment was supported by UNICEF to improve the quality 
of education and measure learning outcomes in literacy and numeracy. The 
joint UN Development Assistance Programme (UNDAP) 2018‐2023 is focusing 
on increased and equitable access to quality education, health, nutrition and 
WASH services. 
187. The Clinton Health Access Initiative (CHAI) has been implementing a 
programme with the government since 2013 to work with food producers to 
combat malnutrition. CHAI is assisting the government to reduce chronic 
malnutrition among infants and pregnant and lactating women.101 CHAI is 
supporting the Africa Improved Foods (AIF) on sourcing local agricultural 
produce to use in its factory which produces nutritious fortified blended foods 
that can be distributed throughout the country.
3.2. Subject of the evaluations
188. This is an activity evaluation of a USD 25 million grant to improve 
literacy and increase the use of health and dietary practices. The midterm 
evaluation will take place in 2018, while the endline will be completed in 2020. 
The midterm evaluation will cover the 2016-2018 (second quarter included) 
timeframe, while the endline evaluation will cover the entire implementation 
period (2016-2020).
189. The HGSF Programme supports students across 104 schools in 
Rwanda’s poorest and most food insecure districts – Nyaruguru and 
Nyamagabe in the south and Rutsiro and Karongi in the west (see map in Annex 
3). Children in the south receive a daily hot meal whereas students in the 
western province are provided a porridge meal. The programme started in 
October 2015 and is anticipated to conclude in late 2020.
190. WFP has planned to reach 83,000 students annually in grades 1-6, and 
reached 99 percent in 2016 and 103 percent in 2017, with similar numbers of 
boys and girls. Over the programme cycle, WFP plans to use 4,657 metric 
tonnes of SuperCereal and 540 metric tonnes of vegetable oil. Additionally, 
maize, beans, salt and sugar are non-USDA commodities and as such purchased 
from other mobilized funds.102
191. The HGSF Programme aims to achieve the McGovern-Dole strategic 
goals of improved literacy of school-age children (MGD SO1) and increased use 
of health and dietary practices (MGD SO2) in the targeted areas through a set 
of interconnected activities, with a view to supporting the government in 
100 WFP Rwanda FY 2015 McGovern-Dole proposal. 101 WFP Rwanda FY 2015 McGovern-Dole proposal. 102 WFP Rwanda FY 2015 McGovern-Dole proposal.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 61
|Page
establishing a national school feeding programme, including building capacity 
at national, district and school levels to facilitate a future handover (see results 
frame in annex). 
192. WFP, together with its implementing government partners MINEDUC 
and MINAGRI as well key implementing NGO partners, is carrying out 
activities to achieve MGD SO1 by: promoting teacher attendance and 
recognition; distributing school supplies and materials; improving literacy 
instruction materials; increasing the skills and knowledge of teachers and 
administrators; providing school meals; developing partnerships with farmer 
groups to supply food to schools; establishing and maintaining school gardens; 
increasing use of health and dietary practices; raising awareness on the 
importance of education; and reducing health-related absenteeism at schools. 
193. WFP and partners carry out activities to achieve MGD SO2 by: raising 
awareness on good hygiene practices; enhancing food preparation and cooking 
practices; building and rehabilitating latrines and water collection systems; 
distributing deworming medication; and building/rehabilitating kitchens, 
cooking areas and storerooms, and providing fuel-efficient stoves. 
194. To ensure the development of a nationally-owned, sustainable school 
feeding programme, WFP and partners carry out activities to achieve MGD 
foundational results by: increasing the capacity of government institutions 
(1.4.1/2.7.1), improving the policy and regulatory framework (1.4.2/2.7.2), and 
increasing government support (1.4.3/2.7.3). To increase engagement of local 
organizations and community groups (1.4.4/2.7.4), WFP and partners train 
PTAs, raise awareness on the importance of education, develop partnerships 
with farmer groups and local cooperatives to supply food to schools, engage 
parents and communities through the establishment and maintenance of 
school gardens, and strengthen school health clubs through training and 
awareness on good health and hygiene practices.
195. The McGovern Dole grant for the five-year programme is USD 25 
million. WFP estimates that the total value of contributions mobilized for this 
project outside of McGovern Dole resources will be around USD 12,1 million. 
These resources have enabled the full implementation of the programme, 
including locally procuring maize and beans with funding from MasterCard.
196. The HGSF baseline study report from July 2016 focused on indicators
that could be measured before project implementation. Some indicators could 
not be measured because their definition is linked to the implementation of 
project activities. Baseline values for each indicator measured against its 
corresponding target, as per the project document, were summarized (see 
baseline report in annex). Following the baseline study, the report continues, it 
is essential that WFP reviews and realigns the targets. In preparation for the 
midterm and endline evaluations, there is also a need for a strong programme 
monitoring component that collects and compiles data from each of the 
beneficiary school and related activities on a regular basis. All indicators, 
including policy-related indicators, require specific project records. 
197. The centralized midterm evaluation of WFP Rwanda’s Country 
Programme (June 2017) noted that although the McGovern Dole grant 
provided much-needed funding for HGSF, it has also led to some unalignment 
with WFP as well as national priorities. The MGD intervention saw a return to 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 62
|Page
providing (for part of the beneficiaries) food imported from the U.S. – a 
condition which was clearly required by the donor. This modality is in 
contradiction with the Government of Rwanda’s own expressed preference but 
also with the logic promoted by WFP’s School Feeding Policy of giving priority 
to helping countries establish and maintain nationally owned programmes 
linked to local agricultural production. However, WFP has sought to address 
this by designing the project in such a way that it will support the local 
production of vegetable oil and CSB+ and contribute to the development of a 
national strategy that if successful will sustain the benefits of USDA support 
beyond the life of the project. The report says that it was too soon to measure 
sustainability/handover in 2016. 
198. The evaluation report concludes that HGSF activities are “completely” 
integrated in government planning and monitoring at local level, with officials 
participating in all steps of the process. The programme promotes the 
participation of girls and the baseline allows monitoring against gender 
indicators as advised by the WFP 2009 Gender Policy (later subsumed by the 
WFP 2015-20 Gender Policy), the report continues. The programme does not, 
however, include a specific approach to address Gender Equality and Women’s 
Empowerment (GEWE) or Sexual and Gender-based Violence (GBV) as 
foreseen in the 2009 policy.
199. A cost-benefit analysis conducted by WFP in collaboration with 
MasterCard in November 2017 concluded that every dollar invested in school 
meals in Rwanda can generate a return of USD 4.8 and 5.6 for home-grown and 
in-kind modalities, respectively, over a child’s lifetime. Finally, the midterm 
and endline evaluations will be guided by the WFP Evaluation Policy 2016-2021 
and the USDA Monitoring and Evaluation Policy 2013. 
4. Evaluation approach
4.1 Scope
200. WFP Rwanda is looking to assign one contract for both a midterm and 
endline evaluation of the 2016-2020 McGovern Dole grant. The programme 
started in late 2015 with the goal of supporting school feeding in 104 schools in 
four districts: Nyaruguru, Nyamagabe, Rutsiro and Karongi in Rwanda (see 
map in Annex 1). 
201. This is an activity evaluation of a USD 25 million grant to improve 
literacy and increase the use of health and dietary practices. The programme 
provides U.S. produced agricultural commodities and financial assistance, and 
supports capacity development and enhanced monitoring and reporting, with a 
key emphasis on sustainability and government ownership. In addition to 
USDA-provided commodities, WFP procures maize and beans locally through 
additional funds raised separately through private donors, such as MasterCard.
202. The midterm evaluation will take place in 2018, while the endline will be 
completed in 2020. The midterm evaluation will cover the 2016-2018 (second 
quarter included) timeframe, while the endline evaluation will cover the entire 
implementation period (2016-2020).
203. The beneficiaries of the programme are 85,000 primary school students 
per year, grades 1-6. Of these, 49 percent are female. Other stakeholders who 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 63
|Page
get access to capacity building activities are Parent-Teacher Associations 
(PTAs), School Management Committees (SMCs), teachers and head teachers, 
storekeepers and cooks.
4.2 Evaluation criteria and questions
204. Evaluation Criteria. The evaluations will address all five OECD-DAC 
criteria (relevance, effectiveness, efficiency, impact and sustainability), as per 
USDA Monitoring and Evaluation policy. 
205. Evaluation Questions. Aligned with the evaluation criteria, the 
evaluations will address the following key questions, which will be further 
developed by the evaluation team during the inception phase. Collectively, the 
evaluation questions aim at highlighting the key lessons and performance of the 
HGSF Programme, which could inform future strategic and operational 
decisions. The four key questions that need to be investigated are:
a) Have literacy rates of school age children improved over the duration of the 
programme? If so, how and why? For example, are students able to read grade￾level text? Are teachers demonstrating new methods of teaching? 
b) Has the use of health and dietary practices increased? If so, how? Has illness￾related absence decreased? Are students washing their hands and are schools 
and school kitchens clean? How are school gardens being used? 
c) What is the level of community-level involvement and participation in decision￾making in school governance mechanisms (PTAs and SMCs)? Particularly, 
what is the level of involvement and participation of men? Also, what is the level 
and sustained continuity of community contributions in cash and in kind?
d) What are the key institutions (i.e. international, national, provincial/district 
and local stakeholders) and governance structures required to effectively 
deliver, implement, and sustain school meal interventions? What relationship 
structures among these institutions yield the most successful and effective 
school meal programmes? Is WFP’s capacity support to smallholder farmers 
and key line ministries appropriate/sufficient to effectively facilitate national 
ownership? Has the provided capacity support increased the government’s 
capacity to own and sustain a national school meals programme?
206. The evaluation questions will be reviewed within the course of the 
inception period, and there will be some differentiation between midterm and 
endline questions. For example, the midterm will also include achievement of 
outputs rather than only outcomes and objectives, whereas the endline will 
focus more on impact.
207. Gender equality and empowerment of women should be mainstreamed 
throughout the evaluation, including disaggregation of all data and considering 
whether gender has been integrated in design, planning, implementation and 
results. Reflecting UNDAP concern on gender equality “Ensure women’s full 
and effective participation and equal opportunities for leadership at all levels of 
decision making in political, economic and public life”, the evaluators are 
specifically required to investigate gender aspects in question c. above. 
208. Table 2 below presents key evaluation criteria and corresponding 
questions: 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 64
|Page
Table 2: Criteria and evaluation questions
Criteria Evaluation Questions
Relevance To what extent is the programme in line with the needs of 
beneficiaries (boys and girls) and partners, including 
government?
To what extent is the activity aligned with community, local 
government, and national government policies and 
priorities?
To what extent is the intervention based on a sound gender 
analysis? To what extent is the design and implementation of 
the intervention gender-sensitive?
Effectiveness To what extent are the outcomes or objectives of the 
intervention likely to be achieved?
What are the major factors influencing progress in 
achievement or non-achievement of the outcomes/objectives 
of the intervention?
To what extent does the intervention deliver results for boys 
and girls?
Efficiency Is the programme implemented in a timely way? Are the 
activities cost-efficient? Is the programme implemented in 
the most efficient way compared to alternatives? Were the 
project strategies efficient in terms of financial and human 
resource inputs as compared to outputs?
Does the monitoring system efficiently meet the needs and 
requirements of the project?
Impact What are the medium-term effects on beneficiaries’ lives?
What are the gender-specific medium term impacts? Did the 
intervention influence the gender context?
Sustainability To what extent is the government taking ownership of the 
programme (e.g. demonstrated commitment and 
contributions)?
What is the demonstrated capacity at central and sub￾national levels to manage the programme? 
Are local communities (PTAs, farmers’ groups, etc.) fully 
involved in and contributing toward school feeding and 
education activities?
Has the policy framework supporting the HGSF been 
strengthened within the project period?
What are the major factors influencing the achievement or 
non-achievement of sustainability of the program?
4.3 Data availability 
209. The evaluations will entail qualitative and quantitative primary data 
collection that the evaluation team will be responsible for. In addition, the 
following is a list of background data and/or information available for the 
evaluation team. It is expected that the team will expand this at inception phase.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 65
|Page
• Baseline report for WFP’s USDA McGovern Dole HGSF Programme 2016-2020, 
including data collection tools;
• School feeding handbook;
• WFP School feeding policy;
• 2016 and 2017 Standard Project Reports (SPRs);
• WFP HGSF semi-annual reports to USDA;
• USDA commitment letter for Agreement;
• Evaluation Plan;
• USDA Monitoring and Evaluation Policy;
• USDA McGovern-Dole Indicators and Definitions Handbook;
• Other government education data/policies as applicable.
210. The evaluation team responsible for the baseline assessment warned 
about the availability and usability of certain data sets. Following the 
recommendations of the baseline team, WFP and its partners have improved 
data collection tools to provide the level of granularity required by the donor 
and to answer most of the evaluations questions. For instance, school records 
now provide attendance information per individual child and teacher, records 
are revised monthly and are subject to random checks. However, during the 
inception phase, the evaluation team will be responsible for controlling the 
quality and reliability of data sets and formulate alternative strategies to fill 
potential data gaps. 
211. The evaluation team is expected to explore key questions c. (gender) and 
d. (institutional preparedness for hand-over) largely through qualitative data 
(although some quantitative data on gender parity is also expected). Key 
question d. will require an analysis of similar experiences in other countries and 
a comparison with the situation in Rwanda. 
212. Even though, at this point, WFP does not envision the use of such data 
collection tools, the evaluation team should also bear in mind that the 
Government of Rwanda requires formal approval of household surveys three 
months before the field phase takes place.
213. Concerning data and information, the evaluation team should: 
• Assess availability, validity and reliability as part of the inception phase of the 
midterm evaluation expanding on the information provided in section 4. This 
assessment will inform the data collection;
• Systematically check accuracy, consistency and validity of collected data and 
information and acknowledge any limitations/caveats in drawing conclusions 
using the data.
4.4. Methodology
214. The independent evaluation team is responsible for developing the full 
methodology during the inception phase. In this stage, the ET should validate 
the methodological approach followed during the baseline assessment and 
propose improvements where required. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 66
|Page
215. Question a. (literacy) will be answered using data collected through the 
EGRA standard test. To conduct the EGRA and adapt it to the local context, the 
selected evaluator should reference the EGRA toolkit103
216. Question b. (health and dietary practices), is likely to be based on data
collected through school and student surveys, direct observation and key 
informant interviews.
217. Key questions c. and d. were not explicitly included in the baseline 
assessment and will require the team to develop an appropriate methodological 
approach at this stage. 
218. Midterm findings on key question d. will inform the development of a 
hand-over strategy from WFP to the Government of Rwanda. It is expected that 
the evaluation team formulates clear recommendations that could help a 
smooth transition to a country-owned home grown school feeding programme. 
At endline, the evaluation team should concentrate on assessing the progress 
made in handing over the activities. 
219. The team should identify potential risks of the approach and mitigation 
measures. The following should be considered and included by the evaluation 
team: 
• Firstly, confirm and define specific evaluation questions that are answered, and 
record them in the WFP Evaluation Matrix;
• Include description of sample categories and identify appropriate sample sizes 
(margin of error 5%, confidence level 95%);
• Design credible data collection instruments;
• Use mixed methods in the evaluation design and data collection (including 
quantitative and qualitative) to ensure a comprehensive design, and the reasons 
for the changes in indicators can be explained. This can include triangulation of 
information through a variety of means, or different evaluation questions being 
answered through different methods and types of data. The use of mixed 
methods should be documented in the inception report; 
• To the extent possible, ensure that data collection tools are consistent with 
baseline tools to ensure comparability;
• WFP anticipates that the consultants will recommend a methodology that will 
likely include carrying out key informant interviews and focus group 
discussions (list of interviews to be agreed upon at inception phase). The 
qualitative data collection will gather information on gender equality, capacity 
strengthening and changes in the institutional context. However, bidding 
companies should also propose a wider variety of methods (including, but not 
limited to most significant change, outcome harvesting, etc.) whenever they feel 
these could be useful in enriching the evaluation products; 
• Ensure the evaluation design considers ways to ensure that the voices of women, 
girls, men and boys are heard and documented;
• Ensure the methodology and evaluation implementation are ethical and 
conform to the UNEG Ethical Guidelines for Evaluation.
220. The following mechanisms for independence and impartiality are 
employed:
103 RTI International. 2016. EGRA Toolkit, Second Edition. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 67
|Page
• Appointment of an Evaluation Manager with no previous involvement with the 
HGSF programme (Daniel Svanlund, WFP M&E Officer);
• Establishment of an Evaluation Committee;
• Establishment of an Evaluation Reference Group.
221. The main risk identified that could affect the methodology proposed for 
the midterm and endline is the weakness of the baseline study (the sample size 
of students was small; some data sets had to be reconstructed due to the 
unavailability of reliable data, e.g. teachers’ attendance). As a mitigating 
measure, the evaluation team should highlight the strength of the evidence 
underlying the findings in the midterm and endline. 
4.5 Quality assurance and assessment
222. WFP’s Decentralized Evaluation Quality Assurance System (DEQAS) 
defines the quality standards expected from this evaluation and sets out 
processes with in-built steps for Quality Assurance, Templates for evaluation 
products and Checklists for their review. DEQAS is closely aligned to the WFP’s 
evaluation quality assurance system (EQAS) and is based on the UNEG norms 
and standards and good practice of the international evaluation community and 
aims to ensure that the evaluation process and products conform to best 
practice. 
223. DEQAS will be systematically applied to these evaluations. The WFP 
Evaluation Manager will be responsible for ensuring that the evaluations 
progress as per the DEQAS Process Guide and for conducting a rigorous quality 
control of the evaluations’ products ahead of their finalization. 
224. WFP has developed a set of Quality Assurance Checklists for its 
decentralized evaluations. This includes Checklists for feedback on quality for 
each of the evaluation products. The relevant Checklist will be applied at each 
stage, to ensure the quality of the evaluation process and outputs.
225. To enhance the quality and credibility of these evaluations, an 
outsourced quality support (QS) service directly managed by WFP’s Office of 
Evaluation in Headquarter provides review of the draft inception and 
evaluation report (in addition to the same provided on draft TOR), and provide:
• systematic feedback from an evaluation perspective, on the quality of the draft 
inception/midterm/endline evaluation report; 
• recommendations on how to improve the quality of the final 
inception/midterm/endline evaluation report.
226. The evaluation manager will review the feedback and recommendations 
from QS and share with the team leader, who is expected to use them to finalise 
the inception/ evaluation report. To ensure transparency and credibility of the 
process in line with the UNEG norms and standards,104 a rationale should be 
provided for any recommendations that the team does not consider when 
finalising the report.
227. This quality assurance process as outlined above does not interfere with 
the views and independence of the evaluation team, but ensures the report 
104 Norm #7 states “that transparency is an essential element that establishes trust and builds confidence, 
enhances stakeholder ownership and increases public accountability.”
WFP Rwanda McGovern-Dole Midterm Evaluation Report 68
|Page
provides the necessary evidence in a clear and convincing way and draws its 
conclusions on that basis.
228. The evaluation team will be required to ensure the quality of data 
(validity, consistency and accuracy) throughout the analytical and reporting 
phases. The evaluation team should be assured of the accessibility of all relevant 
documentation within the provisions of the directive on disclosure of 
information. This is available in WFP’s Directive CP2010/001 on Information 
Disclosure.
229. All final evaluation reports will be subjected to a post hoc quality 
assessment by an independent entity through a process that is managed by 
OEV. The overall rating category of the reports will be made public alongside 
the evaluation reports.
5. Phases and deliverables 
230. The evaluation will be conducted in two stages: a midterm evaluation to 
be conducted between June/July and December 2018, and an endline 
evaluation that will take place between July and November 2020. Although the 
two phases are interconnected steps of the same evaluative exercise, their 
objectives are slightly different as outlined in the following sections.
231. The deliverables and deadlines for each phase are as follows: 
Figure 1: Summary Process Map 
5.1 Midterm evaluation 
232. The objective of the midterm evaluation is to provide an evidence-based, 
independent assessment of performance of the project so that WFP and its 
project partners can adjust course as necessary for the remainder of the project 
term. Specifically, the midterm evaluation will (1) review the project’s 
relevance, effectiveness, efficiency, impact, and sustainability, (2) collect 
performance indicator data for strategic objectives and higher-level results, (3) 
assess whether the project is on track to meet the results and targets, (4) review 
the results frameworks and theory of change, and (5) identify any necessary 
mid-course corrections. The evaluation will rely on the Baseline Study for 
baseline data and critical context necessary to evaluate the project at interim. 
233. The evaluation firm selected for this assignment will develop the 
methodological approach following the indications provided in 4.2 Evaluation 
Questions and 4.4 Methodology. The evaluators should also validate or revise 
the assumptions and risk analysis underlying the project design. 
234. The main deliverables of the midterm evaluation are the following: 
1. Prepare 2. Inception
•Inception Report
3. Midterm 
data 
collection
• Aide memoire / 
debriefing PPT
4. 
Analyze 
data and 
Report
• Midterm 
Evaluation 
Report
5. Endline 
data 
collection
• Aide memoire 
/debriefing PPT
6. Analyze 
Data and 
Report
• Final report
7. 
Disseminate 
and follow￾up
WFP Rwanda McGovern-Dole Midterm Evaluation Report 69
|Page
• Inception report. It must be written following WFP recommended template. The 
evaluators must validate the methodology utilized in the baseline phase and/or 
propose alternative methods to measure the same indicators. This means 
setting out a full study design including what data is being collected and for 
what purpose, how sampling is done (to be determined by the evaluation team), 
how the data is being analysed and triangulated. The inception report must also 
include a data quality assurance plan, and how the evaluators will manage and 
safeguard ethics during the life of the evaluation. Annexed to the inception 
report, the evaluation team should include a detailed work plan including, 
timeline and activities.
• Midterm report, including a first draft, where the final approach, methodology 
and data collection tools are clearly recorded, including their limitations and 
mitigations measures. The report must record all standard and custom 
indicator baseline and midterm values.
• Clean data sets, including quantitative data sets in Excel, statistical software 
code, and transcripts and/or notes from focus group discussions and key 
informant interviews.
• Prepare and present a PowerPoint presentation of main findings and 
conclusions to debrief WFP Rwanda management and other relevant staff 
• 2-page brief containing findings, conclusions and recommendations.
Table 3: Midterm evaluation deliverables
Dates Deliverables 
June/July –
August 2018
• Desk review of key project documents 
• Create a data quality assurance plan
• Review baseline methodology; confirm and finalise evaluation 
questions and evaluation design and methodology (including 
sampling strategy), and draft an inception report for agreement 
• Finalize inception report with the inclusion of Evaluation 
Reference Group’s comments 
• Data collection instruments
• Arrange field visits
September –
October 2018
• Conduct field visits 
• Conduct key stakeholder focus groups and key informant 
interviews and collect data with other suggested methods
• Enter, clean, and analyse data 
October –
early 
December 
2018
• Draft midterm report 
• Seek Evaluation Reference Group’s comments on the draft 
midterm report 
• Finalize midterm report 
• Prepare and present a PowerPoint presentation of main findings 
and conclusions to debrief WFP Rwanda management and other 
relevant staff 
• Prepare and share a 2-page brief with key stakeholders 
5.2 Endline evaluation 
235. The objective of the final evaluation is to provide an evidence-based, 
independent assessment of the performance of the project to evaluate the 
project’s success, ensure accountability, and generate lessons learned. 
Specifically, the final evaluation will: (1) use the same methodology developed 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 70
|Page
for the midterm to measure key indicators, (2) analyses data to compare results 
before and after the intervention, and (3) identify meaningful lessons learned 
that WFP, USDA, and other relevant stakeholders can apply to future 
programming.
236. WFP anticipates carrying out the final evaluation during the final year of 
the USDA-MGD grant between July and November 2020. 
237. The main deliverables of the endline are the following: 
• Inception report. It must be written following WFP recommended template. The 
evaluators must validate the methodology utilized in the baseline phase and/or 
propose alternative methods to measure the same indicators. This means 
setting out a full study design including what data is being collected and for 
what purpose, how sampling is done (to be determined by the evaluation team), 
how the data is being analysed and triangulated. The inception report must also 
include a data quality assurance plan, and how the evaluators will manage and 
safeguard ethics during the life of the evaluation. Annexed to the inception 
report, the evaluation team should include a detailed work plan including, 
timeline and activities.
• Endline report, including a first draft, using WFP recommended template. It 
must set out a detailed methodology section, study design, and any limitations 
or where the study design was compromised. Should detail how data was 
collected, validated and analysed, and how conclusions were drawn. How 
different types of methods were brought together in the analysis. Annexes to the 
final report include but are not limited to a copy of the final ToR, bibliography, 
detailed sampling methodology, maps, a list of all meetings and participants, 
final survey instruments, table of all standard and custom indicator with 
baseline, midterm and endline values, list of supported schools.
• Clean data sets, including quantitative data sets in Excel, statistical software 
code, and transcripts and/or notes from focus group discussions and key 
informant interviews.
• PowerPoint presentation of main findings and conclusions for de-briefing and 
purposes. 
• 2-4 page brief containing findings, conclusions and recommendations written 
for a nontechnical audience that includes photos and graphs or charts.
• Conduct a 1-day workshop to share evaluation findings with key stakeholders.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 71
|Page
Table 4: Endline evaluation deliverables
Dates Deliverables 
July – August 
2020
• Finalise updated evaluation questions and evaluation design and 
methodology (including sampling strategy), and draft an 
inception report for agreement 
• Finalize inception report with the inclusion of Evaluation 
Reference Group’s comments 
• Update data collection instruments
• Arrange field visits
September –
October 2020
• Conduct field visits 
• Conduct key stakeholder focus groups and key informant 
interviews and collect data with other suggested methods
• Enter, clean, and analyse data 
October –
early 
December 
2020
• Draft endline report 
• Seek Evaluation Reference Group’s comments on the draft 
endline report 
• Finalize endline report 
• Prepare and present a PowerPoint presentation of main findings 
and conclusions to debrief WFP Rwanda management and other 
relevant staff 
• Prepare and share a 2-4 page brief with key stakeholders 
• Conduct a 1-day workshop to share evaluation findings with key 
stakeholders
6. Organization of the evaluation
6.1 Evaluation conduct
238. The evaluation team will conduct the evaluation under the direction of 
its team leader and in close communication with the WFP Evaluation Manager. 
The team will be hired following agreement with WFP on its composition. All 
communication between the evaluation team and stakeholders should go 
through or include the evaluation manager.
239. The evaluation team will draw its own conclusions free from political 
influence or organization pressure. The evaluation team will not have been 
involved in the design or implementation of the subject of evaluation or have 
any other conflicts of interest. Further, they will act impartially and respect the 
code of conduct of the evaluation profession. The Evaluation Manager has not 
been part of the project’s implementation.
240. Specifically, evaluators shall respect and protect the rights and welfare 
of human subjects and communities, in accordance with the UN Universal 
Declaration of Human Rights and other human rights conventions. Evaluators 
shall respect differences in culture, local customs, religious beliefs and 
practices, personal interaction, gender roles, disability, age and ethnicity, while 
using evaluation instruments appropriate to the cultural setting. Evaluators 
shall ensure prospective participants are treated as autonomous agents, free to 
choose whether to participate in the evaluation, while ensuring that the 
relatively powerless are represented. Evaluators shall make themselves aware 
of and comply with legal codes (whether international or national) governing, 
for example, interviewing children and young people. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 72
|Page
241. The evaluation team should also guarantee the right to provide 
information in confidence and make participants aware of the scope and limits 
of confidentiality, while ensuring that sensitive information cannot be traced to 
its source (right to confidentiality).
242. Evaluators shall act to minimise risks and harms to, and burdens on, 
those participating in the evaluation, without compromising the integrity of the 
evaluation findings (avoidance of harm).
6.2 Team composition and competencies
243. The Team Leader should be a senior evaluator with at least 15 years of 
experience in research and/or evaluation with demonstrated expertise in 
managing multidisciplinary and mixed quantitative and qualitative method 
evaluations, and additional significant experience in other development and 
management positions. In addition, the team leader should also have prior 
experience evaluating school meals programs, ideally USDA-funded 
McGovern-Dole grants. 
244. The Team leader will also have expertise in designing methodology and 
data collection tools. She/he will also have leadership and communication
skills, including a track record of excellent writing and presentation skills. 
Her/his primary responsibilities will be: i) design the approach and 
methodology; ii) guiding and managing the team; iii) leading the field missions 
and representing the baseline team; iv) drafting and revising, as required, the 
inception report, the end of field work i.e. (exit) debriefing presentation 
midterm and endline reports. 
245. The team must include strong demonstrated knowledge of qualitative 
and quantitative data and statistical analysis will be required. It should include 
both women and men and at least one team member should be familiar with 
WFP’s operations (preferably school feeding). 
246. The team will be multi-disciplinary and include members who together 
include an appropriate balance of expertise and practical knowledge in the 
following areas: 
• School Feeding;
• WASH;
• Primary Education (with a strong knowledge of early primary school reading 
process);
• Food security;
• Gender expertise;
• Some expertise in smallholder farmer support is desirable;
• All team members should have strong analytical and communication skills, 
evaluation experience and some familiarity with Rwanda;
• The team should have knowledge of English and Kinyarwanda. The required 
language of both the midterm and endline reports is English.
247. Team members will bring together a complementary combination of the 
technical expertise required and have a track record of written work on similar 
assignments. 
248. Team members will: i) contribute to the methodology in their area of 
expertise based on a document review; ii) conduct field work; iii) participate in 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 73
|Page
team meetings and meetings with stakeholders; iv) contribute to the drafting 
and revision of the evaluation products in their technical area(s). 
6.3 Security considerations
249. Security clearance where required is to be obtained.
250. As an ‘independent supplier’ of evaluation services to WFP, the 
evaluation company is responsible for ensuring the security of all persons 
contracted, including adequate arrangements for evacuation for medical or 
situational reasons. The consultants contracted by the evaluation company do 
not fall under the UN Department of Safety & Security (UNDSS) system for UN 
personnel. 
251. Consultants hired independently are covered by the UN Department of 
Safety & Security (UNDSS) system for UN personnel which cover WFP staff and 
consultants contracted directly by WFP. Independent consultants must obtain 
UNDSS security clearance for travelling to be obtained from designated duty 
station and complete the UN system’s Basic and Advance Security in the Field 
courses in advance, print out their certificates and take them with them.105
252. However, to avoid any security incidents, the WFP Rwanda Evaluation 
Manager is requested to facilitate that: 
• The WFP CO registers the team members with the Security Officer on arrival in 
country and arranges a security briefing for them to gain an understanding of 
the security situation on the ground.
• The team members observe applicable UN security rules and regulations – e.g. 
curfews, child protection protocols.
7. Roles and responsibilities of stakeholders
a- WFP Rwanda: 
The WFP Rwanda Deputy Director will take responsibility to:
• Assign an Evaluation Manager for the evaluation: Daniel Svanlund, M&E 
Officer.
• Compose the internal Evaluation Committee and the Evaluation Reference 
Group (see below).
• Approve the final TOR, inception and evaluation reports.
• Ensure the independence and impartiality of the evaluation at all stages, 
including establishment of an Evaluation Committee and of a Reference Group 
(see below and TN on Independence and Impartiality). 
• Participate in discussions on the evaluation design and subject, its performance 
and results with the Evaluation Manager and the evaluation team. 
• Participate in debriefing(s). 
• Oversee dissemination and follow-up processes, including the preparation of a 
Management Response to the evaluation recommendations.
The Evaluation Manager: 
• Manages the evaluation process through all phases including drafting this TOR.
• Ensures quality assurance mechanisms are operational.
105 Field Courses: Basic; Advanced 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 74
|Page
• Consolidates and shares comments on draft TOR, inception and evaluation 
reports with the evaluation team.
• Ensures expected use of quality assurance mechanisms (checklists, quality 
support). 
• Ensures that the team has access to all documentation and information 
necessary to the evaluations; facilitates the team’s contacts with local 
stakeholders; sets up meetings, field visits; and provides logistic support during 
the fieldwork.
• Organises security briefings for the evaluation team and provides any 
materials as required.
b- An internal Evaluation Committee has been formed as part of ensuring the 
independence and impartiality of the evaluation (see EC TOR in annex). 
c-USDA, as the main funder of the programme, will be given the opportunity to review 
and comment on the key evaluation products. USDA will also participate in the 
evaluation as a key informant prior to the start of in-country fieldwork. 
d- The Regional Bureau: When not the Commissioning Office, the RB will take 
responsibility to:
• Advise the Evaluation Manager and provide support to the evaluation process 
where appropriate. 
• Participate in discussions with the evaluation team on the evaluation design 
and on the evaluation subject as relevant, as required. 
• Provide comments on the draft TOR, Inception and Evaluation reports
• Support the Management Response to the evaluation and track the 
implementation of the recommendations. 
• While the Regional Evaluation Officer Roberto Borlini will perform most of the 
above responsibilities, other RB relevant technical staff may participate in the 
Evaluation Reference Group and/or comment on evaluation products as 
appropriate. 
e- Relevant WFP Headquarters divisions will take responsibility to:
• Discuss WFP strategies, policies or systems in their area of responsibility and 
subject of evaluation. 
• Comment on the evaluation TOR, inception and evaluation reports, as required. 
f- Other stakeholders (government, NGOs) will review and comment on key 
evaluation deliverables as experts in an advisory capacity (see ERG TOR in annex). 
g- The Office of Evaluation (OEV). OEV, through the Regional Evaluation Officer, 
will advise the Evaluation Manager and provide support to the evaluation process 
when required. It is responsible for providing access to the outsourced quality support 
service reviewing draft ToR, inception and evaluation reports from an evaluation 
perspective. It also ensures a help desk function upon request. 
8. Communication and budget
8.1 Communication
253. To ensure a smooth and efficient process and enhance the learning from 
these evaluations, the evaluation team should place emphasis on transparent 
and open communication with key stakeholders. This will be achieved by 
ensuring a clear agreement on channels and frequency of communication with 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 75
|Page
and between key stakeholders. All communication between the evaluation team 
and stakeholders should go through or include the evaluation manager.
254. As part of the international standards for evaluation, WFP requires that 
all evaluations are made publicly available. Following the final approval of the 
evaluation reports, findings and recommendations shall be shared in various 
ways, including through discussions with WFP senior management and staff 
(primarily to enhance strategic and operational aspects) as well as with key 
partners including USDA, MINEDUC and MINAGRI, as well as World Vision, 
ADRA and relevant UN agencies. WFP will publish both the reports and the 
management responses. The published versions of the evaluation reports must 
be free from proprietary and personal identifying information. 
255. The evaluation team is requested to prepare and present PowerPoint 
presentations (to debrief WFP management to inform strategic and operational 
decision-making) and 2-page briefs on the midterm and endline reports, both 
of which will be published on the WFP website and shared with national 
stakeholders and other WFP country offices currently receiving USDA grants. 
The evaluation team will also organize a one-day workshop at WFP Rwanda 
premises to share the findings and recommendations from the endline 
evaluation with key stakeholders. WFP and the evaluation team may discuss 
further the detailed communication/dissemination plan. 
8.2 Budget
256. For this evaluation, the budget will: 
• Be based on procurement through Long-term Agreements. Rates are guided by 
pre-agreed rates.
• As detailed in the project’s Evaluation Plan, the total combined budget for this 
contract will not exceed USD 300,000.
• Travel/subsistence/other direct expenses should be accounted for as instructed 
in the WFP budget template. 
Please send any queries to the Evaluation Manager, Daniel Svanlund, M&E Officer, 
WFP Rwanda at daniel.svanlund@wfp.org. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 76
|Page
Annex 2: McGovern-Dole Program Objectives and Activities
Some activities repeat across objectives due to the interconnected nature of the results.
Table 18: McGovern-Dole Program objectives and activities
MGD SO1: Literacy MGD SO2: Health and 
Dietary Practices Foundation Results
Interconnected Activities
Promoting teacher 
attendance and recognition
Distributing school supplies 
and materials
Improving literacy 
instruction materials;
Establishing literacy 
promotion activities
Enhancing food preparation, 
storage, and cooking practices
Building and rehabilitating 
latrines and water collection 
systems
Distributing deworming 
medication
Building/rehabilitating 
kitchens, cooking areas, and 
storerooms
Providing fuel-efficient stoves
Developing partnerships with 
farmer groups to supply food 
to schools
-- 
Develop partnerships with 
farmer groups and local 
cooperatives to supply food to 
schools
Training teachers and 
administrators -- Conduct capacity building at 
all levels
Raising awareness on the 
importance of education -- Raise awareness on the 
importance of education
Providing school meals -- 
Support the implementation 
of the national HGSF 
program
-- Raising awareness on good 
hygiene practices
Strengthen school health 
clubs through training and 
awareness on good health 
and hygiene practices
Establishing and maintaining school gardens 
Engage parents and 
communities through the 
establishment and 
maintenance of school 
gardens
Training PTAs
Sources: (1) Evaluation TOR. (2) WFP Rwanda. 2015. Results Framework. PowerPoint.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 77
|Page
Annex 3: Map
Figure 1: McGovern-Dole Program coverage in Rwanda
WFP Rwanda McGovern-Dole Midterm Evaluation Report 78
|Page
Annex 4: Stakeholder Analysis
Table 19: Stakeholder analysis and mapping
Stakeholder Interest in the McGovern￾Dole Program
Involvement in 
evaluation; likely use Who
Internal (WFP) stakeholders
Rwanda CO Responsible for the country 
level planning and operations 
implementation, WFP Rwanda 
has a direct stake in the 
evaluations and an interest in 
learning from experience to 
inform decision-making. It is 
also called upon to account 
internally as well as to its 
beneficiaries and partners for 
performance and results of its 
operation.
Provide input on the scope of 
the evaluations, evaluation 
questions and strategies. 
Provide data and 
documentation and facilitate 
implementation of the 
evaluation.
Participate in discussions on 
the evaluation design and 
subject. 
Participate in debriefing(s).
Support the ET’s contacts 
with local stakeholders; sets 
up meetings, field visits; and 
provides logistic support 
during the fieldwork. 
WFP key staff will participate 
in KIIs. 
Oversee dissemination and 
follow-up processes, 
including the preparation of 
a Management Response to 
the evaluation 
recommendations.
Organise security briefings 
for the ET and provide any 
materials as required.
Ensures quality assurance 
mechanisms are operational.
Approve final inception and 
evaluation reports.
WFP Rwanda 
Deputy Director
Daniel 
Svanlund, M&E 
Officer 
Evaluation 
Committee
Evaluation 
Reference Group
HGSF Project 
Manager
RB Nairobi Responsible for both oversight 
of COs and technical guidance 
and support, RB management 
has an interest in an 
independent/impartial 
account of the operational 
performance as well as in 
learning from the evaluation 
findings to apply this learning 
to other country offices. The 
Regional Evaluation Officers 
support CO/RB management 
to ensure quality, credible and 
useful decentralized 
evaluations.
Advise the Evaluation 
Manager and provide 
support to the evaluation 
process where appropriate. 
Participate in discussions 
with the ET on the 
evaluation design and on the 
evaluation subject as 
relevant, as required. 
Provide comments on the 
draft Inception and 
Evaluation reports.
Support the Management 
Response to the evaluation 
and track the 
implementation of the 
recommendations. 
Regional 
Evaluation 
Officer, Roberto 
Borlini will 
perform most of 
the 
responsibilities.
Additionally, 
other RB 
relevant 
technical staff 
may participate 
in the ERG 
and/or comment 
on evaluation 
products where 
appropriate.
OEV OEV has a stake in ensuring 
that decentralized evaluations 
deliver quality, credible and 
useful evaluations respecting 
provisions for impartiality as 
well as roles and 
OEV, through the Regional 
Evaluation Officer, will 
advise the Evaluation 
Manager and provide 
support to the evaluation 
process when required. It is 
Evaluation 
Officer 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 79
|Page
accountabilities of various 
decentralised evaluation 
stakeholders as identified in 
the evaluation policy.
responsible for providing 
access to the outsourced 
quality support service 
reviewing draft inception 
and evaluation reports from 
an evaluation perspective. It 
also ensures a help desk 
function upon request.
Executive 
Board 
The WFP governing body has 
an interest in being informed 
about the effectiveness of WFP 
programs. 
This evaluation will not be 
presented to the Board but 
the findings may feed into 
thematic and/or regional 
syntheses and corporate 
learning processes.
The EB is made 
up of 36 State 
members.
Partners: 
World 
Vision, 
Gardens for 
Health Int’l, 
Rwanda 
Biomedical 
Centre
NGOs are WFP’s partners for 
the implementation of some 
activities while at the same 
time having their own 
interventions. The results of 
the evaluation might affect 
future implementation 
modalities, strategic 
orientations and partnerships. 
World Vision is a sub-grantee 
focusing on literacy, health 
and WASH. Gardens for 
Health International and 
Rwanda Biomedical Centre are 
key implementing partners.
Review and comment on key 
evaluation deliverables as 
experts in an advisory 
capacity.
Key staff from partners will 
participate in KIIs 
World Vision 
Gardens for 
Health 
International 
Rwanda 
Biomedical 
Centre 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 80
|Page
Annex 5: Additional Relevant Interventions (non-USDA-funded)
Implementer Intervention Notes
Literacy
USAID Literacy, Language, and 
Learning (L3) (2012‐2016): 
improving the quality of education1
Partners with MINEDUC
Health and Nutrition
MINESANTE 12+ Program: targets 114,500 girls 
between the ages of 10‐12 years in 
primary schools across Rwanda, 
with the objective of reducing girl 
drop‐out rates by creating safe 
spaces in schools for girls. 
Supported by DFID, Nike 
Foundation, and PSI
Clinton Health Access 
Initiative (CHAI)
Since 2013, CHAI has been 
implementing a program with the 
government to work with food 
producers to combat malnutrition. 
CHAI assists the government to 
reduce chronic malnutrition among 
infants and pregnant and lactating 
women. CHAI supports the Africa 
Improved Foods on sourcing local 
agricultural produce to use in its 
factory which produces nutritious 
fortified blended foods that can be 
distributed throughout the country. 1
This partnership was included in 
the program proposal but did not 
come to fruition
WASH
Rwanda’s Water and 
Sanitation Corporation 
(WASAC)
Ubuzima WASH project: trains 
Community Health Workers and 
school‐based volunteers through the 
Community‐based Environmental 
Health Promotion Programme 
(CBEHPP). CBEHPP is 
MINESANTE’s approved 
methodology of working with 
communities to help them identify 
and solve their own health and 
hygiene issues. World Vision is 
Rwanda’s largest implementer of 
CBEHPP and outside of the 
McGovern-Dole Program is 
supporting MINESANTE in training 
45,000 CHWs in 15,000 villages to 
reduce hygiene‐related diseases in 
communities and within schools. 1
Cost‐sharing partnership 
between World Vision and 
WASAC, partnering with 
MINESANTE
1 WFP Rwanda FY 2015 McGovern-Dole proposal
WFP Rwanda McGovern-Dole Midterm Evaluation Report 81 |Page 
Annex 6: Results Framework and Foundational Results
Source: WFP Rwanda. 2015. Results Framework. PowerPoint. (“Foundational Results” diagram).
(see next three pages)
WFP Rwanda McGovern-Dole Midterm Evaluation Report 82 |Page 
MGD SO1: Improved Literacy of School-Age Children
MGD 1.1: Improved 
Quality of Literacy 
Instruction
MGD 1.1.1: 
More 
Consistent 
Teacher 
Attendance
MGD 1.2: Improved 
Attentiveness
MGD 1.3: 
Improved 
Student 
Attendance
MGD 1.1.2: 
Better 
Access to 
School 
Supplies & 
Materials
MGD 1.1.3: 
Improved 
Literacy 
Instructional 
Materials
MGD 1.1.4: 
Increased Skills 
and 
Knowledge of 
Teachers
MGD 1.1.5: 
Increased Skills 
and Knowledge 
of 
Administrators
MGD 1.2.1: 
Reduced 
Short-Term 
Hunger
MGD 1.3.5: 
Increased 
Community 
Understanding 
of Benefits of 
Education
MGD 1.3.1: 
Increased 
Economic and 
Cultural 
Incentives 
(Or Decreased 
Disincentives)
MGD 1.2.1.1/1.3.1.1:
Increased Access to Food
(School Feeding)
Rwanda FY2015 McGovern-Dole Proposal: Results Framework #1
6. Provide School 
Meals 
(WFP, MINEDUC, 
schools, NGO)
10. Raising 
Awareness on 
the Importance 
of Education 
(WV)
3. Establish 
Libraries 
and 
Produce 
Books and 
Supplement
ary Reading 
Materials 
(WV)
2.Distribute 
School 
Supplies and 
Materials 
(WV)
4. Training 
Teachers 
(WV)
1. Promote 
Teacher 
Attendance 
and 
Recognition 
(WV)
5. Training 
School 
Administrators 
(WV) 
MGD 1.3.3: 
Improved 
School 
Infra￾structure
MGD 1.3.2: 
Reduced 
Health￾Related 
Absences
MGD SO2: 
Increased Use of 
Health and 
Dietary Practices 
( )
Child Friendly 
Schools 
Program
 (Partner 
activity: 
MINEDUC, 
UNICEF)
Framework Key
Result Achieved 
by WFP or 
Subrecipient
Result Achieved 
by Partner
WFP or 
Subrecipient 
Activity
Partner 
Activity
MGD 1.3.4: 
Increased 
Student 
Enrollment
11. Establish 
Activities to 
Promote Literacy 
7. Develop (WV)
Partnerships 
with Farmer 
Groups to 
Supply Food to 
Schools (WFP)
8. Establish and 
Maintain School 
Gardens (WFP)
12. Training: PTAs 
(WV, WFP, NGOs)
9. Provide 
Fuel￾Efficient 
Stoves, 
(WFP)
See also 
activities 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 83 |Page 
MGD SO2: Increased Use of Health and Dietary Practices
MGD 2.1: 
Improved 
Knowledge of 
Health and 
Hygiene Practices
MGD 2.5: 
Increased Access 
to Preventative 
Health 
Interventions
MGD 2.4: 
Increased Access 
to Clean Water 
and Sanitation 
Services
MGD 2.6: 
Increased Access 
to Requisite Food 
Prep and Storage 
Tools and 
Equipment
Rwanda FY2015 McGovern-Dole Proposal: Results Framework #2
15. 
Building/rehabilita
ting Latrines (WV)
MGD 2.3: 
Increased 
Knowledge of 
Nutrition
MGD 2.2: 
Increased 
Knowledge of 
Safe Food Prep 
and Storage 
Practices
Framework Key
Result Achieved 
by WFP or 
Subrecipient
Result Achieved 
by Partner
WFP or 
Subrecipient 
Activity
Partner 
Activity
14. Training: Food 
Preparation and 
Storage Practices 
(WFP) 
13. Training and 
Awareness on 
Good Health and 
Hygiene Practices
(WV)
18. 
Building/rehabilitati
ng Kitchens, 
Storerooms (WFP)
17. Distribute: 
Deworming 
Medication (WFP 
& partner) 
8. Establish and 
Maintain School 
Gardens (WFP)
9. Provide 
Fuel-Efficient 
Stoves (WFP)
16. 
Building/rehabilita
ting Water 
Collection Systems 
(WV)
13. Training and 
Awareness on 
Good Health and 
Hygiene Practices
(WV)
Ubuzima Project
 (Partner 
activity: 
MINESANTE, 
WV)
12. Training: PTAs (WV, WFP, NGOs)
WFP Rwanda McGovern-Dole Midterm Evaluation Report 84 |Page 
MGD 1.4.4/2.7.4:
Increased Engagement of Local 
Organizations and Community 
Groups
MGD 1.4.3/2.7.3: 
Increased Government Support 
MGD 1.4.1/2.7.1:
Increased Capacity of 
Government Institutions
MGD 1.4.2/2.7.2:
Improved Policy and 
Regulatory Framework
Foundational Results
Rwanda FY2015 McGovern-Dole Proposal: Foundational Results
19. Capacity Building: Local, regional, national 
(WFP, WV)
12. Training: PTAs (WV, WFP, 
NGOs)
Result Achieved 
by WFP or 
Subrecipient
Result Achieved 
by Partner
WFP or 
Subrecipient 
Activity
Partner 
Activity
10. Raise Awareness on 
Importance of Education (WV)
Framework Key
20. Support Implementation of the National Home Grown School Feeding Program
(WFP)
7. Develop 
Partnerships 
with Farmer 
Groups to Supply 
Food to Schools 
(WFP)
8. Establish and 
Maintain 
School Gardens 
(WFP)
13. Training and 
Awareness on 
Good Health and 
Hygiene Practices 
(WV) 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 85 |Page 
Annex 7: Government policies and priorities related to food security, social protection and education
Table 20: Government policies and priorities related to food security, social protection and education
Vision 2020 Vision 2020 is the main policy framing Rwanda’s development priorities. It seeks to Vision 2020 seeks to 
modernise agriculture and increase productivity,106 and transform Rwanda into a middle-income country with 
healthier, educated and more prosperous Rwandans by 2020. 
Multi-Sectoral Food and 
Nutrition Policy and Strategic 
Plan (2013–2018)
This plan addresses stunting through multi-sectoral nutrition activities, including nutrition screening of children 
under age five.107 A more recent strategic plan for agriculture investment (PSTA IV 2018–2023) aims to 
mainstream food security and nutrition throughout strategic programs. Multi-stakeholder platforms such as the 
Inter-Ministerial Coordination Committee have been established to expand nutrition activities jointly 
implemented by government and development partners. 
National Social Protection 
Strategy (2011)
The NSPS prioritises development of the social protection sector to ensure that all poor and vulnerable people are 
guaranteed a minimum income and access to core public services. The Vision 2020 Umurenge Programme (VUP) 
is a large-scale government social protection program that aims to eradicate extreme poverty by 2020.108
Implemented by the Local Administrative Entities Development Agency (LODA) under the Ministry of Local 
Government (MINALOC), VUP targets schools in the poorest areas. It is part of the Economic Development and 
Poverty Reduction Strategy (EDPRS 2), which includes indicators and targets to reduce chronic malnutrition 
among children under two, reduce poverty, and create employment, exports, and 11.5 percent economic growth.109
National Strategy for 
Transformation 
Rwanda’s key poverty reduction strategy focuses on three pillars, of which the social transformation pillar has a 
priority area to ensure quality of education for all, aiming at building a knowledge-based economy (UN CCA 2017, 
cited in Evaluation Terms of Reference).
Education Sector Strategic Plan 
(ESSP) (2010‐2015)
The ESSP aims to improve the quality of education. It prioritises school feeding as a key component of school 
health and nutrition. The Rwanda Education Board (REB) supports the ESSP and works to build teachers’ 
capacities. The Competence-Based-Curriculum developed by the REB and MINEDUC establishes specific 
descriptors for literacy relevant to McGovern-Dole Program goals.110
Guided by the ESSP, now in its fourth cycle, Rwanda has invested significant resources toward improving the quality 
and coverage of all levels of education, as well as toward implementing policies that aim to achieve universal and 
equitable access to basic education for all Rwandan children. The provision of universal, compulsory and free nine 
years of basic education for children aged seven to 15 years has had a significant impact on increasing access, and this 
is now being expanded to 12 years. The curriculum has undergone a major reform, with a new competence-based 
curriculum being phased in from January 2016 (UN CCA 2017, cited in Evaluation Terms of Reference).
106 Government of Rwanda. 2004. Rwanda Vision 2020. 107 USAID. 2018. Rwanda: Nutrition Profile.
108 NISR. 2018. Vision 2020 Umurenge Program (VUP) - Baseline Survey.
109 Government of Rwanda. 2013. Economic Development and Poverty Reduction Strategy 2013 – 2018. 110 (1) Read a variety of texts accurately and fast; (2) Express ideas, messages and events through writing legible texts in good hand-writing with correctly spelt words; (3) 
Communicate ideas effectively through speaking using correct phonetics of words; and (4) Listen carefully for understanding and seeking clarification when necessary. 
REB/MINEDUC. 2015. Competence-Based Curriculum: Summary of Curriculum Framework Pre-primary to Upper Secondary 2015. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 86 |Page 
School Health and Nutrition 
(2013) policy
This policy states, “all Rwandan schoolchildren shall achieve their full development potential by studying in a 
healthy environment in child-friendly schools.” It includes capacity building for teachers and students on school 
health and nutrition; school health clubs help disseminate knowledge.
Social Protection Sector 
Strategy (2018-2024)
The government has set the ambitious goal of moving from low-income status to upper-income status by year 
2050, and through the forthcoming Social Protection Sector Strategy (2018-2024), has made a commitment to 
provide a life-cycle approach to social protection systems. The economy is growing and income inequality is 
decreasing, which strengthens the possibility that the government will indeed move toward its goals and self￾finance social protection, including access to education and food insecurity safety nets for all. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 87
|Page
Annex 8: International Assistance to Rwanda
257. United Nations collaboration. WFP has worked in Rwanda since 1975. 111
Rwanda is one of eight pilot countries in which United Nations agencies coordinate 
closely through One UN. United Nations agencies coordinate their activities toward 
economic and social transformation and transformational governance through the 
United Nations Development Assistance Programme (UNDAP).112 As of April 2018, the 
2018-2023 UNDAP was still being finalised. United Nations inter-agency 
collaboration includes WFP and UNHCR’s joint assistance to Congolese and 
Burundian refugees in Rwanda and WFP, UNICEF and UNESCO collaboration on 
education activities such as creating Child‐Friendly School standards and assessing 
literacy and numeracy. WFP and FAO have collaborated on Purchase for Progress 
(P4P) agricultural support activities to support smallholder farmers to access markets. 
Since 2011, the government has integrated lessons learned from that program to create 
Common P4P (CP4P), which purchases food from smallholder cooperatives for the 
National Strategic Reserve.113 
258. Assistance to the education sector. MINEDUC, Save the Children and 
World Vision Rwanda are partnering with WFP on Literacy Boost, a school feeding 
and literacy program to improve the literacy of 195,000 children in grades 1‐3 in 280 
primary schools in central Rwanda. 114 MINEDUC also worked with the US Agency for 
International Development (USAID) Literacy, Language and Learning (L3) project 
2012‐2016 on improving the quality of education.115 
259. Assistance to the health/nutrition sector. USAID Rwanda’s Feed the 
Future current multi-year strategy works to improve infrastructure, agricultural 
market linkages, nutrition, innovation, and policy through multiple activities.116 Since 
2013, the Clinton Health Access Initiative (CHAI) has partnered with the government 
to work with food producers to produce fortified blended foods to reduce malnutrition 
among infants and pregnant and lactating women.117 CHAI is supporting Africa 
Improved Foods on sourcing local agricultural produce to use in its factory which 
produces nutritious fortified blended foods that can be distributed throughout the 
country. USAID also supports programming in Rwanda under the President’s 
Emergency Plan for AIDS Relief (PEPFAR), the President’s Malaria Initiative, and the 
Global Climate Change initiative.118
111 WFP. 2017d. WFP Rwanda Country Brief. November 2017. 112 UN Rwanda. 2018. UNDAP 2018-2023 for Rwanda. Signed 31 July 2018.
113 WFP. 2014. Purchase for Progress - P4P Rwanda.
114 (1) Evaluation TOR. (2) World Vision. 2016. Rwanda Annual Report 2016. 
115 Evaluation TOR.
116 USAID. 2018. Rwanda: Nutrition Profile.
117 Evaluation TOR; WFP. 2015. WFP Rwanda FY 2015 McGovern-Dole proposal. 118 History of USAID Rwanda. Accessed at https://www.usaid.gov/history-usaidrwanda
WFP Rwanda McGovern-Dole Midterm Evaluation Report 88 |Page 
Annex 9: Evaluation Matrix 
Sub-question Evaluation 
Criteria Analysis/Indicators Data Sources Triangulation
Key Question 1: How appropriate is the operation?
1.1 To what extent are the objectives, targeting, 
activities and transfer modalities…
…coherent with/relevant to the needs of the 
food-insecure population, and have they 
remained relevant? 
(Consider the distinct needs of women, men, 
boys and girls from different groups and 
geographical areas.)
Relevance • Level of access to education and retention at 
program design stage and over time
• Prevalence of poverty and food insecurity by 
district
• Indications of the role that school feeding can 
play in improving access to education
• Extent to which beneficiary and geographical
targeting are in line with the spatial pattern of 
food insecurity and address inclusion / exclusion 
error 
• Coherence with recommendations from past 
assessments, analyses, evaluations
• Extent to which problem analysis and previous 
assessments were used to guide program design 
• Extent that analysis was done to identify (1) 
differentiated needs of targeted groups and (2) 
objectives and components designed to respond 
to those needs 
• Extent to which local stakeholders were 
consulted and involved in program design
• Level of ownership and responsibility accorded 
to stakeholders at program design stage
• Analysis of program results framework
Assessments, analyses 
of the target population 
(e.g., baseline, midterm 
quant survey, outside 
assessments)
Perspectives of target
population (via FGDs, 
KIs)
Opinions of KIs (gov’t, 
partners) 
Programme documents, 
incl. results framework 
Compare program
design and results 
framework against 
recommendations of 
past assessments, 
analyses, evaluations
Compare perspectives 
among KIs
Compare perspectives of 
KIs and target 
population
WFP Rwanda McGovern-Dole Midterm Evaluation Report 89 |Page 
Sub-question Evaluation 
Criteria Analysis/Indicators Data Sources Triangulation
1.2 To what extent are the objectives, targeting, 
activities and transfer modalities…
…coherent with/relevant to national policies and 
strategies, including education, food security, 
nutrition, and gender? 
Relevance
Sustainabilit
y 
• Alignment of objectives, targeting, activities and 
transfer modalities with relevant national 
policies/strategies
• Extent to which the handover strategy is in line 
with the conditions/constraints in terms of 
human and financial capacity at the moment that 
the program was conceived
• Extent to which lessons from other school 
feeding handover situations were taken into 
account in program design 
National 
policies/strategies
WFP corporate and 
program documentation
Opinions of KIs (gov’t, 
NGO partners) 
Compare perspectives 
among KIs
1.3 To what extent are the objectives, targeting, 
activities and transfer modalities…
…complementary to the policies, priorities, and 
interventions of donors, relevant government 
entities, and NGO partners? 
Relevance
Sustainabilit
y 
• Coherence of the objectives and activities with 
those of partners; synergies with other projects 
and with related sectors (e.g., health)
• Degree of consultation between WFP and other 
actors in relevant areas 
• Degree of understanding of the program 
demonstrated by other actors who work in the 
same area 
• Existence of memoranda of understanding and 
their relevance to program objectives/activities 
KIs with gov’t and 
implementing partners
Documentation of gov’t, 
donor, and partner 
strategies and programs 
Compare perspectives 
among KIs, esp. gov’t 
and NGO partners 
1.4 To what extent are the objectives, targeting, 
activities and transfer modalities …
…coherent with/relevant to relevant WFP and 
UN-wide system strategies, policies and 
normative guidance?
Relevance
Sustainabilit
y 
• Coherence of the objectives and activities with 
WFP and UN-wide strategies, policies, and 
normative guidance, e.g., regarding: 
education/school feeding, food security, 
nutrition, capacity development, gender
UN and WFP corporate 
policies as indicated
Program documentation
Compare perspectives 
among KIs, esp. WFP 
staff and UN partners 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 90 |Page 
Sub-question Evaluation 
Criteria Analysis/Indicators Data Sources Triangulation
1.5 To what extent is the overall approach 
(combining direct assistance and capacity 
development) to support the transition toward 
full gov’t management and implementation of a 
national school feeding program, appropriate? 
Relevance
Sustainabilit
y 
Impact
• Appropriateness and extent to which WFP has 
supported the gov’t in selecting a school feeding 
model that best fits with government 
institutions, processes and policy frameworks 
structures and policies
• Quality of the capacity analysis that took place at 
the time of program design and degree to which 
the needs of different institutions/key actors was 
taken into account
• Coherence between the identified needs , 
budgeting in the project document, and actual 
expenditure in areas related to capacity 
development and handover
• Degree to which issues around financial 
sustainability were taken into account in the 
design and implementation of the project
• Extent to which a cost analysis approach was 
employed at the moment when decisions were 
taken about program transitioning and the 
extent to which this has continued to be 
considered
• Extent to which the technical capacity needs of 
WFP were taken into account to adequately 
support a transition process, and were 
reviewed/adjusted over time
KIs with gov’t ministries 
(national and sub￾national levels)
Program documents
Budget documents
Compare perspectives 
among KIs, esp. WFP 
staff, gov’t ministries 
(national and sub￾national levels)
WFP Rwanda McGovern-Dole Midterm Evaluation Report 91 |Page 
Sub-question Evaluation 
Criteria Analysis/Indicators Data Sources Triangulation
Key Question 2: What are the results of the operation?
2.1 What is the level of attainment of planned 
results and performance? (gender￾disaggregated, as data permit)
This Q incorporates these key Qs from the TOR:
--Have literacy rates of school-age children 
improved over the duration of the program? If 
so, how and why? 
--Are students able to read grade-level text? 
--Are teachers demonstrating new methods of 
teaching?
--Has the use of health and dietary practices 
increased? If so, how? 
--Has illness-related absence decreased? 
--Are students washing their hands?
--Are schools and school kitchens clean? 
--How are school gardens being used? 
Effectiveness
(some 
indicators/Q
s are also 
impact)
• Comparison of midterm survey data against 
baseline; focus on USDA McGovern-Dole 
indicators
• Attention to gender-disaggregated data, where 
available
Baseline data (EGRA 
and school survey)
Midterm data (EGRA 
and school survey) (incl. 
data on health and 
dietary practices)
Observation
Interviews with 
informants for survey 
modules
Follow-up all self￾reported midterm 
school survey data with 
in-person interviews to 
elaborate data and 
conduct spot checks 
against school-level 
documentation
WFP Rwanda McGovern-Dole Midterm Evaluation Report 92 |Page 
Sub-question Evaluation 
Criteria Analysis/Indicators Data Sources Triangulation
2.
2 
What is the level of community-level 
involvement and participation in decision￾making in PTAs and SMCs? (key Q from TOR) 
--Particularly, what is the level of involvement 
and participation of men? (key Q from TOR) 
--Also, what is the level and sustained continuity 
of community contributions in cash and in kind?
Effectiveness
Sustainabilit
y 
Impact
• Qualitative analysis of PTA and SMC structure, 
composition, and processes
• Qualitative analysis of the influence of PTAs and 
SMCs on school feeding policies and practices
• Extent of involvement and participation of 
women and men; qualitative analysis of any 
gender dynamics that affect representation and 
decisionmaking
Focus groups with PTAs 
and SMCs
KIs with school-level 
actors and other 
stakeholders who have 
first-hand knowledge of 
this issue
Observation
Compare KII findings, 
FGD findings, and 
observation
2.
3 
To what extent has the program addressed 
gender equality and protection issues?
To what extent has the program influenced
gender equality and protection? 
Effectiveness
Sustainabilit
y 
Impact
• Qualitative analysis of WFP, gov’t, UNICEF, and 
partner perspectives on how the program has 
addressed gender and protection
• Focus groups with students, PTAs, SMCs on 
gender and protection issues that affect them 
and how the program has handled them
• KIIs with stakeholders with knowledge of how 
the program has addressed gender and 
protection
Baseline data (EGRA 
and school survey)
Midterm data (EGRA 
and school survey)
KIIs and FGDs
Observation
Compare WFP 
perspectives on gender 
and protection with 
those of program 
participants and non￾WFP stakeholders
2.
4 
How efficient is the program in achieving results 
to date?
Efficiency • Timing and timeliness of implementation; 
adherence to implementation schedule 
• Cost-efficiency of activities relative to 
alternatives
• Efficiency of program strategies in terms of 
financial and human resource inputs vs outputs
• Adequacy of monitoring system relative to 
program needs and requirements 
Program
documentation, incl. 
budget documentation
Midterm data (EGRA 
and school survey)
KIIs with WFP 
management, financial 
and M&E staff
Compare assessments 
by KIs with performance 
data 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 93 |Page 
Sub-question Evaluation 
Criteria Analysis/Indicators Data Sources Triangulation
Key Question 3: Why and how has the operation produced the observed results?
3.1 What are the key WFP-internal factors 
contributing to achievement or non￾achievement of desired results?
Effectiveness
Efficiency 
Sustainabilit
y 
Impact
• Quality and effectiveness of WFP-internal 
context and processes, including but not limited 
to:
o Staff level
o Staff capacity
o Financial resources 
o Supply chain/ pipeline management
o Quality, timeliness and use of 
monitoring data
o Quality, timeliness, and processes for 
internal communication and 
decisionmaking (HQ-RB-CO-field)
o Technical backstopping from HQ-RB
o Knowledge management
o GEEW
Program documents
KIs with WFP staff and 
external stakeholders
Observation
Comparison of 
assessment of 
contributing factors by 
internal and external 
KIs
3.
2 
What are the key external factors contributing to 
achievement or non-achievement of desired 
results?
This Q incorporates these key Qs from the TOR:
--What are the key institutions (i.e., 
international, national, provincial/district and 
local) and governance structures required to 
effectively deliver, implement, and sustain 
school meal interventions? 
--What relationship structures among these 
institutions yield the most successful and 
effective school meal programs? 
--Has the provided capacity support increased 
the government’s capacity to own and sustain a 
national school meals program? 
Effectiveness
Efficiency 
Sustainabilit
y 
Impact
• Quality and effectiveness of external context, 
relationships/partnerships, and processes, 
including but not limited to:
o Communication, information-sharing, 
and coordination with partners 
o Are the right partners on board – is 
anyone excluded
o Extent of systematization of cooperative 
arrangements 
• Capacity strengths and weaknesses of 
institutional partners 
• Financial resource opportunities and constraints 
of institutional partners 
• Country/regional context: policy framework, 
politics, governance, civil infrastructure, 
business, markets, natural hazards
• Factors inherent to serving remote communities
• GEEW
Program documents
Country/regional 
context information 
KIs with WFP staff and 
external stakeholders
Observation
Comparison of 
assessment of 
contributing factors by 
internal and external 
KIs
Comparison of 
subjective and objective 
information 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 94 |Page 
Sub-question Evaluation 
Criteria Analysis/Indicators Data Sources Triangulation
3.
3 
Has the program yielded any unintended 
results, positive or negative?
Effectiveness
Impact
• Existence/identification of positive results not 
anticipated in the results framework; exploration 
of the extent to which these results can be 
attributed to the program 
• Existence/identification of unintended negative 
results; exploration of the extent to which these 
results can be attributed to the program 
• GEEW
Focus groups
KIs
Program documents, 
incl. results framework 
Observation
Comparison of data 
from all sources; 
exploration with the CO
WFP Rwanda McGovern-Dole Midterm Evaluation Report 95 |Page 
Key Question 4: To what extent does the operation’s implementation strategy include considerations for sustainability?
4.1 How effective are efforts to date to build 
government capacity to fully hand over school 
feeding?
This Q incorporates these priorities identified in 
the TOR: 
Is WFP’s capacity support to smallholder 
farmers and key line ministries 
appropriate/sufficient to effectively facilitate 
national ownership? (key Q from TOR) 
This Q incorporates these priorities identified 
by the CO in the inception phase: 
--How WFP can work more strategically in its 
capacity development efforts with government? 
--What is the progress and status of capacity 
development?
--What capacity development approaches/ 
activities are working/not working? 
--To what extent is the program on track for 
handover readiness and government ownership? 
Relevance
Effectiveness
Sustainabilit
y 
Impact
• Analysis of program documentation regarding 
steps taken and steps planned toward capacity 
development of smallholders and gov’t
• Assessment of the existence, rationale and 
quality of capacity development plans
• WFP perspectives (national and sub-national) on 
gov’t readiness for handover
• Gov’t perspectives (national and sub-national) 
on gov’t readiness for handover 
• Perceptions of staff and stakeholders on 
efficiency (cost, systems, staff, alternatives, etc.). 
• Extent to which resources (human, physical, 
financial, organizational and functional) were 
optimally used in program implementation
• Quality of processes (e.g., supply chain 
management), relevance of the division of 
responsibilities in the management of the 
operation at different levels 
• Existence and quality of coordination that has 
been put in place with partners to optimise 
resources
Program documents
KIIs – both national and 
sub-national level
Compare perspectives of 
WFP and gov’t
WFP Rwanda McGovern-Dole Midterm Evaluation Report 96
|Page
Annex 10: List of Persons Interviewed and Focus Groups
Key Informant Interviews
Name Title M/F Location Date Interviewer
WFP Rwanda
[REDACTED] M&E Officer M Kigali Multiple Jeanne/Bruce
[REDACTED] Program
Manager, HGSF F Kigali Multiple Jeanne/Bruce 
[REDACTED] Manager, School 
Feeding F Kigali multiple Jeanne/Bruce 
[REDACTED] Head of Program F Kigali Jeanne/Bruce
[REDACTED] Head, Nutrition M Kigali Bruce
[REDACTED] M Kigali Jeanne/Bruce
[REDACTED] Former HGSF 
Manager F Jeanne/Bruce 
[REDACTED] HGSF 
Coordinator M Rutsiro District 4/10/18 Bruce/Justin
[REDACTED] HGSF 
Coordinator M Nyamagabe District 8/10/18 Jeanne/Justin
[REDACTED] HGSF 
Coordinator M Nyamagabe District 9/10/18 Bruce/Justin
[REDACTED] HGSF Monitoring 
Assistant M Nyaruguru 11/10/18 Jeanne/Daria 
[REDACTED] HGSF Monitoring 
Assistant F Nyaruguru 11.10.18 Jeanne/Daria 
[REDACTED] HGSF 
Coordinator M Nyaruguru 11/10/18 Jeanne/Daria 
[REDACTED] HGSF 
Coordinator M Karongi district 15/10/18 Bruce/Justin
Subtotal 9M/5F
School - based
[REDACTED] Head teacher M [REDACTED] 3/10/18 Bruce/Jeanne/ 
Justin/Daria
[REDACTED] P.3 Kinyarwanda 
Teacher F [REDACTED] 3/10/18 Bruce/Justin
[REDACTED] P.2 Kinyarwanda/ 
Math Teacher F [REDACTED] 3/10/18 Bruce/Justin
[REDACTED] Teacher/
Storekeeper M [REDACTED] 3/10/18 Jeanne/Daria 
[REDACTED] Teacher/
Storekeeper M [REDACTED] 4/10/18 Jeanne/Daria 
[REDACTED] Kinyarwanda 
Teacher M [REDACTED] 4/10/18 Jeanne/Daria 
[REDACTED]
Teacher/Literacy 
Boost reading 
groups
F [REDACTED] 4/10/18 Jeanne/Daria 
-- PTA member M [REDACTED] 4/10/18 Jeanne/Daria 
[REDACTED] Head teacher M [REDACTED] 4/10/18 Bruce/Justin
[REDACTED] Teacher/school 
hygiene F [REDACTED] 4/10/18 Bruce/Justin
[REDACTED] Teacher/Literacy 
Boost F [REDACTED] 4/10/18 Bruce/Justin
[REDACTED] Teacher/Literacy 
Boost M [REDACTED] 4/10/18 Bruce/Justin
[REDACTED] Teacher/
Storekeeper M [REDACTED] 4/10/18 Bruce/Justin
WFP Rwanda McGovern-Dole Midterm Evaluation Report 97
|Page
[REDACTED] School 
Administrator M [REDACTED] 8/10/18 Bruce/Daria
[REDACTED] Teacher/
Storekeeper M [REDACTED] 8/10/18 Bruce/Daria
-- PTC member/ 
Hygiene F [REDACTED] 8/10/18 Bruce/Daria
[REDACTED] Kinyarwanda 
Teacher F [REDACTED] 8/10/18 Bruce/Daria
[REDACTED] Kinyarwanda 
Teacher F [REDACTED] 8/10/18 Bruce/Daria
-- 
Reading group 
facilitator 
(Literacy Boost)
F [REDACTED] 8/10/18 Bruce/Daria
[REDACTED] Teacher/ 
Storekeeper F [REDACTED] 8/10/18 Jeanne/Justin
[REDACTED] Deputy Head 
Teacher / Teacher M [REDACTED] 8/10/18 Jeanne/Justin
[REDACTED] Teacher/Hygiene F [REDACTED] 8/10/18 Jeanne/Justin
[REDACTED]
Teacher/reading 
group facilitator 
/Literacy Boost 
F [REDACTED] 8/10/18 Jeanne/Justin
-- Health Facilitator 
(deworming) M [REDACTED] 8/10/18 Jeanne/Justin
-- PTA Head M [REDACTED] 10/10/18 Jeanne/Daria
-- Reading group 
facilitator M [REDACTED] 10/10/18 Jeanne/Daria 
-- Reading group 
facilitator M [REDACTED] 10/10/18 Jeanne/Daria 
[REDACTED] Head teacher F [REDACTED] 10/10/18 Jeanne/Daria
[REDACTED] Teacher/Literacy 
boost F [REDACTED] 10/10/18 Bruce/Justin
-- PTA Advisor F [REDACTED] 10/10/18 Bruce/Justin
-- PTA Vice 
President F [REDACTED] 10/10/18 Bruce/Justin
[REDACTED] Head teacher M [REDACTED] 10/10/18 Bruce/Justin
-- Reading group 
facilitator M [REDACTED] 10/10/18 Bruce/Justin
-- Reading group 
facilitator M [REDACTED] 10/10/18 Bruce/Justin
[REDACTED] Teacher/Literacy 
boost F [REDACTED] 11/10/18 Bruce/Justin
[REDACTED] Kinyarwanda 
teacher F [REDACTED] 11/10/18 Bruce/Justin
[REDACTED] Head teacher F [REDACTED] 11/10/18 Bruce/Justin
[REDACTED] Head teacher F [REDACTED] 15/10/18 Jeanne/Daria
[REDACTED] Teacher/Literacy 
boost F [REDACTED] 15/10/18 Jeanne/Daria 
[REDACTED] Teacher/WASH F [REDACTED] 15/10/18 Jeanne/Daria
[REDACTED] Teacher/Home 
garden M [REDACTED] 15/10/18 Jeanne/Daria 
-- Storekeeper F [REDACTED] 15/10/18 Jeanne/Daria
[REDACTED] Kinyarwanda 
teacher F [REDACTED] 15/10/18 Bruce/Justin
[REDACTED] Teacher/Hygiene M [REDACTED] 15/10/18 Bruce/Justin
[REDACTED] Teacher/ 
Storekeeper M [REDACTED] 15/10/18 Bruce/Justin
[REDACTED] Head Teacher M [REDACTED] 15/10/18 Bruce/Justin
WFP Rwanda McGovern-Dole Midterm Evaluation Report 98
|Page
[REDACTED] Head Teacher F [REDACTED] 15/10/18 Jeanne/Daria
-- Storekeeper F [REDACTED] 15/10/18 Jeanne/Daria
-- Cook M [REDACTED] 15/10/18 Jeanne/Daria
-- Cook M [REDACTED] 15/10/18 Jeanne/Daria
[REDACTED] Teacher/Literacy 
boost F [REDACTED] 15/10/18 Jeanne/Daria 
[REDACTED] Teacher/WASH F [REDACTED] 15/10/18 Jeanne/Daria 
[REDACTED] Teacher/Gardens 
for health M [REDACTED] 15/10/18 Jeanne/Daria 
Subtotal 25M/
28F
Government Staff – Field-based
[REDACTED]
Social Economic 
&Development 
Officer (Cell level)
M 
Rutsiro 
District/Ruhango 
sector/Rugesa Cell
4/10/18 Jeanne/Daria 
[REDACTED]
Executive 
Secretary 
(Rugesa Cell)
M Rutsiro/Ruhango/
Rugesa 4/10/18 Jeanne/Daria 
[REDACTED] District Education 
Officer M Rutsiro District 4/10/18 Bruce/Justin
[REDACTED] Sector Education 
Officer M Nyamagabe/Musan
ge Sector 9/10/18 Bruce/Justin
[REDACTED] Sector Education 
Officer M Nyamagabe/Kameg
eri Sector 9/10/18 Bruce/Justin
[REDACTED] Sector Education 
Officer M Nyamagabe/Cyanik
a Sector 9/10/18 Bruce/Justin
[REDACTED] District Education 
Officer M Nyaruguru District 11/10/18 Jeanne/Daria 
[REDACTED] District Education 
Officer M Karongi District 15/10/18 Bruce/Justin
Subtotal 8M/0
F
Cooperatives
-- Coop President M [REDACTED]/ 
Nyamagabe District 9/10/18 Jeanne/Daria 
-- Coop Vice 
President F [REDACTED]/ 
Nyamagabe District 9/10/18 Jeanne/Daria 
-- Coop Accountant F [REDACTED]/ 
Nyamagabe District 9/10/18 Jeanne/Daria 
-- Coop Secretary F [REDACTED]/ 
Nyamagabe District 9/10/18 Jeanne/Daria 
-- Coop President F [REDACTED]/ 
Nyaruguru district 11/10/18 Bruce/Justin
-- Coop Vice 
President M [REDACTED]/ 
Nyaruguru district 11/10/18 Bruce/Justin
-- Coop Secretary M [REDACTED]/ 
Nyaruguru district 11/10/18 Bruce/Justin
-- Coop Accountant M [REDACTED]/ 
Nyaruguru district 11/10/18 Bruce/Justin
-- Coop Advisor F [REDACTED]/ 
Nyaruguru district 11/10/18 Bruce/Justin
-- Coop Advisor M [REDACTED]/ 
Nyaruguru district 11/10/18 Bruce/Justin
Subtotal 5M/5F
Partners
[REDACTED]
Deputy Chief of 
Party, World 
Vision 
M Kigali Jeanne/Bruce 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 99
|Page
[REDACTED]
Literacy Boost 
Coordinator, 
World Vision
F Kigali Jeanne/Bruce 
[REDACTED]
Gardens for 
Health 
International 
Agriculture 
Manager
M Kigali 17/10/18 Jeanne/Bruce 
[REDACTED]
Literacy Boost 
Coordinator/ 
World Vision
M Nyamagabe District 8/10/18 Bruce/Daria
[REDACTED]
Literacy Boost 
Coordinator/ 
World Vision
F Nyaruguru District 8/10/18 Bruce/Daria
[REDACTED]
Health & Hygiene 
Coordinator/ 
World Vision
F Southern Province 8/10/18 Bruce/Daria
[REDACTED] CVA Coordinator M Southern Province 8/10/18 Bruce/Daria
[REDACTED]
Gardens for 
Health District 
Coordinator
M Nyamagabe/Nyaru
guru Districts 9/10/18 Bruce/Justin
[REDACTED]
World Vision/ 
Health and 
Hygiene 
Coordinator
M Karongi / Rutsiro 
districts 16/10/18 Jeanne/Daria/
Justin
[REDACTED]
World Vision/ 
Literacy Boost 
Coordinator
M Karongi district 16/10/18 Jeanne/Daria/
Justin
[REDACTED]
World Vision/ 
Literacy Boost 
Coordinator
M Rutsiro district 16/10/18 Jeanne/Daria/
Justin
Subtotal 8M/3F
Focus Groups 
Focus Group 
Type #M #F District/sector School/other Date Interviewer
Cooks 3 1 Rutsiro/ Manihira [Redacted] 4/10/18 Bruce/Justin
Cooks 2 1 Rutsiro/ Ruhango [Redacted] 4/10/18 Jeanne/Daria
Students 5 5 Nyamagabe/ 
Cyanika [Redacted] 8/10/18 Bruce/Daria
Cooks 2 1 Nyamagabe/ 
Cyanika [Redacted] 8/10/18 Bruce/Daria
Cooks 1 1 Nyamagabe/ 
Musange [Redacted] 8/10/18 Jeanne/Justi
n
Students 3 3 Nyamagabe/ 
Musange [Redacted] 8/10/18 Jeanne/Justi
n
Coop members 6 8 Nyamagabe/ 
Kibirizi [Redacted] 8/10/18 Jeanne/Daria 
Cooks 2 1 Rutsiro [Redacted] 15/10/1
8 Jeanne/Daria 
TOTAL 24 21
WFP Rwanda McGovern-Dole Midterm Evaluation Report 100
|Page
Annex 11: Topical Outlines 
WFP Country Office
Illustrative list of interviewees: HGSF Program Team, M&E Team, Health/ 
Nutrition Officer, Admin/Finance Officer, Logistics Team, Gender Focal Point, 
Deputy Country Director 
1. How the operation was designed? What factors affected/contributed to the quality 
of the design? (probe: analytical work, involvement of counterparts)
2. To what extent was an enabling framework in place for school feeding (national 
law, policies, strategies and priorities)? To what extent is the HGSF program in 
line with national policies and priorities? 
3. Do HGSF activities correspond to the needs of the target groups? Any changes 
since design stage? 
4. Complementarity between the WFP’s work on HGSF and the work of other 
partners?
5. What issues were identified as needing to be addressed for successful transition to 
government ownership? e.g., financial needs, capacity gaps, institutional 
weaknesses, logistical capacity, partnerships with other government ministries, 
awareness/communication with specific partners including parents and 
communities, monitoring capacity)? What actions were designed to address these 
and how effective are these? 
6. Main results of the operation to date? Any unexpected results, positive or 
negative? 
7. What capacity development strategy has been put in place? Status of 
implementation? Opportunities and constraints to capacity development? 
8. Strengths of the HGSF program to date? What has worked well?
9. Constraints/ challenges in implementing HGSF? How have these been addressed?
10. Main external factors that have influenced achievement/ non-achievement of 
results? 
11. Main internal factors that have influenced achievement/ non-achievement of 
results? 
12. What analysis has been done regarding the cost-efficiency and cost-effectiveness 
of the program? What are your impressions of cost-efficiency and cost￾effectiveness?
13. Nature and quality of support for HGSF from within WFP (RB, HQ)? Influence of 
this support on efficiency and effectiveness? 
14. Level of collaboration between UN and government as part of UNDAF/ Delivering 
as One? What has WFP done to ensure synergies between HGSF and programs of 
other UN agencies? 
15. What partner activities complement HGSF to enhance prospects for longer-term 
sustainability? Are these sufficient? 
16. Extent of partnerships with the private sector to address sustainability?
WFP Rwanda McGovern-Dole Midterm Evaluation Report 101
|Page
17. To what extent does the HGSF link with other social safety nets in Rwanda?
18. How are gender issues mainstreamed into the program? Is the knowledge on 
implementation of those policies sufficient among WFP staff and partners? 
19. How are HGSF activities perceived by government, ministries, NGOs and the 
general public? 
20. How does the WFP CO communicate with its stakeholders? Who communicates 
with beneficiaries? Suggested improvements?
21. What should be the priorities for the remainder of the program period? 
Suggestions on how to improve the program?
WFP Rwanda McGovern-Dole Midterm Evaluation Report 102
|Page
Ministry of Education – National Level
Topical outline may also be used for donor interviews
1. How has MINEDUC collaborated with the McGovern-Dole Program? 
2. To what extent are HGSF activities aligned with national education policy? Has 
the WFP HGSF work fed into national policy development? 
3. How was MINEDUC involved in the design of HGSF activities? Is the design 
relevant and realistic? 
4. How have gender issues been taken into account in the program? Is this approach 
appropriate and effective?
5. What is your assessment of the performance of the WFP Rwanda CO and sub￾national offices? 
6. What are the communication and information-sharing processes between WFP 
and MINEDUC? How effective are these processes for coordination and decision￾making?
7. Extent to which WFP has considered and addressed national and local capacity 
opportunities and constraints (probe: design phase, implementation, capacity 
development/handover plan)? 
8. What analysis has been done regarding the cost-efficiency and cost-effectiveness 
of the program? What are your impressions of cost-efficiency and cost￾effectiveness?
9. What capacity development support has WFP provided? Relevance and quality of 
support? Examples of changes as a result of that support?
10. Strengths of the McGovern-Dole Program to date? What has worked well? 
11. Constraints/ challenges of the McGovern-Dole Program to date? How have these 
been addressed?
12. What factors have influenced achievement/ non-achievement of results? 
13. Are the resources and expertise WFP has mobilized, adequate to implement 
HGSF? 
14. Do you see any further opportunities for cooperation with governmental and non￾governmental partners (e.g., in health, education, gender equity)? 
15. To what extent has WFP supported partnerships with the private sector to address 
sustainability?
16. What are MINEDUC’s needs and concerns around sustainability and full 
government ownership of the McGovern-Dole Program? 
17. What should be the priorities for the remainder of the program period? 
Suggestions on how to improve the program? 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 103
|Page
Ministry of Education – Sub-national Level
1. What do you see as the purpose of the McGovern-Dole Program? 
2. What changes have you noted in the schools or students since the program 
started? Differences between boys and girls? 
3. Describe any positive or negative impact of the program on:
a. the school? 
b. wider community? 
c. the people who prepare the meals? 
d. local farmers/producers? 
4. How/to what extent does the McGovern-Dole Program complement other 
educational and social protection initiatives? 
5. What capacity development support has WFP provided? Relevance and quality of 
support? Examples of changes as a result of that support?
6. What are the communication and information-sharing processes between WFP 
and MINEDUC at sub-national level? How effective are these processes for 
coordination and decision-making?
7. Strengths of the McGovern-Dole Program to date? What has worked well? 
8. Constraints/ challenges of the McGovern-Dole Program to date? (probe: logistics, 
human resources, capacity)? How have these been addressed? 
9. What factors have influenced achievement/ non-achievement of results? 
10. How have gender issues been taken into account in the program? Is this approach 
appropriate and effective?
11. What should be the priorities for the remainder of the program period? 
Suggestions on how to improve the program? 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 104
|Page
Other Ministries
General Questions
1. How has your institution collaborated with the McGovern-Dole Program? 
2. How did the design of HGSF activities take place? What was your institution’s 
involvement? Is the design relevant and realistic? 
3. To what extent are HGSF activities aligned with the policies and priorities of your 
institution? 
4. What is your assessment of the performance of the WFP Rwanda CO and sub￾national offices? 
5. What are the communication and information-sharing processes between WFP 
and your institution? How effective are these processes for coordination and 
decision-making?
6. What are your institution’s needs and concerns around sustainability and full 
government ownership of the McGovern-Dole Program? 
7. Are activities by other partners or other agencies sufficient to complement the 
McGovern-Dole Program to enhance sustainability? What additional partnerships 
could be explored?
8. Strengths of the McGovern-Dole Program to date? What has worked well? 
9. Constraints/ challenges of the McGovern-Dole Program to date? How have these 
been addressed?
10. What factors have influenced achievement/ non-achievement of results? 
11. What should be the priorities for the remainder of the program period? 
Suggestions on how to improve the program? 
Ministry-specific Topics
Ministry Additional/Specific Line of Inquiry
Ministry of Agriculture (MINAGRI) --Alignment with PSTA4 
--Role the ministry plays with 
smallholder farmers
Ministry of Infrastructure (MININFRA)
Ministry of Local Affairs (MINALOC) Seek MINALOC perspective on how the 
implementation of the program is 
involving the local authorities as key 
stakeholders
Ministry of Gender and Family 
Promotion (MIGEPROF)
--Alignment of HGSF with nutrition 
guideline/ policy/ strategy 
-- How have gender issues been taken 
into account in the program? Is this 
approach appropriate and effective?
WFP Rwanda McGovern-Dole Midterm Evaluation Report 105
|Page
Implementing Partners, UNICEF
Interviewees: World Vision, Gardens for Health International, Rwanda Biomedical 
Centre, UNICEF119 
1. How has your organization collaborated with the McGovern-Dole Program? 
2. How was your organization involved in the design of HGSF activities? Is the design 
relevant and realistic? 
3. What are the communication and information-sharing processes between WFP 
and your institution? How effective are these processes for coordination and 
decision-making?
4. What factors have influenced the performance of the collaboration during this 
period (positively or negatively)? 
5. What additional opportunities exist for collaboration/synergies with your own 
organization? 
6. Do you see any further opportunities for cooperation with governmental and non￾governmental partners (e.g., in health, education, gender equity)? 
7. What analysis has been done regarding the cost-efficiency and cost-effectiveness 
of the program? What are your impressions of cost-efficiency and cost￾effectiveness?
8. Strengths of the McGovern-Dole Program to date? What has worked well? 
9. Constraints/ challenges of the McGovern-Dole Program to date? How have these 
been addressed?
10. How have gender issues been taken into account in the program? Is this approach 
appropriate and effective?
11. What should be the priorities for the remainder of the program period? 
Suggestions on how to improve the program? 
119 UNICEF supported modelling and scaling up Child‐Friendly School standards, which were adopted as the 
national quality guidelines for school infrastructure and software inputs. UNICEF also supported the Learning 
Achievement in Rwandan Schools (LARS) Assessment to improve the quality of education and measure learning 
outcomes in literacy and numeracy.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 106
|Page
School Administration and Teachers
Illustrative list of interviewees: Administrators, head teachers, teachers
1. What do you see as the purpose of the McGovern-Dole Program? 
2. What records are kept on (1) administration of meals and (2) student attendance? 
(ask to see records; check if gender-disaggregated)
3. What changes have you noted in the school or students since the program started? 
Differences between boys and girls? 
4. Describe any positive or negative impact of the program on:
a. the school? 
b. wider community? 
c. the people who prepare the meals? 
d. local farmers/producers? 
5. Strengths of the McGovern-Dole Program to date? What has worked well? (probe: 
logistics, relationship with the community)
6. Constraints/ challenges of the McGovern-Dole Program to date? How have these 
been addressed?
7. What factors have influenced achievement/ non-achievement of results? 
8. Have deliveries during the last year been regular and complete (all items received 
in the requested amounts)? 
9. Has the school had to interrupt feeding for any extended periods during the last 
year? Why? 
10. Does the school follow the official ration scale and number and timing of meals 
per day? If not, why not? 
11. Are there any other health activities in the school (deworming, malaria 
prevention)? Who implements them? Quality and effectiveness?
12. Suggestions on how to improve the program? 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 107
|Page
Parent-Teacher Associations and School Management Committees
1. What do you see as the purpose of the McGovern-Dole Program? 
2. What is the role of the [PTA/SMC] in the program? 
3. Do parents contribute to the meals (probe: fresh vegetables, condiments, fuel; 
level of participation, difficulties)
4. What changes have you seen in the students since the program started? 
Differences between boys and girls?
5. Describe any positive or negative impact of the program on:
a. the school? 
b. wider community? 
c. the people who prepare the meals? 
d. local farmers/producers? 
6. What has worked well? 
7. Constraints and challenges? How have these been addressed?
8. Suggestions on how to improve the program?
Cooks 
1. What do you see as the purpose of the McGovern-Dole Program? 
2. What is your role in the program? 
3. What changes have you noted in the school or students since the program started? 
Differences between boys and girls?
4. Is the food sufficient?
5. Do parents contribute to the meals (fresh vegetables, condiments, fuel, work)?
6. Who decides on the composition of the meals? 
7. Have deliveries during the last year been regular and complete (all items received 
in the requested amounts)? If not, why not? 
8. Has the school had to interrupt feeding for any extended periods during the last 
year? Why? 
9. Does the school follow the official ration scale and number and timing of meals 
per day? If not, why not? 
10. Has the initiative had an impact on your lives? Has it affected how you are seen/ 
treated by the school or by the community? Describe.
11. Suggestions on how to improve the program? 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 108
|Page
Students (Grade 5 and Higher)
1. What do you like most about the school meals?
2. What do you like least about the school meals?
3. Has the McGovern-Dole Program changed anything for you? (probe: 
concentration, attendance, one extra meal a day, one less meal at home per day, 
extra burden of in-kind or cash contribution)
4. On the days that there are no meals, do you still come to school? Do other children 
come to school? 
5. If you could change something about the school meals, what would that be?
Farmer Groups, Local Cooperatives, Storekeepers
1. What do you see as the purpose of the McGovern-Dole Program? 
2. What is your role in the program? 
3. Benefits/advantages to your of participating in the program (probe: benefits to 
group/cooperative; individual/personal benefits)? What has worked well?
4. Constraints/challenges of participating in the program? How have these been 
addressed?
5. What training or support have you received from WFP (or partners)? Relevance 
and quality of training/support? Examples of changes you’ve made as a result of 
that support?
6. Do you plan to continue participating in the McGovern-Dole Program? Why or why 
not?
7. Suggestions on how to improve the program? 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 109
|Page
Annex 12: Field Schedule
Week 1 - QUANTITATIVE SURVEY PHASE 1 STARTS
Date District/Sector School
Mon 10 Sep Karongi/Gashari [Redacted]
Tue 11 Sep Karongi/Gitesi [Redacted]
Wed 12 Sep Karongi/Murundi [Redacted]
Thu 13 Sep Rutsiro/Manihira [Redacted]
Fri 14 Sep Rutsiro/Kivumu [Redacted]
Week 2 - QUANTITATIVE SURVEY
Mon 17 Sep Nyamagabe/Cyanika [Redacted]
Tue 18 Sep Nyamagabe/Cyanika [Redacted]
Wed 19 Sep Nyamagabe/Musange [Redacted]
Thu 20 Sep Nyaruguru/Ruheru [Redacted]
Fri 21 Sep Nyaruguru/Nyabimata [Redacted]
Week 3 - QUANTITATIVE SURVEY
Mon 24 Sep Nyaruguru/Ngoma [Redacted]
Tue 25 Sep Rutsiro/Ruhango [Redacted]
Wed 26 Sep Rutsiro/Kivumu [Redacted]
Thu 27 Sep Karongi/Murundi [Redacted]
Fri 28 Sep Rutsiro/Manihira [Redacted]
Week 4 - QUALITATIVE SURVEY STARTS
Date Team District/Sector School
Mon 1 Oct Team 1 & 2 Kigali [Redacted]
Tue 2 Oct Team 1 & 2 Kigali [Redacted]
Wed 3 Oct Team 1 & 2 Rutsiro/Kivumu [Redacted]
Thu 4 Oct Team 1 Rutsiro/Manihira [Redacted]
Rutsiro/Ruhango [Redacted]
Fri 5 Oct Return to Kigali
WFP Rwanda McGovern-Dole Midterm Evaluation Report 110 
|Page
Week 7: QUANTITATIVE SURVEY – PHASE 2
Date District/Sector School
Mon 22 Oct Nyaruguru/Kivu [Redacted]
Tue 23 Oct Nyaruguru/Nyabimata [Redacted]
Wed 24 Oct Nyamagabe/Kibirizi [Redacted]
Thu 25 Oct Nyamagabe/Kamegeri [Redacted]
Fri 26 Oct Karongi/Ruganda [Redacted]
Week 5: QUALITATIVE SURVEY
Date Team District/Sector School/District 
Official/ Coop
Mon 8 Oct Team 1 Nyamagabe/Cyanika [Redacted]
Team 2 Nyamagabe/Musange [Redacted]
Tue 9 Oct Team 1 Nyamagabe District [Redacted]
Team 2 Nyamagabe [Redacted]
Wed 10 Oct Team 1 Nyaruguru/Nyabimata [Redacted]
Team 2 Nyaruguru/Ruheru [Redacted]
Thu 11 Oct Team 1 Nyaruguru/Ngoma [Redacted]
Team 2 Nyaruguru/Ngoma [Redacted]
Fri 12 Kigali Notes/Data Analysis
Week 6 - QUALITATIVE SURVEY
Mon 15 Oct Team 1
Team 2
Karongi/Gashari
Rutsiro/Kivumu
[Redacted]
Tue 16 Oct Team 1 & 2 Karongi [Redacted]
Wed 17 Oct Kigali [Redacted]
Thu 18 Oct Kigali [Redacted]
Fri 19 Oct Return to Kigali
WFP Rwanda McGovern-Dole Midterm Evaluation Report 111 
|Page
Annex 13: Methodology
A. Data collection methods and tools
Table 21 summarizes the data collection tools.
Table 21: Overview of main data collection tools
Data collection method(s) Type of data to 
be collected Comments
School administrator /head
teacher survey (ODK) 
USDA McGovern￾Dole indicators
--Survey completed by face-to-face 
interviews with head teachers and a 
review of school records to validate 
responses. 
--Additional qualitative data collected 
from KIs and/or focus groups1 with 
head teachers, teachers, students, 
cooks, storekeepers, PTAs and SMCs. 
--Focus groups disaggregated by gender 
where feasible.
EGRA tool USDA McGovern￾Dole indicators
Administered the EGRA (literacy + WASH 
questions) to third-graders. 
Topical outlines Qualitative data on 
all evaluation 
questions and to 
validate and help 
interpret indicator 
data
Topical outlines for: 
--WFP Kigali and field staff 
--Government ministries 
--District government (District Education 
Officials)
--Implementing Partners (World Vision, 
Gardens for Health International, 
Rwanda Biomedical Centre)
--United Nations Agency Partners 
(UNICEF)
--Schools (Head teachers, teachers, 
Parent-Teacher Association, School 
Meal Committee) 1 Focus groups disaggregated by gender where feasible.
260. The baseline did not establish a value for the McGovern-Dole indicator, Percent 
of parents in target communities who can name at least three benefits of primary 
education. The report noted that parent data could not be collected via a household 
survey, which would require approval by the National Institute of Statistics in Rwanda 
(typically a 3-month process). The MTE followed the approach of the baseline study to 
access parent knowledge, attitudes and practices and perspectives via focus groups 
with PTAs and interviews with PTA members. We also included a question in the head
teacher survey to ask their perception of parent knowledge of the benefits of primary 
education. 
261. See the evaluation matrix (Annex 9) for a detailed description of data sources 
and data collection tools and how these link to evaluation sub-questions. The 
evaluation questions are translated into data collection tools: topical outlines for 
interviews and focus groups, the EGRA tool, and a head teacher survey. The EGRA and 
head teacher survey tools were pre-tested in schools/students outside the sample 
before commencing data collection. The tools were adjusted based on reviewer 
comments and pre-test results before being finalized and deployed. 
262. Quantitative data (EGRA results and head teacher survey data) were collected 
on Android tablets using Open Data Kit (ODK) and Tangerine (RTI) data collection 
software. Final versions of the tools were coded for application onto the Android 
devices and translated into Kinyarwanda. At any time, the tool questions and 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 112 
|Page
responses were viewable on the tablets in Kinyarwanda or in English for quality and 
content review. 
263. The EGRA tool in Kinyarwanda was adapted from the baseline tool. As in the 
baseline, the tool was modified to ensure that the students had no prior exposure to 
the content. It adopted the same standard as in the baseline and was administered to 
third-graders to ensure comparability of results. The ET worked closely with World 
Vision’s early literacy team and WFP to identify experts to adapt the EGRA tool and 
train the team members who will administer it. In addition, the EGRA tool was 
augmented with the required questions needed to collect data for WASH indicators. 
264. TANGO analyst and survey specialist Lloyd Banwart collaborated with World 
Vision to lead the design, programming and training on tablets and software for the 
head teacher survey and EGRA quantitative tools. He travelled to Kigali in September 
for training and technical support and supervision: EGRA training (two days), EGRA 
testing (one day), and training on the head teacher survey tool (two days). National 
team members Justin Tuyiringire, Daria Mutetere and Juventine Mujawase 
participated in all training activities and administered the tools. 
265. Quality control included daily updates to TANGO by the field team. Data were 
uploaded to TANGO twice per week at minimum. TANGO reviewed the data and 
provided feedback on data quality and survey progress to the entire team, and had 
regular virtual consultations with the researchers to troubleshoot data issues while the 
team was still in the field, enabling prompt and efficient resolution of the issues.
266. TANGO concluded the qualitative field mission with a debrief that served as an 
informal validation of preliminary findings. 
B. Sample selection
267. Two kinds of quantitative data were collected from all students in the sample: 
(1) data from applying the EGRA, to assess any changes in reading levels, and (2) data 
to inform the indicator, Percent of students who can identify at least three key health 
and hygiene practices, to assess any changes in health and nutrition practices. 
268. We applied the formula below to arrive at the minimum sample size of students 
to assure statistical accuracy in comparisons across subsamples or survey rounds:
n = Deff* [(Zα + Zβ)2 * (P1*(1-P1) + P2*(1-P2) / (P2 - P1)2]*NR
where:
Variable
Assumed 
value Description
n =
Deff = 2 Design effect for complex sample design (assumed to be = 2)
Zα = 1.282 Z value associated with desired significance level for confidence (90%, one￾tailed)
Zβ = 0.842 Z value associated with desired significance level for power (80%, one￾tailed)
P1 = 50.0% estimated level of an indicator measured as a proportion at the time of the 
first survey or within a comparison group
P2 = 65.0%
expected level of the indicator either at a later survey round or different 
comparison group. (P2 - P1) is the magnitude of change or difference 
across subgroups that the sample is powered to detect (in this case, a 
difference of 30%, or 15 percentage points). 
NR 10.0% Non-response rate
WFP Rwanda McGovern-Dole Midterm Evaluation Report 113 
|Page
269. This formula computes a minimum required sample size of 210 students to 
enable statistically accurate comparisons for a single stratum. The McGovern-Dole 
indicators are program-wide, i.e., based on students in all four districts where HGSF 
is implemented. While it is possible to stratify the sample by grade and achieve 
statistical accuracy, this would require a sample of 210 students in each grade of 
interest (i.e., 210 students in first grade, 210 students in second grade, etc.). Likewise, 
stratifying by district would require samples of 210 students per district. Given that 
the McGovern-Dole indicators are whole-program and that significant additional 
resources would be required to make statistically robust comparisons across grades or 
districts, we used only two strata, stratifying by gender only. Such disaggregation is 
important to be able to make statistically valid comparisons of results for boys and 
girls. As statistically representative gender-disaggregated data are not available for the 
baseline, stratifying at midterm allows us to establish statistically representative data 
points for boys and girls and thus allow meaningful comparisons at endline. This 
requires a minimum sample size of 210 male students and 210 female students; we 
rounded up the sample size to 220 males and 220 females for logistical ease – where 
11 male and 11 female third-grade students are interviewed in 20 schools. 
270. Noting the CO’s interest in disaggregating results to enable exploring the 
possible reasons for any differences, we (1) disaggregated key results by district – with 
the understanding that these results are indicative, not statistically representative; and 
(2) used the preliminary disaggregated results to inform qualitative lines of inquiry, to 
better understand why some districts may be achieving different results than others. 
C. Student sampling frame 
271. All 220 male students and all 220 female students were administered the EGRA 
and the health/WASH questions added to the end of EGRA tool. The sampling frame 
is third-grade students. The process for defining the student sampling frame mirrored 
the systematic random sampling method used in the baseline. The first stage of the 
sample selected 20 schools from the 104 schools receiving support using probability￾proportional-to-size (PPS) systematic random sampling. The second-stage sample 
selection and sampling frame was developed and completed upon arrival in the school 
using the attendance register for that day. In each third-grade class one sampling 
frame was developed for girls and one for boys. Then 11 students were randomly 
selected from each sampling frame. We note that resources did not allow for collecting 
data from a control group/counterfactual for impact.
D. School selection and school-level survey
272. As discussed above, 220 boys and 220 girls were interviewed for two sampling 
strata, totalling 440 students. The number of schools visited for the school surveys 
(20) was a function of how many EGRAs and school surveys the ET estimated could 
be completed in a day. The EGRA/school survey team consisted of three national 
evaluators. We estimated that in one day, one interviewer can administer the EGRA to 
11 students and conduct the survey in one school; therefore, in one day, three 
interviewers in one school can interview 22 students and collaborate in administering 
the school survey. For the sample of 440 students, this equates to approximately 20 
days needed for interviews (440 students/20 schools = 20 days). These assumptions 
held for quantitative fieldwork, which was completed in 20 days. 
273. In the first stage of the two-stage clustered sample, we drew a sample of 20 
schools using PPS method. To define the school sampling frame, the ET used a CO￾provided list of all 104 program schools showing how many girls and boys in each 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 114 
|Page
school’s third-grade class. Before drawing the sample, the CO eliminated from the list 
any schools considered too difficult for the ET to access in the time available for 
fieldwork.120 While the margin of error corresponding to the EGRA and WASH data 
from the two student samples (girls and boys) remains 10 percent, the margin of error 
for the data from the multi-module school surveys is 17 percent. We find the latter 
margin of error reasonable given the survey budget. A lower margin of error would 
require a larger sample for the school surveys, and this is not achievable with the 
current budget.
274. Table 22 shows the schools visited and the number of student respondents, by 
sex.
Table 22: Schools visited and number of third-grade respondents at midterm
School Boys Girls Total
[REDACTED] 11 12 23
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
[REDACTED] 11 11 22
Total 220 221 441
120 The project determined “difficult-to-access schools” based on school proximity to a main road and general 
condition of secondary/dirt roads. The ET balanced its sample between accessible and difficult-to-access schools 
using a mix of random and purposive selection: the first selection was random, and this selection was refined 
using a range of criteria to ensure the sample was a cross-section of schools and a mix of remote and more 
accessible schools. The selection was discussed with WFP. The ET did visit a number of difficult-to-access schools
and only had to omit a few schools from the final sample due to localized road conditions that did not allow for 
safe travel.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 115 
|Page
276. The students in each selected school were administered the EGRA+WASH tool. 
In addition, the EGRA/school survey team administered a school-level quantitative 
survey to head teachers in sample schools to assess progress on school-level indicators. 
277. The draft Word version was sent to WFP/USDA for review and then returned 
to TANGO for finalization in Word and ODK. The EGRA/school survey team 
completed a pre-test of the tools with age-appropriate children after the EGRA 
workshop and before the start of fieldwork. Minor revisions were be made to integrate 
needed adjustments indicated by the pre-test to finalize the tool for fieldwork. This 
was done in close collaboration with World Vision’s Early Reading program team. 
278. The survey team administered the school survey to the head teachers upon their 
arrival at each school. In most cases the team interviewed the head teacher, who 
typically fulfils the role of headmaster/ school administrator. In the few cases where 
the head teacher was not available, the team interviewed a staff member in a 
comparable role with adequate institutional knowledge to respond to the survey 
questions (see details in Table 23). 
Table 23: Head teacher survey respondents
Position of School-level Respondent # Interviewed
Head teacher 15
Deputy head teacher 3
Teacher 1
School administrator 1
Total 20
Source: Midterm head teacher survey
279. While the other team administered the EGRA to students, the head teacher 
reviewed the tool and prepared the data necessary to complete the survey in the 
presence of the survey team. 
280. Preliminary results from EGRA and the school survey were provided to the 
qualitative team prior to the October fieldwork in order to help shape qualitative 
inquiry. The EGRA/school survey team conducted their work n two stages: Stage 1 in 
September, pausing in October to join the full ET for qualitative work, and resuming 
on Stage 2 after the end of the qualitative mission. 
E. Administration of EGRA 
281. The EGRA was administered by the three Rwandan ET members (the 
EGRA/school survey team). The CO provided a vehicle and a driver for the duration of 
their fieldwork.
282. As in the baseline, the MTE worked to minimize interruption of school learning 
activities by coordinating closely with school head teachers and teachers on student 
sampling, warm-up and assessment. The EGRA/school survey team consulted with 
WFP staff, literacy partner (World Vision), and school staff in advance to determine 
the timeliest window for conducting the assessment in each school.
F. Site selection for qualitative team field visits
283. The full ET (EGRA/school survey team + two international evaluators) 
collected primary qualitative data in the October mission. This constituted a “deep 
dive” inquiry at school, sector and district level, covering all districts and 
geographically focused on a subset of the 20 schools in the EGRA/school survey 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 116 
|Page
sample. As noted, data from Stage 1 EGRA/school survey was analysed by TANGO in 
tranches on a rolling basis, in order to inform and fine-tune both the sampling and the 
lines of inquiry of the qualitative work. 
284. The ET qualitative team visited 10 schools, purposively selected based on (1) 
schools where quantitative data analysis been completed; (2) trends and questions 
emerging from the initial quantitative data analysis (3) location, to collect input from 
schools in varied settings and contexts (4) school size and (5) logistical feasibility. 
G. Data analysis 
285. Quantitative analysis was conducted using STATA version 13.0 software. The 
analysis included descriptive analysis such as cross-tabulations, means, and 
significance tests to provide additional context to the underlying components of key 
indicators. This analysis was conducted per module of the head teacher survey tool. 
Baseline data was used to inform program performance, but limitations in the baseline 
data identified by WFP, and identified by the ET, restricted the level of analysis that 
can be completed using this data. The student and school samples drawn at midterm 
are self-weighted, and therefore no sampling weights were calculated or applied in the 
estimation of statistics. The data collection tools were applied using software 
programmed with internal controls to ensure data are consistent and reduce the 
likelihood of data entry errors. During the data analysis stage, all data were reviewed 
for outliers (initially flagged as three standard deviations or more for continuous 
variables). Discrete variables were reviewed for consistency. When potential data 
errors were identified, the analyst spoke with researchers to consult field notes 
associated with the interview(s). If no further information was available, then the data 
point was recoded to missing. 
286. The qualitative analysis applied a matrix approach to record, organize and 
analyse qualitative data and ensure all team members record information consistently 
and in a manner that directly responds to research questions. Capturing qualitative 
data in matrices enabled the identification of important patterns in responses and 
contextual information that may help to explain quantitative or secondary data. Data 
matrices also helped triangulate responses from FGDs, KIIs, in-depth interviews and 
other sources to determine whether information is reliable. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 117 |Page 
H. Limitations and risks
287. Table 24 replicates the table in the inception report for this evaluation, listing potential methodological limitations to the 
evaluation and proposed strategies to minimize their impact. It adds a column to report whether the anticipated limitations were 
encountered in the conduct of the evaluation, and steps taken.
Table 24: Anticipated methodological limitations and mitigation strategies; comments post-evaluation
Possible Limitation as 
Identified in Inception Phase Discussion Mitigation Strategy Comments Post-Evaluation
Quality and availability of 
secondary data, including the 
baseline data set
Secondary data sources/ documents 
typically vary in quality and reliability. 
Some of the data/ information the ET 
requests may not be available or may take a 
significant time investment to acquire. 
Some of the baseline data may vary in 
quality or structure and thus limit 
comparability in subsequent evaluations.
Assess the quality of secondary sources; 
prioritize analysis of research/data 
deemed high-quality; triangulate data 
across sources and provide 
opportunities for diverse stakeholders 
to validate findings. Consult with the 
CO and other stakeholders early in the 
evaluation process to identify data 
needs and gaps to enable primary data 
collection to address gaps and quality 
issues.
The presence of non-response data for 
reading comprehension in the baseline 
data reduced the comparability from 
endline to baseline. To address this 
constraint, baseline values were re￾estimated. 
Respondent bias Respondent bias is an inherent risk in any
evaluation. For example, respondents may 
wish to report in a way they think will 
favour them in terms of new or continued 
program benefits or positive recognition.
The ET will solicit perspectives from a 
range of stakeholders and take 
anticipated biases into account during 
analysis; the team will seek a balance of 
perspectives. The methodology will rely 
on a cross-section of information 
sources (e.g., stakeholder groups, 
beneficiaries) and using a mixed￾methods approach to ensure 
triangulation of information through a 
variety of means. 
Strategy implemented as planned. No 
issues encountered.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 118 |Page 
Budget and time constraints The scope of the evaluation and time in the 
field are determined by resources available. 
The scope must therefore be clearly defined 
and delimited.
Use the inception phase to determine, 
per consultation with WFP/USDA 
stakeholders, evaluation priorities and 
special points of interest (e.g., thematic, 
geographic, strategic). Ensure clear 
expectations of all parties on the main 
evaluation questions and the extent to 
which these can feasibly be investigated 
given existing data/ information and 
the time and resources available for 
collecting and analysing this and new 
information.
The evaluation budget and in-country 
time were adequate. The ET found the 
timeline between fieldwork and the 
submission of the draft report to be 
challenging, compounded by delays in 
receiving information due to office 
absences, and the receipt of needed 
data well into the drafting period 
Generalizability of findings As noted above, resources and time 
determine how much ground the ET can 
cover. This is not necessarily problematic, 
however when framing findings it is 
important to identify any limitations on 
generalizability of findings.
Specific limitations of the program will 
be noted in the evaluation report and 
presentations of findings. Again, 
agreement and understanding of all 
stakeholders regarding the scope of the 
evaluation is important to interpreting 
the findings, analysis, and 
recommendations in the appropriate 
context.
Strategy implemented as planned. No 
issues encountered.
Turnaround time for approval of 
evaluation tools 
Conformance with the proposed schedule is 
contingent on the timely review and 
approval of the EGRA/school survey tool 
and the topical outlines. The schedule is 
tight, so there is little room for delays from 
any party.
TANGO will ensure regular, frequent 
communication with the CO so that any 
adjustments by any party to anticipated 
timelines/due dates are shared 
immediately and the calendar can be 
adjusted to accommodate the 
requirements of all sides as best as 
possible. Certain activities must take 
place in August: the EGRA workshop 
(Aug 31), sampling, and drafting and 
programming the EGRA and school 
survey; TANGO and the CO must work 
concurrently to the inception report 
review to obtain the information 
needed for these tasks.
Strategy implemented as planned. No 
issues encountered. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 119 |Page 
Mobility The team may encounter mobility 
limitations such as those due to weather, 
terrain, or security issues. These constraints 
could affect the sampling and field schedule.
At this time, neither the rainy season 
nor any political or security issues are 
anticipated to constrain mobility. The 
Evaluation Team will communicate 
regularly with the CO regarding any 
changes that would affect the activity/ 
travel plan.
Strategy implemented as planned. No 
issues encountered.
288. With regard to the request from WFP Rwanda to discuss the EGRA margin of error: The confidence interval for all 
EGRA indicators is presented for the midterm analysis, and the ‘re-estimated’ baseline EGRA indicators. The confidence 
interval reflects the range that there is 95% confidence the actual population value will fall. The confidence interval across the 
EGRA indicators is relatively small. In other words, there is not a lot of variance across the third-grade student population(s), 
and the reader can ascertain that there is a high probability that the actual population values fall within these intervals. The 
limitations encountered and actions taken were:
• The ET requested the baseline data from WFP to undertake unpaired longitudinal statistical tests of EGRA indicators. Upon 
reviewing the raw EGRA baseline data, the midterm team was not able to replicate the EGRA values reported in the baseline 
report. This is true for all data presented in the tables below. For the current draft report, TANGO provided “re-estimated”
values based on the baseline data from WFP (see tables below). 
• A primary obstacle identified by the ET with the data was incomplete reading comprehension data as part of section 5 of the 
EGRA tool. In practice this is a series of 5 questions to test the students reading comprehension skills. The questions should be 
applied to all students who completed reading of the section 5 narrative. From a total of 402 third graders who began section 
5 of the baseline tool, there were a total of 243 responses to the first comprehension question, a total of 215 responses to the 
second comprehension question, a total of 51 responses the third comprehension question, a total of 12 responses to the fourth 
comprehension question, and a total of 5 responses to the fifth comprehension question. The increase in non-responses across 
section 5 comprehension questions makes the estimation of the indicator “Percent of students who, by the end of two grades of 
primary schooling, demonstrate that they can read and understand the meaning of grade level text” difficult. A common 
practice is to take students who have 3 out of 5 comprehension questions correct and count them toward comprehension. 
• For comparability purposes, TANGO based the ‘comprehension’ portion of this indicator on the correct response to 
comprehension questions 1 and 2. This assumption presents a substantial limitation to the comparability with the midterm 
data – however the ET feels this assumption is conservative (the baseline value may be higher than the “re-estimated” point 
estimate, but it is unlikely to be lower). The midterm team appreciates the difficulties of field research and realizes the 
identified differences may be a result of miscommunication (wrong data and/or explanation of data estimation techniques). 
As such the ET welcomes a conversation with the baseline researcher to further clarify the differences outlined.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 120 |Page 
Table 25: Re-estimated values for reading comprehension
Baseline Midterm
Indicator Reported Re-estimatedA 
Confidence
Lower 
Bound
Interval
Upper 
Bound n
Point 
Estimate
Confidence
Lower 
Bound
Interval
Upper 
Bound n
 
Percent of students who, by 
the end of two grades of 
primary schooling, 
demonstrate that they can 
read and understand the 
meaning of grade level textA
49.0 41.5 36.7 46.4 402 59.0*** 55.3 64.5 441
 Male students 42.0 39.8 29.8 46.6 201 56.8*** 50.2 63.4 220
 Female student 57.0 43.3 36.4 50.2 201 62.9*** 56.5 69.3 221
 
Difference between re-estimated baseline data and midterm data are statistically significant at the 10% (*), 5 %(**) or 1 %(***) level. A The midterm evaluation team was not able to replicate the point estimates reported in the baseline report with the data shared by WFP. See limitations for further 
description. 
Table 26: Re-estimated data for EGRA results
Indicator Baseline Midterm
Indicator Reported Re-estimatedA
Confidence
Lower 
Bound
Interval
Upper 
Bound
# of 
Tasks
Mean 
per 
Minute
Confidence
Lower 
Bound
Interval
Upper 
Bound
# of 
Tasks
Reading Letters/Sounds 11.0 16.4 15.6 17.3 100 49.0*** 46.6 51.5 95
Reading Syllables 25.0 45.6 42.5 48.7 100 55.8*** 53.3 58.3 100
Familiar Words 11.0 21.1 19.5 22.7 50 29.9*** 28.7 31.1 50
Unfamiliar words 7.0 14.8 13.5 16.0 50 33.7*** 32.7 34.6 50
Correct words in text/story 10.0 20.1 18.5 21.8 65 22.8** 21.3 24.4 56
n n/a 402 441
 
Difference between re-estimated baseline data and midterm data are statistically significant at the 10% (*), 5 %(**) or 1 %(***) level. A The midterm evaluation team was not able to replicate the point estimates reported in the baseline report with the data shared by WFP. See limitations for further 
description.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 121 
|Page
I. Ethical safeguards
289. All TANGO staff, consultants, and officers associated with this evaluation 
complied with TANGO’s policies and procedures, including TANGO’s Code of Ethics 
and Conduct. The ET also received refresher information as needed on ethical research 
safeguards and child and youth protection, based on UNICEF guidance and WFP 
policies and standards. 
290. The ET ensured appropriate ethical considerations were in place for all 
interviews by informing all interviewees of the purpose and duration of the interview, 
how they were identified to participate in the interview, informing interview 
participants of their rights, providing guarantees that specific interview findings will 
remain confidential and that all information provided will be used to assess the 
program, with no direct attribution to the interviewee. Finally, all interviewees were 
informed that they may choose not to participate, and gave their verbal consent to 
participate before the interview moved forward. The ET also obtained verbal consent
before taking any photographs. There were neither interview nor photo refusals. 
Enumerators introduced themselves to the head teacher, teachers and students, and 
obtained consent before starting activities at the school. They treated administrators, 
teachers, students and others respectfully and talked to students in a friendly and 
supportive manner.
291. TANGO has conformed to WFP and UNEG ethical standards in the conduct of 
this evaluation. We safeguarded and ensured the observation of ethical practices at all 
stages of the evaluation cycle. This has included ensuring informed consent, protecting 
privacy, confidentiality and anonymity of participants, ensuring cultural sensitivity, 
respecting the autonomy of participants, ensuring fair recruitment of participants 
(including women and socially excluded groups) and ensuring that the evaluation 
results in no harm to participants or their communities.
292. Table 27 describes ethical issues that were considered in the evaluation 
preparation/design, data collection, data analysis, reporting and dissemination, and 
how they were monitored and managed during the implementation of the evaluation. 
Table 27: Ethical issues and safeguards implemented
Ethical issue Safeguard/measure
1. Interviewing child/youth 
respondents (child protection 
issue) – for EGRA assessment 
and student interviews/ focus 
groups
1. Ensured that a teacher was present, to monitor the interview 
and ensure that the child felt safe and comfortable.
2. Reviewed World Vision child protection protocols and signed
statements of compliance.
3. Consulted with WFP program staff for any additional 
guidance.
2. Reliability and accuracy of 
school-level data 
2. During school visits, EGRA/school survey team reviewed
survey responses and spot-checked supporting documentation 
to validate data accuracy. Individual student performance on 
EGRA tests were not shared with teachers. 
3. Compliance with 
Evaluation Code of Conduct
3. As contractually required, all consultants read and agreed to 
conform to the WFP Code of Conduct and Ethical Standards 
and the UNEG Code of Conduct for Evaluation in the UN 
System.
4. Data protection 4. Tablets were password secure and data were uploaded as 
soon as possible to a remote secure server.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 122 
|Page
J. Quality assurance
293. A foundation of quality control is effective training on data collection 
techniques, methodologies, and the technology used to collect data. The entire team is 
highly experienced in all methodologies employed in this evaluation, so the training 
elements focused on reviewing specific data collection tools, and data entry and 
protocols for ODK surveys with the national researchers. The ET members, with the 
support of HQ-based TANGO analysts and survey specialists, were the primary 
developers of both the topical outlines and the school survey tools, and provided 
orientation and training to national researchers on all tools. The ET, including 
quantitative and qualitative researchers, was small, with team members engaged in 
both qualitative and quantitative activities. This increased the reliability of the data – 
ensuring tools were applied equally across different regions, settings, and schools. 
294. Sample size determination and sampling procedures were verified by Mark 
Langworthy, a TANGO partner and economist with extensive experience in 
quantitative surveys. A HQ-based TANGO analyst programmed the ODK survey tools 
into computer tablets and provided in-person training and real-time support to data 
collection and quality assurance for the EGRA/school survey. Supervision and 
coordination with the quantitative team were maintained by remote data quality 
control to identify any problems or anomalies, plus regular voice communication 
between the enumerators and the team leader. The team uploaded data to the TANGO 
server daily. TANGO reviewed the data and provided feedback on data quality and 
survey progress and highlighted specific issues twice per week. Data quality checks 
confirm strong internal validity for the quantitative tools. Equally, the quantitative 
results were largely in line with qualitative data – suggesting strong external validity. 
295. It is the duty of the team leader to ensure the protection of human subjects and 
their confidentiality, and training interviewers in obtaining verbal consent from 
participants. To ensure the security of the data, TANGO follows standard operating 
practices such as locked files, password secured databases and the handing over of all 
hard copies (where applicable) to TANGO. Participating individuals were assigned 
identification numbers, and this number is the only identifier to appear on any data 
collection tools such as surveys, written notes, transcripts and labels on audio tapes. A 
single list linking the names of participating individuals and their identification 
numbers is kept in a protected file that is only accessible to a small number of TANGO 
senior staff. 
296. Quality assurance was maintained in the reporting process by a TANGO Quality 
Assurance Manager who reviewed content to ensure that it met the criteria laid out in 
the TOR, that all comments by stakeholders were addressed, and that the report 
content fairly reflects the findings of the qualitative team and the quantitative data. 
Qualitative interviews with adults were recorded with the permission of the 
interviewees, and the recording was used to verify the accuracy of the interviewer 
notes.
297. Accountability to affected populations links to WFP’s commitments to include 
beneficiaries as key stakeholders in its work. As such, WFP is committed to ensuring 
gender equality and women’s empowerment (in the evaluation process, with 
participation and consultation in the evaluation by women, men, boys and girls from 
different groups. The final evaluation report’s findings, conclusions and 
recommendations reflect the ET’s GEEW analysis as appropriate.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 123 |Page 
Annex 14: Supplementary Tables
 Table 28: Student reading and comprehension (third-grade), baseline and midterm
 Baseline Midterm
Indicator Reported Re-estimateda 
Confidence
Lower 
Bound
Interval
Upper 
Bound n
Point 
Estimate
Confidence
Lower 
Bound
Interval
Upper 
Bound n
 
Percent of students who, by 
the end of two grades of 
primary schooling, 
demonstrate that they can 
read and understand the 
meaning of grade level texta
49.0 41.5 36.7 46.4 402 59.0*** 55.3 64.5 441
 Male students 42.0 39.8 29.8 46.6 201 56.8*** 50.2 63.4 220
 Female students 57.0 43.3 36.4 50.2 201 62.9*** 56.5 69.3 221
 
Differences between re-estimated baseline data and midterm data are statistically significant at the 10 percent (*), 5 percent (**) or 1 percent (***) level. a The ET was not able to replicate the point estimates reported in the baseline report with the data shared by WFP. See limitations section for further 
description. 
Table 29: Words correct per minute (detailed), third-grade, by district
Fluency
Words correct 
per minutes
% of students in wcpm range
Karongi Nyamagabe Nyaruguru Rutsiro
0 14.5 20.9 13.6 24.5
1 to 15 14.5 12.7 15.5 14.5
16 to 30 26.4 32.7 40.9 30.0
31 to 40 13.6 14.5 17.3 19.1
41 to 56 30.9 19.1 12.7 12.7
n 110 110 110 111
Difference between male and female students at midterm are statistically significant at the 10% 
(*), 5%(**) or 1%(***) level.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 124 |Page 
Table 30: Reading and comprehension at midterm, by district
Indicator
Point 
Estimate
Midterm Target n
Percent of students who, by the end of two grades of 
primary schooling, demonstrate that they can read 
and understand the meaning of grade-level text 51.5 69.0 441
Karongi district 41.8 n/a 110
Nyamagabe district 51.8 n/a 110
Nyaruguru district 58.2 n/a 110
Rutsiro district 54.1 n/a 111
Source: Midterm EGRA survey
 Table 31: Students' home support and time allocation at midterm
 Indicator Point Estimate
 Percent of students to get help reading from parents for 
homework/reading
83.2
Male students 87.3
Female students 78.7
 Percent of students to have enough time to study and do 
homework
88.0
Male students 89.1
Female students 86.9
n 441
 
No significant difference between male and female students at midterm is observed at a p<0.05 level or 
lower.
Source: Midterm EGRA survey
WFP Rwanda McGovern-Dole Midterm Evaluation Report 125 |Page 
Table 32: Student attentiveness, by grade, baseline and midterm
Point Estimate
Baseline Midterm Target n a
Percent of students in classrooms 
identified as attentive by their teachers 
as attentive
60.0 57.4 80.0
First grade 66.0 n/a 2116
Second grade 58.0 n/a 1804
Third grade 59.8 n/a 1844
Fourth grade 51.8 n/a 1667
Fifth grade 51.4 n/a 1540
Sixth grade 50.9 n/a 805
Weighted total 57.4 57.4 9776
a Reported number of total students in each grade 
Source: Midterm head teacher survey
Table 33: School administrator use and application of teaching techniques at 
midterm
% of Head
teachers Target # of Head
teachersa
School administrators and officials who 
received trainings or certifications as a 
result of USDA assistance 95.0 93 99
n 20 104 a Values extrapolated from sample data
Source: Midterm head teacher survey
WFP Rwanda McGovern-Dole Midterm Evaluation Report 126 |Page 
Table 34: Health and hygiene practices at midterm, by district
 Indicator
Point 
Estimate
Midterm Target n
Percent of students who can identify at least three key 
health and hygiene practices 
49.2 80 441
Karongi district 48.2 n/a 110
Nyamagabe district 47.3 n/a 110
Nyaruguru district 60.9 n/a 110
Rutsiro district 40.5 n/a 111
Source: Midterm EGRA survey
Table 35: Health and hygiene practice utilisation/application at midterm
Indicator Point Estimate
Midterm
Percent of students to regularly practice: 
Other personal hygiene (i.e., bathing) 91.4
Handwash with soap after toilet 42.0
Handwash before eating 36.5
Drink clean/treated water 15.2
Use of clean water 10.2
Brush teeth 8.4
Eating a balanced diet 5.0
Avoid open defecation 4.8
Eat well-cooked food 4.8
Wash fruits and vegetables 2.0
n 441
Source: Midterm EGRA survey
WFP Rwanda McGovern-Dole Midterm Evaluation Report 127 |Page 
 Table 36: Parents' knowledge of education benefit and storekeepers' knowledge of 
good practices, baseline and midterm
 Point Estimate
 Indicator Baseline Midterm Target n
 
Percent of parents in target communities 
who can name at least three benefits of 
primary education n/a 69.8a 90.0 20
 
Percent of cooks and storekeepers who 
can identify at least three safe food 
preparation and storage practices 92 85.0 95.0 20
 a A proxy indicator asking school head teachers to estimate this proportion
Source: Midterm head teacher survey
WFP Rwanda McGovern-Dole Midterm Evaluation Report 128 |Page 
Table 37: School meals distributed, actual versus planned, FY 2018
Indicator
1 Oct 2017 – 31 
Mar 2018 
Actual a
FY 2018 
1 Apr – 30 Sep
Actual
FY 2018 
Planned
% Achieved 
(actual/ 
planned)
Number of daily school meals 
provided to school age 
children as a result of USDA 
assistance
6,526,756 7,539,818 14,066,574 94%b
Number of daily school meals 
provided to school age 
children as a result of USDA 
assistance (females)
41,696 40,878 43,215 95% 
Number of daily school meals 
provided to school age 
children as a result of USDA 
assistance (males)
43,296 42,712 39,890 107%
Number of school-aged 
children receiving daily school 
meals as a result of USDA 
assistance (new)
13,665 615 11,136 6% 
Number of school-aged 
children receiving daily school 
meals as a result of USDA 
assistance (continuing)
71,327 82,975 71,970 115%
Number of school-aged 
children receiving daily school 
meals as a result of USDA 
assistance
84,992 83,590 83,106 101%
Source: Semi-annual reports for April and October FY 2018. a Values given for reference only. Per the CO, the 1 Apr – 30 Sep period is the more accurate figure to use in 
assessing performance against target. We have therefore used this figure to compute percent achievement, 
though see the exception in note (b). b We calculated this value differently from the other indicators in this table, as it appears logical that this indicator 
is intended to be cumulative, to include 1 Oct 2017 – 31 Mar 2018 and 1 Apr 2018 – 30 Sep 2018. Were we to 
apply the method used for the other indicators (see note (a)), the total would be 14,962,000 and achievement 50 
percent.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 129 |Page 
Table 38: Commodity distributions, expected v. actual
Mid-year reporting 
timeline
Vegetable oil distributions: 
Southern provinces
CSB+ distributions: 
Western provinces
Expected 
(NMT)
Actual 
(NMT)
% 
Achieved
Expected 
(NMT)
Actual 
(NMT)
% 
Achieved
FY18 Q3(a) 55.19 51.73 94% 706.95 444.46 63%
FY18 Q1-2(b) 43.12 66.52 154%1 384.48 326.22 85%2
FY 17 Q3-4(b) 42.00 41.25 98% 349.44 326.22 93%
FY 17 Q1-2(b) 27.86 18.29 66% 181.88 135.21 74%
FY 16 Q3-4(b) 36.53 28.54 78%3 299.75 148.10 49%3
FY 16 Q1-2 (from 21 Dec)(b) Not started N/A Not started N/A
Total: 204.70 206.33 101% 1922.50 1380.21 71%
Source (a): Rwanda(MGD15)_Semi Annual Report April 1 2018_ September 30 2018 final 
Source (b): FY 2016-2018 Indicator Tracking Excel Spreadsheets “Commodity Direct Distribution” 1 Higher enrolment than anticipated; balances (14MT) for academic term starting April at schools to ensure no 
pipeline break.
2 Slower rate of consumption owing to structural challenges like access to schools during the rainy season, resulting in 
delivery delays. WFP is working with government to resolve.
3 Food had already been distributed for Q2 in southern provinces and in Q3 for western and southern provinces for 
the school year.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 130
|Page
Annex 15: Bibliography
Abadzi, H. 2012: Developing Cross-Language Metrics for Reading Fluency 
Measurement: Some Issues and Options. Global Partnership for Education 
Working Paper Series on Learning No. 6. 
http://documents.worldbank.org/curated/en/611321468338475370/Developing￾cross-language-metrics-for-reading-fluency-measurement-some-issues-and￾options
Government of Rwanda. 2013. Economic Development and Poverty Reduction 
Strategy 2013 – 2018.
Government of Rwanda. 2004. Rwanda Vision 2020.
Ipsos, Inc. 2016. Baseline Study: Home Grown School Feeding Program 2016-2020. 
July. Report authors P. Mukiri and A. Kaburu.
MasterCard. 2017. The School Feeding Investment Case: Cost-Benefit Analysis in 
Rwanda. Report prepared by MasterCard for World Food Programme.
MINAGRI, NISR, WFP. 2015. Rwanda Comprehensive Food Security and 
Vulnerability Analysis. March 2016.
National Institute of Statistics Rwanda. 2018. Vision 2020 Umurenge Program 
(VUP) - Baseline Survey.
National Institute of Statistics Rwanda. 2015. Rwanda Poverty Profile Report 
2013/14--Results of Integrated Household Living Conditions Survey (EICV4).
Republic of Rwanda, Ministry of Education. 2016 Education Statistical Yearbook.
Republic of Rwanda, Ministry of Education. 2014. National School Health Policy, 
Kigali 2014.Republic of Rwanda. National Cooperative Policy in Rwanda toward
Private Cooperative Enterprises and Business Entities for Socio-Economic 
Transformation. Revised version [1]. Kigali, January 15, 2018.
Republic of Rwanda, Ministry of Education. 2013. Education Sector Strategic Plan, 
2013/14 – 2017/18. 
Rwanda Education Board/MINEDUC. 2015. Competence-Based Curriculum: 
Summary of Curriculum Framework Pre-primary to Upper Secondary 2015.RTI 
International. 2016. EGRA Toolkit, Second Edition. 
https://www.globalreadingnetwork.net/resources/early-grade-reading￾assessment-egra-toolkit-second-edition
Sachs, J., et al. 2018. SDG Index and Dashboards Report 2018.
SDG Center for Africa and Sustainable Development Solutions Network. 2018. Africa 
SDG Index and Dashboards Report. July.
United Nations Development Programme. 2016. Human Development Report 2016: 
Human Development for Everyone.
WFP Rwanda McGovern-Dole Midterm Evaluation Report 131
|Page
UNHCR. ND. Operations Plan: Rwanda 2018. 
United Nations High Commissioner for Refugees. 2018. Operational Update: 
Rwanda, August 2018.
United Nations Rwanda. 2018. UNDAP 2018-2023 for Rwanda. Signed 31 July 2018.
United States Agency for International Development. 2018. Rwanda: Nutrition 
Profile.
United States Agency for International Development. 2015. Early Grade Reading 
Barometer. https://earlygradereadingbarometer.org/uganda/results. 
United States Department of Agriculture, Foreign Agricultural Service. NDa. Foreign 
Agricultural Service Strategic Plan (2019-2022). 
United States Department of Agriculture, Foreign Agricultural Service. NDb. FY 2019 
Food Assistance Proposal Guidance and Request for Applications.
World Bank Group. 2018. Rwanda Overview.
World Food Programme. 2018a. WFP Rwanda Country Brief. March 2018.
World Food Programme. 2018b. McGovern-Dole Programme Semi-Annual Report 
Narrative, October 1, 2017 – March 31, 2018.
World Food Programme. 2018c. Country Programme Standard Project Report 2018 
(Rwanda).
World Food Programme. 2017a. WFP Strategic Plan (2017-2021).
World Food Programme. 2017b. Country Programme – Rwanda – (2013-2018). 
Standard Project Report 2017.
World Food Programme. 2017c. Operation Evaluation. Rwanda Common Country 
Programme 200539, Mid-Term Evaluation (2013-2016). Evaluation Report.
Prepared by IRAM. 
World Food Programme. 2017d. WFP Rwanda Country Brief. November 2017.
World Food Programme. 2017e. Country Programme Standard Project Report 2017 
(Rwanda). 
World Food Programme. 2017f. Gender Action Plan narrative, Rwanda Country 
Office, January 2017.
World Food Programme. 2016a. Country Programme – Rwanda – (2013-2018). 
Standard Project Report 2016.
World Food Programme. 2016b. Food and Safety Net Assistance to Refugee Camp 
Residents and Returning Rwandan Refugees. Standard Project Report 2016.
World Food Programme. 2015. WFP Rwanda FY 2015 McGovern-Dole Proposal. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 132
|Page
World Food Programme. 2014. Purchase for Progress - P4P Rwanda.
World Food Programme. 2013. Revised School Feeding Policy Promoting Innovation 
to Achieve National Ownership. 
World Vision. 2016. Rwanda Annual Report 2016. 
WFP Rwanda McGovern-Dole Midterm Evaluation Report 133
|Page
List of Acronyms 
ADRA Adventist Development and Relief Association
CBEHPP Community-based Environmental Health Promotion Program
CCP Common Country Programme
CFSVA Comprehensive Food Security and Vulnerability Analysis
CHAI Clinton Health Access Initiative
CHW Community Health Worker
CO Country Office
CP4P Common P4P
CSP Country Strategic Plan
CSR Country Strategic Review of Food Security and Nutrition
CVA Citizen Voice and Action
DDP District Development Plan
DEO District Education Officer
DEQAS Decentralized Evaluation Quality Assurance System
DFID Department for International Development
EB Executive Board
EC Evaluation Committee
EDC Education Development Center
EDPRS 2 Economic Development and Poverty Reduction Strategy
EGRA Early Grade Reading Assessment
ERG Evaluation Reference Group
ET Evaluation Team
ESSP Education Sector Strategic Plan
FY Fiscal Year
GII Gender Inequality Index
HGSF Home Grown School Feeding
HQ Headquarters
LODA Local Administrative Entities Development Agency
MDG Millennium Development Goal
MGD McGovern Dole
MIGEPROF Ministry of Gender and Family Promotion
MINAGRI Ministry of Agriculture and Animal Resources
MINALOC Ministry of Local Affairs
MINEDUC Ministry of Education
MININFRA Ministry of Infrastructure
MINISANTE Ministry of Health
MT Metric ton
MTE Midterm evaluation
NMT Net metric ton
NST National Strategy for Transformation
ODK Open Data Kit
OEV (WFP) Office of Evaluation
PEPFAR President’s Emergency Plan for AIDS Relief
P4P Purchase for Progress
PPS Probability-proportional-to-size
PRRO Protracted Relief and Recovery Operation
PTA Parent-Teacher Association
WFP Rwanda McGovern-Dole Midterm Evaluation Report 134
|Page
RB Regional Bureau
RBC Rwanda Biomedical Centre
REB Rwanda Education Board
RTI Research Triangle Institute
SABER Systems Approach for Better Education Results
SDG Sustainable Development Goal
SEO Sector Education Officer
SMC School Management Committee
SO Strategic Objective
SPR Standard Project Report
TOR Terms of Reference
UN CCA UN Common Country Analysis
UNDAP United Nations Development Assistance Programme
UNDSS UN Department of Safety and Security
USAID United States Agency for International Development
USDA United States Department of Agriculture
VUP Vision 2020 Umurenge Programme
WASAC Water and Sanitation Corporation
WASH Water, Sanitation and Hygiene
WCPM Words correct per minute
WFP World Food Programme
World Food Programme Rwanda
http://www1.wfp.org/countries/rwanda
[Place, Month
b6/download/
https://docs.wfp.org/api/documents/cdcbe4be57914c65b0dbcbd201196a
and Year, Report number]