Decentralized Evaluation
Final Evaluation of WFP’S USDA FY 13 McGovern-Dole International Food for 
Education and Child Nutrition Programme’s Support in Afar and Somali Regions 
in Ethiopia 2013–2017
Evaluation Report - Final
15 June 2018
WFP Ethiopia
Evaluation Manager: Tewelde Birhan Girma
Prepared by
Muriel Visser, Team Leader
Denis Alder, Senior Evaluator
Rita Bhatia, Senior Evaluator
Gadissa Bultosa, Senior Evaluator
Doe-e Berhanu, Research Coordinator & Evaluator
Christine Fenning, Research Analyst
Decentralized evaluation for evidence-based decision making
Agreement:FFE-663-2013/026-00 
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
i
ACKNOWLEDGEMENTS AND DISCLAIMER
The Evaluation Team expresses its gratitude to the Ethiopia WFP Country Office Staff and
the Staff of the Ministry of Education for their support to the inception and field missions.
We would also like to gratefully acknowledge the work of the survey team comprised of: Mr.
Lemi Sime Demie, Mr. Kassahun Eyayu Werku, Ms. Hawa Alo Nur, Ms. Fatuma
Mohammed Humed, Mr. Ali Ebrahim Dawed, Mr. Abdurahman Seid Mohammed, Mr.
Daniel Shiferaw Asfaw, Mr. Sendere Mohammed Hussien, Ms. Tigist Eshetu Kasaye, Mr.
Daniel Kifle Wolde, Ms. Sintayehu Anberber Terefe, Mr. Shumet Korsa Tefa, Mr. Tesfaye
Gedu Moges, Mr. Nur Ma’alin Husen, Ms. Liyu Chala Bosa, Ms. Sintayehu Alene Alemu, Mr.
Juneydi Ali Mohammed, and Mr. Kedir Ahmed Ali who carried out the data collection in
Afar and Somali regions.
The opinions expressed in this report are those of the Evaluation Team, and do not
necessarily reflect those of the World Food Programme or the United States Department of
Agriculture. Responsibility for the opinions expressed in this report rests solely with the
authors. Publication of this document does not imply endorsement by WFP of the opinions
expressed.
The designations employed and the presentation of material in maps do not imply the
expression of any opinion whatsoever on the part of WFP concerning the legal or
constitutional status of any country, territory or sea area, or concerning the delimitation of
frontiers. 
DISCLAIMER: This publication was produced at the request of the United States Department of 
Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views 
expressed in this publication do not necessarily reflect the views of the United States Department of 
Agriculture or the United States Government.
Accessibility Note: An accessible version of this document can be made available by contacting 
fas.monitoring.evaluation@usda.gov 
MGD school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
ii
TABLE OF CONTENTS 
Executive Summary ..................................................................................................................... v
1. Introduction ................................................................................................................ 1
1.1 Overview of the evaluation subject......................................................................................... 2
1.2 Context .................................................................................................................................... 4
1.3 Evaluation methodology and limitations ............................................................................... 6
2. Evaluation findings .................................................................................................. 12
KQ 1 – How appropriate was the programme?............................................................................ 12
KQ 2 – What were the results of the programme?........................................................................15
KQ 3 – What factors affected the results?....................................................................................28
KQ 4 – To what extent are the project results sustainable? ........................................................ 33
3. Conclusions and recommendations .........................................................................36
3.1 Overall assessment/conclusion ............................................................................................ 36
3.2 Recommendations ................................................................................................................40
ANNEXES .................................................................................................................................. 43
Annex A Terms of Reference................................................................................................... 44
Annex B Methodology.............................................................................................................72
Annex C Theory of Change......................................................................................................85
Annex D Full Evaluation Matrix .............................................................................................90
Annex E Organisation of the evaluation............................................................................... 102
Annex F Team Roles and Responsibilities ........................................................................... 104
Annex G Detailed evaluation timeline .................................................................................. 108
Annex H Detailed field work schedule .................................................................................. 110
Annex I List of persons met during the Inception Mission.................................................. 112
Annex J People interviewed (observation and main field visits).......................................... 113
Annex K The McGovern-Dole Programme in Ethiopia......................................................... 116
Annex L Other school feeding programmes in Ethiopia...................................................... 140
Annex M Map ......................................................................................................................... 144
Annex N Gender .................................................................................................................... 145
Annex O Equity ...................................................................................................................... 149
Annex P School Health and Nutrition ................................................................................... 151
Annex Q Nutrition ................................................................................................................. 154
Annex R Case Study 1: The GoE Emergency School Feeding Programme .......................... 159
Annex S Case Study 2: Girls’ Initiative Programme – Take-Home Ration ......................... 165
Annex T The Survey – Data Collection Tools and Methods.................................................. 171
Annex U Survey Instrument ................................................................................................... 177
Annex V Stakeholder analysis............................................................................................... 185
Annex W Approach to interviews and school level observations .......................................... 189
Annex X Results of EMIS data analysis ............................................................................... 200
Annex Y Field Survey Results .............................................................................................. 204
Annex Z Assessment of progress against SABER framework .............................................. 217
Bibliography............................................................................................................................. 219
List of Acronyms ......................................................................................................................227
Tables
McGovern-Dole International FFE and Child Nutrition Programme Beneficiaries in Afar and Somali
regions ..................................................................................................................................................... 3
Budgetary situation of the programme ................................................................................................... 4
 ToC Assumptions ..................................................................................................................................... 8
 Planned and achieved sample distribution ............................................................................................. 9
 Latrine availability in Afar and Somali schools..................................................................................... 23
Enrolment for Boys and Girls and Gender Parity Index....................................................................... 26
 Grade completion by gender for sampled FFE and non-FFE schools .................................................. 28
Statistical significance of performance differences associated with gender and FFE status ............... 28
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
iii
 Evaluation Questions .............................................................................................................................76
Evaluation Questions (showing additions)........................................................................................... 90
Evaluation Matrix.................................................................................................................................. 92
Mapping of TOR questions onto the evaluation matrix ....................................................................... 99
 Evaluation Timeline .............................................................................................................................102
 Support and Information Requirements for the Evaluation...............................................................103
 Evaluation team roles and responsibilities ..........................................................................................106
McGovern-Dole planned school feeding activities .............................................................................. 118
 Planned food requirements for the McGovern-Dole programme ....................................................... 119
Planned beneficiaries for the McGovern-Dole programme................................................................. 119
 Outputs: targets and results .................................................................................................................124
Outcomes: targets and results..............................................................................................................132
Beneficiaries: targets and results .........................................................................................................136
Nutrition Composition of School Meal per day per Child ................................................................... 156
 Strengths, weaknesses and challenges................................................................................................. 165
 FFE beneficiaries in Afar and Somali................................................................................................... 166
 Grade completion by gender for sampled FFE and non-FFE schools ................................................. 167
List of schools to be sampled................................................................................................................ 174
 Detailed stakeholder mapping .............................................................................................................185
Ethiopian and Gregorian years ...........................................................................................................200
 Data files contributing to EMIS analysis (Afar and Somali Regions) .................................................201
 Enrolment for Boys and Girls and Gender Parity Index, from EMIS data ........................................ 203
 Grade repeats (non-completion) for Boys and Girls, from EMIS data............................................... 203
 Planned and achieved sample distribution ......................................................................................... 204
 Number of sample schools, by enrolment class.................................................................................. 205
 Grade completion by gender for sampled FFE and non-FFE schools ................................................ 206
 Statistical significance of performance differences associated with gender and FFE status ............. 206
 Child Questionnaire Analysis .............................................................................................................. 207
 Analysis of Questions on School Facilities...........................................................................................210
 For Schools in FFE, food supplies, THR and impact on attendance and attentiveness......................214
 School meals information for non-FFE schools ..................................................................................215
 Staff attending short courses or workshops in the sample schools .....................................................216
 Policy Goals and Status ........................................................................................................................217
Figures
Figure 1 Theory of Change...................................................................................................................................... 7
Figure 2 Theory of Change....................................................................................................................................73
Figure 3 Project-level results framework............................................................................................................. 86
Figure 4 Reconstructed ToC diagram.................................................................................................................. 88
Figure 5 Project level results framework............................................................................................................139
Figure 6 GoE funding for the ESFP (2015-2018) in USD millions .................................................................... 161
Figure 7 Distribution of In-Programme and Out-of-Programme Primary Schools in Afar and Somali, 2013 172
Figure 8 Map of selected IP (●) and OP (●) sample school pairs ............................................................................ 173
Figure 9 Distribution of sample schools by enrolment size .............................................................................. 204
Figure 10 Distribution of sample schools by enrolment size .............................................................................. 205
Figure 11 Child interviews in process..................................................................................................................208
Boxes
Box 1 Overview of the 2017 Food Diversion in Somali Region.......................................................................... 20
Box 2 The effect of THR on girls’ education and welfare..........................................................................................26
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
iv
Box 3 Evaluation’s approach to Triangulation and Complementarity ...................................................................75
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
v
Box 4 Evidence of the benefits of HGSF in Ethiopia ........................................................................................142
Box 5 Composition of ESFP meals per day.......................................................................................................160
Box 6 Examples of school name variations...................................................................................................... 202
Maps
Map 1 Food for Education coverage in Somali and Afar region .......................................................................144
Map 2 WFP Assisted Food for Education and Girls’ Initiative Programme woredas, 2011 ...............................158
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
vi
Introduction
Executive Summary
1. This is the end-of-project evaluation of the McGovern-Dole Food for Education and
Child Nutrition Programme (FFE - 663-2013/026-00; project number 200253) of the United
States Department of Agriculture (USDA), under which the World Food Programme (WFP)
supports school feeding (SF) in two arid Regions of Ethiopia. The programme’s value is
USD40,707,805 for a four-year period which started in January 2014. Commissioned by the
WFP Country Office for Ethiopia, this is the first formal evaluation and covers the period
from 2013 to December 2017.
2. The evaluation’s objective was to assess to what extent the USDA McGovern-Dole￾funded activities contributed to the anticipated outcomes of improved attendance and
attentiveness, and improved equity in primary education access, in the Afar and Somali
regions. The evaluation was conducted for the purpose of accountability and learning. The
primary users of this report are stakeholders directly involved in implementing the
programme. These include WFP Ethiopia and its main implementing partner, Ethiopia’s
federal Ministry of Education (MoE), the Local Education Development Partner Group, and
USDA as the donor.
3. The programme involves daily school meals provided to 289,000 primary school
children, and capacity building aimed at supporting increased dietary and health practices
and improved student attendance. The capacity building component also includes activities
to support the capacity of the Government to develop a National School Feeding Programme
(NSFP). The programme provides children with one hot meal per day, corresponding to 647
calories per day. In addition, a monthly Take-Home Ration (THR) of two litres of vegetable
oil is provided to girl pupils as an incentive to attend school. This programme is part of a
larger set of actions supporting school feeding (SF) and promoting food security in Ethiopia.
4. Ethiopia has seen strong economic growth in the past two decades. However, food
insecurity still remains high, with an estimated 8.5 million food-insecure Ethiopians
currently requiring relief assistance, following the third consecutive poor/failed rains in the
southern drought belt. About 38 percent of Ethiopian children under five are stunted, 10
percent are wasted, and 24 percent are underweight. Rural populations are highly exposed to
undernutrition and micronutrient deficiencies due to poverty and food insecurity. The whole
of Afar and Somali Regions are classified as ‘priority one’ or ‘priority two’ national hotspots
requiring urgent humanitarian response, with approximately one third of the population in
Somali Region and just over one quarter of the Afar population requiring food assistance.
5. While Ethiopia has made strong progress towards achieving universal primary
education, high dropout rates, especially in pastoralist and emerging regions, are poverty￾related and reflect the fact that both boys and girls work or take care of cattle to support the
family. Learning outcomes are not keeping pace and there are regional and gender disparities
in basic education performance.
Methodology
6. The evaluation used a quasi-experimental design where WFP intervention schools
were compared with non-intervention schools in Afar and Somali regions through a survey
which covered a total of 1,080 students in 90 randomly selected schools as planned. This was
complemented by an analysis of Education Management Information System (EMIS)
Statistics of the Ministry of Education comparing FFE schools with non-FFE schools, a desk
review and analysis of documents and data, semi-structured interviews, focus group
discussions, and observations during several field visits to both regions. Primary data
collection was conducted in February 2018. The evaluation design was informed by a theory
of change and accompanying assumptions. Evaluation questions covered criteria of
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
vii
relevance, effectiveness (and impact), efficiency, sustainability and coherence. Gender
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
viii
Equality and the Empowerment of Women (GEEW) was mainstreamed in the evaluation
design and data collection methods. The evaluation also included two mini-case studies, one
of the separate Emergency School Feeding Programme (ESFP) introduced by the
Government of Ethiopia (GoE) and one of the Take-Home Rations – Girls Initiative
Programme within the McGovern-Dole operation.
7. Significant limitations of the evaluation included the absence of any baseline study or
data, limitations in the EMIS data (received only for 2016-2017), challenges in identification
of school locations due to faulty lists, and insecurity issues which interfered with data
collection. The absence of baseline data meant that a double-difference analysis between
baseline and endline was not possible. To ensure that sampling met criteria, overall
approximately 20 percent of schools were replaced from a pre-established reserve list as
necessary, and the evaluation team are confident that the survey findings are robust.
Key Findings
8. Appropriateness: The interventions have appropriately focused on areas which are
acknowledged to be among the poorest in the country, where people live in a context marked
by poverty and hunger and face significant challenges including prolonged drought; and
issues like girls’ education remain pertinent. The relevance and importance of the WFP￾supported SFP to children’s needs were confirmed and appreciated by beneficiaries and other
stakeholders at various levels.
9. The McGovern-Dole FFE programme is generally well aligned with national policies
and strategies, including in the areas of education and social protection, except with the
Government’s priority of home-grown school feeding, as the food is currently imported from
the United States. The McGovern-Dole FFE Programme is coherent with WFP’s overarching
policies on SF and nutrition. While it is also broadly in line with WFP’s gender policy, only a
basic gender analysis was conducted prior to project implementation, though lessons from a
different project were taken into consideration. The programme, however, is currently not
actively collaborating with other United Nations (UN) or Non-Governmental Organisation
(NGO) programmes in the project areas.
10. Generally, the programme’s design is appropriate. The intervention includes
important aspects which are essential contributors to its success, namely Water and
Sanitation activities and strengthening of Parent Teacher Associations. However, it is
questionable whether school gardens in arid zones are a realistic endeavour.
11. Results: Quantitative and qualitative data consistently underscore significant and
important output, outcome and impact level results and provide a convincing case for the
importance of school feeding for areas that are severely affected by food insecurity. The
evidence demonstrates that school feeding, supplemented by specific interventions targeted
at girl students, improves inclusiveness, participation and achievements in education.
Specifically, the statistical analysis comparing schools with school feeding with those without
shows that:
• In both Afar and Somali regions, enhanced school enrolment and a more favourable
Gender Parity Index is associated with FFE. GPI was 88 percent in Afar and
90 percent in Somali for FFE schools, but only 78 percent in Afar and 69 percent in
Somali for non-FFE schools.
• Grade repetition rates are consistently about 10 percent lower in FFE schools than in
non-FFE schools, with girls being slightly more advantaged than boys. FFE schools’
pass rates were 10.7 percent for boys and 12.3 percent for girls better than non-FEE
schools in Afar; in Somali, pass rates were 9.8 percent for boys and 12.1 percent for
girls better.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
ix
• The child survey showed improved indicators for FFE schools across most factors
including attendance, meal frequency, food consumption scores and attentiveness.
They showed access to and uptake of take-home rations close to 100 percent for girls,
with a high proportion (88-95 percent, Afar-Somali) using the oil at home directly for
cooking.
• The survey also showed that overall facilities tended to be better in FFE than in non￾FFE schools; they indicated little external support for school meals outside of the FFE.
Some 66 percent in Afar and 87 percent in Somali of FFE schools had kitchens, whilst
for non-FFE schools, none were reported in the Afar sample, and only 7 percent in
Somali. Correspondingly, few children reported eating at school in the non-FFE
schools. Latrine facilities also tended to be better in FFE schools. Of the FFE schools,
42 percent in Afar and 23 percent in Somali reported receiving one or more
infrastructure improvements in 2016-17, whilst for non-FFE schools, the ratios were
17 percent in Afar and 13 percent in Somali.
12. Factors affecting results: The school selection process was participatory but
because of government desire to have an equitable distribution it resulted in a selection of
schools that are geographically spread out. This has complicated support to the schools. It
makes frequent monitoring costly and also reduces efficiency in conducting school visits.
13. While WFP generally has very clear guidelines and an efficient procurement system in
place, issues with the timeliness of delivery were found due to regular pipeline breaks, and
the first semester delivery has been consistently late. There were some complaints about the
quality of the food (too close to its ‘best before’ date and often damaged oil tins causing the
oil to go rancid in storage).
14. The GoE ownership of and commitment to SF, at both federal and regional levels, have
increased, and various high-level persons in government became champions for SF. An SF
policy is yet to be put in place, but progress has been made in terms of committing to an SF
strategy in line with GoE’s international, regional and national commitments (such as
inclusive right to food, health and education, expansion of local school meal programmes
using home grown food where possible and effective governance and sustainable financing of
SFPs). The Emergency School Feeding Programme (ESFP), which is largely implemented
with GoE’s own funds, is also a good complementary programme to WFP’s SFP and builds on
many of the lessons learned from the previous and current McGovern-Dole programmes.
15. The external operating environment has at times made the implementation of the
McGovern-Dole FFE interventions difficult or caused disruption, e.g. schools being
geographically spread out, severe droughts, school closures, or conflict-induced displacement
in 2017. Extreme poverty and the specific conditions in the regions made it difficult for the
programme to mobilize community resources that were assumed to be available for the
implementation of an integrated package of support. In addition, high turnover of
government staff at all levels has presented a challenge to the efficiency and reduced the
effectiveness of staff to properly manage the programme. Critical gaps appear when those
trained leave. Staff turnover at senior federal government level has made it challenging for
WFP to make progress on strengthening the policy environment on school feeding, and has
undermined its high-level advocacy efforts on the issue of transitioning to a national school
feeding programme. Due to limited resources, there is weak monitoring and reporting, both
from the side of WFP and of Regional Bureaus of Education.
16. Sustainability: The evidence reviewed strongly suggests that with the end of the
McGovern-Dole FFE Programme in Afar and Somali regions the many positive effects on
households – which live in extremely precarious conditions – cannot be sustained as it would
effectively require households to have the means to fill the gap (financially) that is being left
through the loss of income which the SF and THR represent. The Government is not yet
ready to finance and efficiently run a national school feeding programme, though there is
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
x
clear evidence of a growing interest and commitment to establish a sustainable national SFP
in Ethiopia.
Overall Conclusions
17. The evaluation’s overall assessment is that the direct SF component of the McGovern￾Dole-funded SF intervention has many very positive features, including a demonstrable
impact on attendance, enrolment, and completion, particularly by girls, in both Afar and
Somali regions. However, it also raises some concerns. On the positive side, the core activity
of providing nutritious daily school meals appears to be generally well-designed, well￾implemented and effective. Food consumption scores are higher, perceived attentiveness
better and absenteeism lower in FFE schools. Ethiopia’s experience of emergency SF suggests
that key elements in terms of systems and capacity are in place. WFP’s work on SF is highly
credited with strengthening government capacity for planning and implementation of school
feeding interventions. Deployment of technical assistance is considered very relevant and
appropriate at both regional and federal levels. WFP through the McGovern-Dole funds has
provided financial and technical support to the regional education management system and
to the NSFP. There is evidence of learning between different models of SF as the McGovern￾Dole model is mirrored in both the Home Grown School Feeding Programme (HGSFP) and
the ESFP.
18. However, key challenges remain to be addressed. These include the nature of funding,
which to date on the GoE side has come from additional allocations on an emergency basis
and as such is not sustainable. Long lead times in terms of procurement, logistics and
organization because of various constraints related to internal capacity and the bureaucracy
(of the MoE) are a major challenge and jeopardise the purpose of addressing emergency
needs. Furthermore, the programme needs a more effective and reliable M&E and reporting
system. While changes in attitudes by communities to gender are in evidence at community
level, there are concerns whether these benefits will be sustained beyond the lifetime of the
intervention given the extreme poverty and stress on populations due to drought and conflict
which put children, and girls in particular, at risk. The review of the underlying theory of
change and its accompanying assumptions confirms this summary conclusion, which is
reflected in the practical recommendations which follow.
Recommendations
Recommendation Timing Responsibility
Strategic issues
R1 Prioritize fundraising for the continuation of school feeding
and a THR for girls to the schools that were covered under the
McGovern-Dole FFE programme in Afar and Somali regions as a
matter of absolute priority.
by the new school
year (September
2018)
WFP CO with
support from the
WFP RB
R2 Prioritize finalization of the national strategy and use it as the
basis to develop an implementation guideline with different types
of school feeding scenario, including a separate guideline for the
pastoralist context.
during 2018 WFP CO with
support from a
consultant (in
collaboration with
MoE)
R3 Develop and implement an adequately funded advocacy
strategy that builds on the key findings of this evaluation and
previous strategic work to scale up political and financial
commitment to SF in Ethiopia. This could include developing short
learning papers based on the findings of this evaluation.
by December 2018 WFP CO with
support from the
WFP RB
R4 Advocate with the GoE to ensure that the government policies
and strategy include an incentive for girls’ education in food￾insecure /pastoral societies using funds from the Productive Safety
Net Programme (PSNP).
in the course of
2018 and 2019
WFP CO
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
xi
Recommendation Timing Responsibility
Design of future WFP SF interventions
R5 Conduct an independent assessment of needs of vulnerable
populations for the next McGovern-Dole FFE programme and use
the findings of this assessment to identify suitable design options
for school meals in pastoralist areas.
in the start-up
phase of the next
McGovern-Dole SF
programme (and at
the latest by mid￾2019)
WFP CO with
external
consultancy
support
R6 Include continued investment in government technical
capacity for the logistical management of school feeding at federal
and regional levels in all future SF support by WFP in Ethiopia
in time for the new
SF programme
WFP CO
R7 Ensure future school feeding interventions include multi-year
evaluations in the design of the programme with baseline, midline,
follow-up and endline surveys, and recommendations for
adjustments as appropriate during implementation.
In time for the new
SF programme
WFP CO
R8 Ensure that future SF in Ethiopia by WFP includes attention to
specific strategies, targets and indicators for increasing the
participation of women and girls in SF design and implementation
stages.
by July 2018 WFP CO with
support from the
WFP RB
R9 Ensure that the selection of beneficiary schools under the next
phase of McGovern-Dole support to SF in Ethiopia is based on a
clustered approach so that the distances between schools do not
make monitoring overly onerous or complicated.
in time for the new
SF programme
WFP CO with
support from the
WFP RB
Operational issues
R10 Strengthen the monitoring and reporting capacity of WFP,
and regional and woreda level BoEs in the area of SF so that the
data collection allows for efficient management of SF.
by July 2019 WFP CO with
support from the
WFP RB
R11 Ensure improved coordination with other education sector
stakeholders and working in complementarities for greater impact
and critically assess capacity of communities to support the CHILD
approach.
by July 2019 WFP CO with
support from an
external consultant
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
1
1. Introduction
1. This Evaluation Report (ER) presents the findings, conclusions and
recommendations of the end-of-project evaluation of the school feeding programme (SFP)
implemented by the World Food Programme (WFP) in Ethiopia with the support of the
McGovern-Dole Food for Education Programme (FFE) of the United States Department of
Agriculture (USDA).
2. The programme provides a meal to selected schools in Afar and Somali regions, a
take-home ration (THR) for girls, and technical and material inputs to the strengthening of
government institutions in support of school feeding (SF).
3. The scope of the end-of-project evaluation complies with the requirements of the
terms of reference (TOR) (Annex A). The purpose of the evaluation is to assess to what
extent the USDA McGovern-Dole-funded activities contributed to the anticipated outcomes
of improved attendance and attentiveness, and improved equity in primary education
access, in the Afar and Somali regions. The evaluation covers the period from 2013 to
December 2017, with project implementation starting in January 2014. The evaluation was
commissioned by the WFP Country Office (CO) for Ethiopia.
4. This is the first formal evaluation of this programme. A final evaluation of the
previous McGovern-Dole-funded SFP (Programme number: FFE 6632008/006-00) was
not conducted. The normal requirement for a mid-term evaluation was waived at the time
of agreement negotiation.
5. The original purpose of the evaluation was to inform a subsequent design phase for
the “strategic planning for school feeding in the two pastoralist regions of the country
during the design of WFP’s support to the national school feeding programme” (Annex A:
TOR, ¶ 5). However, delays in contracting have meant that the evaluation has only been able
to inform the new McGovern-Dole application to a very limited extent as the data collection
phase overlapped with the McGovern-Dole application process.
6. The evaluation has dual and mutually reinforcing objectives as follows:
• Accountability – to review the performance and results of the McGovern-Dole FFE
operation by assessing the results of the project against its main objectives.
• Learning – to determine the reasons as to why certain changes did or did not occur,
and derive best practices and pointers for learning. The evaluation will provide the
evidence base for future operational and strategic decisions.
7. The primary users of this ER are stakeholders directly involved in carrying out the
evaluation.1 In addition to the evaluation team (ET) itself, these include WFP Ethiopia and
its main implementing partner, Ethiopia’s federal Ministry of Education (MoE); the Local
Education Development Partner Group; USDA; WFP Headquarters (HQ) and Office of
Evaluation (OEV). The ER is also of direct interest to the participants in the Internal
Evaluation Committee (IEC) and Evaluation Reference Group (ERG). A full stakeholder
analysis is provided in Annex V.
1 The users of the eventual evaluation report and its findings and recommendation will be a broader group –
see stakeholder analysis in Annex V.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
2
1.1 Overview of the evaluation subject
8. The subject of the evaluation is WFP USDA’s McGovern-Dole International FFE and
Child Nutrition Programme (FFE 663-2013/026-00; project number 200253), which is a
WFP operation implemented in Ethiopia’s Afar and Somali regions (Map 1) with activities
at national level focusing on capacity development.
9. The Programme was approved on the 9th of November 2012. This McGovern-Dole
grant accounts for 13.27 percent of the WFP Ethiopia Country Programme (CP) 200253
(2012–2015) budget of USD306,641,528 (WFP, 2011d). Other CP priorities include: i)
increase the capacity of Ethiopia’s disaster risk management system; ii) enhance natural
resource management in food-insecure communities and resilience to weather-related
shocks; iii) facilitate access to HIV care, treatment and support; and iv) promote
opportunities for livelihood diversification and improved access to food markets. Project
implementation started in January 2014 for a period of three years which was later
extended to four years (see Annex K).
10. The purpose of McGovern-Dole International FFE and Child Nutrition Programme
interventions is to ensure equal access is provided for boys and girls at primary school with
a focus on the marginalized food-insecure areas and vulnerable children. The envisioned
outcomes include: i) increasing school enrolment and attendance in the Afar and Somali
regions to achieve the government target of 100 percent enrolment; and 2) improving the
management capacity of school health and nutrition programmes and of school
infrastructure through participatory community planning.2 Intermediate programme
results aim at contributing towards improving health and dietary practices. Major outcome
indicators for the programme include: the percentage change in enrolment, attendance rate,
and attentiveness and Gender Parity Index (GPI). Contrary to some of the other McGovern￾Dole programmes in the region (Kenya, Rwanda) this programme does not target
improving literacy or numeracy levels, because the MoE has an intervention – the General
Education Quality Improvement Programme (GEQIP) – funded by several donors which
has been focusing on improving quality since 2008.
11. The key counterpart ministry for the programme is the MoE, which oversees the
implementation of the programme. The Bureaus of Education (BoE) for Afar and Somali
Regions are responsible for the overall management and implementation of the programme
in their respective regions, including handling food transportation from WFP warehouses to
the schools. The non-food activities that contribute to education, water, sanitation and
hygiene (WASH), construction, health and dietary objectives are implemented by WFP in
collaboration with the BoEs. WFP also engages with each Region’s Bureau of Urban
Development on construction-related activities such as improving water and sanitation
facilities in programme schools. The programme has only government partners.
12. The programme covers 292,249 children (of whom 45 percent are girls) in 590
schools in Afar and Somali (see Table 1 below). This represents just under one third of the
primary school children in these two regions (which hold 1 million pupils in total), and five
percent of the total of 19 million primary school children in the country. In line with gender
equality and women’s empowerment (GEEW) considerations in each targeted school, all
children benefit from the school meals and all girls receive a THR. The school feeding does
not officially cover pre-school children.
2 Essential to this is the use of a participatory community planning tool called Child in Local Development
(CHILD) which had earlier been used in other regions of the country and under this programme was adapted
for the pastoralist regions.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
3
McGovern-Dole International FFE and Child Nutrition Programme
Beneficiaries in Afar and Somali regions
Beneficiary schools and pupils Afar Somali Totals
Number of schools in the programme 361 229 590
Percentage of schools with McGovern￾Dole FFE support versus all primary
schools in the region
47% 19% 33%
On-site feeding beneficiaries (boys and
girls)
93,983 198,266 292,249
Female SF beneficiaries and percentage of
total beneficiaries
43,119 (46%) 88,410 (45%) 131,529 (45%)
Percentage of pupils in Afar and Somali
receiving McGovern-Dole SF
48% 22% 29%
Girls’ THR 43,119 88,410 131, 529
13. The programme provides children with about one third of their daily calorie intake
(see Annex Q) along with complementary activities such as deworming and health and
nutrition education. It provides one hot meal per day (composed of 120 gr of corn soya
blend (CSB+),3 6 gr vegetable oil and 3 gr salt4) for all targeted children, corresponding to
504 calories per day. According to WFP School Feeding and Nutrition Policy, September
2010 (WFP, 2010b), the midday school meal should meet 30-45 percent of the energy
requirements and micronutrient requirements. The ingredients for the meals are provided
in kind to the schools.
14. The SFP in Ethiopia seeks to address gender parity and equity through the provision
of school meals, a THR that specifically targets girls, and capacity development which
includes attention to gender issues. All girl pupils in the schools covered by the programme
receive a THR of eight litres of vegetable oil per semester. The THR is conditional on 80
percent attendance by beneficiary girls (and according to records reviewed by the ET was
being provided to over 95 percent of the eligible girls). Additional measures include
provision of facilities like latrines, and provision of water and sanitation. A more detailed
gender analysis can be found in Annex N.
15. The provision of meals and the THR takes up the bulk of the programme budget.
However, the programme also includes a substantial capacity development component of
USD1.8 million (six percent of the total programme budget) which comprises activities such
as: teaching materials and equipment; financial and technical support to regional level
Education Management Information System (EMIS), adaptation of the CHILD manuals to
pastoralist communities; provision of financial and technical support to water and
sanitation; capacity development for school health and nutrition (SHN) education;
promotion of health clubs and school gardens; building and rehabilitation of school
facilities; provision of energy-saving stoves; training in food handling activities; and
strengthening of SHN planning and management capacity at woreda and school level.
3 CSB+ contains corn, soy, dried skimmed milk and Vitamins and minerals and CSB has all other ingredients
except dried milk powder.
4 WFP has mobilized resource from other donors for the procurement of salt.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
4
16. A full overview of McGovern-Dole International FFE and Child Nutrition Programme
activities and targets can be found in Annex K. There have been three changes to the
programme:
• The ration size for CSB was modified from 150gm to the actual size of 120gm
retroactively, to address the imbalance between the released resources and beneficiary
numbers.
• The type of ration was changed from CSB to CSB + when the programme was extended.
• During the extension phase the programme was amended to include financial and
technical support to the national school feeding programme (NSFP).
17. The FFE programme has a total budget of USD40.7 million, up from the original
commitment of USD26.5 million in 20125 (see Table 2). The programme involves the
distribution of a total of 12,450 MT of CSB, 6,110 MT of CSB+, and 4,830 MT of vegetable
oil (i.e. a total of 23,390 MT). The anticipated timeframe of the programme was from
January 2013 through to December 2015. The programme had one extension and two
budget revisions (BR).
Budgetary situation of the programme
Original commitment BR1 BR2
Additional commitment N/a USD 2,209,835 USD11,999,970
New total USD26,500,000 USD 28,709,835 USD40,709,850
Date November 2012 March 2015 July 2016
18. The McGovern-Dole FFE and Child Nutrition operation provides approximately 50
percent of all the school meals in Ethiopia (this includes other donors and government
meals). A WFP/MoE Home Grown School Feeding (HGSF) pilot programme is on-going in
Oromia and SNNPR regions with support from other donors and regional government
funding. In addition, in 2014-2107, the MoE also ran an Emergency School Feeding
Programme (ESFP) in drought-affected regions of the country in schools that are not
covered by WFP. Originally envisioned as a larger programme, it was funded by the
Government of Ethiopia (GoE) to the tune of USD27 million. Further details on these school
feeding initiatives can be found in Annex L. Annex L includes information on the previous
evaluations of WFP-supported school feeding in Ethiopia and summarizes the main lessons
learned.
19. A map is included in Annex M.
1.2 Context
20. Government policies and priorities. Ethiopia has implemented successive
national plans focused on poverty reduction and the acceleration of economic growth. It is
currently implementing the second phase of its Growth and Transformation Plan (GTP II)
(Government of Ethiopia, 2010b), which will run to 2019/20. The plan envisages Ethiopia
achieving lower-middle-income status by 2025 by modernizing the agricultural sector,
expanding industrial development with primary focus on light manufacturing, and making
a significant shift in export development, while implementing the Climate Resilient Green
Economy (CRGE) strategy in all sectors to ensure sustainability. The industrial sector is
5 2013 Operational Plan for Agreement # 663-2013/026-00 (11/9/2012)
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
5
expected to grow by an average of 20 percent, transforming the country into a
manufacturing hub and creating jobs. Modernizing the agricultural sector is deemed critical
in terms of the development of the manufacturing sector and in enhancing food and
nutrition security and reducing rural unemployment (Government of Ethiopia, 2016b). A
draft policy for SF has been prepared with support from WFP and is awaiting approval.
Responsibility for SF is formally recognized in the structures of government and at central,
regional and woreda level staff is assigned to supporting SF activities.
21. Economy and poverty. Ethiopia has seen strong economic growth in the past two
decades and an average economic growth rate of 11 percent, well above the Sub-Saharan
average. However, economic growth is slowing down due to sluggish performance in
agriculture and the stagnation of industry. Ethiopia has made strong strides in the
reduction of poverty. The poverty headcount declined from 38.7 per cent in 2005 to
29.6 per cent in 2010/11 and is estimated to have declined further to 23.5 per cent in 2016.
Between 2000 and 2014, the Human Development Index (HDI) for Ethiopia improved by
an annual average of 3.2 percent, from 0.284 in 2000 to 0.4442 in 2014.
22. Food security and nutrition. Higher economic growth has brought about an
overall decline in food insecurity. However, food insecurity still remains high, with an
estimated 8.5 million food insecure Ethiopians currently requiring relief assistance to meet
basic food needs following the third consecutive poor/failed rains in the southern drought
belt which has put severe strain on families. The updated national ‘hotspot’ classification,
shows an increase in the number of priority woredas requiring urgent humanitarian
response, with 228 out of 461 priority woredas classified as ‘priority one’ (very severe). All
woredas in Afar and Somali, where the McGovern-Dole FFE operation has been
implemented, are classified as ‘priority one’ and ‘priority two’ hotspots. In Somali Region,
1.7 million (31 percent of the region’s population) require food assistance while in Afar the
number stands at 449,987 (26 percent of the region’s population) (Government of Ethiopia
& Humanitarian Partners, 2017).
23. About 38 percent of Ethiopian children under five are stunted, 10 percent are wasted,
and 24 percent are underweight. The feeding practices of only 7 percent of children age 6-23
months meet minimum acceptable dietary standards. Vitamin A, iron, zinc and other
micronutrient deficiencies are widespread, with 57 percent of children aged 6-59 months
and 24 percent of women aged 15-49 suffering from anaemia (Central Statistical Agency &
The DHS Program, 2016). Rural populations are highly exposed to undernutrition and
micronutrient deficiencies due to poverty and food insecurity.
24. Education. The GoE has made progress towards universal primary education. The
Net Enrolment Rate (NER) for primary has increased from 21.6 percent in 1995/96 to
93.7 percent in 2014/15. However, grade 1-8 dropout rates increased by almost one
percentage point in 2015/2016 to 10.7 percent compared to the previous year, and failed to
meet the 1 percent target in the Education Sector Development Programme (ESDP IV)
(Government of Ethiopia, 2016f), (Government of Ethiopia, 2015a). High dropout rates,
especially in pastoralist and emerging regions, are poverty related and reflect that children,
both boys and girls, work or take care of cattle to support the family – a fact which has
become more predominant due to the recent drought. Learning outcomes are not keeping
pace and there are also regional and gender disparities in basic education proficiency.
25. Gender dimensions. Women in Ethiopia account for 50 percent of the population
and the Ethiopian Constitution and its National Policy on Women guarantee them gender
equality and the protection of human rights in various spheres of life. However, women do
not equally participate in and benefit from development and progress. Ethiopia’s standing
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
6
in the Global Gender Gap Index (115 out of 144 countries and an overall score of 0.656)
shows that women’s participation in key sectors and their role in decision-making are still
not at par with that of men (World Economic Forum, 2017).
26. In the education sector, national strategies to ensure equal access to education have
contributed to increasing the number of enrolled girls and boys across different regions.
However, the GPI indicates gaps at all levels of education. The GPI for Afar Region is 0.9
and is almost equivalent to the national GPI which is 0.91 while the GPI for Somali is the
lowest in the country at 0.86 (Government of Ethiopia, 2016f). Gender disparities are
widely attributed to societal gender roles and socio-economic challenges, including girls’
responsibilities for household chores and a lack of gender-sensitive facilities and services
around schools (UN Women, 2014). Three million Ethiopian children remain out of school,
many of whom are girls. A significant number of out-of-school children are from pastoralist
and semi-pastoralist areas in Ethiopia. The nomadic lifestyle of the populations in these
areas, combined with conflict and drought, makes girls particularly prone to being taken out
of school when families come under stress (Atem Consultancy Service, 2012).
27. International assistance. Total development assistance to Ethiopia was USD3.9
billion in 2013.6 Since 2004, official development assistance (ODA) to Ethiopia has
increased by 66 percent in real terms. While humanitarian aid has declined from 20 percent
of ODA to 10 percent in the past decade the humanitarian caseload is still significant, and
often recurs in the same regional hotspots from one year to the next, many of which are
pastoralist and nomadic in nature. A number of actors support school feeding in Ethiopia
(details can be found in Annex L).
28. WFP’s work in Ethiopia. WFP has been working in Ethiopia since 1965. WFP’s
CP in Ethiopia is based on a country strategy for 2012-2015. The CP prioritizes various
areas including work on nutrition, relief and support to the Productive Safety Net
Programme (PSNP), and climate solutions, in addition to the FFE Programme. The CP is in
line with the United Nations Development Assistance Framework (UNDAF) and is designed
to support the Government’s growth and transformation agenda. The CP aims to improve
access to and equity in primary education, especially in chronically food-insecure pastoralist
areas.7 Other current WFP activities in Ethiopia include two Protracted Relief and Recovery
Operations (PRROs), supporting the livelihoods of communities in the Southern pastoral
belt that are facing acute food insecurity, currently aggravated by the El Niño crisis, and
providing food assistance and livelihood interventions for refugees in 13 camps across
Ethiopia.
1.3 Evaluation methodology and limitations
Mixed methods
29. In line with requirements from the TOR (Annex A), and documented in the
Inception Report (IR), the overall approach to data collection has been a combination of
secondary and primary data collection and analysis and has included:
• Documentation analyses: qualitative and quantitative.
• Key informant interviews (KIIs) and focus group discussions (FGDs) at various levels
with key stakeholders.
6The latest year for which OECD/DAC data are available.
7 The CP also includes support to PSNP etc.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
7
• An analysis of national EMIS data (2013-2016) of the performance of WFP schools
compared to non-school feeding schools. Details and results can be found in Annex
X.
• Administration of a survey covering pupils, teachers and school principals with
closed and open questions, targeting school- and woreda-level interviewees/
informants and covering intervention and control schools. Full details and results
are in Annex Y.
• Observations8 of school feeding preparation and provision of meals in WFP schools.
A theory-based approach – working from the theory of change to test the key assumptions
30. The different data collection methods were combined and linked to the theory of
change (ToC) (see Figure 1).
31. Annex C includes a more detailed discussion of the ToC and a version of the ToC
showing causal linkages. Annex K provides targets for the indicators linked to the
Programme. The ToC directly informed the full evaluation matrix presented in Annex D.
The circled numbers in Figure 1 refer to the underlying assumptions in the ToC which are
listed in Table 3 below and which were tested through the evaluation.
Figure 1 Theory of Change
8 Observation could only be done up to the end of January as this was the end of the school feeding phase.
Even if stocks remained in the schools at that time, the plan was to distribute them to beneficiaries because of
the expiry date of CSB and oil. However, during the visit of the ET in February 2018, some of the schools were
still providing school feeding in Somali Region, and the ET could observe food preparation and provision of
meals.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
8
 ToC Assumptions
Number Assumption
1 Food will be delivered in a timely manner and in the required quantities.
2 Sufficient funds will be made available for the programme by the donors.
3 The Government of Ethiopia will have sufficient capacity to secure implementation.
4 The education system is effective in absorbing new students.
5
Other partners are able and willing to work together with WFP to implement the school
health and nutrition components of the programme.
6 Incentives for school feeding are not outweighed by other factors (early child marriage,
drought).
7 The national policy environment is conducive to school feeding.
8 Government owns the programme and is willing to provide the resources (human and
financial) for implementation.
32. Using this approach, the evaluation found that some assumptions were misplaced or
inaccurate; that other assumptions should have been identified but were not; or that
assumptions about causality were proving correct, underscoring appropriate design. See the
discussion of the assumptions in section 3.1.
Using a quasi-experimental design
33. The evaluation was conducted using a quasi-experimental design9 where WFP
intervention schools were compared with non-intervention schools.
34. The sample survey was designed to address the evaluation questions relating to
impact and effectiveness through a quantitative, evidence-based and statistically sound
approach. At the same time, it collected qualitative data from a significant number of key
informants involved in the sampling which could be analysed qualitatively for recurrent
themes and observations. The field sampling also provided the framework for more
discursive and purely qualitative KIIs with school principals and FGDs with Parent Teacher
Associations (PTAs) during the fieldwork.
35. The sample size of 90 schools estimated in the project proposal was retained, as
being consistent with prior information to give 10 percent sampling error for 95 percent
confidence intervals on worst-case binomial estimates (full details on sampling of the
schools are in Annex T). This comprised 30 schools in Afar region, and 60 schools in Somali
region, the 1:2 split ratio being proportional to the number of government primary schools
in the region (573 Afar, 1207 Somali), according to prior information. The level of sampling
was also consistent with the timetable and resources available to the evaluation. Annex Y
includes the results tables and analysis from this survey.
36. In addition to the sample survey, data on educational indicators (enrolment, grade
completion, by gender) for all primary schools were provided by the MoE EMIS system for
the 2016-17 academic year (September-July). This was not a sample but a census
(100 percent coverage) of schools and therefore was expected to provide comprehensive
information on the impact, if any, of the programme on these basic educational indicators.
37. Full outlines of the survey approach and of the survey instrument used are provided
in Annex T and Annex U. The results of the survey can be found in Annex Y. The results for
the EMIS data study are given in Annex X.
9 The TOR required the ET to establish whether a quasi-experimental design was feasible for the final
evaluation. The inception phase established that this was possible.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
9
Sequencing of the inquiry
38. Data collection and analysis were sequenced to ensure that secondary data were used
to maximum effect. This was done as follows:
• An initial analysis of the EMIS data to compare statistics from EMIS for WFP (i.e.
SF) schools with other primary schools in Ethiopia. This planned to cover the
academic years from 2013-14 to 2016-17 for those EMIS indicators for which national
statistics were available. A limitation was that despite repeated requests only data for
2016-17 were received, with significant limitations that are noted in Annex X. The
contribution of the EMIS data to the overall analysis is therefore relatively limited.
• Documentary analysis to identify preliminary answers to the evaluation questions.
This allowed the team to identify gaps and to pursue these gaps during the interview
process and further documentation and data collection at the field work stage.
• A survey of WFP primary school and control schools, in accordance with the
approach described in Annex H of the IR (Annex T of this report). This was
conducted during February 2018. In total 45 pairs of in- and out-of-programme
schools were selected (90 schools total) with 12 students per school (1080 students in
total). Table 4 shows the distribution of the sample.
 Planned and achieved sample distribution
Planned sample Achieved sample
Region IP OP Total IP OP Total
Afar 15 15 30 24 6 30
Somali 30 30 60 30 30 60
Total 45 45 90 54 36 90
% 50% 50% 100% 60% 40% 100%
(IP = In FFE programme, OP = Out of FFE programme)
It will be noted from Table 4 that the achieved non-intervention sample (OP) for Afar
was lower than planned. This was due to a combination of factors: prior information
for OP status based on 2013 data was not reliable; in-field substitution in Afar was
difficult due to high programme coverage; and tight time constraints on the field
sampling prevented more complete scoping out of alternative IP-OP pairings. On the
other hand, in Somali, in-field substitutions worked effectively and a balanced
sample was achieved, in spite of the necessity of making substitutions from the
original lists.
• In-depth interviews with key informants at federal, regional, woreda, and school
levels to obtain insights from key partners as well as from beneficiaries on the
different dimensions covered by the evaluation.
39. The evaluation drew on these data sources where they are likely to have most utility.
The evaluation matrix (Table 11 of Annex D) shows which data sources provided
information against the identified indicators.
Qualitative research
40. In terms of the qualitative side, the team carried out observation visits in schools in
Afar and Somali regions during a week-long visit in January 2018. The observation visits
focused on reviewing the arrangements for school feeding in targeted schools (i.e.
observation of preparation of meals, storage and handling as well as meal time). This was
followed by a two-week data collection phase in February in the same regions during which
the team conducted interviews with regional education authorities, woreda education staff
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
10
and heads of kebeles, and visited schools to interview teachers, pupils, parents and support
staff (cooks, members of parent teacher associations and other relevant persons).
41. The evaluation approach also included two mini-case studies:
• A study of the Emergency School Feeding Programme to inform the evaluation’s
assessment of sustainability and impact dimensions and to input into the lessons
learned (and recommendations) (see Annex R).
• A study of the Take-Home Rations – Girls Initiative Programme which examined
impact on household income and spending patterns, as well as on changing norms
about girls’ education. It also examined the extent to which the benefits are sustained
over time and beyond the duration of the programme; compared beneficiary girls
with non-beneficiaries (male pupils) and how these fare by comparison; and
considered unintended effects/outcomes, as well as the positive and negative
consequences of the intervention. (See Annex S.)
42. The qualitative ET had planned to visit a non-WFP school where emergency school
feeding (ESF) was in operation, but it could not take place due to time constraints and no
ongoing ESFP during the evaluation.
43. For the in-depth interviews at school level, interviews were sought with the head
teacher, the co-ordinator of school meals, a group of beneficiaries of the THRs (i.e. girls), a
group of non-beneficiaries of THRs (i.e. boys), parents of THR beneficiaries and parents of
non-THR beneficiaries, the cooks and the PTA. See Annex T for further details.
44. In each kebele and at woreda level, the team sought interviews with the local
education authorities, specifically any officials responsible for school feeding, and staff of
UNICEF, Save the Children International (SCI) and Italian Cooperation working in the area
of school feeding.
45. At regional level, in-depth interviews were undertaken with the relevant regional
authorities and WFP staff. Where possible this was done before the school visits to
maximize the opportunity for further investigation of issues at the school level. At national
level the interviews were with key informants from WFP, the MoE, UNICEF, SCI, and the
donor, USDA.
46. A systematic approach to consultation was included in all phases of the evaluation:
• The Evaluation Manager (EM) was informed of the arrival of the ET and their
planned field visits to Somali and Afar region and stakeholders meetings.
• The itineraries for the field visits were shared with the EM to inform the respective
WFP sub-offices and the Government departments.
• The survey team informed the EM about the field work itineraries and requested
letters of introduction for the teams from the Education Department.
• An exit debriefing while planned could not take place due to last minute logistic
challenges on the CO side. However, the evaluation team shared key qualitative
findings with the CO in a separate note.
• A debriefing to Government and wider stakeholders including the external reference
group at the end of the field work also did not take place
47. As part of the overall approach to the evaluation the ET ensured both triangulation
and complementarity between methods. Validity and reliability were ensured through a
rigorous evaluation design which is further discussed in Annex B.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
11
Limitations and risks
48. During the inception work of the ET it emerged that school feeding would have
ended by the time of the initially planned dates for the field mission visits to Afar and
Somali (in February 2018). The ET had to move a part of the field work forward to January
2018 to ensure observation of school meals could be part of the evaluation approach.
49. Distances in the geographical areas covered by the evaluation were considerable and
inefficiencies were the result of poor information sharing with the ET. Valuable time and
resources were expended on visiting schools in Afar region that were found to be closed.
The survey team faced considerably challenges because lists of schools provided by WFP
could not be matched to actual locations. A substantial number of schools were found not to
exist, and had to be replaced by schools on a reserve list (see below). These issues were the
result of the lack of a current and complete list of schools with geographical coordinates and
their programme status. The data used were from a study in 2013, and clearly in the
interim some schools had moved, merged, or ceased to operate. The current available lists
from WFP and MoE lacked geo-data and could only be matched by school name, which
proved to be very problematic as detailed in Annex X (p. 201). Overall approximately
20 percent of schools had to be substituted due to closure (as sampling was done during a
holiday period for some areas), non-existence, or incorrect programme status. In Afar,
locating substitute out-of-programme schools was particularly difficult because of higher
programme coverage in that region.
50. Security issues also posed a risk to the timely and orderly conduct of the evaluation.
To mitigate this risk, the survey sampling included a 20 percent margin in terms of the
selection of schools to visit. When it was not possible to visit a specific school because of
security concerns, then this school was replaced by another already sampled school without
affecting the quality of the evaluation.
51. The work of the qualitative team was compromised in Afar region because regional
authorities had not been informed of the ET’s visit, which meant that the targeted number
of schools could not be reached and some informants were not available for interviews.
Ensuring quality
52. WFP has developed a Decentralised Evaluation Quality Assurance System (DEQAS),
informed by the norms and standards for evaluations developed by the United Nations
Evaluation Group (UNEG). The DEQAS forms a specific set of guidance materials based on
WFP’s Evaluation Quality Assurance System (EQAS) and its Evaluation Policy. The guide
sets out process maps with in-built steps for quality assurance and templates for evaluation
products, as well as checklists for feedback on quality. DEQAS has been systematically
applied during this evaluation, with relevant guideline documents having been provided to
the ET.
53. Mokoro’s internal Quality Support (QS) System was integrated into the evaluation
process in line with the company’s commitment to delivering quality products and
adherence to the principles of independence, credibility and utility. Evaluation products
were shared with the QS experts prior to submission. (See Annex F, Table 15.)
54. There was no potential conflict of interest in the performance of this evaluation.
None of the ET members were involved in the preparation or direct implementation of the
WFP-supported school feeding activities in Ethiopia.
55. The team adopted a careful and thorough approach to the ethics of the evaluation,
complying with standard 3.2 of the UNEG Norms and Standards (UNEG, 2016). While
supportive and collegiate in its working relations with WFP, it was strictly neutral and
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
12
unbiased. It requested consent from all interviewees and focus groups before proceeding
with discussions, and assured them of full confidentiality: while names are listed in Annex
J, no view or statement is attributed to a named individual, or presented in such a way that
an individual can be traced as its source. The evaluation complied fully with GoE and WFP
guidelines on contact with children (UNEG, 2008). There were some ethical issues in
including schools in the survey that did not benefit from the FFE programme, as part of the
counterfactual sample. The evaluation team emphasized to the schools concerned that
participation was voluntary and that they were free to refuse (as indeed were all
respondents in the sample, who were similarly briefed and appropriately substituted if they
did not wish to participate). The issue of perceived lack of benefits for schools that had not
benefited from school feeding was handled by explaining that the original selection criteria
were based on relative need at the time of programme start-up in 2013, and that the current
situation is open to review, which the survey seeks to inform. The survey provided an
opportunity for the non-programme schools to feed back comments and views to the DoE
and WFP.
2. Evaluation findings
56. This section presents the evaluation findings against the Key Questions (KQ) and
Evaluation Questions (EQ) set out in Table 9 of Annex B. Further details can be found in
Table 19, Table 20 and Table 21 of Annex K, as well as in Annex R and Annex S, which
discuss two case studies, in Annex X, which presents the results from the EMIS data
analysis, and in Annex Y ,which includes the field survey results.
KQ 1 – How appropriate was the programme?
EQ 1 – To what extent are the interventions appropriate to the needs of boys,
girls, and parents of school-age children?
57. The interventions have appropriately focused on areas which are acknowledged to be
among the poorest in the country, where people live in a context marked by poverty and
hunger. In these areas, populations face significant challenges including the prolonged
period of droughts. Poverty is the major obstacle to girls’ education in Ethiopia, particularly
among girls from food-insecure pastoralist areas. Food shortage and insufficient income
deter many parents from sending their daughters to school, underscoring the importance of
specific interventions for girls which in the case of this programme include the girls’ THR
(Annex S). The inclusion of WASH and school health and nutrition interventions – though
not the direct responsibility of WFP – was very appropriate given their importance as part
of a comprehensive approach to nutrition and SF. The relevance and importance of the
WFP-supported SFP to children’s needs were confirmed and appreciated by beneficiaries
and other stakeholders at various levels. Interviews with beneficiaries in all locations
consistently underscored the importance of the interventions in alleviating hunger and
providing essential services.
58. CSB is widely accepted (and liked). The appropriateness of the food choice can be
questioned given that the food is imported, therefore having an impact on sustainability,
and given that lack of water is a major challenge at schools.10 The fact that the food is
imported also has implications for the efficiency of the programme given that the same
10 In Bangladesh WFP and stakeholders decided to use biscuits in their school feeding programme to address
the issue of lack of water, kitchens and cooking facilities and WASH.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
13
volume of funding could be used to cover more schools.11 In addition, if the food had been
produced locally, this would have a positive effect on the economy and on job creation, and
would therefore have contributed to the alleviation of poverty.
59. The ET found that the design and the school selection process were participatory.
Through consultations with communities the BoE selected the initial schools. However, it
did not take account of potential logistical challenges that would arise from following the
government criterion of promoting equity by selecting schools over a large area (the entire
region). A checklist was used that included criteria such as the level of community
participation (in providing water, firewood, and the cook’s salary) and drop-out rates. The
result was that selected schools were spread out over a vast area.
EQ 2 – To what extent are interventions aligned with relevant national
policies? Was programming sensitive to context?
60. The programme is well aligned with national policies, including in the areas of
education (Government of Ethiopia, 2010a and Government of Ethiopia, 2015a) and social
protection (Government of Ethiopia, 2013c). The ESDP V (2015-2020) (Government of
Ethiopia, 2015a) articulates Ethiopia’s plan to reaching the sustainable development goals
for education12 and aligns with the Second GTP (Government of Ethiopia, 2016b). The
ESDP V recognizes the challenge of ensuring equitable access to education especially in
rural areas. The Government’s Social Protection Policy (Government of Ethiopia, 2013c)
lists six objectives, including objective 3.3 to “increase access to equitable and quality
health, education and social welfare services and build human capital to break
intergenerational transmission of poverty”. The McGovern-Dole-supported WFP SF
programme aims to encourage participation in basic education and also to help improve
education quality, and in this respect it is clearly in line with national education and social
protection objectives.
61. While the WFP SFP is well aligned with national policies in terms of encouraging and
working towards providing equitable access to education for all children, the programme
under evaluation does not reflect the Government’s priority of home-grown school feeding,
as the food is currently imported, an issue that raises questions over its sustainability (see
section 3, ¶173).
62. To date, a national SF strategy is at final stage, to be endorsed by the Council of
Ministers. Government has not yet committed to what would be relatively high recurrent
costs for regular SF interventions.
Coherence with WFP policies and strategy
63. The McGovern-Dole-supported SF programme in Ethiopia is consistent with WFP’s
global policies on nutrition (WFP, 2012a), school feeding (WFP, 2013f), and gender13 (WFP,
2009c, WFP, 2015c).
64. The McGovern-Dole programme is consistent with both WFP's SF Policy and WFP's
Nutrition Policy. The revised SF policy characterises its new emphases as follows:
"While continuing to advocate for the universal adoption of school feeding programmes that help
increase children’s access to learning opportunities and improve their health and nutrition status,
11 See also the 2017 Ethiopia Investment Case report (WFP, 2017d). 12 SDG 4: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.
http://www.undp.org/content/undp/en/home/sustainable-development-goals.html
13 For a discussion on gender see EQ 5 and Annex N.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
14
WFP will focus increasingly on helping countries to establish and maintain nationally owned
programmes linked to local agricultural production." (WFP, 2015c)
65. The policy's emphasis is on supporting "access to learning opportunities", which is
also the focus of the McGovern-Dole programme in Ethiopia, contrary to other McGovern￾Dole programmes, which also put a strong focus on the quality of education (see Annex K).
66. WFP’s Nutrition Policy (WFP, 2012a) focuses on five distinct but related areas, one
of which is to strengthen “the focus on nutrition in programmes without a primary nutrition
objective and, where possible, linking vulnerable groups to these programmes” (WFP,
2012a: 7-8). School feeding falls under this category. Nutrition support in SF is viewed in
terms of the provision of the target group’s nutrient needs. The food provided through the
SF intervention significantly reduces micronutrient deficiencies and conforms to WFP’s
normative guidance on nutritional standards (see Table 22 of Annex Q).
Sensitivity to context
67. Generally, the programme’s design is appropriate. The intervention includes
important aspects which are essential contributors to its success, namely ensuring access to
water and sanitation in the predominately arid areas where the programme was
implemented. It included a strong focus, through THR, on the prevailing issues of
challenges to girls’ participation in education. However, it is questionable whether school
gardens in arid zones are a realistic endeavour. Woreda-level interviews, as well as parent
interviews, converged to highlight that in conditions of extreme poverty and given the
prevailing insecurity the capacity of families to participate is extremely limited.
68. Some elements of the design were ambitious given the resources and the reality of
the context. As is noted in the discussion on EQ 4, the assumptions around the possibility of
establishing school gardens in extremely arid areas were ambitious to start with, and proved
to be even more challenging with the continuing drought in these areas. The design also
assumed considerable involvement of communities in mobilizing resources to support
priorities identified through school level planning. The level of realism of this assumption
needs to be questioned in light of the extreme poverty in these regions and the prevailing
drought which severely limits the capacity of communities to contribute in any form.
EQ 3 – To what extent was the intervention design based on a sound analysis
of gender and equity, and sensitive to GEEW? Were other issues, including
protection and accountability towards affected populations, adequately
factored in?
69. When the programme was designed, the 2009 WFP gender policy was still in force
(WFP, 2009c). This policy introduced a shift from a women-centred approach to a broader
analysis of challenges and opportunities in the lives of women, men, girls and boys. WFP’s
latest Gender Policy (2015–2020) (WFP, 2015c) was adopted towards the end of the review
period. For a more detailed discussion on gender in Ethiopia see Annex N.
70. Only a superficial gender analysis was conducted before the programme started. The
baseline data is poor and there is no prior evaluation to build on. However, the THR
intervention built on a prior project and on an evaluation of this intervention (WFP, 2011a)
and could therefore incorporate important lessons learned (see Annex S).
71. There is no complaints mechanism in place yet, although biannual meetings and
monitoring visits have provided an opportunity to hear about issues.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
15
KQ 2 – What were the results of the programme?
EQ 4 – To what extent have the planned outputs and outcomes been attained?
Have there been any unexpected results (positive or negative)?
What are the results of the programme?
72. Following the requirements of the TOR, EQ4 asks about the extent to which planned
outputs and outcomes have been attained.14
73. Overall, there is evidence – corroborated by qualitative and quantitative findings –
that WFP has achieved highly satisfactory levels of performance both at output and at
outcome levels. In fact, for many indicators – as will be seen below – the programme
consistently over-performed, reaching more beneficiaries, and providing more facilities
than had been planned.15
Attainment of outputs
74. Adapt CHILD manuals for pastoral communities: The approach to school feeding
under the current grant included using the CHILD16 approach in the pastoralist
communities in an effort to help transform schools into local development centres for their
communities through participatory community planning and resource mobilization. Prior
to this, the CHILD approach had only been implemented in the highland areas of north￾central Ethiopia. A revised manual was produced with adaptations to the pastoralist
context, identifying effective methods to mobilize the pastoralist communities to participate
in school development programmes, and including activities that could demonstrate to the
communities the concept of schools as development centres.17
14 There were some challenges in answering these questions definitively because of the multiple divergences
between output and outcome indicators and targets in the baseline survey (WFP, 2013a), WFP’s grant
proposal to USDA (WFP, 2011c), the USDA Commitment Letter to WFP (USDA, 2012b), and WFP’s biannual
progress reports as explained in Annex K. For the MGD school feeding programme under review, the
outcomes achieved at the end of the previous phase in FY 2011/12 were to be taken as baseline figures for
continuing activities, and WFP conducted a baseline survey in that same year for the new indicators that were
included in the current MGD intervention. However, a review of the baseline and performance targets set both
in WFP’s grant proposal to USDA and in the USDA Commitment Letter to WFP shows that the findings of the
2011/2012 baseline survey and the reported outcomes of the previous MGD intervention did not sufficiently
inform the design of the MGD programme under review, as the output and outcome targets set for some
indicators are inconsistent with the findings of the baseline survey and outcomes achieved at the end of the
previous MGD intervention. It has not been possible to find clear written statements as to why these various
changes were made and they are further complicated by apparent inconsistencies and errors in some of the
data, which has made assessing the attainment of “planned” outputs and outcomes challenging.
15 Annex K, Table 19 and Table 20 set out the planned outputs and outcomes respectively as shown in the
USDA Commitment Letter to WFP (USDA, 2012b) and its second modification (USDA, 2016b), and show the
performance data reported by WFP for each six-month period from 1 April 2014 up to 30 September 2017.
16 CHILD is a community-led participatory planning approach that aims to transform schools to the local
development centre in their community. The approach is aimed to improve the school premises and
community ownership of education, and to make environmental improvements that support the FFE activity
as well as improve the awareness of environmental sustainability of children and their families, with the
overall aim of contributing to increased school enrolment and attendance. It is a planning tool to build the
planning capacity of the education sector at the grassroots level to integrate children’s education into
development programmes.
17 There appear to be discrepancies between what was reported in the biannual reports and what the CO is
saying actually happened. According to the first biannual progress report (01/04/2014 – 30/09/2014), the
adaption of CHILD manuals for pastoralist communities was done in the previous MGD SFP (01/10/2013 –
30/05/2014), not the MGD SFP under review. However, according to the CO, “the activity was initiated” in
2014 during the previous MGD intervention but the translation, printing, and dissemination continued
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
16
75. Provide training on CHILD planning tools: To help develop the planning skills of
the education sector offices at regional, woreda and school levels, WFP has provided
training on CHILD planning to selected schools. The schools that received the training were
selected based on their accessibility for regular monitoring, community interest to
participate in the planning and implementation process, schools with a large student
population, and proximity of the schools to community centres. The target was to train 460
school directors, PTA members, and health and agricultural extension workers in the first
year of the SFP (2014). Due to an increase in the number of CHILD schools and high
turnover of trained people, there was a need to train more people than initially targeted,
with WFP training a total of 579 individuals over the course of three years (2014 – 2016).
However, the reported figure is not disaggregated by sex and it is therefore not apparent
how many if any of the trainees were women. The ET has observed that the overwhelming
majority of school directors, PTA members, and woreda extension workers are men,18
which presumably means that the training has also disproportionately reached men. In the
case of the education sector this is a reflection of the low numbers of female staff. However,
in the case of the PTAs the underlying reasons may also reflect social norms.
76. Promote school gardens: A total of 145 school gardens were reportedly established,
which is much higher than the target of 35.19 School gardens were part of the CHILD
approach, and sought to address how to use the school compound for different activities,
including garden composting and wood lot tree planting. As part of this activity seeds and
agricultural tools were procured and distributed through the Regional Bureaus of
Agriculture (BoAs). However, the high reported figure is incongruent with what was
observed during the evaluation mission to Afar and Somali as well as what Regional BoEs
and schools indicated. Out of the nine schools the ET visited for qualitative assessment, only
the model school in Gursum woreda, Somali region, had a functioning garden, which had
been in place for seven years.20 The other schools had either started and then abandoned
their gardens due to water shortages or had not even established a garden due to the
prevalent arid conditions.
77. As part of the school gardening package, livestock production was piloted in 10
selected schools21 in Somali region with the aim of generating income as well as of
supplementing the meals, and to ensure sustainability of the SFP. This intervention which
was not part of the original design of the McGovern-Dole programme included support
from FAO and other stakeholders. However, the fifth biannual progress report was the first
and only time that this activity was mentioned and there are no documents that clarify how
through the end of September 2015. This is further complicated by the fact that the last reporting period for
the previous MGD intervention and the first reporting period for the MGD intervention under review are
overlapping.
18 Out of the nine schools the ET visited for qualitative assessment, only one had a female director. Most of the
schools had all male teachers and very few women represented in PTAs. The woreda and regional BoE staff
that the ET met with were also all men.
19 The figures provided in the biannual progress spreadsheet reports add up to 145. However, the narrative
reports indicate that the total number of gardens established and maintained is 72, though this is still higher
than the target of 35. 20 Deghale School has used part of the ETB 150,000 prize money it won last year for being a model school to
expand the school garden. The school garden produces tomatoes, chillies and fruits, which the school sells to
purchase school materials (pens, pencils, notebooks) for poor students. Last season’s earnings from the sale of
produce from the school garden amounted to ETB 3,600. 21 There is no information on the criteria used to select the ten schools. Deghale School was provided with ten
goats, five oxen and one cow, which died from snakebite. The school has sold some of its oxen and goats for
income and has purchased new livestock.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
17
it was added to the school gardening component of the SFP and whether and why it was
discontinued.22
78. Provide energy saving stoves: To reduce schools’ cooking fuel consumption and the
environmental impact as well as minimize the labour needed to collect firewood, WFP
planned to provide 300 fuel-efficient stoves during the last three years of the SFP.
According to the McGovern-Dole biannual progress reports, WFP has exceeded the target
and has provided 353 stoves, with additional stoves distributed during the last period of
2017.
79. Build/rehabilitate schools: Targets for canteen construction were exceed. WFP
supported the regional BoEs in developing standard designs for storage, kitchens and
canteens and constructing one model canteen in each region (two in total), inclusive of
kitchen and dining room, as well as furnishing them with the necessary tables and chairs.
While the initial plan was for each canteen to have a holding capacity of 500, the USDA
Modification II to Commitment Letter to WFP (USDA, 2016b) reduced the holding capacity
to 200 and approved the construction of 40 additional low-cost kitchens and canteens in
the two regions, bringing the target figure of canteen construction to 50.23 WFP’s reports
indicate that it has exceeded the target and constructed 87 kitchens and canteens, two of
which are the model canteens with the rest being the low-cost ones. The McGovern-Dole
intervention also set out to construct 30 latrines, and under this output indicator the
biannual reports show that 19 latrines have been constructed.
80. Provide technical support to improve water and sanitation facilities: Almost all the
targets for improving water and sanitation facilities were exceeded by the programme. The
programme provided financial support to the regional BoEs to work with the regional Water
and Sewage Bureaus in improving WASH facilities in schools. WFP targeted constructing
20 latrines but achieved a higher output of 23. WASH technical support was provided to 90
schools (against the 70 that were planned). The construction of water ponds was slightly
less than the target of 83, with 75 water ponds having been constructed. In terms of the
number of schools using an improved water source, while the target was set at 267, WFP
reported that 366 schools now have access to clean water as a result of the McGovern-Dole
intervention, which means the target was exceeded by 25 percent. It should be noted that
the provision of sanitation facilities can only be fully effective if there is access to water and
soap is available. The programme does not fund the provision of soap, but school
informants and field observations suggest that a limited supply is provided by the local
health post and/or purchased by the schools from earnings they get by selling the empty
THR oil containers.
81. Develop SHN planning and management capacity: Target figures for capacity
development were exceeded under the programme. WFP supported MoE in developing an
SHN training of trainers (ToT) manual and provided the ToT on SHN at the regional level to
the relevant bureaus, including education, health and agriculture (BoE, BoH, BoA) and the
water bureaus. The regional bureaus then cascaded the training downwards to school
directors, teachers and woreda FFE focal persons. While the programme had set out to train
a total of 160 government counterpart staff, by the end of September 2017 it had managed
22 In feedback to the draft report WFP clarified that the initiative had been delayed in the past two years due to
recurring drought in the region.
23 According to the CO, the holding capacity of the canteens was reduced from 500 to 200 since the space was
not enough to accommodate sufficient numbers of tables and chairs for 500 students. Keeping the original
holding capacity would have required doubling the size of the canteen, which would also have meant
additional cost.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
18
to train a total of 266 people. The evidence reviewed suggests that the majority of the
participants of these activities were men although exact figures are not available.24
82. Promote health and hygiene clubs: As part of the effort to improve health and
hygiene, WFP supported the establishment of health and hygiene clubs in 315 SFP schools
(against the target of 172) and provided them with mini-media materials (microphones) to
disseminate health and hygiene messaging around the school. However, interviews with
students and observations at schools underscored a lack of proper WASH facilities,
particularly hand-washing facilities near latrines and kitchen/dining areas, as well as a
shortage of water and soap, which has led to students not being able to practise what they
have learned in the hygiene clubs.
83. Promote health and nutrition education: As part of the CHILD approach, WFP
planned to develop a total of eight (and achieved five) different booklets and information
and communication materials on nutrition, health and hygiene practices that could be used
for the purposes of facilitating SHN activities in CHILD schools and as a manual for
trainings. WFP held five trainings and advocacy meetings on SHN (the target was four) at
national and regional levels to establish the regional-level SHN steering committee and
develop the national SHN strategy and action plan. The regional BoHs and the health
faculty of Semera University and Jigjiga University in Afar and Somali regions provided the
SHN trainings. Training in child health and nutrition was provided to 813 (279 female, 534
male) health club members, teachers, school directors and woreda BoE staff, and training
on school health and nutrition was provided to 852 students and teachers.25 Here again,
mainly due to high turnover of trained staff at woreda and school levels, the total number of
people trained greatly exceeded the plan for all four years of the programme.26 WFP’s
biannual reports do not report clearly and consistently on the regional breakdown of the
total number and categories of people trained and do not disaggregate participants by sex.
84. Almost all of the schools the ET visited for the qualitative assessment had health and
hygiene posters with messaging in local language displayed around the school, particularly
around the kitchen/cooking areas and in classrooms.27 Stakeholders interviewed stressed
the importance of the trainings, although some commented that trainings were too short.
85. Provide financial and technical support to the regional education management
system: As part of the effort to build the capacity of GoE in SFP management, two technical
assistants (TAs) were hired at the start of the McGovern-Dole intervention to support Afar
and Somali regional BoEs in programme implementation. The TAs have supported the BoE
FFE focal persons in all aspects of the McGovern-Dole SFP implementation, including
conducting trainings, programme monitoring and reporting in support that was much
24 The reporting does not provide sex-disaggregated figures for the number of people trained nor does it
disaggregate how many of those trained were teachers, school directors, woreda focal persons or supervisors
from regional bureaus. The biannual reports also do not present a clear picture on the regional breakdown for
the total number of people trained. Based on the fact that the stakeholders at the regional, woreda and school
levels are predominantly men, it is safe to assume that the recipients of the SHN training were also mostly
men.
25 The USDA Modification II to Commitment Letter to WFP (USDA, 2016b) added the child health and
nutrition training indicator while at the same time keeping the school health and nutrition training indicator
under the same activity. WFP has been reporting on both indicators in all its biannual reports since then.
26 The number of people trained in child health and nutrition exceeded the target of 156 by 521 percent and
those trained in school health and nutrition exceeded the target level of 150 by 568 percent. Sex disaggregated
data were not available.
27 The exception was one of the schools the ET visited in Kebribeyah woreda, Somali region, where a big pile of
posters was left on the school director’s desk, covered with dust.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
19
appreciated and is considered to have strengthened the capacity to manage the programme.
WFP also developed an online database system to improve the SFP monitoring and
reporting in both regions and provided training on the system to the region’s EMIS
Directorate and SF focal persons at woreda Education Offices. However, the database is not
functional due to the inability of the regional BoE to get to link the database to the online
system.
86. Provide financial and technical support to the NSFP: USDA Modification II to
Commitment Letter to WFP (USDA, 2016b) added to the original programme design the
provision of financial and technical support to the Federal MoE, which increased WFP’s
investments in strengthening government capacity at the strategic and operational levels.
While the core purpose of supporting the Federal MoE was to enable a transition from
donor-supported SFP to government-owned national SFP, it did not succeed in achieving
this goal due to government inability/reluctance to take on the commitment. This also
meant that WFP’s plan to develop a monitoring and evaluation system for the NSFP did not
materialize. However, WFP has invested in the capacity of the government by providing
technical assistance for the development of the National School Feeding Strategy and
ensuring the inclusion of school feeding in the National Social Protection Policy.28
87. Provide school meals: This core business of the programme has performed at 83.5
percent of planned output levels for the number of metric tons of food provided and at only
54.51 percent for number of days school meals are provided.
88. WFP experienced several pipeline breaks due to the delay of food releases from the
donor, which have caused delayed provision of meals and the dispatch of food commodities
close to their “best use by” dates. Food commodities that were close to the best by date were
distributed as THR to communities to avoid wastage. Five out of the seven biannual
progress reports this evaluation has reviewed indicate that there were disruptions to the
provision of school meals due to pipeline breaks.29 The two major disruptions happened
during the third (01/04/2015 – 30/09/2015) and sixth (01/10/2016 – 31/03/2017)
reporting periods. There was no meal provision during the third reporting period due to
delays in receipt (on the donor side) and subsequent delays in dispatch of food
commodities; and food was only provided to 20 percent of schools (only 39,600 students) in
Somali region during the sixth reporting period as a result of delayed arrival of food
commodities, which was the result of commodities being held at the Port of Djibouti for
several months before USDA was notified (January – April 2016) and of the delay caused by
the off-cycle extension which delayed the shipment of commodities
89. To offset the pipeline break in Afar during this reporting period, school children in
four woredas were provided meals for 53 feeding days with donations AICS (Italian
Cooperation) had provided for ESF. Another issue that led to a pipeline break in Somali
region during the sixth reporting period and compounded the problem was the diversion of
28 As part of the technical support to the government, WFP planned to develop three school feeding plans,
which actually are annual working plans for the SFP, and developed an additional three activity plans for the
two Regional BoEs and the Federal MoE. WFP had also planned to develop four guidelines and training
materials; however, the output results reported in the biannual progress reports reveal double counting of
results by repeating outputs that are also reported under the activities of promoting health and hygiene clubs
and promoting health and nutrition education. The double counted output results include the production of an
SHN implementation manual and a training manual on food handling and management, a visual guideline on
kitchen management, hygiene and sanitation, SHN guidelines for health and hygiene clubs, and an SHN ToT
manual.
29 The only two biannual progress reports that did not report on disruption of school meal provision due to
pipeline breaks are the first (01/04/04 – 30/09/2014) and fourth (01/10/2015 – 31/03/2016) reports.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
20
food commodities by the BoE from 38 targeted schools to 167 non-targeted emergency￾affected schools without the consent of WFP or the donor (see Box 1). This action resulted
in 40,276 students not having access to school meals during that period. In addition, 42 MT
of CSB+ was damaged during transportation from Djibouti when the truck overturned on
the way to Somali region.
Box 1 Overview of the 2017 Food Diversion in Somali Region
In 2015 and 2016, the two main rainy seasons that supply over 80 per cent of Ethiopia’s agricultural
yield were not successful. The multi-agency Belg assessment conducted in December 2016 estimated
that 2 million students were affected by the drought. According to the report, Somali, Oromia and
SNNPR were the worst-affected regions. The report further confirmed a number of school closures
due to the emergency situation, with Somali region having the highest number of schools closed.
The situation led the GoE to allocate emergency funding for school feeding to the worst affected
regions and led to the introduction of an ESFP (more details in Annex L).
In February 2017, the Somali Regional BoE diverted 532 MT (approximately one sixth of the
allocated food commodities) from the McGovern-Dole SFP schools to 51 schools which it had
identified as being worst affected by the drought. The BoE used the 22 million birr allocated by MoE
under the ESFP to procure food for additional schools, including 65 schools that were not reached
with the diverted commodities. With these funds, the BoE procured 800MT of locally blended food,
140 quintals of sugar, 90q salt, 90q tealeaf.
In subsequent explanations the regional government stressed that the diversion took place in
circumstances of extreme urgency. However, the fact that the diversion took place without consent
being sought from WFP or from the donor violated the agreement that had been signed with the
Government of Ethiopia for the McGovern-Dole FFE programme. Because of this WFP immediately
suspended all food delivery to Somali region when it discovered the diversion and pending
clarification. As the clarification was not forthcoming WFP then commissioned an independent
verification of the diversion (WFP, 2017e).
The independent verification concluded that the reallocation was justified given that the targeted
schools were the most affected by the drought and had had to close as a result of it. It also found that
the distribution to the newly identified priority schools had been well organized, had taken place in
an orderly fashion, and had included training and the provision of all essential non-food items
(NFIs). Additional observations of importance by the verification team included the high level of
community involvement and mobilisation in the ESFP efforts and consistent community reports
that the food had made a significant difference. As noted in the verification report, “to avoid further
suffering of children, as soon as the verification mission was complete and with the support of
USDA, WFP lifted the suspension of dispatch of FFE commodities”. When food distribution
resumed, WFP took over the transportation of food commodities from the central warehouse to
schools. This procedure continued in place until the time of this evaluation (close to the end of the
McGovern-Dole programme).
The regional government almost immediately presented a formal apology to WFP and the donor for
the breach in procedures and provided assurances that events of this kind would not be repeated.
Nonetheless, for WFP and for the donor the fact that procedures were not followed and that the BoE
took a unilateral decision without consultation, and did not replace the food, created a rift and
jeopardized the trust established between partners.
This situation continued to be of great concern to the regional authorities in Somali when the ET
visited as it was understood by the latter that it was because of these events that Somali region was
excluded from the next McGovern-Dole school feeding proposal.
Source: WFP Programme Unit Addis Ababa and Jijiga Area Office (March 2017). Food diversion
fact verification report – Food for Education Programme ( WFP, 2017e) and interviews by the ET.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
21
90. Provide teaching material and equipment: WFP planned to provide teaching
material and equipment to 182 schools in years 2015 and 2016. In fact, the project
distributed material every year to a total of 334 schools, which represents 183 percent of the
planned distribution. The materials included text books, science and geography teaching
kits, and first aid kits. The evaluation noted that there were materials in all schools, coming
from different sources. Feedback from schools was that the materials were useful but it was
not possible to assess the specific utility of the material provided by WFP.
91. Provision of non-food items to schools: Under this activity the project planned to
provide kitchen shelves and pallets for food storage, cooking pots, plates and other utensils.
The activity also included provision of motor bikes to support the monitoring and
supervision activities of the BoE. 1218 schools received utensils, which represents 142
percent of the planned distribution. The plates and other utensils are managed by Food
Management Committees (FMCs) in the schools.
92. Take-home rations: The number of THRs distributed fell short of the planned
targets in most years and was affected by pipeline breaks, similarly to the school meal
provision. A full discussion of the THR is provided in Annex S. A monthly THR of two litres
(8 litres per semester) of vegetable oil is provided to girl pupils as an incentive to attend
school, based on a minimum attendance rate of 80 percent as an incentive to attend school
and to encourage parents to send their girls to school. The targeted beneficiary girl students
for the THRs were 90,243 for the first three years and 128,783 for the fourth year of the
programme. Comments and notes from interview respondents to the survey, as well as
during the qualitative interviews in schools and with communities (also discussed elsewhere
in this report) consistently emphasized the great and critical importance of the THR to girls
(together with the significant importance of the food to all pupils).
93. Training in food preparation and storage practices: WFP planned to train 1400
cooks over the course of the project. This number was exceeded and a total of 1892 cooks
were trained. The survey findings – which compared WFP schools with the HGSF school –
highlight the importance of the training provided by WFP which has been focused on
schools that receive school meals through the McGovern-Dole programme. Schools that are
not part of a school meals programme (non-FFE schools) receive very little capacity
building with the exception of the work that USAID has done in literacy and language
training, making WFP the main sponsor of capacity building to these schools (naturally this
capacity building also includes other areas such as WASH, SHN, CHILD, etc.).
94. Training in food handling and management for government counterparts: WFP
planned to train 55 government staff in the supervision of food handling and management.
Over the period almost three times this number were covered (134 staff). Staff interviewed
during the school visits demonstrated a good understanding of the food handling practices,
although issues were apparent in storage practices, for example the lack of pallets in half of
the store rooms that were covered by the observation visits (five out of ten schools).
Attainment of outcomes
95. This section discusses the findings from the survey and the qualitative interviews and
observations visits to schools. It covers the six main outcome areas that were identified for
the intervention: access and equity in education; alleviation of short-term hunger;
promoting an essential package of interventions and infrastructure in schools;
strengthening of government capacity; promoting school health and nutrition
interventions; and increasing attentiveness and learning.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
22
Improving access and equity of education for boys and girls, and children from rural
areas
96. The statistical analysis of the survey findings shows that the McGovern-Dole￾supported schools have higher grade completion rates than non-intervention schools. The
quantitative survey data, which are presented in detail in Annex Y, show various statistically
significant results. Notably, they show that grade completion rates in FFE schools are
significantly higher than in non-FFE schools (Table 7 and ¶124 below), which confirms that
SF contributes to improved access to and retention at school. The difference between FFE
and non-FFE schools is of the same magnitude (approximately 10 percent) in Afar and
Somali.
97. There are also differences between Afar and Somali regions, with attendance being
lower in Afar than in Somali, which is consistent with the completion data presented in
Table 7 below. In Somali, however, attendance at non-FFE schools is notably lower than in
FFE schools. The qualitative research corroborates the quantitative findings that SF
contributes to improving access as intended, that it attracts children from other schools,
and that it makes it possible for children to come to school rather than engage in pastoral
activities. For more details see the discussion on EQ 6 below.
Alleviating short-term hunger and retaining boys and girls in school
98. Results regarding eating at school are very consistent with the FFE status, as
recorded from the schools’ own records. For non-FFE schools, children rarely eat at school,
whilst for FFE schools, they report that they always eat at school (Annex Y, Table 35,
question CQ8). This is a striking difference, suggesting that sources of school meals outside
those provided by the FFE have relatively little impact on meal provision, as children in
non-intervention schools are reporting that in Afar, 79 percent, and in Somali 98 percent
never eat at school, whilst in the in-programme schools, they always eat in school.
99. Survey findings show that children in non-FFE schools rarely eat at school, whilst
children in FFE schools report that they always eat at school. An interesting finding on
eating patterns suggests that boys in Afar might actually depend on FFE as their main food
source. It is also in non-FFE schools in Afar where a higher percentage of boys report being
‘quite tired’ as opposed to their peers in FFE schools. Thus, for boys in Afar there appears to
be dependence on the FFE as their main food source where available (see Annex Y), as they
rarely eat at home.
Promoting essential packaged interventions and infrastructure at school
100. Increased access to requisite food prep and storage tools and equipment: WFP has
played an important role in the provision of infrastructure for schools, through
improvement of kitchens and storage facilities, as well as through latrine facilities. The
efforts have also included strengthening WASH facilities, where progress has been made
but where the ET’s visits to schools still underscored significant challenges.
101. The importance of this support shows up in the analysis of the survey data
comparing FFE and non-FFE schools. Food stores are associated with the FFE
programme status. In Afar half the FFE schools have stores; of those that do not, several
note the use of a classroom for storage. By contrast, non-FFE schools in both Afar and
Somali do not in general have stores. In Somali, 90 percent of the FFE schools report having
a store. The distinction for kitchens is even clearer between FFE and non-FFE schools. In
Afar, 66 percent and in Somali 87 percent of FFE schools have kitchens, whereas for non￾FFE schools, the proportions are much lower, none in Afar and only 7 percent in Somali.
Survey results also show that latrine facilities are better in FFE schools in both regions as
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
23
can be seen from Table 5. It should be noted that most schools had separate facilities for
boys and girls.
 Latrine availability in Afar and Somali schools
Concrete slab latrines Separate latrines No latrines
Afar FFE 79% 75% 21%
Somali FFE 100% 87% -
Afar non-FFE 16.7% (33.3% had earth pit
latrines)
66.7% 50%
Somali non-FFE 83% (1 school had an earth pit
latrine)
70% 13%
102. Provision of energy saving stoves: Similar to the promotion of school gardens, no
outcome indicator has been set for the provision of energy-saving stoves to show if using
these stoves has led to the desired result of reduced impact on the environment and less
time spent collecting firewood. Qualitative findings show that fuel-saving stoves are widely
accepted, but more so in Somali than Afar. Cooks interviewed at one school in Afar
expressed concerns about the design of the fuel-saving stove, which they said had a very
small and inconvenient opening for putting in the wood, and said they had put away the
stoves and were using the traditional open fire cooking method.
Increased access to clean water and sanitation services: While access to water is better in
FFE schools than non-FFE schools, substantial room for improvement remains. Water
supply and storage methods were diverse and with no consistent picture distinguishing FFE
and non-FFE schools. Piped water supply was available in about 20 percent of schools, river
or stream water in some 17 percent, boreholes and hand pumps in a similar proportion; rain
water harvest was reported in Somali region by around 30 percent of schools, but not
mentioned in Afar region. The remainder relied on storage in Rottos (plastic tanks), Birkas
(concrete tanks), drums or smaller containers. Delivery by tanker, animal or hand-carried
containers was relied on by some 10-12 percent of schools across both regions. Strikingly,
however, in Somali region 6 out of 30 of the non-FFE schools (20 percent) reported having
no water source or storage at all, whereas all FFE schools had some water supply and
storage.
103. Qualitative findings show that only a minimal number of schools have adequate
WASH. Out of the nine schools the ET visited for qualitative assessment, only the model
school in Somali was equipped with a good standard kitchen, canteen, storage, latrines and
hand-washing facilities. There was no adequate water provision at the other eight schools,
with only half of them providing improvised washing stations outside the kitchen (a plastic
jerry can with a hole to trickle water out) for hand washing (mostly just rinsing without
soap) when water was available from nearby birkas or could be purchased from the town
water supply. The lack of water and sanitation facilities has impacted hygiene in the schools
as students were observed eating their meals without washing their hands. Students also
said that although they have been taught that they should wash their hands after using the
latrine, they do not do so because there are no hand-washing facilities by the latrines and
because of the persistent water shortage. The ET has also observed the impact the absence
of a canteen in most of the schools is having not only on hygiene but also on the
learning/teaching process. With the exception of the model school in Somali that has a
standard canteen and Eliwuha school in Adaa’r woreda of Afar that has turned an old
classroom into a designated dining space, the other seven schools either have their students
eat their meals sitting on the ground outside, inside their individual classrooms, or turn one
of the classrooms into a dining room during meal times, which interferes with the learning/
teaching process.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
24
104. Promotion of school gardens: There is no outcome indicator for the promotion of
school gardens. While the output figure reported for the number of school gardens is much
higher than the target (see ¶76 above), qualitative findings show that except for the model
school in Somali, none of the other schools visited had school gardens. Given the extremely
arid conditions of both regions and the persistent water shortage, school directors and
teachers stated that it never was realistic for them to have gardens.30
Promoting school health and nutrition interventions
105. Survey results show that training and capacity building activities have taken place
and varied in size; they also show – as can be expected – that these activities are
predominantly focused on FFE schools (see Table 40 of Annex Y).
106. The qualitative assessment showed that most schools do not have comprehensive
SHN interventions, though there are some notable exceptions, for example, the model
school the ET visited.
107. Schools struggle to implement SHN trainings mainly due to water scarcity. In some
cases, despite the trainings on proper food storage practices, schools struggle to find a
suitable space to store food commodities, where strong winds have damaged or destroyed
school infrastructure.
Strengthening the Government’s capacity for planning and implementing SF
108. WFP plays a significant role in strengthening government capacity and working with
the government in strengthening capacity in the areas of nutrition, preparation of school
meals, administration of FFE, gender issues and support for girls, as well as school health.
Survey results show that WFP is the main sponsor of workshops and training courses, apart
from USAID who do literacy and language training. WFP’s technical assistance is
considered very relevant and appropriate, at both the Federal and Regional levels.
109. The Government’s Emergency SF is a strong example of government capacity (see
Annex R). The design and rolling out of this programme which has gradually expanded is
fully modelled on WFP’s ‘way of working’ and has included training of staff in the case of the
Somali ESFP (evidence collected by the ET suggested this was not the case in Afar region).
Important issues related to delays in procurement and other logistical challenges remain to
be further improved. While it had other negative consequences for school feeding in Somali,
the unauthorized re-allocation of SF in Somali (see Box 1 above) showed some of the
strengths of capacity built, including a concern for targeting and securing a minimum
quality of interventions (by including training etc.).
110. Areas for improvement include increasing the duration of the trainings which some
interviewees have commented were too short, and further improving supervision to
maximize results of training. More generally, the high turnover of trained school and
woreda staff is a challenge for the proper implementation of the SF programme.
111. Increased engagement of local organizations and community groups: The
2011/2012 baseline survey showed that no schools had CHILD planning teams in place at
the start of the SFP intervention. The first, second and fourth McGovern-Dole biannual
progress reports provided outcome figures of 70 percent, 5 percent and 10 percent
respectively, bringing the total percentage increase in the number of schools with CHILD
30 The dire water situation was clear during school visits with some of the schools having to rely on water
transportation by donkey over a distance of more than 20 km. Given the costs involved, only six jerry cans of
water could be afforded which were used for cooking and cleaning plates.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
25
planning teams in place to 85 percent and exceeding by a big margin the 20 percent target.
In all schools visited by the ET there was active participation of parents and presence of
community members. Participation of community and pupils is evident from survey
findings which show that in Somali, 37 percent of girls and 15 percent of boys bring
firewood to school to support the SFP while the numbers are lower in Afar at 20 percent,
with no consistent differentiation between the sexes (see Annex Y).31
112. Some continued challenges in the implementation of the CHILD approach in both
Afar and Somali regions were noted during the in-depth visits. According to the Regional
BoEs, school directors and PTA members contacted during the evaluation, although the
CHILD manual was adapted to the local context and training was provided on the approach,
the CHILD package was less successful than anticipated because it did not succeed in
mobilizing resources from the community for its implementation. The high mobility of
communities, especially during drought seasons, has made it generally difficult for schools
to have the expected level of community participation and engagement for this approach to
be transformative. This is reflected in the fact that the complete essential package that is
promoted as part of the McGovern-Dole intervention32 is only in place in one of the nine
schools visited in both regions.33 Visits showed that there are very few partners in Somali
and even fewer in Afar that could step in to fill the gap in essential package provision in SFP
schools.
Increasing attentiveness and learning
113. Survey results show that children in FFE schools do better in terms of attentiveness
and learning. There is a small but consistent correlation between SF and attentiveness (see
¶123 below and Annex Y). This suggests that the prospect of a meal functions as an
incentive for children to come to school, and that the nutritious nature of the meals
contributes to concentration in class. Stakeholders reported that children are more
energetic, are ‘bigger’ and tend to win in scholastic competitions, and are aware of
importance of micronutrients.
EQ 5 – What have been the gender and equity dimensions of the programme’s
results?
114. Questions CQ17 – CQ19 of the field survey concern the THR, principally oil. The
answers are consistent with expectations: girls in FFE schools report 100 percent received
the ration, whereas boys did not. Frequency was reported to be 2-3 times per semester,
again consistent with the protocol for the THR. Most of the oil (88-95 percent of replies)
was used for cooking, with a small proportion reporting it was sold or traded.
115. There are clear gender- and equity-related indicators included in the programme’s
results framework (see Figure 5 in Annex K), and field survey results and EMIS data
confirm that the intended outcomes of improved attendance and increased enrolment of
girls, for example, have been met as illustrated in the following paragraphs and in the
discussion of EQ 6 below.
31 In some of the schools visited, the cooks hired to prepare the school meals were responsible for collecting
and bringing firewood while in most schools the provision of firewood was the responsibility of the
community, either sending each of their children to school with one stick of firewood every day or raising
funds from the community to deliver to the school on camel back large supplies of firewood.
32 The essential package includes water, sanitary latrines, deworming, micronutrient supplementation, and
health, nutrition and hygiene education.
33 The school that has all the components of the essential package as part of the MGD intervention is Deghale
School in Somali, which is one of the two MGD model schools.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
26
116. Table 6 below shows the enrolment and Gender Parity Index (GPI) for the schools
included in Table 30 (Annex X) from the EMIS data set. The GPI is more favourable in the
FFE than non-FFE schools. In Afar there is a 10 percent difference in the GPI between FFE
and non-FFE schools. The difference is particularly marked in Somali, where FFE schools
have an overall GPI of 90 percent, whereas in the non-FFE schools it is only 69 percent.
Table 7 and Table 8 below further corroborate this point.
Enrolment for Boys and Girls and Gender Parity Index34
Boys
FFE Schools
Girls Total GPI Boys
Non-FFE Schools
Girls Total GPI
Afar 39,431 34,805 74,236 88% 46,802 36,369 83,171 78%
Somali 36,618 32,801 69,419 90% 304,042 211,073 515,115 69%
Source: EMIS data
117. The survey data (see Annex Y) shows that attendance is lower in Afar than in Somali,
which is consistent with the completion data presented in Table 7 below. In Somali,
however, attendance at non-FFE schools is notably lower than in FFE schools.
Box 2 The effect of THR on girls’ education and welfare
118. Observations also showed improved levels of confidence by girls which were visible
during interviews and also reported by other stakeholders. Stakeholders’ views converge in
the finding that attitudes within communities towards girls’ education and early marriage
have started to change/have changed. Girls are more valued now. Some fear that if the SF
programme and THR end, the effect of this behaviour and attitude change might plateau,
while others say that the achievements will be lost altogether.
34 Enrolment ratio of girls to boys
Data from the survey and the interviews with various stakeholders underscore that the introduction
of the THR has greatly increased girls’ enrolment, maintained school attendance, prevented school
dropout, and narrowed gender gaps in the target schools. The initiative motivated parents and the
communities to send their daughters to school. In-depth interviews suggest that from the
perspective of the community the THR is leading to a decrease in (the risk of) early marriage and is
increasing the income transfer to food insecure households.
The survey data on programme status and grade completion rates shows improvement for the girls
is greater than for the boys, with the girls in the non-FFE schools having higher dropout rates than
for boys. In Afar completion rates are lower than in Somali region for both FFE and non-FFE
schools, and the non-completion rate for girls also notably worse than in Somali. Table 7 reflects
the data.
The effect of THR is evident in the reducing gender gap. The data from WFP shows the GPI has
improved and the enrolment and attendance and literacy have significantly gone up. The GPI for
Afar Region is 0.9 and is almost equivalent to the national GPI (0.91), while the GPI for Somali
region is at 0.86.
THR has brought an effect on income transfer to parents and education benefits to their daughters.
Most parents stated that the programme has contributed to supplementing household food income
to cover the cost of learning materials, clothing for their school age children and of course diet
diversity.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
27
119. The ET found no evidence of a negative impact of the THR on boys. There were some
accounts of families with many girls sharing oil with families who have only boys.
120. A few other gender-related issues were observed at schools:
a) the PTAs’ are composed mainly of men;
b) teachers are predominantly men – some schools do not have a single female teacher;
and
c) in some schools, boys eat first and girls eat later and feeding takes place in a
disorganized manner that marginalizes girls.
EQ 6 – What has been the intended or unintended impact of the programme?
121. Survey and interview data corroborate in finding important and significant impact
level effects of the programme in terms of Food Consumption Scores (FCS), attentiveness
and attendance, and grade completion (with important additional effects on girls). Each will
be discussed in turn below.
122. In both Afar and Somali there are higher FCS35 for both boys and girls for the FFE
schools than for the non-FFE schools. As with other indicators, nutritional status appears to
be somewhat better in Somali than in Afar. Detailed results can be found in Annex Y.
123. As noted above, children in FFE schools have a small but consistently higher level of
attentiveness. In Afar and Somali FFE schools, ‘good’ is the most common category, whilst
for non-FFE schools it is ‘average’ in Afar and for boys in Somali, but remains ‘good’ for
girls in Somali.
124. In addition, the evaluation finds that there is a large difference in grade completion
between FFE and non-FFE schools, with a greater effect on girls than on boys. Table 7
below shows the numbers of students who enrolled and completed their grade year in 2009
(Ethiopian calendar, 2016/17 Gregorian calendar). The confidence limits for the estimated
grade completion rates from the sample are shown. Table 8 contrasts the main differences
from Table 7 and shows the statistical significance of the effects. Of note are the large
differences in grade completion between the FFE and non-FFE schools, with the FFE
schools performing much better. This difference is of the same magnitude in Afar and
Somali (about 10 percent). The improvement for the girls is greater than for the boys, with
the girls in the non-FFE schools showing higher dropout rates than for boys. In Afar
completion rates are lower than in Somali for both FFE and non-FFE schools, and the non￾completion rate for girls is also notably worse than in Somali.
125. Some unintended results include overcrowded classrooms as SF attracts pupils
coming from other schools, which is further aggravated by the conversion of classrooms
into storage spaces or eating spaces. Interviews also made it clear that SF puts a strain on
teaching staff and school management as they accumulate teaching duties with duties
related to the management/supervision of school feeding. In addition, there are some very
young children in school who are either coming to school as pre-school children (attracted
by the school feeding) or being brought in by their older sisters and brother to be able to
share meals. In both cases, these children are often mixed with the higher classes which
impacts on the teaching-learning process.
35 The FCS is calculated conventionally according to WFP standards from the recall questions for food groups
eaten in the last seven days, as shown in the Survey Instrument (Annex U).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
28
 Grade completion by gender for sampled FFE and non-FFE schools
(2009 Ethiopian calendar, Sep 2016 – Jul 2017)
95% confidence limits
In
Region FFE Schools Gender Enrolled Passed % Pass ±% Lower Upper
Afar Yes 24 Boys 2,535 1,997 79% 1.6% 77% 80%
Girls 2,360 1,760 75% 1.8% 73% 76%
No 6 Boys 1,127 767 68% 2.7% 65% 71%
Girls 918 572 62% 3.1% 59% 65%
Somali Yes 30 Boys 11,521 11,032 96% 0.4% 95% 96%
Girls 10,793 10,422 97% 0.3% 96% 97%
No 30 Boys 6,585 5,662 86% 0.8% 85% 87%
Girls 4,484 3,787 84% 1.1% 83% 86%
Statistical significance of performance differences associated with
gender and FFE status
based on data in Table 7
Region Comparison Difference
∆%
Probability
∆% due to
chance
Significant
difference?
Afar Boys - Girls, In FFE 4.2% 0.00051 ***
Boys - Girls, Not in FFE 5.7% 0.00654 **
FFE - non-FFE, Boys 10.7% < 0.00001 ***
FFE - non-FFE, Girls 12.3% < 0.00001 ***
Somali Boys - Girls, In FFE - 0.8% 0.00175 **
Boys - Girls, Not in FFE 1.5% 0.02561 *
FFE - non-FFE, Boys 9.8% < 0.00001 ***
FFE - non-FFE, Girls 12.1% < 0.00001 ***
Statistical significance: * ≥ 95%, ** ≥ 99%, *** ≥ 99.9% probability that observed difference not
due to chance.
KQ 3 – What factors affected the results?
EQ 7 – How efficiently was the programme implemented?
126. The school selection process was participatory. The government focus on equitable
distribution36 resulted in a selection of schools that are geographically spread out. This has
complicated support to the schools. It makes transportation challenging; contributes to
higher monitoring costs; reduces the possibility of exchange and support between schools;
puts additional challenges on government in terms of supervision and support; and also
reduces chances of efficiency in conducting school visits.
127. While WFP generally has very clear guidelines and an efficient procurement system
in place, issues with the timeliness of delivery were found due to regular pipeline breaks,
and the first semester delivery has been consistently late.
128. It was also reported that delivered food is often close to its ‘best before’ date, related
to its being imported from outside the country and delays in delivery internally within the
country (which are a reflection of capacity challenges on the government side) which cause
36 This is reported to be an important principle in Ethiopia to ensure that populations all benefit from the
available resources.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
29
an issue as the food then easily expires while in storage. Furthermore, damaged oil tins
cause the oil (imported from the USA) to go rancid in storage. The tins seem to be of poor
quality and long transport over road damages them easily.
129. There have also been challenges in communication between WFP and the regional
education authorities in the Somali region, which the ET believes played a role in the
unauthorized re-allocation of food to different schools (see Box 1 above).
130. Many of these issues should surface during WFP monitoring visits, but the Ethiopia
CO is struggling with human and financial resources to make these visits more frequent and
the choice has been made to increase the delivery of services (as is seen by consistently
surpassing targeted numbers) rather than increasing oversight. In addition, WFP’s
approach has been to focus on capacity building of regional, district and cluster supervisors
who should provide the monitoring and technical support as part of institutionalization and
sustainability. However, many schools reported not receiving regular monitoring visits,
neither by the Government nor by WFP.
131. The ET did not find evidence of collaboration or complementarities with the work of
other UN or NGO partners, or even collaboration with other WFP programmes in Ethiopia
although opportunities could exist, for example, for complementarities with WFP’s
engagement in social protection. However, as elaborated under KQ1, complementarity with
other government initiatives, e.g. the children’s government, is evident in a few schools.
132. In terms of cost-effectiveness, the 2017 Ethiopia Investment Case report (WFP,
2017d) highlights the cost-effectiveness of school feeding overall as an investment which
yields significant returns. The study found that for the in-kind SF modality (which is used in
Afar and Somali regions) every dollar invested in the school meals programme brings an
economic return of $3.1 over the lifetime of the beneficiary. As noted in the study, the
overall cost per beneficiary for in-kind SF is substantially higher than the HGSF model
which was also assessed.37 The fact that the GoE has been able to use local (in-country)
procurement for its ESFP (including in the regions covered by this evaluation) suggests that
future SF interventions in the region can move progressively to a HGSF model which will
have lower costs and bring benefits to local farmers and cooperatives.
EQ 8 – To what extent did internal/external factors and processes contribute
to the changes and results achieved?
External factors
133. The GoE ownership of and commitment to SF, at both federal and regional levels,
have increased through the ESFP, and various high-level persons in government have
become champions for SF, including H.E. the former First Lady of Ethiopia. While the
government SF policy remains to be approved (delays are reportedly because of reluctance
to commit to the financial commitments it entails), progress has been made in terms of
committing to an SF strategy in adherence to its international, regional and national
37 For the in-kind model, the cost per child was calculated at USD424. The largest contributors to that cost
were commodity cost at USD206, other operational and admin cost at USD100 and external transport at
USD71 per beneficiary over her/his lifetime. The in-kind modality resulted in a gender equality benefit for
girls because of the take-home ration component. This was calculated to be USD79. By comparison with the
HGSF, every dollar invested in the school meals programme brings an economic return of USD6.7 over the
lifetime of the beneficiary and when the impact to the farmers is included the benefit increases to USD7.2.
The cost of the HGSF programmes is almost half of the in-kind programme, thereby increasing the cost￾benefit ratio. An additional value of USD84 is created for local participating farmers for each student over the
programme period of eight years.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
30
commitments (such as inclusive right to food, health and education, expansion of local
school meals programmes using home grown food where possible and effective governance
and sustainable financing of SFPs) which is a positive move. The ESFP, which is largely
implemented with GoE’s own funds, is also a good complementary programme to WFP’s
SFP and builds on many of the lessons learned from the previous and current McGovern￾Dole programmes. It is also indicative of strong commitment to SF.
134. The African Union’s (AU’s) commitment to SF and strengthened continental
ownership of the HGSF approach has been an important factor in enhancing commitment
to SF. Following the 2016 AU Summit and the decision made by Heads of Government to
promote SF as part of their commitment to Human Development, there has been a focus on
ensuring that African countries follow up on the commitment made in 2016 by scaling up
SF across the continent and allocating funds from the national budget to school feeding.
March 1st was declared as the African Day of School Feeding by the AU, member states and
partners at the 2016 Summit, which has ensured continued advocacy for SF involving multi￾sectoral players.
135. The impact of the Indian Ocean Dipole-induced drought, which has been further
compounded by below average spring rains, has left many in the country in dire
humanitarian situations. The condition is particularly worse in the lowland drought belt,
including Afar and Somali regions, where spring rains have been poor or have completely
failed for the third consecutive season, leading to implications for the achievements of the
McGovern-Dole SFP objectives. As can be understood from the outcome and impact level
findings above, feeding schools fare better than non-feeding schools during droughts in
terms of maintaining a higher level of attendance, lower level of drop-out rates, and
minimal or no school closures, although some movement of populations has nonetheless
taken place because of the drought. Furthermore, the upsurge of conflict around the border
areas of Somali and Oromia regions has had an impact on all districts along the regional
borders, exacerbating further the drought-related humanitarian need. Conflict-induced
displacements in 2017 have led to school disruptions and affected the expected outcomes of
increased enrolment, improved attendance, improved access to food by school children,
improved attentiveness, and more girls being educated, thus minimizing the positive impact
the school feeding programme could have had on communities suffering from the drought.
136. High turnover of government staff at all levels, including senior federal level
management as well as regional, woreda and school level administration, has presented a
challenge to the efficiency of the SFP and reduced the effectiveness of staff to properly
manage the programme. Staff turnover at senior federal government level has made it
challenging for WFP to make the progress it had expected on strengthening the policy
environment on school feeding and has undermined its high-level advocacy efforts on the
issue of transitioning to a national SFP.
137. The design of the SFP was based on assumptions concerning the need for a strong
coordinated response in order for the SFP to be effective. However, coordination and
complementary working have been very weak, as evidenced by the absence of the full
essential package in almost all SFP schools. The fact that there are very few education sector
partners working in Somali and even fewer in Afar (particularly in woredas where the
McGovern-Dole SFP is implemented) has hindered WFP’s efforts to form partnerships to
promote the essential package in SFP schools, particularly in improving learning, WASH
and school infrastructure. The perspective voiced by some education sector partners is that
WFP could do better in terms of creating synergies with partners that have ground presence
in the two regions by recognizing the complexities of SF and how it interacts with what
others are collectively trying to achieve in education rather than speaking about it as if it is
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
31
“just bringing in food.” Especially in WASH, the magnitude of the problem in the arid and
drought-affected regions of Afar and Somali calls for a well-coordinated response that
leverages resources.
Internal factors
138. WFP is recognized for its leadership and strong technical capacity in SF, being
referred to by many, including beneficiary communities, as “the backbone” of the SFP in the
country, without which many would not have had the chance to go to school. The CO has
developed a strong relationship with MoE and regional BoEs. This is in part a reflection of
WFP’s long engagement in SF in the country but it is also the result of deliberate
investments WFP has made to further strengthen this relationship as well as build the
capacity of the Ministry and the Regional Bureaus to manage and potentially transition the
SFP. WFP is credited for supporting efforts to strengthen government capacity and the
technical assistance it has provided through the McGovern-Dole programme is considered
very relevant and appropriate.
139. School targeting has not taken into consideration clustering for impact and has been
politically motivated to some extent, with the Regions deciding which woredas would be
included in the initial list of schools to be considered for selection. The government’s view
on equity over need in terms of which schools get targeted has led to SFP resources being
spread too thinly and presented a challenge for WFP to concentrate its inputs for maximum
programme impact.
140. The programme design has not taken into consideration the impact drought has on
the ability of communities to contribute towards the SFP. While community engagement
has been robust for the most part, the requirements placed on communities to pay the
salaries of cooks, construct a kitchen/shed and provide firewood and water necessary for
meal preparations have been excessively cumbersome and put an additional burden on
families during times of drought. The inability of communities to meet these obligations has
caused disruptions to the provision of school meals and in a way minimized the success of
the programme. In the same way, the CHILD package has been less successful than
anticipated because it did not succeed in mobilizing resources from the community in part
for the same reasons.
141. There is weak monitoring and reporting by WFP and by the Regional BoEs. The very
limited resources under the McGovern-Dole programme for monitoring and technical
support have kept WFP from sufficiently staffing the project and have prevented it from
conducting frequent and thorough monitoring of all SFP components. Discrepancies in
reporting and lack of clarity on processes and procedures start at school and woreda levels,
compromising the overall quality and integrity of the reporting. Delays by BoEs in
distributing SF registers38 and logbooks to schools have also created a challenge to timely
reporting. Furthermore, in the absence of a mid-term evaluation, there has not been a
reassessment of the programme during implementation and there has been no systematic
lesson learning.
142. The capacity of the regions to run the SFP efficiently has been rather weak. While
transferring the responsibility of direct implementation of activities from WFP to the
government was done as a way of building the capacity of the government to manage the
SFP and instil a sense of ownership, it has at times compromised the quality and timeliness
of the activities and has negatively impacted results. With the responsibility of non-food
38 The registers are printed by WFP and handed to the education authorities for distribution.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
32
item (NFI) procurement transferred to the regional BoEs, there is a decline in the quality
and quantity of NFIs provided to the schools, with schools not having enough utensils such
as bowls, spoons and cups compared to the number of students and/or the utensils being of
very poor quality (non-durable) so that they break easily or sometimes melt when the hot
CSB porridge is served in them.
143. There are transport and logistical challenges due to the vast geographic coverage of
the programme, the regions’ inadequate transport capacity and weak logistics management,
and delays in getting the food commodities into the country. Pipeline breaks, with food
commodities sometimes arriving at the school one or two months late during the first
semester, have made school-level programme planning and management rather difficult
and have also impacted the performance of the SFP. The delivery of CSB close to its “best by
date” meant that it had to be distributed as THR instead of being prepared and served to the
students at school as per the plan. The quality of the tins that the oil comes in has also been
an issue, as the tins are not sufficiently resistant to the road conditions and arrive at the
school badly damaged and leaking, which not only means less oil available for school meal
preparation and THR but also presents a food safety and hygiene issue.
144. The diversion of food commodities by Somali BoE (see Box 1 above) not only
disrupted the SFP for a month and led to WFP taking over the transportation of food
commodities once the SFP resumed in March 2017, but also affected the trust between
partners which risks affecting the continuity of SF support to this region in spite of the
obvious needs of its population – a need which has been aggravated by the devastating
effects of the drought and the eruption of conflict.
EQ 9 – What was the quality of the monitoring and reporting system? Did
this enhance or impair the performance of the programme?
145. This evaluation has thoroughly reviewed the monitoring and reporting framework
used for this McGovern-Dole project (see in particular the commentary on performance
indicators and data availability in Annex K).
146. In general, the evaluation finds that arrangements for monitoring and lesson
learning have not been adequate. There are various reasons for this.
147. The previous McGovern-Dole grant was not evaluated, which reduced the possibility
for learning from the earlier phase, although the study on THR39 was useful in informing
the design of the THR as part of the present intervention and various lessons learned in a
less formal manner informed the design.
148. The TA baseline survey conducted at the start of the programme suffered from
considerable weaknesses (WFP, 2013a). As such it has not been an adequate tool for
monitoring. In addition, the requirement for a mid-term evaluation was waived at the time
of agreement negotiation and WFP submitted the UNDAF Ethiopia mid-term review report
(Universalia, 2014) in lieu of a formal mid-term evaluation. However, the UNDAF mid-term
review did not specifically report on the McGovern-Dole SFP as it reflected the general
performance of the UN programme in Ethiopia. The combination of not having an end-of￾intervention evaluation, an inadequate baseline and a mid-term report that does not
specifically reflect the McGovern-Dole interventions has reduced the possibility for learning
and mid-course adjustments.
39 Girls’ Initiative Impact Assessment, January 2011, Addis Ababa (WFP, 2011a).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
33
149. In terms of regular monitoring arrangements, the evaluation found that reports have
been produced regularly based on data collected by WFP but that there have been
weaknesses in monitoring, in the quality of data and in the reporting which have limited the
utility of the reports. The WFP CO is supposed to conduct monitoring visits to all target
schools once a semester. Interviews at school level suggest that this is not done consistently.
In addition, the monitoring of the SFP focused mainly on the school meals aspect of the
programme and to the ET’s knowledge no regular monitoring of capacity building activities
that are implemented by the BoE took place.40 As noted in Annex K there are
inconsistencies in WFP’s records/reports (e.g. names of schools) as well as inconsistencies
in terms of reporting outputs, outcomes and beneficiaries. WFP has encountered difficulties
in obtaining accurate and detailed school level records/data, with some not being submitted
for months, reportedly due to lack of clarity on processes and procedures, and others
lacking crucial information, for example the names of girls are provided but not how many
days these girls have attended. Visits by the ET to schools also highlighted that some
schools do not seem to have the standard WFP logbooks for keeping a detailed record of
THR.
150. Finally, it should be noted that the final evaluation of the programme was
commissioned very late which limited its utility in terms of feeding into the early thinking
about the next McGovern-Dole grant.
KQ 4 – To what extent are the project results sustainable?
EQ10 – To what extent will household food security for school-going boys and
girls be sustained without/beyond USDA/WFP funding?
151. Food insecurity still remains high in Ethiopia, with an estimated 8.5 million food
insecure Ethiopians currently requiring relief assistance to meet basic food needs following
the third consecutive poor/failed rains in the southern drought belt (Government of
Ethiopia, 2016b). There is clear documentary evidence that poverty and food insecurity are
key barriers to children’s school attendance and that this results in high dropout during
difficult times in the pastoralist region of Somali and Afar. The ET observed the livelihoods
and food insecurity and coping mechanisms and hardships of the pastoralist region during
their visit. This was further complemented by interviews with children, parents and
teachers who clearly stated that the school feeding serves to alleviate hunger and keeps
children in school. There is also documentary evidence that provision of meals for children
in school reduces the costs of food for the whole family and offsets the cost to the family
caused by children attending school not being able to provide labour.41 Interviews with
beneficiaries consistently confirmed that having access to a daily meal for their children
allowed them to save on the cost of meals and use some of the income freed up by this for
small investments and diet diversity, e.g. in livestock like chickens, which can offer
additional food (i.e. eggs) to the family over a certain period of time. The THR provides an
additional source of income either by making it possible for families to spend money on
other critical needs, or when the oil is sold or bartered to supplement income. However, it
does not seem realistic to say that SF can contribute directly to household food security.
152. The FFE programme has demonstrated positive achievements as highlighted above
and in Annex R and Annex S, and has likely had important protective benefits for
40 Clarification is being sought from the WFP CO.
41 Marginalized, food-insecure people typically spend the majority of their income on food. Smith, L., (2003).
“The use of household expenditure surveys for the assessment of food insecurity”.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
34
beneficiaries who might otherwise have been at risk of child abuse that frequently occurs
during migration/ displacement. The evidence suggests that school feeding and the THR
have been a life line for the children and this has made a csignificant difference in the lives
of children and in particular girls in the pastoralist regions.
153. Many children we spoke to said that the school meals helped them focus and
concentrate in class. Many said they were willing to stay back longer for extra classes and
that has a direct impact on their performance. Teachers underscored the importance of
school feeding in promoting attendance and ensuring concentration. These findings are also
backed up by the results of the EMIS analysis and the quantitative findings which clearly
show that SF schools have significantly higher completion rates than comparable non-SF
schools. The data also showed that this effect was particularly strong for girls, and in the
Somali region (see Annex Y).
154. The evidence reviewed strongly suggests that with the end of the SFP these effects on
households – which live in extremely precarious conditions – cannot be sustained as it
would effectively require households to have the means to fill the financial gap that is being
left through the loss of income which the SF and THR represent. Thus, schools that have
had a long history of feeding have said unequivocally that without feeding their school
performance would drop significantly. Other studies in Ethiopia including the 2016/17
assessment (which included Somali and Oromia) clearly showed that SF reduced school
closure during the drought (of 400 that closed, 158 reopened in Somali, and in Oromia of
137 that closed, 41 reopened) and brought children back to school (Government of Ethiopia,
n.d.-a). Government officials stated that at best if SF and the THR end, the effect on
enrolment and attendance will plateau, as there is some evidence of changing values around
girls’ education. However, most were of the opinion that end of support by McGovern-Dole
will effectively mean that SF and THR will be discontinued and the achievements to date
will be lost.
EQ 11 – To what extent has the programme prepared the Ethiopian
Government and the education system to ensure that they can continue
school feeding in the Afar and Somali region without / beyond USDA/WFP
funding? To what extent will the GoE be able to mobilize and sustain funding
for school feeding for the Afar and Somali regions?
155. There is clear evidence of a growing interest and commitment to establish a
sustainable national school feeding programme in Ethiopia. The collaboration between
WFP and MoE to pilot an HGSF programme in SNNPR and Oromia region in 2012, which
has seen the regional government contributing a matching fund which has grown eightfold
(from 2 million to 16 million Birr (2018)), testifies to this. Other actors are also supporting
SF, for example Ye Enat Weg, a charitable organisation which initiated SF in 2014 under the
leadership of H.E. the former First Lady of Ethiopia with the objective of improving the life
of impoverished children in Addis Ababa.
156. GoE officials informed the ET that though there is no formal SF policy, there is high￾level national commitment. The Parliament, the previous prime minister42 and other senior
government officials all in principle appreciate the importance of SF. The former First Lady
is also an advocate of SF as mentioned above. There are several policies such as education,
SHN, and nutrition which integrate SF (see ¶60ff). Key national policies underscore the
importance of SF in the country.
42 The prime minister resigned on 15 February 2018.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
35
157. During the interviews at senior level within WFP and GoE, it was promising to note
the commitment at the highest level and a substantial financial contribution to HGSF and
ESFP. In addition, this year for the first time the Government has decided to use USD4
million from the World Bank grant for the improvement of girls’ education for SF. An
important achievement in the past years has been the establishment of a separate
directorate on the school improvement programme Quality Education for All.
158. The analysis based on the World Bank SABER43 (Systems Approach for Better
Education Results) tool reported overarching policies for SF in alignment with national
level policies (SABER, 2015). The evaluation’s own assessment of progress against the
SABER baseline (see Annex Z) suggests that Ethiopia has made considerable progress from
2015 to 2018, particularly in increasing commitment at the highest levels of government, in
providing resources through provincial budgets as well as considerable resources from
emergency government funding (related to the drought), and in moving forward with an
HGSF model in regions of the country where this is appropriate. Ethiopia’s experience of
emergency SF suggests that key elements in terms of systems and capacity are in place. The
meetings held with WFP Regional Bureau’s focal person in SF provided the ET with the
procedures, budget and systems of ESF. This is a significant achievement. Now in its third
year, the ESFP has seen a good evolution. Nonetheless, key challenges remain to be
addressed. These include the nature of funding, which in part comes from additional
allocations on an emergency basis and as such is not sustainable, although some regional
governments have been allocating funding to SF from their regular budget. Long lead times
in terms of procurement, logistics and organization because of various constraints related to
internal capacity and the bureaucracy (of the MoE) are a major challenge and defeat the
purpose of addressing emergency needs.
159. WFP’s work on SF is strongly credited with strengthening the government capacity
for planning and implementation of SF. Amongst other activities WFP has supported the
MoE in the development of the national SF strategy, and in the drafting of region-specific
standards for kitchens, stores and canteens. In addition, WFP has supported capacity
strengthening of regional BoEs to work with universities on research and advocacy.44
Deployment of technical assistance is considered very relevant and appropriate at both the
regional and federal levels. WFP through the McGovern-Dole funds has provided financial
and technical support to the regional education management system and to the NSFP.
There is evidence of learning between different models of SF as the McGovern-Dole model
is mirrored in the HGSF and ESFP.
160. WFP’s work is recognised by senior government officials. The biggest achievement is
the establishment of a directorate in the MoE, “School Improvement Programme”, under
which the SF falls. This has enhanced the visibility of SF in Ethiopia and the accountability
of the Government and made it possible to roll out the ESFP. There is a growing recognition
of financial commitment by the Government, although no fixed budget line is allocated to
SF and no funding has been forthcoming at regional level in Somali and Afar, seemingly due
to competing priorities. There have been increasing monetary and non-monetary
contributions from woredas, kebeles and communities in these areas. Overall the HGSF
experience is still being consolidated but it is seen as the future way forward.
43 SABER’s five policy dimensions for school feeding are: Policy Frameworks; Financial Capacity; Institutional
Capacity and Coordination; Design and Implementation; and Community Roles. The scale ranges from latent
to emerging to established to advanced.
44 Research on sustainability of school feeding in Somali region was ongoing at the time of writing with Jigjiga
University as was the development of a documentary film on SF with Somali television.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
36
3. Conclusions and recommendations
3.1 Overall assessment/conclusion
161. The McGovern-Dole programme has contributed at output, outcome and impact
levels to the education of girls and boys in the areas where school feeding has been
provided. In addition, the programme has seen WFP successfully engaging at policy level
with the government, including in terms of developing technical and management capacity
and contributing to stronger commitment to SF. The assessment below highlights many of
these strengths and also identifies a number of areas where improvement might be
envisioned.
162. Appropriateness: The McGovern-Dole programme has successfully targeted
children in the most needy areas of the country. Across the spectrum, interventions have
been relevant to the beneficiaries at the time of design and continued to be relevant in
implementation as confirmed by the stakeholder interviews. The programme appropriately
sought to involve communities and parents in the design, identification of priorities and
management, although with some challenges in practice, in terms of the spread of schools.
163. Attention to gender was part of the design, with a specific focus on the inclusion of
interventions to address challenges to girls’ participation in education which built on
previous experience with the provision of THR. This strategy is appreciated and considered
pertinent by beneficiaries and has contributed to increasing awareness of gender issues,
although at decentralized levels the representation of women in decision-making (e.g. on
PTAs) and as beneficiaries of training has been limited.
164. No specific complaints mechanisms has been put in place for beneficiaries to voice
possible grievances although the assumption was that this would be done through the PTA
and other project monitoring mechanisms. Protection of beneficiaries is not assured and the
imminent closure of project activities without any guarantees of continuity is likely to put
beneficiaries at significant risk as school feeding and THR provide protection against the
risk of child abuse, child labour and early marriage and pregnancy for girls (see Annex S).
165. The intervention was and continues to be well aligned with the broad development
priorities and the sectoral policies of the GoE, which have included increasing recognition of
the importance of school feeding. WFP’s engagement in SF became somewhat less aligned
with the choice to use imported food, although efforts have been made through other WFP
interventions to roll out models of SF that include local purchasing, for which support has
increased. The programme is well aligned with the UNDAF and with the policies of WFP.
166. Results: Quantitative and qualitative data collected by the evaluation underscore
significant and important output, outcome, and impact level results of school feeding and
provide a convincing case for the importance of school feeding for areas that are severely
affected by food insecurity. The evidence demonstrates that school feeding, supplemented
by specific interventions targeted at girl students, improves inclusiveness, participation and
achievements in education. Specifically, the statistical analysis comparing schools with
school feeding with those without shows that:
• In both Afar and Somali regions, enhanced school enrolment is associated with FFE,
with schools with school feeding having a significantly more favourable GPI
compared to those without school feeding.
• Grade repetition rates are consistently lower in FFE schools in Somali region than in
non-FFE schools, although there is limited evidence of this effect in Afar.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
37
• Completion rates are significantly higher for FFE schools than for non-FFE schools
with a difference in the magnitude of 10 percent in Afar and Somali. This
improvement while significant for all students, is particularly high for girls.
167. Additional important outcomes include higher food consumption scores for boys and
girls in FFE schools, as well as better perceived attentiveness and lower absenteeism in FFE
schools.
168. WFP has played a significant role in improving the capacity of government to plan
and implement school feeding. In doing so it has built on its previous experience but has
also made important additional efforts. The provision of technical support to the federal
and regional education offices – an innovation under the present McGovern-Dole funding –
has been particularly important. Government capacity has been demonstrated through the
rolling out of the government-funded ESFP which was modelled on the WFP experience.
Although there is still room for improvement in certain aspects, the ESFP successfully
identified populations that were severely affected and included an integrated package of
training, support and monitoring, as well as the provision of locally procured food to
schools at a critical time of need. Challenges include delays in terms of delivery to schools
related to weak logistics and procurement.
169. In terms of improving facilities, FFE schools have experienced improvements in
terms of classrooms, kitchens, storerooms, water storage and other conditions, due to a
combination of WFP inputs and inputs by government. However, observation visits also
underscore the sub-optimal conditions under which meals are often prepared and served
and the on-going challenges in the provision of a coordinated/integrated package of support
to schools, and critical shortages remaining particularly in the provision of water. The
anticipated mobilization of partners to support the CHILD approach has not materialized.
170. Changes in attitudes by communities towards gender are in evidence at community
level, with stakeholder consultations and beneficiary interviews consistently highlighting
that girls’ education is considered more important now than it was in the past, although
there were concerns voiced about whether these benefits would be sustained beyond the
lifetime of the intervention given extreme poverty and stress on populations due to drought
and conflict which put children and girls in particular at risk.
171. Effectiveness and factors that affected the results: The programme has been
relatively well funded. Beneficiary numbers have corresponded to what was planned, targets
on most activities were exceeded (and in some cases were substantially higher) or reached,
and the programme has benefited approximately equal numbers of boy and girl pupils.
There have been some constraints in terms of provision of food resulting in food being
provided to schools on just over half the planned days although it reached 83 percent of the
output levels.
172. Effectiveness has been enhanced by strong government commitment and leadership
and good coordination and working through government systems, as well a strong drive for
SF through the commitment of the African Union. Ownership of the school feeding is
globally strong. Effectiveness has been reduced by the lack of clustering of schools (which
produced inefficiencies in terms of logistics and complicated support and monitoring),
insufficient attention to quality take-over by government staff, the high turn-over of
government staff, transport and logistical challenges for the government services that have
taken over areas that were previously WFP responsibilities (suggesting that take-over has
not been as successful as expected), and weak monitoring and reporting on WFP’s side.
External factors have also impacted on the results, in particular the prevailing drought in
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
38
the target areas, which has affected the capacity of communities and local partners to
provide support and engage with the programme in the manner that was anticipated.
173. Sustainability: Progress towards sustainability has been made through continued
emphasis on capacity building and establishment of systems and processes for the
management of school feeding. However, some elements of the project design were over￾ambitious, such as the assumption about the capacity of communities to provide
complementary inputs into the programme, or simply not realistic, such as the plans to
establish school gardens in extremely arid regions.
174. Government funding for ESF testifies to the important commitment by the
Government. However, there is no clear perspective of immediate or medium-term financial
sustainably for school feeding. Due to continued stress in the pastoralist regions (through
drought and conflict) combined with the precarious situation of households, and because
schools/government will not continue providing the meals and THR in the absence of WFP
due to financial constraints, achievements are likely to be lost if support is not continued in
the two regions.
175. There is little doubt that investing in school feeding is beneficial and that the whole
community can widely benefit from such programmes. It is very evident that all
constituents value the importance of school feeding. The challenge however is finding the
financial resources to make this programme possible.
176. This evaluation used a theory-based approach and identified underlying assumptions
as part of its ToC. Some of these assumptions did not hold, and point to weaknesses that
need to be considered in any future design and that have informed the formulation of the
recommendations presented below.
Discussion of the assumptions underlying the ToC
177. The evaluation identified a number of assumptions underlying the ToC (see section
1.3). This section of the report reviews each of the assumptions and provides an assessment
of the extent to which these assumptions held true.
• Assumption 1 stated that food would be delivered in a timely manner and in the
required quantities. However, the evaluation found this was not the case. There were
disruptions and diversions of food to different schools in Somali and Afar Regions.
• Assumption 2 related to sufficient funds being made available for the programme by
the donors. The evaluation findings show that funding was a challenge in particular
for monitoring and capacity building.
• Assumption 3 reflected the belief that the Government of Ethiopia would have
sufficient capacity to secure implementation. In practice, limited capacity and high
turnover of government staff has been a big challenge. Nonetheless, the GoE has
made a significant financial investment in school feeding although there is still not a
regular budget line for school feeding in the government budget.
• Assumption 4 related to the education system being effective in absorbing new
students. Evidence suggests this has not been a big problem, though in some cases
schools have been overcrowded because school feeding attracts children from other
areas, and this was compounded by limited infrastructure as classrooms were used as
storage and eating areas and teachers in some schools had to oversee the feeding.
• Assumption 5 suggested that other partners would be able and willing to work
together with WFP to implement the school health and nutrition components. This
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
39
assumption did not hold, and was likely very ambitious. In practice government is
the only partner in McGovern-Dole school feeding with support from the community
in providing water, firewood, cooks and support to other activities. There have been
no substantial partnerships or even synergies with WFP’s own activities in related
areas.
• Assumption 6 concerned the idea that incentives for school feeding would not be
outweighed by other factors (early child marriage, drought). The evidence suggests
that school feeding was able to overcome most of these other factors and has
attracted children to school although the drought and conflict have affected the SFP.
The evidence also suggests that THR has had a positive impact on the risk of child
marriage.
• Assumption 7 relates to the national policy environment being conducive to school
feeding. This assumption has held. Over the period commitment to school feeding
has grown although areas in need of strengthening remain.
• Assumption 8 related to the Government owning the programme and being willing to
provide the resources (human and financial). The assumption has held partially as
there is room for the financial commitment of the GoE to be further increased.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
40
3.2 Recommendations
Recommendation Timing Responsibility Justification
Strategic issues
R1 Prioritize fundraising for the
continuation of school feeding
and a THR for girls to the
schools that were covered under
the McGovern-Dole FFE
programme in Afar and Somali
regions as a matter of absolute
priority.
by the new
school year
(September
2018)
WFP CO with
support from the
WFP RB
High levels of food insecurity and continued drought in both Afar and Somali regions –
as well as the very positive outcome and impact level findings of this evaluation –
justify continued external support to SF and THR to maintain attendance and
enrolment, to promote completion and to support girls’ education and gender equality.
Support to Afar and Somali regions should be continued as a matter of priority to
ensure that the progress which this report has highlighted is sustained and that the
gains in terms of education, gender, nutrition and changing social norms with respect
to girls’ education are not lost. This is also essential from the perspective of protection
of vulnerable populations (see Recommendation 6).
R2 Prioritize finalization of the
national strategy and use it as
the basis to develop an
implementation guideline with
different types of school feeding
scenario, including a separate
guideline for the pastoralist
context.
during 2018 WFP CO with
support from a
consultant (in
collaboration with
MoE)
Future interventions of SF in Ethiopia need to recognize that different conditions exist
in different areas of the country that may make certain models and aspects of school
feeding challenging. To guide this design, and to support the GoE in further rolling out
SF, WFP should develop a typology of districts and different types of SF scenarios. For
example, in Afar and Somali, HGSF is not likely to work due to low production of crops
in the area, limited financial capacity of local government to allocate funds and manage
food procurement, and climate conditions/water scarcity which have made introducing
school gardens very difficult in some areas.
R3 Develop and implement an
adequately funded advocacy
strategy that builds on the key
findings of this evaluation and
previous strategic work to scale
up political and financial
commitment to SF in Ethiopia.
This could include developing
short learning papers based on
the findings of this evaluation.
by December
2018
WFP CO with
support from the
WFP RB
The findings of this evaluation and earlier work45 should be used to scale up advocacy
with government and partners to include SF into the safety net/social protection
programme and to ensure that there is a financial commitment to SF beyond the
education budget only. The advocacy efforts should be supported by a series of short
‘lesson learning’ papers on the impact level findings of this evaluation which can be
drafted on the basis of the annexes of this report and could cover the experience with
THR, and the Government’s experience with the HGSF and the ESFP. These learning
papers should be widely disseminated in Ethiopia and in the region, as well as at the
level of the African Union. This requires a specific advocacy strategy by WFP as well as
dedicated financial and technical resources for the purpose of advocacy.
R4 Advocate with the GoE to
ensure that the government
policies and strategy include an
in the course
of 2018 and
2019
WFP CO This evaluation has highlighted how a well selected and targeted THR provided over a
sustained period of time can make a significant contribution to advancing girls’
education and gender equality. WFP should develop a standalone publication from the
45 Including the findings of the 2017 Ethiopia Investment Case report (WFP, 2017d).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
41
Recommendation Timing Responsibility Justification
incentive for girls’ education in
food-insecure /pastoral societies
using funds from the Productive
Safety Net Programme (PSNP).
findings of this assessment to be used for the purpose of policy and decision making
(see also Recommendation 3).
Design of future WFP SF interventions
R5 Conduct an independent
assessment of needs of
vulnerable populations for the
next McGovern-Dole FFE
programme and use the findings
of this assessment to identify
suitable design options for
school meals in pastoralist
areas.46
in the start￾up phase of
the next
McGovern￾Dole SF
programme
(and at the
latest by
mid-2019)
WFP CO with
external
consultancy
support
In situations of extreme vulnerability and need SF is a safety net intervention and
appropriate measures need to be taken to ensure that populations are not put at risk
when the intervention ends, and that beneficiaries have access to mechanisms for
voicing their concerns during project implementation. In the assessment of the ET, the
termination of the SF interventions under the current conditions of extreme drought
puts girls and boys at significant risk of dropping out of school and girls in particular at
risk of early marriage and exploitation. WFP’s future SF programmes need to be
independently reviewed at the design stage to ensure that strategies for sustainability,
exit, and continued support by government and others are realistic.
R6 Include continued
investment in government
technical capacity for the
logistical management of school
feeding at federal and regional
levels in all future SF support by
WFP in Ethiopia
in time for
the new SF
programme
WFP CO SF programmes need to continue to prioritize government capacity building and
technical assistance so as to continue to encourage gradual takeover by government.
This will require striking an appropriate balance between funds used for school
feeding/THR and the funds needed to guarantee quality training, supervision and
support (which under the current programme were not sufficient). Particularly critical
are the reviewing and strengthening arrangements for logistic management by regional
government so that government response capacity can be increased/scaled up.
Continued use of TA based in government institutions (at central and decentralized
levels) is likely to be a relevant strategy for the next programming period.
R7 Ensure future school feeding
interventions include multi-year
evaluations in the design of the
programme with baseline,
midline, follow-up and endline
surveys, and recommendations
for adjustments as appropriate
during implementation.
In time for
the new SF
programme
WFP CO The current programme was not informed by an evaluation of the previous phase, and
the baseline had considerable limitations. It would be important therefore to ensure
that future WFP SF interventions follow good practice and include baseline, midline
and endline assessments and that the implementation can be informed by findings of
these surveys/assessments.
46 This is in line with WFP’s recognition that food assistance is fundamental to protecting the basic right to life. Hunger can cause and exacerbate existing
protection risks and, in crisis settings, people’s consumption and availability of food is altered. Vulnerability to food insecurity is therefore often linked with
vulnerability to protection risks. WFP’s specific commitment to protection is outlined in its 2012 Policy on Humanitarian Protection.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
42
Recommendation Timing Responsibility Justification
R8 Ensure that future SF in
Ethiopia by WFP includes
attention to specific strategies,
targets and indicators for
increasing the participation of
women and girls in SF design
and implementation stages.
by July 2018 WFP CO with
support from the
WFP RB
This should include commitments and targets from government at different levels for
ensuring that women are represented on all decision-making bodies, as well as the
inclusion of indicators that measure women’s representation and participation in the
revised monitoring framework (see Recommendation 4).
R9 Ensure that the selection of
beneficiary schools under the
next phase of McGovern-Dole
support to SF in Ethiopia is
based on a clustered approach
so that the distances between
schools do not make monitoring
overly onerous or complicated.
in time for
the new SF
programme
WFP CO with
support from the
WFP RB
The evaluation has highlighted multiple challenges arising from the spread-out nature
of the schools under the current grant. This has made monitoring and support
extremely complex and has impacted on the efficiency of the operation and on the
capacity to engage with the more demanding aspects of the project around support to
PTAs, developing partnerships, monitoring the THR, among others. The next grant
should include a condition that schools be selected in a more clustered manner to
facilitate monitoring and support. The selection of specific areas to cluster from could
be based on indicators of food security and educational performance.
Operational issues
R10 Strengthen the monitoring
and reporting capacity of WFP,
and regional and woreda level
BoEs in the area of SF so that
the data collection allows for
efficient management of SF.
by July 2019 WFP CO with
support from the
WFP RB
The evaluation has highlighted various challenges in the monitoring of SF. The next
phase of the programme needs to strengthen the capacity, logistics and systems
required to effectively monitor the programme and to ensure that the monitoring
results are used to inform decision-making.
R11 Ensure improved
coordination with other
education sector stakeholders
and working in
complementarities for greater
impact and critically assess
capacity of communities to
support the CHILD approach.
by July 2019 WFP CO with
support from an
external
consultant
Linkages are critical and WASH has to be an integral component of SF programme.
WFP should ensure that it comprehensively maps opportunities for collaboration and
seeks out non-conventional partners where appropriate. A partnership assessment by
an external consultant could assist in facilitating this work. In addition, the evaluation
findings point to the importance of assessing community capacity to provide support to
the programme, especially in regions that are under duress due to drought or other
factors.
43
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
ANNEXES
Annex A Terms of Reference
Annex B Methodology
Annex C Theory of Change
Annex D Full Evaluation Matrix
Annex E Organisation of the evaluation
Annex F Team Roles and Responsibilities
Annex G Detailed evaluation timeline
Annex H Detailed field work schedule
Annex I List of persons met during the Inception Mission
Annex J People interviewed (observation and main field visits)
Annex K The McGovern-Dole Programme in Ethiopia
Annex L Other school feeding programmes in Ethiopia
Annex M Map
Annex N Gender
Annex O Equity
Annex P School Health and Nutrition
Annex Q Nutrition
Annex R Case Study 1: The GoE Emergency School Feeding Programme
Annex S Case Study 2: Girls’ Initiative Programme – Take-Home Ration
Annex T The Survey – Data Collection Tools and Methods
Annex U Survey Instrument
Annex V Stakeholder analysis
Annex W Approach to interviews and school level observations
Annex X Results of EMIS data
Annex Y Field Survey Results
Annex Z Assessment of progress against SABER framework
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
44
Annex A Terms of Reference
FINAL EVALUATION
WFP’S USDA McGovern - Dole International Food for Education and Child Nutrition Programme’s
Support in Afar and Somali regions in Ethiopia 2013 to December 2016
1. Introduction
1. These Terms of Reference (TOR) are for end of project evaluation of the school feeding programme that
has been implemented by the World Food Programme (WFP) in Afar and Somali regions. The evaluation
is commissioned by the WFP Ethiopia country office (CO) and will cover the period from 2013 to
December 2017.
2. The TOR was prepared by WFP Ethiopia based on reviews of various documents vis-à-vis the project and
consultations with stakeholders. The purpose of the TOR is twofold. Firstly, it provides key information
to the evaluation team and guides them throughout the evaluation process; and secondly, it provides key
information to stakeholders about the proposed evaluation.
2. Reasons for the evaluation
3. The justifications for the evaluation being commissioned are as presented below.
2.1 Rationale
4. The current United States Department of Agriculture (USDA) McGovern – Dole (MGD) support towards
school feeding programmes in the Afar and Somali regions would have come to an end in December
2016. However, the initiative has now received an extension through 2017. The end of project evaluation
is therefore being undertaken now to assess the extent of the contribution of the USDA MGD-funded
activities to equity in access to primary education, contribution towards stabilizing attendance for both
boys and girls, and narrowing the gender gap in primary education in Afar and Somali regions.
5. This evaluation comes at a critical time in WFP’s engagement with the government in developing a
national school meals programme. Undertaking such a rigorous evaluation will help determine and
identify the definite lessons of the project in the two regions. Furthermore, the findings will be
instrumental in strategic planning for school feeding in the two pastoralist regions of the country during
the design of WFP’s support to the national school feeding programme.
2.2 Objectives
6. Evaluations in WFP serve the dual and mutually reinforcing objectives of accountability and learning:
7. Accountability – The evaluation will rigorously assess and provide evidence on the performance and
results of the MGD school feeding operation. As such, the final evaluation will assess the results of the
project against the following objectives: 1) Increase in enrollment and attendance in the two regions to
achieve the government target of 100% enrollment, 2) Improvement in management capacity of school
health and nutrition programmes, 3) Improvement in school infrastructure by gauging the effectiveness of
a participatory planning tool called Children in Local Development (CHILD) in contributing towards
improving the school environments and increasing community engagement in schools in the pastoralist
communities.
8. The evaluation will also assess and provide evidence on the extent to which the different activities
planned under the agreed intermediate results have contributed towards achieve an increase in use of
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
45
health and dietary practices of school age children in Afar and Somali region. Since the intended results
and achievements of the programme included in the commitment document, this planned evaluation will
assess and report on the performance and achievements of the programme for enabling WFP Ethiopia to
meet its accountability to donors, beneficiaries, and other stakeholders.
9. Learning – The evaluation will determine the reasons as to why certain changes did or did not occur,
derive best practices and pointers for learning. It will provide evidence-based findings to inform
operational and strategic decision-making. In addition, the findings and recommendations will be actively
disseminated to main stakeholders and learnings will be incorporated into relevant lesson sharing systems
including in the design of new school feeding interventions.
2.3 Evaluation stakeholders and users
10. The different stakeholders both inside and outside of WFP have an interest in the results of the evaluation.
Some of these stakeholders will be asked to play a role in the evaluation process. Table 1 below provides
a preliminary stakeholders’ analysis, which should be deepened by the evaluation team during the
inception phase.
11. Accountability to affected populations is tied to WFP’s commitment to include beneficiaries as key
stakeholders in WFP’s work. As such, WFP is committed to ensuring Gender Equality and Women’s
Empowerment (GEEW) in the evaluation process through participation and consultation of women, men,
boys and girls from different groups.
12. The primary users of this evaluation are internal (WFP) and external (partners and donors) stakeholders.
The WFP Ethiopia CO internally, and its partners and donors externally will use the findings and
recommendations of this evaluation in their respective decision-making processes. The partners and
donors who are external users of the evaluation findings and recommendations include:
• The Federal Ministry of Education is the principal government point of contact for the project
implementation and strategic consultations with WFP.
• The Planning and Resource Mobilization Directorate is responsible for the overall coordination
functions, reporting and liaising as well as monitoring of the operation.
• The Ministry of Finance and Economic Development leads the major coordination mechanisms for
interventions under United Nations Development Assistance Framework (UNDAF) and is contact
point for coordination, planning, budget allocation, finance transfer and reporting through its regional
and woreda structures.
• The Regional Education Bureaus of Afar and Somali manages the food distribution, implementation
of complementary capacity development activities and monitoring and reporting.
• The school directors, food management committee and CHILD47 planning teams (where applicable)
are the main actors in the programme implementation.
47 CHILD is an acronym for Children in Local development. It is CHILD is Community- led participatory planning
approach that aimed to transform school to the local development center in their community. The approach is
aimed to improve the school premises, community ownership of education and to make environmental
improvements that support the FFE activity as well as improve the awareness of environmental sustainability of
children and their families. The concept is further developed to the planning tool to build the planning capacity of
the education sector at the grassroots level to integrate children education in to development programme.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
46
• Given the core functions of the Regional Bureau (RB), the RB is expected to use the evaluation
findings and recommendations to provide strategic guidance, programme support, and oversight.
• WFP HQ may use evaluations for wider organizational learning and accountability.
• Office of Evaluation (OEV) may use the evaluation findings, as appropriate, to feed into evaluation
syntheses as well as for annual reporting to the Executive Board.
• As WFP Ethiopia is accountable to inform the donors and beneficiaries at large about the
performance and results of the project, the target beneficiaries and communities as well as the donor
(USDA) are considered as users of this evaluation. This evaluation will make the target beneficiaries
and communities aware of the relevance, benefits, effectiveness and sustainability of the project. The
donor may use the evaluation as an instrument for decision making for future funding in Ethiopia
along with drawing lessons for making financing decisions for similar projects elsewhere.
TOR Table 1: stakeholders and interest in the evaluation
Stakeholders Reasons for interest in the evaluation and the
likely uses of the evaluation report for the listed
stakeholders
INTERNAL STAKEHOLDERS
WFP Ethiopia Country Office (CO) WFP Ethiopia CO has a direct stake in the
evaluation and is a primary stakeholder with an
interest in learning from the experience to inform
decision-making. The evaluation is also important
for internal accounting and informing its
beneficiaries and partners about the performance
and results of the operation.
WFP Regional Bureau Nairobi (RB) The RB is responsible for both oversight of COs
and technical guidance and support, and therefore
has an interest in an independent account of the
operational performance as well as in learning
from the evaluation findings to apply the lessons
to other country offices.
WFP Headquarters (HQ) WFP HQ will be users of the findings with an
interest in the lessons that emerge from
evaluations, particularly as they relate to WFP
strategies, polices, thematic areas, or delivery
modality with wider relevance to WFP
programming.
Office of Evaluation (OEV) OEV has a stake in ensuring that decentralized
evaluations deliver quality, credible and useful
evaluations respecting provisions for impartiality
as well as the roles and accountabilities of various
decentralized evaluation stakeholders as identified
in the evaluation policy.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
47
Stakeholders Reasons for interest in the evaluation and the
likely uses of the evaluation report for the listed
stakeholders
WFP Executive Board (EB) The WFP governing body are potential users of
the evaluation. The EB may have an interest in
being informed about the effectiveness of WFP
operations. This evaluation will not be presented
to the EB but its findings may feed into annual
syntheses and into corporate learning processes.
EXTERNAL STAKEHOLDERS
Government - Federal Ministry of Education,
Federal Ministry of Finance and Economic
development, Regional Education Bureau,
Regional Finance and Economic Development
Bureau
The Government is a primary stakeholder and
user of the evaluation with a direct interest in
understanding whether WFP activities in the
country are aligned with its priorities, harmonized
with the action of other partners and meet the
expected results. In particular, the findings and
issues related to capacity development, handover
and sustainability will be of interest.
Beneficiaries (school directors, food management
committees, CHILD planning teams, student,
school community)
The beneficiaries are the schools (including the
management structures), the students (boys and
girls) and their families. They are primary
stakeholders, providers of information and users
of the evaluation with the right to know whether
their voice and views are reflected correctly in the
report. The beneficiaries should not just serve as
data providers but need to have a larger role as a
main stakeholder and should be consulted
throughout the process. The outcome will also
help the community to understand 1) the role of
the community in the programme and 2) how their
engagement will further be strengthened.
Donor (USDA) This programme is funded by USDA and they are
primary stakeholders and users of the evaluation
with an interest in knowing whether their funds
have been spent efficiently and if WFP’s work has
been effective and contributed to the agreed
results.
United Nations Country Team (UNCT) The school feeding programme is part of
UNDAF. The UNCT therefore has an interest in
ensuring that WFP operation is effective in
contributing to the UN concerted efforts.
Civil Society Organizations Some NGOs like Save the Children have recently
engaged in school feeding and are seeking
technical support from the programme. The
outcome of this evaluation will provide evidence
and information for effective implementation and
design of their programme in these regions.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
48
3. Context and subject of the evaluation
3.1 Context
13. With a total population of over 100 million people growing at a rate of 2.88% per annum, Ethiopia is the
second most populous nation in Africa. Around 84% of the Ethiopian population lives in rural areas and
are mostly engaged in small scale agriculture and pastoralist activities.
14. During the past decade, Ethiopia has implemented successive national plans to accelerate economic
growth with focused efforts on poverty reduction, resulting in an average of 11% economic growth.
Ethiopia also managed to reduce population living under poverty from 38.7% in 2004/05 to 26% in 2013
(GTP/Annual Progress Report 2012/2013). However, the economic growth is slowing down compared to
the last decade with the most recent data, for 2012/13, revealing a GDP growth rate of 9.7% from 11%
GDP in year 2011. Shortfalls in the performance of agriculture and the stagnant development of the
industry sectors48 are the main reasons for the slowdown of economic growth.
15. In the education sector, the government has continued to expand access to achieve universal primary
education in line with the Education for All (EFA) goals. Among the different strategies employed,
expansion of primary schools through Alternative Basic Education (ABE) is an important one. The
children who enrolled in ABE increased from 11,000 in 1996 to 32,048 in 2014. Primary School
Enrollment has risen from 3.7 million in 1999 to 18 million in 2013/2014. Ethiopia has made progress in
the Net Enrollment Rate (NER) for primary schools (Grades 1–8), from 22% in 1994 to 92.6% in
2013/201449. At national level, 21.4% of pupils were enrolled in grade 1 in 2005 E.C (2012/13). Of those
students, 3% dropped out (left school) before attending the final exam which would enable them to
transfer to grade 2 in 2006 E.C (2013/14). Many boys and girls, particularly in emerging regions and
pastoralist areas drop out of school at early grades because of poverty. It is estimated that about 3 million
primary age children remain out of school (ESDP IV). A significant number of out of school children are
from emerging regions where most pastoralist and semi-pastoralist in Ethiopia are located (UNICEF,
2012). Beyond primary education, the education and training policy developed in 1994 has been
strengthened through a multiyear Education Sector Development Programme (ESDP). Since the first
phase (1997), the country has registered progressive achievements in general education, technical and
vocational education and higher education sectors.
16. The two regional states (Afar and Somali), which have benefited from the MGD supported School
Feeding Project, have made progress over the last four years. For example, the Somali region has seen
some successes in creating access to primary education. The net enrollment rates in primary education
have progressively increased from 50.9% in 2010/11 to 63.7% in 2011/12, and to 84.64% in 2015/16
(Education Abstract). However, the progress was slow in the predominantly pastoralist region of Afar
with 55.49% net enrollment, which fell behind the high national average. Although the progress in Afar
compared to Somali region is lower, the improvement is evident with a 20% increase in net enrollment
from 2011/12 to 2015/16, of which 23.16% was an increase for boys and 16.19% for girls.
48 National human development report 2014, Ethiopia
49 Education annual abstract EC 2006 (2013/2014)
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
49
17. Despite the progress that has been achieved during recent years within the education sector, the sector still
faces persistent challenges that, among others, include tradeoffs between access, quality and regional
disparities. The high dropout rates in almost all levels especially in emerging regions and pastoralist areas
are mainly because of poverty. The Gender Parity Index (GPI) for Afar is 0.9 and is almost equivalent to
the national GPI of 0.91 while the GPI for Somali is 0.83 (Education abstract 2015/2016). There has also
been progress with dropout rate reduction - from 16% in 2012/13 to 9.9% in 2015/16 nationally. The
reduction in dropout rates for girls and boys were 15% and 17% respectively.
18. Food insecurity and poverty, though overall declining, remain at high rates, and an estimated 7 million
school age children are living in food insecure parts of the country. Malnutrition is a serious public health
concern for the country with 44% of the children under five stunted. Vitamin A, iron, iodine and zinc
deficiencies can be found in many parts of Ethiopia.50 Malnutrition and hunger also hamper
concentration in class and diminish the opportunities to develop the full mental potential of a child.
19. Ethiopia is currently experiencing its worst drought in 30 years according to the United Nations, with
levels of acute need across all humanitarian sectors having already exceeded levels seen in the Horn of
Africa drought of 2011. As per the Federal Ministry of Education (MoE) of Ethiopia and the UN Office
for the Coordination of Humanitarian Affairs (OCHA) report, the current expanded food insecurity,
malnutrition and disrupted livelihoods have greatly affected six regions of the country. As a result, the
education sector has also been seriously affected disrupting the children’s education. As per the rapid
assessment conducted by the MOE, Regional Bureau of Education Cluster in August 2015, over 3 million
school children, mainly from the poorest quintile of the population are affected by the emergency in six
regional states (Afar, Ethiopia Somali, Oromia, Amhara, Tigray, Southern Nations Nationalities and
People (SNNP) and Dire Dawa).
20. Recognizing the gradual spill over impact of the drought on the students in affected areas, the MoE
developed an education in emergency response plan as an integral part of the broader government-led
response to the El Niño crisis. The ministry required over USD 53 million to provide educational
supplies, WASH facilities, school feeding programme, psychosocial support and establishing temporary
learning spaces to prevent the 3 million children from risk of dropping out of schools. However, response
from the development partners for the government’s education emergency response plan was almost nil
and since then, the Federal Ministry of Education has allocated USD 27.5 million to provide school
feeding for the 3 million children affected by the drought for one semester (February - June 2016).
3.2 Subject of the Evaluation
21. WFP, in collaboration with the Ethiopian MoE, has been implementing school feeding interventions for
20 years. Over this period, the intervention has successfully contributed to the increase in school
enrollment and attendance, the decrease in the gender gap in enrollment and the improved ability of
pupils to concentrate in class. The school feeding intervention in Ethiopia is responding to the
government Growth and Transformation Plan (GTP) and Education Sector Development Plan (ESDP IV)
strategy of expanding access and reducing inequalities in access to primary education. The programme is
also part of the United Nations Development Assistance Framework (UNDAF) 2012-2016 which places a
50 Ethiopia Mini Demographic and Health Survey ,2014
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
50
premium on basic social services outcome of improving access to quality education and alleviating short￾term hunger and malnutrition. School feeding is included in the social protection policy as social safety
net instrument, in the national school health and nutrition strategy as school based health and nutrition
interventions and in the national nutrition programme as one of the initiatives to strengthen nutrition
sensitive interventions in the education sector.
22. WFP supported school feeding interventions in Ethiopia are aimed at assisting primary school children
attend school without feeling hungry and at the same time support the communities to effectively plan for
a more child-friendly school environment. About 450,000 pupils in the four regions (Afar, Oromia,
SNNPR, Somali regions) are currently receiving one third of their kilocalories from the programme. Out
of these, 263,000 children are in Afar and Somali regions get support from MGD. Whereas, the rest
beneficiaries in Oromia and SNNPR receive support from other bilateral and private donors.
23. In line with the ESDP, the WFP school feeding programme in Ethiopia prioritizes the pastoral areas of
Afar and Somali regions, where access to education and gender balance continue to be behind other
regions. Afar and Somali regions were selected because they had a low gross enrolment rates of 43.7%
and 75.1% respectively in the 2011-12 school year and was a concern as it was affecting Ethiopia’s ability
to achieve universal primary education and eliminating gender disparity at primary education by the year
2015.
24. USDA, through the MGD International Food for Education and Child Nutrition Programme is supporting
school feeding in the two regional states of Afar and Somali with USD 40.7 million over a period of four
years in support of 289,000 students in 583 primary schools. This accounts for 52% of WFP School
Feeding programme targeted schools in the country currently. The programme provides one hot meal per
day for all targeted children. In addition to the school meal, it provides an additional incentive (vegetable
oil as a take-home) for girls in the pastoralist regions with a condition of 80% attendance to encourage
parents to send their children to school on regular basis. The meal is composed of corn soya blend,
vegetable oil and salt which is equivalent to 647 Kcal per day, while the take home ration consists of 8
liters of vegetable oil per semester.
25. The multi-year funding further includes a large capacity building component of about USD 1.3 million.
The funding is used to develop institutional, human and infrastructural facilities. The capacity building
includes improving the hygiene and sanitation practices in the kitchen, developing school health and
nutrition planning and implementation capacity of the education sector, promoting health and hygiene
clubs, promoting school gardens, providing energy saving stoves and providing training to improve the
food handling and management capacity of the implementing partners.
4. Evaluation Approach
4.1 Scope of the Evaluation
26. The evaluation will cover the whole period of the USDA MGD project from 2013-2016 in Somali and
Afar regions. The evaluation will assess the performance and results of the programme, the processes of
the programme implementation and sustainability of achieved results in the two regions supported with
USDA funding. The analysis will also be gender disaggregated to clearly outline the different factors
affecting or affected by gender dynamics, and involve a detailed gender analysis.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
51
4.2 Methodology
27. The final evaluation will use the internationally agreed upon criteria: relevance, effectiveness, efficiency,
impact and sustainability. Although impact is a long-term effect or outcome of an intervention/project,
this evaluation should try to single out the contribution of this four-year project in the intervention areas
by considering the counterfactual of what would happen if the project had not been implemented. Any
intended or unintended effects/outcomes along with the negative and positive consequences due to the
intervention should be assessed.
28. WFP is aware that identification of a control group might not be possible at this stage. If no control group
can be identified, during the inception phase the selected firm will need to develop an alternative
approach to guarantee a rigorous assessment of the results. Such method will be validated by the
Evaluation Committee with the support of the Evaluation Reference Group before the beginning of the
field phase. The Evaluation Reference Group consists of the relevant stakeholders including USDA (see
annex 5).
29. The methodology proposed should also clearly outline how GEEW principles were integrated and
addressed in the design, planning and implementation and what results were achieved.
30. The evaluation team will be required to develop a theory of change for the programme based on the
attached results framework (see Annex 3) and evaluate the different causal pathways. The evaluation will
use mixed methods (quantitative and qualitative) and triangulate information from different methods and
sources to enhance the reliability of findings.
31. The bidding consulting firms, using their specific expertise and experience, will develop and submit a
technical proposal indicating a concise but clear methodology to be employed for the evaluation using the
attached templates.
32. The quantitative and qualitative data to be used for this evaluation will be collected from a combination of
desk review and primary data collection from representative sample schools from both intervention and
non-intervention schools in the two pastoral regions (Afar and Somali). The sample from non-intervention
schools will be used to establish attribution of results to the MGD programme. Selection of the control
group must be performed using a matching method of choice that guarantees similar observable
characteristics in the treatment and control groups. Data from comparison and intervention schools will be
used to present a counterfactual of what the results would have been without the MGD intervention. In
this regard, it will be important for proposals to outline a methodology that provides a robust
counterfactual. Ideally, a quasi experimental design should be adopted to clearly address attribution of the
results to the MGD project.
33. The bidding consulting firms/companies should provide calculations and justifications for an adequate
sample size for both intervention and comparison schools that is statistically representative while putting
into consideration financial and time constraints. In sampling, the baseline line will be expected to ensure
a 95% confidence level. In line with this, the firms/companies should propose a methodology that
provides the most robust findings with the available and generated data while considering any data gaps.
34. During the inception phase, the consulting firm will develop a detailed methodology including:
identifying a counterfactual group, sample design, sample size calculations, and method of analysis. The
viability of the methodological design of the evaluation (together with the feasibility of including a
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
52
credible control group) should be assessed at this stage and eventual changes suggested to ensure the
robustness and credibility of the exercise.
35. Therefore, the quantitative and qualitative data which require for answering the evaluations
criteria/questions and for purpose of triangulations will be collected from the sampled schools. The
evaluation team should clearly outline in an evaluation matrix, what questions will be answered using
which methods and sources of information. This will be developed during the inception stage and agreed
on by all stakeholders.
36. In addition to the data that will be collected at the school level, it is also expected that the selected
evaluation team collect data from relevant, regional-level stakeholders for filling data gaps and for
triangulation of information from various sources. The evaluation will also use the baseline data to
compare the results before and after the intervention. WFP is aware of gaps and uneven quality in the
baseline data. The selected evaluation team will need to assess the data gaps and develop a methodology
to reconstruct baseline values.51 The evaluation should ensure gender analysis and integrate it well into
the methodology.
37. Bidding companies are expected to present the proposed methodology for the evaluation in their
respective proposals. Bidders are encouraged to come up with alternative and cost-effective evaluation
methods that provide sound evidence-based recommendations to inform future programming and policy
formulations. The methodology should take into consideration realities on ground including geographical
constraints, data gaps and time constraints and ensure that they offer a realistic and robust approach that
provides the necessary evidence to draw conclusions and lessons from the programme.
38. The bidding companies are expected to expound on the specific approach and methodology that will be
used to answer all the evaluation questions below. Additionally, the consultant should illustrate how the
comparison between intervention and comparison schools will be undertaken. Gender analysis should be
part of all the relevant evaluation questions.
4.3 Evaluation Criteria and Questions
51 Regional Bureau will provide support to work with the evaluation team on developing sound definitions of
indicators
Efficiency
• To what extent did the project attain peak pipeline performance?
• What were the complementarities between the programmes interventions and
Relevance
• To what extent were interventions appropriate to the needs of boys, girls, and parents of
school-age children?
• To what extent were interventions aligned with relevant national policies, including
sectoral policies and strategies?
• To what extent were interventions coherent with WFP strategies, policies and normative
guidance?
• To what extent were the interventions aligned with the needs of other key stakeholders?
• To what extent was the intervention design based on sound gender analysis?
• To what extent was the design and implementation of the intervention sensitive to
GEWE?
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
53
interventions of relevant humanitarian and development partners as well as other WFP
country office interventions in the country? How did these complementarities contribute
to savings and efficiency?
• Did the interventions within the programme offer the best value for money for WFP?
• To what extent were project management practices and tools adequate to implement the
project?
• Was project resource adequate and available on time to implement the activities as
planed?
• What are the main lessons learned in terms of future of similar projects?
• What are some of the key best practices from this project?
Effectiveness
• What are the main expected and unexpected results of the programme?
• To what extent has this programme achieved the planned outcomes and objectives on:
• Alleviating short term hunger and retaining boys and girls in school
• Promoting essential packaged interventions and infrastructure at school
• Strengthening the governments capacity for planning and implementing school
feeding
• Increasing attentiveness and learning
• Promoting school health and nutrition interventions in primary school for boys
and girls
• Improving access to and equity of education for primary school boys and girls
particularly for children from rural areas of emerging regions and underserved
areas.
• Did the assumptions hold true?
• To what extent did internal/external factors and processes contribute to the changes and
results achieved:
• Internal factors (within WFP’s control): the processes, systems and tools in place to
support the operation design, implementation, monitoring/evaluation and reporting;
the governance structure and institutional arrangements (including issues related to
staffing, capacity and technical backstopping from RB/HQ); the partnership and
coordination arrangements; etc.
• External factors: the external operating environment; the funding climate; external
incentives and pressures and overall sustainability of the programme.
• To what extent is the country taking ownership of the programme? (e.g. demonstrated
commitment and contribution to the programme);
• What is the national readiness to implement the programme? (e.g., demonstrated capacity
at central and sub-national levels to manage the programme?
Impact
• What are the changes that the intervention contributed to?
• Counterfactual analysis of what those outcomes would be in the absence of the
intervention project?
• Cause – effect relationships of key results – What are the impact pathways?
• What are the intended and unintended results/outcomes?
• What are the negative and positive consequences of the intervention?
Sustainability
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
54
• To what extent will the GoE be able to mobilize and sustain funding for school feeding for
the Afar and Somali regions?
• To what extent has the programme prepared the Ethiopian Government and the education
system to ensure that they can continue school feeding in the Afar and Somali region
without / beyond USDA/WFP funding?
• To what extent will household food security for school going boys and girls be sustained
without / beyond USDA/WFP funding?
•
4.4 Data Availability
34. The evaluation will review and use all available secondary data which are related to the programme such
as standard projects reports, previous evaluation, monitoring reports, baseline reports, UNDAF reports
and special reports. In addition, the project results framework (see annex 3); the corporate and project
specific indicators. The government EMIS and policy documents; and other UN agency documents will
also be reviewed as the data sources for the evaluation.
35. The bidding consulting firms should be cautious that there may be data gaps and the available secondary
data will be of poor quality and therefore indicate strategies for remedial in its respective proposals as
well as in proposing alternative methods that will enable filling data gaps or improving the quality of
available secondary data.
36. Some of the cited examples of data gaps and poor quality that the bidding consulting firms should be
aware of and should devise strategies or select appropriate methods for remedies are:
• Limited or unreliable datasets in the schools and government EMIS (Education Management
Information System)
• Limited quality of baseline data
• Some of the data are only available in local languages
• High staff turnover resulting in limited institutional memories
• Lack of data for some of the indicators during the baseline stage
• Incomplete indicators for measuring results in the logframe
• Poor quality of monitoring and progress reports - output and outcome data
• Lack of assumptions and risks in the project document or results framework
4.5 Quality Assurance
37. WFP’s Decentralized Evaluation Quality Assurance System (DEQAS) defines the quality standards
expected from this evaluation and sets out processes with in-built steps for quality assurance. DEQAS is
closely aligned to the WFP’s evaluation quality assurance system (EQAS) and is based on the UNEG
norms and standards and good practice of the international evaluation community and aims to ensure that
the evaluation process and products conform to best practice.
38. DEQAS will be systematically applied, where applicable, to this evaluation. The Evaluation Manager will
be responsible for ensuring that the evaluation progresses as per the DEQAS step by step process guide
and for conducting a rigorous quality control of the evaluation products ahead of their finalization.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
55
39. WFP has developed a set of quality assurance checklists for its decentralized evaluations. This includes
checklists for feedback on quality for each of the evaluation products. The relevant checklist will be
applied at each stage, to ensure the quality of the evaluation process and outputs.
40. This quality assurance process does not interfere with the views and independence of the evaluation team,
but ensures the report provides the necessary evidence in a clear and convincing way and draws its
conclusions on that basis.
41. The evaluation team will be required to ensure the quality of data (validity, consistency and accuracy)
throughout the analytical and reporting phases. The evaluation team should be assured of the accessibility
of all relevant documentation within the provisions of the directive on disclosure of information. This is
available in WFP’s Directive (#CP2010/001) on Information Disclosure. The evaluation team should be
assured of the accessibility of all relevant documentation within the provisions of the directive on
disclosure of information. Refer to WFP Directive (#CP2010/001) on Information Disclosure.
42. Concerning the quality of data and information and evaluation products, the evaluation team should
systematically check accuracy, consistency and validity of collected data and information and
acknowledge any limitations/caveats in drawing conclusions using the data. If the expected standards are
not met, the evaluation team will, at its own expense, make the necessary amendments to bring the
evaluation products to the required quality level.
5. Phases and Deliverables
43. The evaluation will proceed through the following phases. The deliverables and milestones for each phase
are as follows:
44. Preparation: This phase of the evaluation will be carried out by the Evaluation Manager. It includes the
preparation of the TOR, selection of the evaluation team, and contracting of the evaluation company.
45. Inception: This phase will be conducted by the evaluation team with guidance from the evaluation
manager. It concludes with an inception report detailing how the team intends to conduct the evaluation
with an emphasis on methodological and planning aspects.
46. Deliverable: Inception Report. This report focuses on methodological and planning aspects of the
evaluation. The report will be prepared by the evaluation team leader before embarking into the actual
evaluation work and will get endorsement from WFP. It will present the evaluation methodology; the
sampling technique; evaluation matrix - showing how each question will be answered; data collection
tools and sources of data. It will also present the division of tasks amongst team members as well as a
detailed timeline for the evaluation mission and for stakeholders’ consultation. The inception report will
also encompass details of a concrete operational plan for the evaluation. In essence, the report will
provide the CO and the evaluation team with an opportunity to verify that they share the same
understanding about the evaluation and clarify any misunderstanding at the outset.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
56
47. Data collection and analysis. This covers data collection and analysis activities, including field work.
The evaluation team will conclude the data collection phase with a presentation of the early findings of
the evaluation for the WFP Ethiopia CO, the Government of Ethiopia, USDA and other relevant
stakeholders.
48. Deliverable: debriefing workshop and Powerpoint presentation. The debriefing workshop is meant to
be an opportunity for the evaluation team to present early findings of the data collection and a preliminary
analysis of the data, but also to clarify pending issues.
49. Reporting: The evaluation team will analyze the data collected during the desk review and field work,
conduct additional consultations with stakeholders as required to eventually draft an evaluation report.
The draft evaluation report will present the findings, conclusions and recommendations of the evaluation.
Findings should be evidence-based and relevant to the evaluation objectives. Data will be disaggregated
by sex and the findings and conclusions will highlight differences in results of the feeding and non￾feeding schools on basic results indicators. There should be a logical flow from findings to conclusions
and from conclusions to recommendations. Recommendations will be limited in number, actionable and
targeted to the relevant users. These will form the basis of the WFP management response to the
evaluation. The Evaluation Reference Group (ERG) will review the draft evaluation report and provide
comments to address any factual errors. The evaluation team will incorporate ERG’s inputs into a final
evaluation report.
50. Deliverables: Draft Evaluation Report; Raw Data Sets; and Final Evaluation Report.
51. Dissemination and follow-up: The evaluation report and brief will be shared with relevant stakeholders
and users of the evaluation. The WFP CO will elaborate a management response to the evaluation
recommendations by providing actions that will address each recommendation and indicate an estimated
timeline for implementing corrective actions.
52. Deliverable: The Evaluation Brief. This is the product that the evaluation team should deliver to WFP.
An Evaluation Brief should be a two to three page a stand-alone summary - describing the evaluation
design, key findings, conclusions, recommendations and lessons learned. This document will serve to
inform any interested stakeholders of the final evaluation and should be written in a language easy to
understand by non-evaluators, with appropriate graphics and tables.
TOR Table 2: Activity timeline by evaluation phase
Activities Date Responsible
PHASE I – PREPARATION
• Floating bid, identification, and
recruitment of the evaluation company
• Hold pre-contract meeting with winning
company to ensure that the proposed
evaluation team members will be available
throughout the evaluation exercise.
• Signing service contract
June – September
2017 WFP Ethiopia CO,
Regional Bureau
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
57
Activities Date Responsible
PHASE II – INCEPTION
• Provide introductory briefing on the
programme, expectations and requirements
for the evaluation; and provide background
materials to the evaluation team.
• Desk review, initial consultation, drafting
of the inception package (including
methodology and evaluation mission
planning)
• Develop data collection instruments,
(developing questionnaires including
household questionnaire on the community
understanding of the value of education)
• Develop and finalize the sampling frame
and design for programme (intervention)
and non-programme (a
comparison/control) groups
• Submission of draft inception package
• Quality assurance of the inception package
and providing comments/recommendations
for improvement.
• Submission of the final draft inception
package and endorsement
• Preparation of the evaluation mission
(including setting up meetings, arranging
field visits, etc.)
September, 2017 WFP Ethiopia CO,
RB, Evaluation
Reference Group,
Decentralized
Evaluation Quality
Support service,
Evaluation Team
PHASE III: DATA COLLECTION
• Recruit data collectors/enumerators
• Training of data collectors/enumerators
• Data collection (quantitative and
qualitative data)
• Interviews with key internal and external
stakeholders, project site visits, etc.
• Abide by protocol for data management
and data quality assurance (For example:
Conduct weekly check-in of data collected
by enumerators)
October –
November, 2017
Evaluation team
PHASE IV– REPORTING
• Data cleaning and analysis
• Evaluation report drafting
• Quality Assurance of draft evaluation report
• Stakeholders comments on evaluation report
• Revision of the Evaluation report
• Final Evaluation report
• Evaluation brief
November –
December, 2017 Evaluation team,
WFP, Evaluation
Reference Group
PHASE V – DISSEMINATION OF REPORT
Final evaluation report and briefing dissemination
and follow up
January 2018. WFP Ethiopia CO
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
58
6. Organization of the Evaluation
6.1 Evaluation conduct
53. A company that has experience with WFP programme evaluation will be identified through Long-Term
Agreement (LTA) by the Country Office to independently manage and conduct the evaluation.
54. The company will provide an Evaluation Team (ET) in line with the LTA. To ensure an impartial and
objective evaluation, the WFP staff should not be part of the evaluation team or participate in meetings
with external stakeholders. The evaluation team should also not have been involved in the design or
implementation of the subject of evaluation or have any other conflict of interest. Further, they will act
impartially and respect the code of conduct for the evaluation.
6.2 Evaluation Management
55. The evaluation will be managed by Evaluation Manager (EM) who will be part of the evaluation team but
not part of WFP CO. The responsibilities of the EM will include:
a. Act as the contact point in WFP for the evaluation team leader throughout the evaluation process
b. Convene on behalf of the chair, the evaluation reference group and evaluation committee and
provide secretariat services
c. Consolidate library of information needed for the evaluation before start of inception
d. Lead the development of a Communication and Learning Plan
e. Take responsibility for the administrative and logistical needs of the evaluation
f. Organize and facilitate evaluation team orientation meetings
g. Coordinate with the evaluation team to Prepare field site visit schedules in line with the
requirements set out in the inception report and organize meetings (including a briefing and
debriefing) and site visits;
h. Comment on and quality assures the evaluation products in compliance with DEQAS
i. Share inception and evaluation report with the outsourced Quality Support Advisory service (DE
QS) for feedback on the quality from an evaluation perspective
j. Provide systematic and constructive feedback to the evaluation team leader on the basis of his/her
review and of the feedback from the DE QS.
k. Share the final draft inception and evaluation report with the Evaluation Reference Group and
relevant stakeholders to review and comment on the drafts.
l. Collate all comments received on inception and evaluation report in a matrix
m. Submit quality assured inception and evaluation report for approval of the CD/DCD
n. Facilitate/support the development of a management response and dissemination of the evaluation
report and its findings with all evaluation stakeholders.
56. Evaluation Manager may be an Ethiopian national or an international - in which case s/he will be home￾based and will remotely manage the evaluation following the WFP DEQAS and in close consultation with
the WFP CO evaluation focal person.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
59
6.3 Evaluation Conduct
57. The team will conduct the evaluation under the direction of the evaluation team leader in close
collaboration with the WFP evaluation focal point. The team will be hired by the company following
agreement with the CO on its composition.
6.4 Team composition
58. The evaluation team is expected to include 3-4 members, including the team leader. It should include
women and men of mixed cultural backgrounds and Ethiopian Nationals. The estimated number of days is
expected to be in the range of 60 for the team leader; 40 for the national evaluators.
59. Team competencies: The team will be multi-disciplinary and include members who together include an
appropriate balance of expertise and practical knowledge in:
• Statistical analysis
• School feeding
• Safety nets/social protection
• School health and nutrition
• Capacity development
• Gender analysis
60. Experience with evaluating FFE projects or other comprehensive school feeding projects a plus
61. All team members should have strong analytical and communication skills; evaluation experience and
familiarity with the country or region.
62. The team leader will have technical expertise in one of the technical areas listed above and demonstrated
experience in leading similar evaluations. S/he will also haveexpertise in designing methodology, data
collection tools and undertaking systematic qualitative and quantitative analyses. She/he will also have
leadership and communication skills, including a track record of excellent writing and presentation skills.
63. Her/his primary responsibilities will be: i) defining the evaluation approach and methodology; ii) guiding
and managing the team during the evaluation process; iii) leading the evaluation mission and representing
the evaluation team in meetings with stakeholders; iv) drafting and revising, as required, the inception
package, and evaluation report in line with DEQAS;
64. The team members will bring together a complementary combination of the technical expertise required
and have a track record of written work on similar assignments. Team members will: i) contribute to the
design of the evaluation methodology in their area of expertise based on a document review; iii) conduct
field work; iv) participate in team meetings and meetings with stakeholders; v) contribute to the drafting
and revision of the evaluation products in their technical area(s).
7. Roles and Responsibilities of WFP Stakeholders
7.1 Ethiopia WFP CO
65. The Ethiopia CO Management (Director or Deputy Director) will take responsibility to:
o Assign a WFP internal evaluation manager for the evaluation.
o Compose the internal evaluation committee and the evaluation reference group (see annex 5).
o Approve the final TOR, inception and evaluation reports.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
60
o Ensure the independence and impartiality of the evaluation at all stages, including
establishment of an evaluation committee and of the reference group.
o Participate in discussions with the evaluation team on the evaluation design and the evaluation
subject, its performance and results with the evaluation manager and the evaluation team
o Organise and participate in at least two separate debriefings, one internal and one with external
stakeholders
o Oversee dissemination and follow-up processes, including the preparation of a management
response to the evaluation recommendations
66. Evaluation Manager:
o Manages the evaluation process through all phases including drafting this TOR
o Ensure quality assurance mechanisms are operational
o Consolidate and share comments on draft TOR, inception and evaluation reports with the
evaluation team
o Ensures expected use of quality assurance mechanisms
o Ensure that the team has access to all documentation and information necessary to the
evaluation; facilitate the team’s contacts with local stakeholders; set up meetings, field visits;
provide logistic support during the fieldwork; and arrange for interpretation, if required.
o Organise security briefings for the evaluation team and provide any materials as required
67. An internal evaluation committee will be formed internally as part of ensuring the independence and
impartiality of the evaluation. The composition of the committee and their specific TORs will be
determined in line with WFP guidance before inception. The team will ideally be chaired by the DCD and
have the evaluation manager and relevant programme officer as a minimum. Overall responsibilities will
include but not be limited to reviewing evaluation deliverables (TOR, inception report, draft and final
evaluation reports by providing input to evaluation process and commenting on evaluation products).
They will also be responsible for submitting the final documents for approval to the chair of the
committee. The terms of reference for the evaluation committee are presented in annex 6.
68. An evaluation reference group will be a joint internal and external working group that will support a
credible, transparent, impartial and quality evaluation process. Their primary responsibility will be
reviewing and commenting on evaluation TOR, draft IR and ER and act as experts in an advisory
capacity, without management responsibilities. The group will be chaired by the DCD and membership
will include both internal and external stakeholders. The exact participants will be determined before
inception. They will review the evaluation products as further safeguard against bias and influence. The
terms of reference for the evaluation reference group are presented in annex 5.
7.2 Regional Bureau
69. The Regional Bureau management will take responsibility to:
• The M&E officer or the Regional Evaluation Officer will be the focal point for this
evaluation.
• Participate in discussions with the evaluation team on the evaluation design and on the
evaluation subject as relevant.
• Provide comments on the draft TOR, Inception and Evaluation reports
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
61
• Support the Management Response to the evaluation and track the implementation of the
recommendations.
7.3 Relevant WFP Headquarters divisions
70. Relevant WFP Headquarters divisions will take responsibility to:
• Discuss WFP strategies, policies or systems in their area of responsibility and subject of
evaluation.
• Comment on the evaluation TOR and draft report.
7.4 Other Stakeholders (Government, NGOs, UN agencies)
71. Other Stakeholders (Government, NGOs, UN agencies) will be part of the Evaluation Reference Group
and will provide inputs into the evaluation and provide comments on the deliverables as key partners and
contributors to the programme.
7.5 The Office of Evaluation (OEV)
72. OEV will advise the evaluation manager and provide support to the evaluation process where appropriate.
It is responsible to provide access to independent quality support mechanisms reviewing draft inception
and evaluation reports from an evaluation perspective. It will also ensure a help desk function upon
request from the Regional Bureaus.
8. Communication and Budget
8.1 Communication
73. To enhance learning from this evaluation, the evaluation team should place emphasis on transparent and
open communication with key stakeholders. For example, this might include ensuring a clear agreement
on channels and frequency of communication with and between key stakeholders.
74. Communication with evaluation team and stakeholders should go through the evaluation manager.
75. Following the approval of the final evaluation report, dissemination will be broad and workshops will be
conducted both internally and with partners, looking at the recommendations and the way forward
8.2 Budget
76. Budget: The evaluation will be funded from the evaluation fund reserved by the country office for this
purpose. Procurement will be through the WFP LTA. However, the budget will be proposed by applicants
in their financial proposal which will be submitted together with the technical proposal.
Important Notes:
• The bidding companies should submit the technical and financial proposals using the appropriate
templates for LTA firms. The technical proposals should include all the technical and team
composition and competency aspects as outlined in the TOR. The financial proposal should have the
financial details that the firm proposes for the evaluation.
• The submission of the proposals should adhere to the dates stipulated on the bidding document.
• The winning firm will submit Performance Bond.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
62
• The payments will be effected in three phases/installments: submission and endorsement of Inception
Package (first payment); completion of data collection and submission of draft report (second
payment); and a final report is submitted and also gets approval and acceptance (third/final
payment).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
63
TOR Annex 1: Map of school feeding areas
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
64
TOR Annex 2: Background documents to be reviewed
Key documents to be reviewed includes:
1. COUNTRY PROGRAMME ETHIOPIA 200253 (2012–2016)
2. WFP Ethiopia Country Strategy, 2011-2015
3. WFP School Feeding Policy, 2013
4. Ministry of Finance and Economic Development (MoFED) (2010). Growth and Transformation Plan,
2010/11-2014/15.
5. MoFED GTP Annual Progress Report, 2013, 2014
6. Education Sector Development Plan IV (ESDPIV) 2010/11-2014/15
7. National School Health and Nutrition Strategy, Ministry of Education, October 2012
8. United Nations Development Assistance Framework(UNDAF), 2012-2015
9. Afar and Somali Region Annual Work Plan (2005-2007)
10. National Nutrition Programme (NNP), June 2013- June 2015
11. School Feeding Annual Reports 2012-2014
12. Education Management Information System (EMIS)
13. UN Agencies Programme Implementation Manual (PIM), December 2013
14. UNDAF Midterm Review, 2013
15. Project-Level Results Frame work and the Evaluation Plan
16. Baseline evaluation report
17. Baseline evaluation data set
18. USDA M&E policy
19. USDA standard indicator handbook
Provide
teaching
materials
and
equipment
Teachers
trained by
Ministry of
Education
Increased Capacity of Government Institutions
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
TOR Annex 3: Results Framework
Project – Level Result Framework
Key
65
Improved
Attentiveness
Improved
student
Attendance
Result obtained
by Coordination
Result Obtained by
Ministry of Education
Increased Increased
Economic Access Student
and cultural Enrolment
incentive (or
Decreased
Disincentives)
Take Home Take Home
Rations Rations
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
66
TOR Annex 4: Results and Indicators
RESULTS AND INDICATORS
Increased Capacity of
Government Institutions
Number of government staff in relevant ministries/offices trained to
monitor the safety of food in school feeding programmes (maintain full
knowledge skill set)
Better Access to School
Supplies and Materials
Number of schools receiving school supplies and materials as result of
USDA assistance
Improved Attentiveness Percent increase in number of students in classrooms identified as
attentive by their teacher
Increased Access to Food
(School Feeding)
Number of daily school meals (breakfast, snack, lunch) provided to
school age children because of USDA assistance
Percent of students: girls regularly (80%) attending USDA supported
classrooms/schools
Percent of students: boys regularly (80%) attending USDA supported
classrooms/schools
Number of individual benefiting directly from USDA funded
intervention
Increase economic and cultural
incentive or disincentive
Increased gender equity in primary education
Increased Use of Health and
Dietary Practices
Percent of schools in target communities that clean cooking and eating
equipment, consistent with accepted standards, prior to use
Increased Engagement of Local
Organizations and Community
Groups
Percent increase in the Number of schools with CHILD planning team
in place
Improve knowledge of safe
health and hygiene practice
Percent increase in the number of schools in target communities that
clean cooking and eating equipment consistent with accepted standards
prior to use
Increased Knowledge of Safe
Food Preparation and Storage
Practices
Percentage increase in the number of people at school, district and
regional level trained in food management and handling
Increased Access to Clean
Water and Sanitation Services
Percent increase in the number of target school with year-round access
to a clean and safe water source
Increased Access to Requisite
Food Preparation and Storage
Tools and Equipment
Percent increase in the Number of target schools with ‘standard’ food
preparation and storage equipment
Increased Student Enrollment Percent increase in net enrolment rate in school because of USDA
assistance in Somali
Percent increase in net enrolment rate in school as a result of USDA
assistance in Afar
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
67
TOR Annex 5: Terms of Reference Evaluation Reference Group
1. Context: WFP in collaboration with the Ethiopian Ministry of Education has been implementing
school feeding interventions for 20 years. Over this period, the intervention has successfully
contributed to the increase in school enrollment and attendance, the decrease in the gender gap in
enrollment and the improved ability of pupils to concentrate in class. USDA, through Mc Govern -
Dole International Food for Education and Child Nutrition Programme is supporting school feeding in
the two regional states of Afar and Somali with USD 40.7 million over a period of four years (2013-
2017) in support of 289,000 students in 583 primary schools. This accounts for 52% of WFP School
Feeding Programme targeted schools in the country currently.
2. Purpose: The overall purpose of the ERG is to support a credible, transparent, impartial and quality
evaluation process in accordance with WFP Evaluation Policy 2016-2021. ERG members review and
comment on evaluation TOR and deliverables. The ERG members act as experts in an advisory capacity,
without management responsibilities. Responsibility for approval of evaluation products rests with the
Deputy Country Director as Chair of the Evaluation Committee.
3. Tasks: the ERG will review the evaluation products and provide comments to the evaluation team
4. Responsibilities by phase and time commitment:
ERG members’ responsibilities by
Evaluation Phase
Estimated
time required
Approximate dates
Phase 1: Planning -- --
Phase 2: Preparation
• Review ToR and provide feedback
ensuring that the ToR will lead to a useful
evaluation output and provide any
additional key background information to
inform the finalization of the TOR.
• Identify source documents useful to the
evaluation team.
1 day June - July 2017
Phase 3: Inception
• Meet with evaluation team (together
and/or individual members) The ERG is a
source of information for the evaluation,
providing guidance on how the evaluation
team can design a realistic/practical,
relevant and useful evaluation.
• Assist in identifying and contacting key
stakeholders to be interviewed, identifying
and accessing key documentation and data
sources, and identifying appropriate field
sites. This is important in their role of
safeguarding against bias.
• Review and comment on the draft
Inception Report
1 day September 2017
Phase 4: Data collection and analysis
• Act as key informant during the data
collection stage.
• Assist the evaluation team by providing
sources of information and facilitating data
access.
1.5 days October 2017
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
68
• Attend the validation /debriefing meeting
conducted by the evaluation team at the
end of the fieldwork.
Phase 5: Report
• Review and comment on the draft
evaluation report, specifically focusing on
accuracy and on quality and
comprehensiveness of evidence base
against which the findings are presented,
and conclusions and recommendations are
made. Particular attention should be given
to ensuring that the recommendations are
relevant, targeted, realistic and actionable.
The ERG must respect the decision of the
independent evaluators regarding the
extent of incorporation of feedback
provided to them by the ERG and other
stakeholders, as long as there is sufficient
transparency in how they have addressed
the feedback, including clear rationale for
any feedback that has not been accepted.
2+2 days November 2017
Phase 6: Disseminate and Follow-up
• Disseminate final report internally and on
websites of ERG members as relevant;
• Share as relevant evaluation findings
within respective units, organizations,
networks and at key events;
• Provide input to management response and
its implementation (as appropriate).
2 days January 2018
5. Procedures of Engagement:
• The Evaluation manager will notify the ERG members the time, location and agenda of meeting at least
one week before the meeting, and share any background materials for preparation
• ERG meetings will be held via electronic conference call/Skype.
• ERG members, representing their organizations will also be interviewed by the evaluation team during the
inception and data collection phases. This will be indicated in the evaluation schedule, and ideally
confirmed prior to the commencement of the data collection phase
• For each of the key evaluation products (Terms of Reference, Inception Report, Evaluation Report), the
ERG members will provide feedback electronically to the Evaluation Manager. For the Inception Report
and Evaluation Report the Evaluation Manager will consolidate all feedback for forwarding to the
Evaluation Team and will ensure that these have been appropriately responded to by incorporating them in
the reports or providing rationale where feedback is not incorporated.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
69
TOR Annex 6: Terms of Reference Evaluation Committee
1. Context: WFP in collaboration with the Ethiopian Ministry of Education has been implementing
school feeding interventions for 20 years. Over this period, the intervention has successfully
contributed to the increase in school enrollment and attendance, the decrease in the gender gap in
enrollment and the improved ability of pupils to concentrate in class. USDA, through Mc Govern -
Dole International Food for Education and Child Nutrition Programme is supporting school feeding in
the two regional states of Afar and Somali with USD 40.7 million over a period of four years (2013-
2017) in support of 289,000 students in 583 primary schools. This accounts for 52% of WFP School
Feeding Programme targeted schools in the country currently.
2. Purpose: The overall purpose of the evaluation committee is to ensure a credible, transparent, and quality
evaluation process in accordance with WFP Evaluation Policy 2016-2021. It will achieve this by
supporting the evaluation manager through the process, reviewing evaluation deliverables (terms of
reference, inception report, baseline report and endline report) and submitting them for approval by the
DCD who will be the chair of the committee.
3. Responsibilities of the Evaluation Committee: the EC is responsible for selecting and contracting the
evaluation team and approving all the evaluation products (terms of reference, inception report, baseline
and endline report of the evaluation).
4. Activities by phase and estimated time per EC member (excluding the EM)
Phase Activities Estimated time
1. Planning • Nominates an evaluation
manager.
• Decides the evaluation
budget.
• Decides the contracting
method well in advance to
enable the evaluation
manager to plan for the next
phase of the evaluation.
1/2 day - end of June
2. Preparation • Reviews the TOR on the
basis of:
o The External
Quality Support
advisory service
feedback
o Evaluation
Reference Group
comments
o The EM responses
documented in the
comments matrix
• Approves the final TOR.
½ to 1 day - early July
3. Inception • Briefs the evaluation team
including an overview of the
subject of the evaluation.
• Informs the design of the
evaluation during the
inception phase as key
stakeholders of the
2 days – September
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
70
evaluation.
• Supports the identification
of appropriate field visit
sites on the basis of
selection criteria identified
by the evaluation team
noting that the EC should
not influence which sites are
selected.
• Reviews the draft IR on the
basis of:
o The external
Quality Support
advisory service
feedback
4. Data Collection and
Analysis
• The evaluation committee’s
members:
o Are key informants
during the data
collection
o Act as sources of
contextual
information and
facilitating data
access as per the
needs of the
evaluation.
o Attend the
validation/debriefin
g meeting, and
support the team in
clarifying/validatin
g any emerging
issues and
identifying how to
fill any
data/information
gaps that the team
may be having at
this stage.
o Facilitate access to
stakeholders and
information as
appropriate
o Attend debriefing
meeting with
Evaluation Team.
2 days – October 2017
5. Reporting • Reviews the draft ER on the
basis of:
o The external
Quality Support
advisory service
feedback
o ERG comments
o The Evaluation
2 days – November 2017
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
71
team responses
documented in the
comments matrix
• Approves the final ER.
6. Dissemination and
Follow-up
• Facilitate preparation of the
management response to the
evaluation recommendations
• Approve the Management
Response
• Disseminate evaluation
results
• Make the report publicly
available
• Is finally responsible to
ensure periodic follow up
and updating of the status of
the implementation of the
recommendations.
1 day – January 2018
5. Procedures of Engagement
• The Deputy Country Director will appoint members of the evaluation committee
• The Evaluation manager will notify the members of the time, location and agenda of meetings at least one
week before the meeting, and share any background materials for preparation.
• Approval can be made via email on the basis of submission to the EC chair after endorsement by all EC
members
• EC meetings will be held face-to face and/or via electronic conference call/Skype and/or email depending
on the need, the agenda and the context.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
72
Annex B Methodology
Proposed approach and methodology
Mixed methods
1. In line with requirements from the TOR (Annex A), and documented in the IR,
overall approach to data collection has been a combination of secondary and primary data
collection and analysis. This has allowed the evaluation to maximize the use of secondary
data, reducing the burden on stakeholders of primary data collection and allowing for the
latter part of the data collection process – which focused on primary data – to cover those
questions/issues that could not be answered from the secondary data. The approach has
ensured that the evaluation covered the range of issues that the TOR require be
investigated. It has also maximized the evaluation’s utility for the purposes of lesson
learning – an important aspect of the TOR and a consideration which was emphasized to
the ET during the Inception mission briefings by the CO and the MoE stakeholders.
2. As envisaged in the TOR, the evaluation has adopted a mixed-methods approach, in
a complementary manner. This has included:
• Documentation analysis, both qualitative, and quantitative
• Key Informant (KI) interviews and focus group discussions (FGDs) at various levels
with key stakeholders
• An analysis of national EMIS data (2013-2016) of the performance of WFP schools
compared to non-school feeding schools. Details and results are in Annex X.
• Administration of a survey covering pupils, teachers and school principals with
closed and open questions that targets school and woreda level interviewees/
informants and which covers intervention and control schools. Full details and
results are in Annex Y.
• Observations of school feeding preparation and provision of meals in WFP schools
A theory based approach – working from the ToC to test the key assumptions
3. The different data collection methods were combined and linked to the ToC (Annex
C), in a manner that allowed us to explore whether the underlying assumptions in the ToC
have been met.
4. Annex C includes a discussion of the ToC and a fuller version of the ToC which shows
causal linkages. The principal purpose of this approach was to analyse the understanding of
causality implied in programme design and, by identifying the assumptions underlying the
ToC, to determine key factors or issues likely to explain the degree to which the programme
has achieved its objectives. Through the assumptions identified in the ToC analysis, the ToC
directly informed the full evaluation matrix presented in Annex D. The circled numbers in
the figure referred to the underlying assumptions in the ToC were as follows:
1. Food will be delivered in a timely manner and in the required quantities- However
this was not the case. There were disruptions and diversions of food to different
schools in Somali Region.
2. Sufficient funds will be made available for the programme by the donors- Funds for
the capacity building activities were limited
3. The Government of Ethiopia will have sufficient capacity to secure implementation￾limited capacity and high turnover of the government staff was a big challenge.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
73
4. The education system is effective in absorbing new students- This was not a big
problem, though the infrastructure was limited- class room used as the storage and
eating areas and teachers in some schools had to oversee the feeding.
5. Other partners are able and willing to work together with WFP to implement the
school health and nutrition components of the programme- Government is the only
partner in McGovern-Dole school feeding with support from the community in
providing water, fire wood, cooks and
6. Incentives for school feeding are not outweighed by other factors (early child
marriage, drought)- to some extent yes, drought and conflict has effected the SFP.
7. The national policy environment is conducive to school feeding. Yes
8. Government owns the programme and is willing to provide the resources (human
and financial) for implementation. There is a commitment but ownership is limited.
The resources provided by the government are for the emergency school feeding
programme only.
Figure 2 Theory of Change
Note: circled numbers in the diagram refer to the underlying assumptions in the ToC given above.
5. Using this approach, the evaluation found that some assumptions were misplaced or
inaccurate; that other assumptions should have been identified but were not; or that
assumptions about causality are proving correct, underscoring appropriate design. For
example – it was reported that there was a break in pipeline and delays in food deliveries,
funding constraints, capacity of the partners, high turnover of school staff, lack of water,
drought and conflict etc.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
74
Using a quasi-experimental design
6. The evaluation was conducted using a quasi-experimental design52 where WFP
intervention schools will be compared with non-intervention schools.
7. The sample survey was designed to address the evaluation questions relating to
impact and effectiveness through a quantitative, evidence-based and statistically sound
approach. At the same time, it collected qualitative data from a significant number of key
informants involved in the sampling which can be analysed qualitatively for recurrent
themes and observations. The field sampling also provided the framework for more
discursive and purely qualitative KIIs with school principals and FGDs with PTAs during
the fieldwork.
8. The sample size of 90 schools estimated in the project proposal has been retained, as
being consistent with prior information to give 10 percent sampling error for 95%
confidence intervals on worst-case binomial estimates (full details on sampling of the
schools are in Annex T of the IR). This comprised 30 schools in Afar region, and 60 schools
in Somali region, the 1:2 split ratio being proportional to the number of government
primary schools in the region (573 Afar, 1207 Somali), according to prior information. The
level of sampling was also consistent with the timetable and resources available to the
evaluation. Annex Y gives the results tables and analysis from this survey.
9. In addition to the sample survey, data on educational indicators (enrolment, grade
completion, by gender) for all primary schools was provided by the MoE EMIS system for
the 2016-17 (academic year Sep-Jul). This was not be a sample but a census (100 percent
coverage) of schools and therefore was expected to provide comprehensive information on
the impact, if any, of the programme on these basic educational indicators. The results for
the EMIS data study are given in Annex X.
10. A full outline of the survey approach and the survey instrument used are provided in
Annex U.
Sequencing of the inquiry
11. Data collection and analysis was sequenced to ensure that secondary data are used to
maximum effect. This was as follows:
• An initial analysis of the EMIS data to compare statistics from EMIS for WFP (i.e.
school feeding) schools with other primary schools in Ethiopia. This planned to cover
the academic years from 2013-14 to 2016-17 for those EMIS indicators for which
national statistics are available, but in the end (after 2 months of repeated requests)
MoE was only able to provide data for 2016-17, with significant limitations that are
noted in Annex X. The contribution of the EMIS data to the overall analysis is
therefore relatively limited.
• Documentary analysis to identify preliminary answers to the evaluation questions.
• A survey of WFP primary school and control schools, in accordance with the
approach described in Annex T. This was conducted during February 2018. In total
45 pairs of in- and out-of-programme schools were selected (90 schools total) with 12
students per school (1080 students in total).
• In-depth interviews with key informants to gain additional insights.
52The TOR required the ET to establish whether a quasi-experimental design was feasible for the final
evaluation. The inception phase established that this was possible.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
75
The evaluation has drawn on these data sources where they are likely to have most utility.
As already mentioned, the evaluation matrix shows which data sources will provide
information against the identified indicators.
Triangulation and complementarity
12. As part of the overall approach to the evaluation we ensured both triangulation and
complementarity between methods (see Box 3). We also triangulated within methods where
appropriate (e.g. comparing the perspectives of different stakeholders interviewed – see the
final column of the evaluation matrix (Annex D).
Box 3 Evaluation’s approach to Triangulation and Complementarity
Evaluation questions and evaluation matrix
13. The team has reviewed the evaluation questions (EQs) as presented in the TOR, and
has reworked those to produce a succinct set of logically sequenced questions. These are
summarized in Table 9 below. Table 10 in Annex D shows the correspondence between the
evaluation questions as presented in the TOR and those questions in Table 9. In this process
the issues covered by the TOR evaluation questions were retained (but in some cases
grouped together). In addition, questions were added to cover the following aspects which
at the inception phase were identified as being important, but which had not been
sufficiently reflected in the TOR questions:
• The extent to which gender and equity were considered at the design and
implementation levels
• An assessment of the sustainability of benefits for beneficiaries
• Effectiveness of monitoring and evaluation
14. The full evaluation matrix in Annex D provides further details on how each of these
questions have been answered. Table 9 cross-references each of the questions to the key
OECD/DAC evaluation criteria of relevance, effectiveness, efficiency, sustainability and
impact.
15. As noted above, the EQs were directly correlated with the ToC analysis. For each EQ,
the detailed matrix shows the analysis and indicators that were used to answer it; the main
sources of information for this purpose; and how the findings on each question were
triangulated. Wherever appropriate, gender dimensions are factored into the sub-questions,
judgement criteria and indicators for each EQ.
Methods can be combined in different ways:
‘Triangulation’: confirming and corroborating results reached by one method with other
results reached by another method. For instance, when beneficiaries of a project’s service state
that they judge it good (or bad); this can be cross-checked by collecting quantitative data on
coverage and accessibility of the service.
‘Complementarity’: results obtained by a method help better understand those obtained by
another method. In-depth theory-based approaches may help understand reasons why a project
led to unexpected results; qualitative methods may help clarify concepts and define variables;
and large-scale data sets may be analysed by multivariate and case-based methods.
Source: Stern et al, 2012
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
76
 Evaluation Questions
Key Question 1: How appropriate was the programme? Evaluation
criteria
EQ1. To what extent were interventions appropriate to the needs of
boys, girls, and parents of school-age children?
relevance
EQ2. To what extent were interventions aligned with relevant
national policies, including sectoral policies and strategies,
with WFP strategies, policies and normative guidance, and
with the needs of other key stakeholders? Was programming
sensitive to the context?
relevance
internal coherence
external coherence
EQ3. To what extent was the intervention design based on sound
analysis of gender and equity, and sensitive to Gender Equality
and Women’s Empowerment (GEEW)? Were other cross￾cutting issues, including protection and accountability towards
affected populations, adequately factored in?
relevance
Key Question 2: What are the results of the programme?
EQ4. To what extent have planned outputs and outcomes been
attained? Have there been any unexpected results (positive or
negative)?
effectiveness
EQ5. What have been the gender and equity dimensions of the
programme's results?
effectiveness
EQ6. What has been the (intended or unintended) impact of the
programme?
Impact
Key Question 3: What factors affected the results?
EQ7. How efficiently was the programme implemented? efficiency
EQ8. To what extent did internal/external factors and processes
contribute to the changes and results achieved?
efficiency
effectiveness
internal and
external coherence
EQ9. What was the quality of the monitoring and reporting system?
Did this enhance or impair the performance of the
programme?
efficiency
effectiveness
Key Question 4: To what extent are the project results
sustainable?
EQ10. To what extent will household food security for school going
boys and girls be sustained without / beyond USDA/WFP
funding?
sustainability
EQ11. To what extent has the programme prepared the Ethiopian
Government to continue school feeding in the Afar and Somali
region without / beyond USDA/WFP funding? To what extent
will the GoE be able to mobilize and sustain funding for school
feeding for the Afar and Somali regions?
sustainability
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
77
Key Question 5: What are the main lessons that can be
learned from this project?
EQ12. What lessons from this project should influence future
programmes (including good practices to be emulated and
weaknesses to be mitigated)?
16. In order to answer the evaluation questions in a thorough manner, and to ensure
sufficient attention to issues related to gender, sustainability and impact, the ET has
conducted two mini-cases studies as part of the evaluation approach. These mini-studies
have put emphasis on specific areas where we believe lesson learning might be particularly
beneficial.
• A study of the Emergency School Feeding Programme to inform the evaluation’s
assessment of sustainability and impact dimensions and to input into the lessons
learned (and recommendations). The findings from this study have complemented
the control and intervention comparison.
• A study of the Take-Home Rations-Girls Initiative Programme- has focused on the
impact of this activity of the programme at pupil and household levels. The study has
looked at the sustainability of impact of the Take-Home Ration on beneficiary pupils
(girls) and their families, in the form of direct benefits on household income and
spending patterns, but also any evidence of changing norms about girls’ education. It
has further looked at the extent to which the benefits are sustained over time and
beyond the duration of the programme. The study also looked at non-beneficiaries
(male pupils) and how these fare by comparison. The study has also considered
unintended effects/outcomes, as well as the positive and negative consequences of
the intervention.
Data Collection Methods and Tools
Overview
17. This section explains the different instruments to be employed and the approach to
triangulating evidence from different sources.
Data collection instruments
Document/literature review and review of secondary data
18. A substantial library of secondary data has been compiled in the course of the
Inception phase with the support from the CO and MoE (see the Bibliography at the end of
this report). The secondary data included a range of: project design documents; agreements
governing implementation; project planning, monitoring and reporting documents53
(internal and to the donor); WFP internal data (providing insight into country level
performance, pipelines, disbursements, cost breakdown); and external reports by other
stakeholders which provided insight into the overall context and other programmes. Gaps
in terms of documentation were identified and followed up.
53In particular, the evaluation will carefully analyse available monitoring data on the programme. Important
sources will be CO reports on output and outcome data relative to baselines, as well as WFP Standard Project
Reports (SPRs) and – where available and reliable – MoE data from the electronic Standard Project Reporting
system (eSPR).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
78
19. The secondary data analysis also included an analysis of key indicators from the
EMIS data base. This was requested from the MoE as a summary of enrolments and
completion by girls and boys for primary schools in Afar and Somali for years 2012/13-
2016/17. Annex X gives details of the data received and the analysis that it was possible to
perform with it. The data was received relatively late (mid-January 2018) and in more
limited form, being only for the 2016/17 academic year. As noted above, the evaluation
matrix shows which EQs will use secondary data as a support.
20. Secondary data analysis was done ahead of the field work and was initiated during
the inception phase.
21. As part of the preparation process the team analysed the available documentation for
preliminary responses to evaluation questions. The output of this was a populated
evaluation matrix with selected preliminary answers to the evaluation questions which the
team complete during the data collection phase with key findings from that phase.54This has
allowed the ET to have a clear overview of the gaps that still need to be addressed through
the primary data collection.
22. As noted elsewhere, there was no formal baseline survey for this project. The
evaluation approach for impact assessment, as fully described in Annex T, is to rely on
alternate sources. UNICEF and MoE undertook a comprehensive schools survey in 2013
and map data from this, together with codes for schools in and out of feeding programs is
available. The survey itself uses pairwise controls between programme and non-programme
schools to provide an alternative method of assessing impact. Annex Y gives the results of
the comparative analysis.
23. In addition to providing preliminary answers to the evaluation questions, the
secondary data analysis has allowed the team to produce internal products which has input
into the primary data collection.
24. A detailed field visit programme has been prepared (see Annex H). The field visits
included:
• Observation of school feeding in progress55 (see Annex W for observation guidelines)
in January 2018. This was not originally part of the proposed methodology in the TOR. 
It arose from the need to collect data before school feeding ends at the end of January
2018.
• Administration of a combined open and closed response survey at school level (see
Annex T and Annex Y for details on the survey). This took place from 12- 28 February
2018. This was delayed by a week due to late arrival of pupils and teachers after the
schools reopened on February 6, 2018.
• In-depth interviews with key stakeholders at all implementation levels took place as
planned for February 2018–interviews at federal, regional level, woreda, kebele level
54 This section refers to various internal products and reflects the fact that the Inception Report is as much a
document to guide the working of the team as it is a document for the client to understand how the ET will
implement the study. It should be noted that these internal products will not be shared with the client. They
will, however, be scrutinized by the QA team who will provide feedback and suggestions. Some of the internal
products may become part of the final evaluation report.
55 Observation could only be done up to the end of January as this was the end of the school feeding phase.
Even if stocks remained in the schools at that time, the plan was to distribute to beneficiaries because the
expiry date of CSB and Oil. However, during the visit of ET in February 2018, some of the schools were still
providing school feeding in Somali Region, and ET could observe the feeding and the in-depth qualitative
work.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
79
and with schools (including beneficiaries) to inform in-depth understanding as well
the various case studies of WFP and Emergency School Feeding schools, and the case
study on take-home rations. These case studies are in Annex R and Annex S.
25. Observation of SMP allowed the evaluation – among other aspects - to gather
information on conditions under which school feeding is provided in practice, to assess
nutrition and hygiene practices,56 to verify to what extent selected beneficiaries have been
able to apply the knowledge acquired during training, to verify compliance with WFP
guidance on school feeding, and to assess gender dynamics around the school feeding
process. School visits and data collection had initially been planned for February 2018.
However, during the inception visit it became clear that school feeding formally ends in
January 2018 (at which time some of the food will also expire). In light of this, the ET re￾arranged and reviewed the distribution of team inputs over the evaluation period for the
field work so that a first set of schools were visited in January 2018 for the purpose of
observing school feeding in progress. In consultation with the WFP country office and
keeping in mind the distances, logistics, time and the security, two woredas from Afar
[Adaa’r and Chifra] and two from Somali region (Jijiga and Kebribeyah) were selected from
the sampled woredas for the quantitative survey. As a second step, 2 schools from Afar and
2 from Somali region from the 4 woredas were selected upon arrival at the site (schools
were not selected in advance to ensure that the visit is not overly prepared). Though initially
planned to visit 3 schools in each region, due to logistics and time constraints only 2 schools
in each region were observed. A checklist was developed to observe the organisation,
preparation of the meals, capacity of cooks, store keepers, water- sanitation, water source,
facilities and process for hand washing, availability of latrines (distance and gender
segregation) and interaction with cooks and other staff and the children to seek their views.
26. The survey has enabled the team to gather quantitative and qualitative information
from schools, kebeles and woredas in the main geographical areas and has been used to
compare WFP schools with control schools (and for the purpose of sustainability with
Emergency School Feeding Schools if this proves feasible). The Survey Instrument (SI)
developed during the inception phase (see Annex U) was used without further modification.
27. The survey included Focus Group Discussions (FGDs) of Parent Teacher Associations
(PTA) and Key Informant Interviews (KII) with school principals within the quantitative
survey framework, enabling triangulation of quantitative and qualitative insights. The
survey also captured some information on what other initiatives are on-going in the regions
to be visited which may have an impact on the programme. This information will be used to
establish whether the changes might be due to other intervening factors.
28. In-depth qualitative interviews provided the team a grounded understanding of WFP
SF activities, including beneficiaries, at different levels. This provided key information on
various issues of interest highlighted in the TOR including SF management,
complementarities, sustainability, unintended outcomes, positive and negative
consequences, and impact on beneficiaries. The interviews also provided an understanding
of other initiatives that are going on in the overall environment and which might have an
influence on the outcomes of the project (i.e. as an input into the contribution analysis and
the counterfactual).
29. Schools were paired between FFE and non-FFE schools within the same locality for
close comparison, and randomly selected by region. School and child level interviews were
56 Cooking and storage facilities and arrangements will be inspected, as well as latrines and other sanitation
and hygiene facilities.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
80
undertaken, with 90 schools, 12 children per school from grades 2-4 (6 boys, 6 girls, 2 each
from each grade), giving 1080 child interviews. The sample of 12 children per school is the
minimum possible within the tight constraints of the survey that allows statistical
comparisons across gender (2 per gender) and grade (2 per grade in 3 grades). Grade 1 is
not sampled as many children will be below a minimum age for comprehension of the
questions (8 years). Sampling within grades is by random selection of classes, and for
children within classes it will be randomly from the class registers. Specifically designed
random number tables were supplied to the survey teams to facilitate this.
30. Data was collected on paper forms, as there are extensive amounts of qualitative data
(text material) to be collected which cannot be reasonably types onto small tablets. Paper
forms also allow more rapid entry of tabular data from the school registers of enrolment,
attendance, and marks by grades. Paper forms also facilitated translation into Amharic and
Somali. The Survey Instruments in English are given in Annex U. Data entry was done by
the local survey partners (B&M Consulting) using a forms and transaction software
(CSPRO).
31. The datasets were reviewed and processed by the international consultant
responsible (Denis Alder) using R statistical software and Excel. Apart from primary
screening done at data entry, further validation included range checks for outliers, digit
preference checks and distribution checks and tests, all using R software. The analysis of
the data is presented in Annex Y, and includes a statistical comparison of significant
differences for pass rates by grades related to gender and FFE programme status (Annex Y,
Table 34 and Table 35), an analysis of frequency responses by region, FFE status and
gender for the child questionnaire (Annex Y, Table 36), school staff student ratios and
gender balance and facilities such as latrines, water supply, school feeding facilities (Table
6, Annex Y), details of recent receipts of FFE supplies, and teachers observations on impact
on attendance and attentiveness, by region, for in-FFE schools (Table 38, Annex Y),
observations on school meals support for non-FFE schools (Table 39, Annex Y), and
capacity building for school staff through workshops and short courses (Table 40), again for
FFE and non-FFE schools, by region, with gender balance information.
32. Qualitative data collected on the survey forms was re-entered with the quantitative
data using the R package and used to elucidate the quantitative analysis and underlying
processes and themes. However, most of the qualitative interviews will be collected outside
the formal survey. These will be reviewed and interpreted subjectively to determine the
main themes and issues. The statistical analysis will likely highlight schools that exemplify
key issues or opportunities, whose narratives will also be reviewed subjectively. As with all
the data, all material will be anonymised, and the source of exemplified narratives will not
be identified.
33. For the in-depth interviews, the team visited and 4 schools in Somali region by 3
members of the ET and 1 school in Afar region by 2 members of the ET. The woredas and
schools were selected from amongst those included in the survey in consultation with the
sub-offices, strongly influenced by logistics to maximize the utility of the time in the field.
School feeding observations were taken place in January in 2 school of Afar and the
information was feed into the analysis. One of the schools visited in the Somali region is a
model school. However, in Afar region- only one school could be visited due to tight
timeline, logistics, distance and timing of the schools. Despite this- the school though
officially opened, there were no children and no feeding due to absence of teachers and the
expiry of food commodities.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
81
34. The qualitative ET had planned to visit a non-WFP school where emergency school
feeding was in operation, but it could not take place due to time constraints and currently
no ESFP was in operation. So the case study is based on interviews with Government
stakeholders, WFP and UNICEF and SCI.
35. For the in-depth interviews at school level, interviews were sought with the head
teacher, the co-ordinator of school meals, a group of beneficiaries of the take-home rations
(i.e. girls), a group of non-beneficiaries of take-home rations (i.e. boys), parents of take￾home ration beneficiaries and parents of non-take-home ration beneficiaries, the cooks and
the PTA. See Annex T for further details.
36. In each kebele and at woreda level, the team sought interviews with the local
education authorities, specifically any officials responsible for school feeding, staff of
UNICEF, Save the Children International and Italian Cooperation working in the area of
school feeding.
37. At regional level, in-depth interviews were done with the relevant regional
authorities, and WFP staff. This was either done before/after the school visits to maximize
the opportunity for further investigation of issues at the school level. At national level the
interviews were with key informants from WFP, the MoE, UNICEF, SCI and USDA as a
donor.
38. For key categories of interviews, men and women (boys and girls) were interviewed
separately.
39. The interviews at the school level were conducted through an interpreter. Though it
was planned to include female interpreters for girls’ pupils, but this was not possible.
40. Annex H gives details of the proposed evaluation mission schedule.
Key informant and stakeholder interviews and FGDs
41. KII and FGDs were done both as part of the survey, as well as through the separate
field work. The range of interview targets is indicated in the stakeholder analysis (Annex V)
as well as in the more detailed guidance on the field work in Annex W). By default,
interviews were treated as confidential. Interview findings were systematically written up by
team members using a standard template and shared through a compendium in a
confidential section of the team’s e-library. The compendium enabled interview notes to be
easily searched by topic, and facilitated triangulation of different interviewee recollections
and perspectives.
42. The field work (both the survey and the in-depth qualitative field work) included
FGDs with beneficiaries (with separate groups for women/girls) and with WFP,
Government of Ethiopia, and other staff involved in delivering programmes. To gain the
opinions and views of as many members of the focus group as possible, a participatory
approach was used where appropriate.
43. Guidelines for different categories of interview and focus group are shown in Annex
W and are linked to the evaluation matrix.
Consultation strategy, communication and feedback
44. As part of the inception mission the team has already had meetings with Government
(including the State Minister), in addition to meetings with the CO and with selected
members of the external reference group. These interviews have been important in ensuring
understanding of the process, in assessing the feasibility of the approach, and in securing
commitment to the next steps of the evaluation. An exit presentation at the end of the
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
82
inception highlighted progress made and next steps. As part of the inception the survey
team continued work in the field beyond the inception exit meeting and has presented to
the CO of the detailed thinking around the survey process. The approach presented in this
IR reflects those consultations and the contributions from the CO.
45. The evaluation continued in a consultative fashion. A systematic approach to
consultation during the phases followed the following:
• Informing the EM of the arrival of the ET and their planned field visits to Somali and
Afar region and stakeholders meetings.
• The itineraries for the field visits were shared with the EM to inform the respective
WFP sub-offices and the Government departments.
• The survey team informed the EM about the field work itineraries and requested
letters of introduction for the teams from the Education Department.
• An exit debriefing could not take place due to delay in undertaking the survey by one
week. Though a meeting was planned to share preliminary findings and impressions
of the field visits by the qualitative ET, but this could not take place given conflicting
meetings in the CO. The evaluation team shared some findings to be included in the
next phase of the McGovern-Dole proposal to be submitted in March. So there was
no formal debriefing with Aide Memoir and or a PowerPoint presentation of
preliminary findings and conclusions to seek clarification and validation.
• A debriefing to Government and wider stakeholders including the external reference
group at the end of the field work also did not take place.
Regular communication with the Evaluation Manager and feedback on the evaluation
progress by the team leader was in place.
Limitations and risks
46. Several limitations at the inception phase need to be noted:
a) Only limited documentation had been made available to the team prior to the
inception visit. This was subsequently remedied. However, it has meant that the
team has worked under considerable pressure catch up in terms of its reading and
analysis of the documentation. Further analysis was done while comments on the IR
were awaited, and prior to the field work. This additional analysis may result in
changes to the evaluation data collection tools.
b) As no evaluation was undertaken at the end of the previous McGovern-Dole
intervention, and in the absence of a well-established baseline, the evidence base at
the start of the evaluation is very thin. However, the use of a quasi-experimental
survey design with paired FFE and non-FFE schools in similar localities provides a
rational basis for evaluation of the probable impacts of the programme.
c) During the inception visit it emerged that school feeding will have ended by the time
the field mission visits Afar and Somali (in February 2018). This did constrain the
team’s understanding of key dimensions of SF. Therefore, the ET had to move a part
of the field work forward to January 2018 to ensure observation of school meals
could be part of the evaluation approach. Ideally this should have taken place when
the IR was approved.
d) Distances in the geographical areas covered by the evaluation were considerable.
Valuable time and resources were while visiting the schools in Afar region- and were
found closed. The survey was planned as a simple random sample stratified by
regions, with in/out of programme schools in paired clusters. It was realised that
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
83
prior information on FFE status was imperfect, so a randomised list of potential
substitutes was used to adjust the survey samples in the field while maintaining
objectivity. However, in Afar it proved difficult to find the necessary number of out 
of programme schools, whilst in Somali, the very large distances, advice on security
factors in some areas, and limited time for the survey mean that not all parts of the
region were covered. The map in Figure 10, Annex Y, shows the final survey coverage
relative to distribution of all government primary schools.
e) It will be noted that there were considerable difficulties in planning and executing
the survey due to the inadequacies of prior information held both by WFP CO and
MoE EMIS datasets. A particular difficulty is the constant and considerable
variation in Woreda, and especially school names, as noted in Annex X, Box 6 and
text. Table 29 in Annex X lists the data sets used to derive the prior information. In
the field, protocols were established and applied to substitute schools from a
randomised list if for any reason a school was closed, could not be located or
inaccessible due to security concerns, or was found to not be of the expected FFE
status. This worked well in Somali where a balance of 30 FFE and 30 non-FFE
schools were sampled. In Afar, time proved too short to allow sufficient non-FFE
schools to be located – many that were out of programme from prior information,
turned out to be in-programme when interviewing was done. Although some
substitutions were done, these too proved to be in programme, so that in the end, the
Afar sample comprised 24 in-programme and only 6 counterfactual out-of￾programme schools. Nonetheless, in spite of the small counterfactual sample,
significant differences were found in grade completion rates between FFE and non￾FFE schools in Afar, as well as Somali with a larger, more balanced sample.
47. The evaluation was on a very tight timeframe. While the evaluation took place and be
finalized before the 31st of March 2018 as required. The ET and WFP were aware of these
constraints. To gain time, some of the phases of the evaluation were overlap. Thus, while the
IR was being completed, the survey tools were finalised and the survey teams were being
mobilized. In an identical fashion the evaluation report writing started while the data
collection was being finalized. Also, efforts were made to highlight to the stakeholders,
reviewing the various products of the report that timely feedback was critical to ensuring
that the subsequent phases of the evaluation could proceed as planned.
Ensuring quality
48. WFP has developed a Decentralised Evaluation Quality Assurance System (DEQAS –
see WFP, 2016f), informed by the norms and standards for evaluations developed by
UNEG. The DEQAS forms a specific set of guidance materials based on WFP’s Evaluation
Quality Assurance System (EQAS) and its Evaluation Policy. The guide sets out process
maps with in-built steps for quality assurance and templates for evaluation products, as well
as checklists for feedback on quality for evaluation products. DEQAS will be systematically
applied during this evaluation, with relevant guideline documents having been provided to
the ET.
49. Mokoro’s internal Quality Support (QS) System was integrated into the evaluation
process in line with the company’s commitment to delivering quality products and
adherence to the principles of independence, credibility and utility. Evaluation products
were shared with the QS experts (Stephen Lister and Alistair Hallam) prior to submission.
Both experts have deep familiarity with WFP and EQAS, making them well placed to review
deliverables and advise on evaluation methodology, as well as to provide technical insights
to complement the team’s evaluation assessments. (See Annex F, Table 15.)
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
84
50. There was no potential conflict of interest in the performance of this evaluation.
None of the ET members were involved in the preparation or direct implementation of the
WFP-supported school feeding activities in Ethiopia.
51. The team adopted a careful and thorough approach to the ethics of the evaluation,
complying with standard 3.2 of the United Nations Evaluation Group (UNEG) Norms and
Standards (UNEG, 2016). While supportive and collegiate in its working relations with
WFP, it was strictly neutral and unbiased. It requested consent from all interviewees and
focus groups before proceeding with discussions, and assured them of full confidentiality:
while informants’ views may be quoted and their names were listed in an annex to the
evaluation report, no view or statement are attributed to a named individual, or presented
in such a way that an individual can be traced as its source. The team thus encouraged all
informants to be frank and accurate in their assessments of programme performance. It
complied fully with GoE and WFP guidelines on contact with children (UNEG, 2008).There
were some ethical issue in including schools in the survey that did not benefit from the FFE
programme, as part of the counterfactual sample. The evaluation team did emphasize to the
schools concerned that participation is voluntary, and they were free to refuse (as indeed
are all respondents in the sample, who will be similarly briefed and appropriately
substituted if they do not wish to participate). The issue of perceived lack of benefits for
schools that have not benefitted from school feeding will be handled by explaining that the
original selection criteria were based on relative need at the time of programme start-up in
2013, and that the current situation is open to review, which the survey seeks to inform.
The survey provides an opportunity for the non-programme schools to feedback comments
and views to the DoE and WFP and therefore represents a positive for the school, though
the future status of any given school in any forthcoming SFPs will not depend on the survey
responses, which will in any case be anonymized and not attributable to any particular
school or person.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
85
Annex C Theory of Change
Introduction
1. To understand the underlying logic of the McGovern-Dole programme and to ensure
that the evaluation team identifies assumptions that will be investigated as part of the
evaluation process, a key task for the evaluation has been to prepare a theory of change
(ToC).
2. The drafting of the ToC was informed by the project documentation and by
discussions with WFP and other stakeholders. A draft version was shared with WFP and the
reference group and comments received fed into the final version of the ToC that is shown
in Figure 1 (in the main report).
Observations and consequences for the implied ToC
3. WFP’s grant application to McGovern-Dole is structured in terms of a project results
framework (PRF) which is replicated in the evaluation TOR and a project performance plan
which includes indicators against which progress will be measured (a summary version of
which is also part of the terms of reference).57 For ease of reference the PRF is reproduced
on the next page (Figure 3)
4. The PRF formed the basis for reconstituting the ToC which was a key task for the
evaluation (see Figure 1). The PRF identifies a number of higher level results (which we
have broadly interpreted as outcomes) to be achieved by WFP as well as those that are to be
achieved in collaboration with other partners.
5. Looking at the highest level of outcomes in the results framework diagram (Figure 3
below) it is immediately obvious that these essentially cover two main areas of outcome,
namely education outcomes (essentially improving attendance and attentiveness) and
health and nutrition outcomes (essentially reducing health related absences and increasing
health and dietary practices).
6. However, the use of a number of horizontal arrows in the diagram shows quite
clearly that, while the education outcomes (attendance and attentiveness) are presented at
the same level as the health and nutrition ones, there is the assumption that improved
health and dietary practices lead to reduced health-related absences which in turn
contribute to the two main outcomes (attendance and attentiveness).
57 Both are part of the USDA commitment letter dated November 9th, 2012 (USDA, 2012b).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
Figure 3 Project-level results framework
86
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
87
7. It is interesting to note that contrary to SF interventions supported by McGovern￾Dole in other countries (e.g. Nepal and Kenya), this results framework does not explicitly
bring out outcomes in the area of literacy (education quality). We have been given to
understand that this is because the Ministry of Education has other programmes that
address this. We also would surmise that this choice makes sense given that the starting
point in the targeted regions (Afar and Somali) in terms of getting children (and especially
girls) to go to school was low, and that therefore getting children into school is a first
priority, with the understanding that ensuring that this ultimately should also contribute
(with the support from other actors) to better quality education.
8. It is also noticeable that the results framework does not bring out the fact that the
project design explicitly seeks to address issues around producing more equitable education
(which school feeding contributes to in general by removing a barrier to children going to
school) and through the Take-Home Rations as an intervention to encourage girls’
attendance.
9. Returning to the results framework, under these higher level outcomes are a number
of areas of activities and outputs that produce each of the outcomes. In the case of improved
attentiveness there are three groupings of what we have interpreted as outputs/immediate
outcomes which related to provision of school supplies, teacher training (by the MoE), and
provision of school meals. Feeding into student attendance are two sets of
outputs/immediate outcomes related to the provision of take-home rations and enrolment.
Finally, for the health related outcome there is a set of four different outputs/intermediate
outcomes identified which relate to improving food preparation and handling, storage and
equipment, water and sanitation, and health and hygiene. Three out of four of these (the
exception being the provision of energy saving stoves/non-food items) are explicitly
highlighted as being results that are obtained through coordination (i.e. through the inputs
and support from other actors working in this area).
10. Two sets of foundational results at the bottom of the diagram are essential strategies
to be employed to achieve the results and outcomes. These relate to increased capacity of
government institutions at different levels, and increased engagement of local organizations
and community groups. These also imply important elements of collaboration and
coordination.
The reconstructed ToC
11. In drafting the ToC we have used the principle of a causal chain of inputs through
activities to outputs, immediate and intermediate outcomes and impacts. This follows the
2016 WFP corporate ToC for school feeding. Our reconstituted ToC has thus sought to bring
out the logical progression from these inputs, through activities, direct outputs, outcomes to
impact. It should be noted that we divided outcomes into the more immediate outcomes
which are in the left-hand column, and longer term outcomes which are in the right hand
column.
12. In drafting the ToC we have for the purpose of coherence and simplicity merged
some of the wording from the PRF areas, and consolidated certain activity areas into fewer
groupings. We have also added in the ToC diagram in the far left hand column an overview
of the starting point which shows what it was that the inputs provided through the
operation sought to address.
13. On the other side of the ToC we have also brought out a more explicit focus on
broader areas of impact, and have sought to bring out the gender and equity dimension
more strongly (compared to how it is presented in the results framework.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
88
14. In the detailed version of the ToC that is presented below (Figure 4) we have also
included a set of arrows which shows what parts of the diagram explicitly link to other parts
of the logical sequencing. The arrows show an approximate representation of causality.
Figure 4 Reconstructed ToC diagram
15. Finally, in line with the logic of a ToC we have identified key assumptions which are
the numbered figures in the ToC diagram. We have sought to identify the main assumptions
and to keep the number of assumptions manageable. The assumptions have then been used
to inform the questions and sub-questions in the evaluation matrix. The main assumptions
are as follows:
1. Food will be delivered in a timely manner and in the required quantities.
2. Sufficient funds will be made available for the programme by the donors.
3. The Government of Ethiopia will have sufficient capacity to secure implementation.
4. The education system is effective in absorbing new students.
5. Other partners are able and willing to work together with WFP to implement the
school health and nutrition components of the programme.
6. Incentives for school feeding are not outweighed by other factors (early child
marriage, drought).
7. The national policy environment is conducive to school feeding.
8. Government owns the programme and is willing to provide the resources (human
and financial) for implementation.
16. Clearly, the `visual representation presented in the ToC is just one way in which the
underlying logic of the operation can be presented, and there may be arguments to position
specific elements slightly differently (for example, enrolment as an intermediate outcome
may also be positioned as an immediate outcome which then leads to attendance and
attentiveness). However, it should also be remembered that the ultimate objective of an
exercise such as this is not to get the perfect ToC but rather to have a version of the
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
89
underlying logic that allows the evaluation to investigate the relevant different dimensions
of the programme and to do justice to the underlying design intentions and to reporting on
the achievements and areas of challenge.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
90
Annex D Full Evaluation Matrix
Main evaluation questions
1. Table 10 below lists the main evaluation questions (EQs) and shows which evaluation
criteria are most relevant in assessing each EQ. Definitions for the evaluation criteria are
given below the table, and the full evaluation matrix is in Table 11.
2. The questions posed in the TOR (Annex A) have all been incorporated in the
evaluation matrix. but some additional questions have been added for completeness. The
added questions are shaded thus in the table. Table 12 below shows where each question in
the TOR is incorporated in the evaluation matrix as elaborated by the evaluation team.
 Evaluation Questions (showing additions)
Key Question 1: How appropriate was the programme? Evaluation criteria
EQ1. To what extent were interventions appropriate to the needs of boys, girls,
and parents of school-age children?
relevance
EQ2. To what extent were interventions aligned with relevant national policies,
including sectoral policies and strategies, with WFP strategies, policies and
normative guidance, and with the needs of other key stakeholders? Was
programming sensitive to the context?
relevance
internal coherence
external coherence
EQ3. To what extent was the intervention design based on sound analysis of
gender and equity , and sensitive to GEEW? Were other cross-cutting
issues, including protection and accountability towards affected populations
adequately factored in?
relevance
Key Question 2: What are the results of the programme?
EQ4. To what extent have planned outputs and outcomes been attained? Have
there been any unexpected results (positive or negative)?
effectiveness
EQ5. What have been the gender and equity dimensions of the programme's
results?
effectiveness
EQ6. What has been the (intended or unintended) impact of the programme? impact
Key Question 3: What factors affected the results?
EQ7. How efficiently was the programme implemented? efficiency
EQ8. To what extent did internal/external factors and processes contribute to the
changes and results achieved?
efficiency
effectiveness
internal and external
coherence
EQ9. What was the quality of the monitoring and reporting system? Did this
enhance or impair the performance of the programme?
efficiency
effectiveness
Key Question 4: To what extent are the project results sustainable?
EQ10. To what extent will household food security for school going boys and girls
be sustained without / beyond USDA/WFP funding?
sustainability
EQ11. To what extent has the programme prepared the Ethiopian Government
and the education system to ensure that they can continue school feeding in
the Afar and Somali region without / beyond USDA/WFP funding? To what
extent will the GoE be able to mobilize and sustain funding for school
feeding for the Afar and Somali regions?
sustainability
Key Question 5: What are the main lessons that can be learned from this
project?
EQ12. What lessons from this project should influence future programmes
(including good practices to be emulated and weaknesses to be mitigated)?
all
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
91
Evaluation criteria
3. The standard OECD DAC evaluation criteria are relevance, effectiveness, efficiency,
sustainability and impact, for which we use the following definitions.
Relevance The extent to which the objectives of an intervention are consistent with
beneficiaries' requirements, country needs, global priorities and
partners' and donors' policies.
Effectiveness The extent to which the intervention’s objectives were achieved, or are
expected to be achieved, taking into account their relative importance.
Efficiency A measure of how economically resources/inputs (funds, expertise, etc.)
are converted to results.
Sustainability The continuation of benefits from an intervention after major assistance has 
been completed. The probability of long-term benefits. The resilienceto 
risk of the net benefit flows over time.
Impact Positive and negative, primary and secondary long-term effects
produced by an intervention, directly or indirectly, intended or
unintended.
4. As regards “impact” and “results”, the evaluation follows the EQAS preferred usage
in which:
• “result” and “effect” are practically synonyms, and results can be at the output,
outcome and/or impact levels, while
• “impact” (as above) refers to lasting and significant effects at the goal and outcomes
level of the logical framework (results-chain).
5. As regards efficiency and effectiveness the evaluation follows the technical guidance
note (WFP, 2013e) which adopts the DAC definition of effectiveness as a measure of the
extent to which an aid activity attains its objectives (the relationship between subsequent
levels in the logical framework: activities, outputs, outcomes, and impact), and a broad
definition of efficiency as a measure of the relationship between inputs and results (outputs,
outcomes, and impact).
6. We employ the additional criterion of coherence as follows:
Coherence The consistency of policy/programme elements with each
other (do they complement each other in a positive way?)
7. This can be applied as internal coherence to the different elements of a school
feeding programme, and as external coherence to the consistency of the school feeding
programme with other related programmes.
Full evaluation matrix
8. Table 11 below is the full evaluation matrix which is the guiding framework for the
evaluation. It underpins the discussion guides for interviews and FGDs that are presented in
Annex W.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
92
Evaluation Matrix
Specific questions Analysis/indicators Main sources of
information
Triangulation approach
Key Question 1: How appropriate was the programme?
EQ1. To what extent were
interventions appropriate to
the needs of boys, girls, and
parents of school-age
children?
• Assessment of needs
of target population at
design stage, and
significant trends
• Check of alignment of
programme’s
strategies with those
needs, at design and
currently
• Analysis of data
(reflecting the situation
at the start of the
programme and other
assessments) of needs
of girls, boys, women
and men in the target
population
• Expressed views of
target population (girls,
boys, women and men)
as recorded at design
stage, since, and
during mission field
work
• Analytical opinions of
expert informants
(national and regional
governments, DPs,
other actors).
• Compare needs as
summarised in formal
documentation with
those expressed by
target group.
• Compare needs as
interpreted in the
design and
implementation of the
programme with the
interpretation of expert
analytical informants
Strength of evidence: good
EQ2. To what extent were
interventions aligned with
relevant national policies,
including sectoral policies and
strategies, with WFP
strategies, policies and
normative guidance, and with
the needs of other key
stakeholders? Was
programming sensitive to the
context?
• Check of alignment of
programme’s
objectives, targeting
and activities with
those stated/
prioritised in national
policies on education,
food security and
nutrition and gender
(including gender
elements of sector
policies)
• Check of alignment of
programme’s design
objectives and
targeting (and any
subsequent revisions
thereof) with
corporate WFP and
UN strategies,
policies and
standards: school
feeding, resilience,
nutrition, gender.
• Was the design based
on specific analysis of
the contexts in Afar
and Somali Regions?
• Programme
documentation
• National policy and
strategy documentation
• WFP and UN corporate
policy and strategy
documentation
• Interviews
• Compare the views of
GoE, WFP, DP and
other informants
• Compare issues as
summarised in formal
documentation with
those expressed by
key informants.
Strength of evidence: Good
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
93
Specific questions Analysis/indicators Main sources of
information
Triangulation approach
EQ3. To what extent was
the intervention design based
on sound analysis of gender
and equity , and sensitive to
GEEW? Were other cross￾cutting issues, including
protection and accountability
towards affected populations
adequately factored in?
• Analysis of
programme's priorities
and gender and
equity strategies
compared with
national, WFP and
other relevant policy
and strategies
• Analysis of
programme design
against WFP and UN
policies on protection
and accountability to
affected populations
• Programme
documentation
• GoE, DP, WFP and UN
corporate
documentation
• Opinions of target
group on relevant
gender issues, as
expressed at the
design, in subsequent
consultations and/or
during mission field
work
• Interviews with key
informants from GoE,
DP, WFP, UN and
other actors
• Compare issues as
summarised in formal
documentation with
those expressed by
target group.
• Compare the views of
GoE, WFP, other UN
and DP informants
Strength of evidence:
Good, documentation
mostly available.
Remaining information to
be collected through
interviews.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
94
Specific questions Analysis/indicators Main sources of
information
Triangulation approach
Key Question 2: What are the results of the programme?
EQ4. To what extent have
planned outputs and
outcomes been attained? –
including:
• Improving access to
and equity of
education for primary
school boys and girls
particularly for
children from rural
areas of emerging
regions and
underserved areas.
• Alleviating short term
hunger and retaining
boys and girls in
school
• Promoting essential
package
interventions
• Improving WASH and
school feeding
infrastructure in
schools
• Promoting school
health and nutrition
interventions in
primary school for
boys and girls
• Strengthening the
government's
capacity for planning
and implementing
school feeding
• Increasing
attentiveness and
learning
Have there been any
unexpected results (positive or
negative)?
• Comparison of most
recent output data
with baseline and
targets
• Comparison of most
recent outcome data
with baseline and
targets
• Qualitative analysis
by GoE, WFP, DP
and other federal and
local observers/actors
of outcome-level
performance
• Analysis of
government
implementation the
Emergency School
Feeding programme
as an indicator of
capacity
• Qualitative analysis of
the views expressed
by beneficiaries at
local level (parents,
pupils, community
learners)
(logframe indicators
related to percentage of
pupils who are attentive,
number of school meals
provided, cleaning of
cooking and eating
equipment, percentage of
schools with CHILD
planning team, persons
trained by type of training,
and enrolment rates)
• WFP performance data
• Analysis of EMIS data
comparing schools
inside and outside of
the programme
• GoE data on
Emergency School
Feeding programme
• Survey
• Interviews at federal,
regional, woreda and
school level
• Programme
documentation and
Government reports
• Cross-check recorded
output and outcome
data with
programme/governmen
t documentation and
informants in GoE and
at schools visited in
field
• Triangulate views on
the key outcomes
between different
informant groups
• EMIS, WFP monitoring
data and survey results
will be triangulated to
evaluate data reliability
and consistency.
Strength of evidence:
Moderate. WFP
performance data has been
provided. Not possible to
assess the strength of the
EMIS data which remains
to be provided by
government. Field work in
January will provide initial
insights into the strength of
records at local level.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
95
Specific questions Analysis/indicators Main sources of
information
Triangulation approach
EQ5. What have been the
gender and equity dimensions
of the programme's results?
• Analysis of output￾and outcome-level
performance data
compared with design
targets
• Qualitative analysis
by GoE, WFP, DP
and NGO observers
of programme’s
gender equality and
equity performance
against WFP and
GoE criteria
• Qualitative analysis of
interviews with
beneficiaries
• Analysis of the impact
of the take-home
rations on girls and
boys and at
household level
(logframe indicators
related to gender
disaggregated analysis of
percentage of pupils who
are attentive, number of
school meals provided,
cleaning of cooking and
eating equipment,
percentage of schools with
CHILD planning team,
persons trained by type of
training, and enrolment
rates)
• WFP performance data
• WFP internal reporting,
and
documentation/reports
by other partners
• Analysis of EMIS data
• Survey
• Interviews, including
with the participants in
national level policy
forums
• Cross-check recorded
performance data and
survey data with
informants in GoE and
at schools visited in
field
• Compare WFP
perceptions of gender
equality and protection
performance with those
of GoE and DP, NGO
informants
Strength of evidence:
Moderate. WFP
performance data has been
provided. Not possible to
assess the strength of the
EMIS data which remains
to be provided by
government. Field work in
January will provide initial
insights into the strength of
records at local level.
EQ6. What has been the
(intended or unintended)
impact of the programme?
• What are the changes
that the intervention
contributed to?
• Counterfactual
analysis of what those
outcomes would be in
the absence of the
intervention
programme?
• Cause – effect
relationships of key
results – What are the
impact pathways?
• What are the intended
and unintended
results/outcomes?
• What are the negative
• WFP performance data
• WFP project reports
• Analysis of EMIS data,
comparison schools
inside and outside the
programme
• Survey
• Interviews, including
with the beneficiaries
• Cross –check
performance data,
survey and stakeholder
perceptions
• Compare analysis of
EMIS data with the
results of the survey
Strength of evidence:
Moderate. WFP
performance data has been
provided. Not possible to
assess the strength of the
EMIS data which remains
to be provided by
government. Field work in
January will provide initial
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
96
Specific questions Analysis/indicators Main sources of
information
Triangulation approach
and positive
consequences of the
intervention?
insights into the strength of
records at local level.
Key Question 3: What factors affected the results?
EQ7. How efficiently was
the programme implemented?
To be analysed in terms
of:
• logistics
efficiency –
timeliness of
deliveries,
pipeline breaks
etc.
• extent to which
complementaritie
s were achieved
between the
programme’s
interventions and
interventions of
relevant
humanitarian and
development
partners as well
as other WFP
country office
interventions in
the country? How
did these
complementaritie
s contribute to
savings and
efficiency?
• cost-efficiency –
relevant unit cost
comparisons
• to what extent
were programme
management
practices and
tools adequate to
implement the
programme?
• were programme
resources
adequate and
available on time
to implement the
activities as
planned?
• Assessment of WFP
SPRs and other
reporting for
commentary on
internal factors
positively or negatively
affecting performance:
including staffing
levels, financial
resources, pipeline
issues
• Programme reporting
and other relevant
WFP documentation
• Reports by GoE and
other DPs on events
and trends during the
review period
• Qualitative
assessment by
GoE, WFP and
community/school
level informants of
positive or
negative influence
of internal WFP
factors
• Compare assessment
by responsible WFP
personnel and views of
external stakeholders
and observers and
compare views at
different levels (federal,
regional, woreda,
schools)
Strength of evidence: Good
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
97
Specific questions Analysis/indicators Main sources of
information
Triangulation approach
EQ8. To what extent did
internal/external factors and
processes contribute to the
changes and results
achieved:?
• Internal factors
(within HQ control):
the processes,
systems and tools in
place to support the
programme design,
implementation,
monitoring/evaluation
and reporting; the
governance structure
and institutional
arrangements
(including issues
related to staffing,
capacity and technical
backstopping from
RB/HQ); the
partnership and
coordination
arrangements; etc.
• External factors: the
external operating
environment; the
funding climate;
external incentives
and pressures etc.
Examine whether
assumptions in the
(implicit) theory of change
held true.
• Project time-line
• Programme reporting
and other relevant
WFP documentation
• Reports by GoE and
other DPs on relevant
political and policy
events and trends
during the review
period
• Interviews
• Compare assessment
of factors by WFP CO
and field staff
• Compare assessment
of factors by WFP and
GoE staff
• Compare assessment
of factors by WFP staff
and community/school
level informants
Strength of evidence: Good
EQ9. What was the quality
of the monitoring and reporting
system? Did this enhance or
impair the performance of the
programme?
• Review quality of
WFP, McGovern-Dole
and GoE monitoring
and reporting against
key objectives of the
programme and
standards of good
practice
• Analyse content,
timeliness and
external perceptions
of monitoring and
reporting
arrangements and the
extent to which these
have been used to
inform decision
making
• Determine whether
monitoring reports are
just a procedural
• WFP reports and M&E
systems
• Records of meetings
between WFP and
GoE and of key
decisions taken
• SABER
• Interviews with WFP
staff, GoE, and
external stakeholders
at different levels
• Compare assessment
by WFP staff and GoE
Strength of evidence: Good
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
98
Specific questions Analysis/indicators Main sources of
information
Triangulation approach
statement of
performance data or
offer any analysis of
issues affecting
performance
• Assess to what extent
M&E information was
used to adapt and
improve
implementation
• Assess to what extent
there was flexibility in
programme
implementation
Key Question 4: To what extent are the programme results sustainable?
EQ10. To what extent will
household food security for
school going boys and girls be
sustained without / beyond
USDA/WFP funding?
• Analysis of evidence
collected through in￾depth interviews with
beneficiaries of school
feeding and take￾home rations
• Analysis of
documentary
evidence from other
regions where school
feeding has ended
(e.g. under the
emergency school
feeding programme)
• Interviews
• Document review
• Document review and
analysis of financial
data to judge the
trajectory of sector
funding against
components with
commitments, track
record, political
outlook…
Strength of evidence: Good
EQ11. To what extent has
the programme prepared the
Ethiopian Government and the
education system to ensure
that they can continue school
feeding in the Afar and Somali
region without / beyond
USDA/WFP funding? To what
extent will the GoE be able to
mobilize and sustain funding
for school feeding for the Afar
and Somali regions?
• Analysis of
programme reporting
on planning and
implementation of
appropriate steps
towards handover and
sustainability
• Evidence from GoE's
emergency SF
programme on
GO=OE capacity and
the adaptation of
McGovern-Dole
approaches or other
appropriate
approaches
• Programme
performance
documentation
• SABER
• Analysis of funding
trends by GoE to
school feeding
• Interviews
• Focus group
discussions during
mission field work
• Compare the views of
WFP, GoE and other
policy and programme
observers
• Compare assessment
in Addis Ababa and
regional capitals with
that in sample
communities and
schools
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
99
Specific questions Analysis/indicators Main sources of
information
Triangulation approach
Key Question 5: What are the main lessons that can be learned from this programme?
EQ12. What lessons from
this programme should
influence future programmes
(Including good practices to be
emulated and weaknesses to
be mitigated)?
• draw together
analysis from
previous EQs
• based on findings
against the previous
EQs.
• compare this
programme's
experience with others
in Ethiopia and
elsewhere of which the
evaluators have
knowledge
Strength of evidence: Good
Incorporation of evaluation questions from the TOR
9. TOR requirements included:
35.....The evaluation team should clearly outline in an evaluation matrix, what questions will be
answered using which methods and sources of information. This will be developed during the
inception stage and agreed on by all stakeholders.
38. The bidding companies are expected to expound on the specific approach and methodology
that will be used to answer all the evaluation questions below. Additionally, the consultant should
illustrate how the comparison between intervention and comparison schools will be undertaken.
Gender analysis should be part of all the relevant evaluation questions
10. Table 12 below lists all the evaluation questions that appeared in the TOR and shows
how they are now incorporated in the evaluation matrix (Table 11 above). In some cases
questions have been merged, or treated as part of the analysis required to answer a higher￾level question. The sequence of questions has been modified to match the logical sequence
of the analysis by the evaluation team, and some have been listed under a more appropriate
evaluation criterion. Nonetheless, all questions in the TOR have been factored into the
evaluation matrix.
 Mapping of TOR questions onto the evaluation matrix
EQs as they appear in the TOR Where incorporated
Relevance
To what extent were interventions appropriate to the needs of boys, girls, and parents
of school-age children?
EQ1
To what extent were interventions aligned with relevant national policies, including
sectoral policies and strategies?
EQ2
To what extent were interventions coherent with WFP strategies, policies and
normative guidance?
EQ2
To what extent were the interventions aligned with the needs of other key
stakeholders?
EQ2
To what extent was the intervention design based on sound gender analysis? EQ3
To what extent was the design and implementation of the intervention sensitive to
[GEEW]?
EQ3
Efficiency
To what extent did the project attain peak pipeline performance? EQ7
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
100
EQs as they appear in the TOR Where incorporated
What were the complementarities between the programmes interventions and
interventions of relevant humanitarian and development partners as well as other WFP
country office interventions in the country? How did these complementarities contribute
to savings and efficiency?
(including analysis/indicators
column)
Did the interventions within the programme offer the best value for money for WFP? as above58 (reference to cost￾efficiency)
To what extent were project management practices and tools adequate to implement
the project?
EQ7
(including analysis/indicators
Was project resource adequate and available on time to implement the activities as column)
planed?
What are the main lessons learned in terms of future similar projects? EQ12
What are some of the key best practices from this project EQ12
Effectiveness
What are the main expected and unexpected results of the programme? EQ4
To what extent has this programme achieved the planned outcomes and objectives on:
• Alleviating short term hunger and retaining boys and girls in school
• Promoting essential packaged interventions and infrastructure at school
• Strengthening the governments capacity for planning and implementing
school feeding
• Increasing attentiveness and learning
• Promoting school health and nutrition interventions in primary school for boys
and girls
• Improving access to and equity of education for primary school boys and girls
particularly for children from rural areas of emerging regions and underserved
areas.
EQ4
Did the assumptions hold true? EQ8 (analysis: "Examine
whether assumptions in the
(implicit) theory of change held
true.")
To what extent did internal/external factors and processes contribute to the changes
and results achieved:
• Internal factors (within WFP’s control): the processes, systems and tools in
place to support the operation design, implementation, monitoring/evaluation
and reporting; the governance structure and institutional arrangements
(including issues related to staffing, capacity and technical backstopping from
RB/HQ); the partnership and coordination arrangements; etc.
• External factors: the external operating environment; the funding climate;
external incentives and pressures and overall sustainability of the programme.
EQ8 (incorporated in
analysis/indicators column)
To what extent is the country taking ownership of the programme? (e.g. demonstrated
commitment and contribution to the programme);
EQ11
58 As noted in Mokoro's technical proposal for this evaluation: 
Our perspective on efficiency and VFM will be based on the definitions and guidance provided in
the EQAS technical note on efficiency (prepared for OEV by Stephen Lister and Robrecht Renard
of Mokoro). If the MGD intervention records allow unit cost calculations, we will seek to
benchmark unit costs for SF in Afar in Somali with unit costs for other SF interventions in
Ethiopia, and possibly with remote-area programmes run by WFP in other countries. It will not
be possible to answer literally whether “best value for money“ has been achieved, since that
would require knowledge of all other options’ costs, but a practical assessment whether the
programmes costs are reasonable compared to others should be feasible. Judgements should take
account of all costs, not only those that accrue directly to WFP.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
101
EQs as they appear in the TOR Where incorporated
What is the national readiness to implement the programme? (e.g., demonstrated
capacity at central and sub-national levels to manage the programme?
EQ11
Impact
What are the changes that the intervention contributed to? EQ4
Counterfactual analysis of what those outcomes would be in the absence of the
intervention project? Analysis under EQ4 (links to
Cause – effect relationships of key results – What are the impact pathways? theory of change approach)
What are the intended and unintended results/outcomes? EQ4
What are the negative and positive consequences of the intervention?
Sustainability
To what extent will the GoE be able to mobilize and sustain funding for school feeding
for the Afar and Somali regions?
EQ11
To what extent has the programme prepared the Ethiopian Government and the
education system to ensure that they can continue school feeding in the Afar and
Somali region without / beyond USDA/WFP funding?
EQ11
To what extent will household food security for school going boys and girls be
sustained without / beyond USDA/WFP funding?
EQ10
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
102
Annex E Organisation of the evaluation
a. Team composition and work plan
1. An overview of the skills of each of the team members in line with the TOR was
provided at the proposal stage. As noted in the proposal the team is highly complementary.
It brings together expertise with: WFP processes and systems (Muriel Visser (MV)); WFP in
Ethiopia (Rita Bhatia) (RB), impact evaluations (Denis Alder (DA), Gadissa Bultosa (GB)
and MV); solid expertise in school feeding (MV and RB) and in nutrition (RB), quantitative
data collection (DA, GB) and qualitative data collection (MV, RB, Doe-e Berhanu (DB));
experience with gender issues and ethical considerations in research design and
implementation (RB, DA); and excellent skills in communication and reporting (MV,
supported by other team members). An overview of specific responsibilities and coverage of
technical topics is provided in Annex F.
2. Team coordination will be a constant feature of the evaluation process and will be
assured, amongst others in the following manner:
• The team worked together Ethiopia for five days during the inception phase and was
joined by the Quality Assurance (QA) manager of Mokoro for that period. This has
ensured a shared understanding of the task, provided an opportunity to review team
processes and deliverables.
• A one-day team workshop while in Ethiopia which was also joined by the QA. This
has further developed the understanding of the assignment, and ensured that the
team has worked together on developing the methodology.
• A team Dropbox provides a shared platform for documentation and for team drafts.
3. The team leader has already initiated close consultation and co-ordination among all
team members. This will be maintained throughout the assignment through regular e-mail
and Skype communications. Annex F provides a full specification of team members’
primary roles and specific tasks, as well as the deliverables to which they contribute and the
dates by which their inputs
b. Timeline and data collection schedule
4. The proposed timeline for the key activities and deliverables for the baseline and end
line is found below. This timeline was agreed with the CO at the proposal stage. A more
detailed timeline is found in Annex G.
 Evaluation Timeline
Phases, Deliverables and Timeline Responsible Dates
Phase 2 - Inception
In-country inception mission ET, WFP Ethiopia
CO, EM
27-29 Nov
In-country internal team workshop ET 30 Nov
In-country follow-up preparatory work on survey W/c 4 December
Submission of draft inception report (IR) to EM ET 15 Dec59
Revise draft IR based on feedback received by DE QS and EM ET 10 Jan
Submission of revised IR based on DE QS and EM QA ET 10 Jan
Submits the final IR to the internal evaluation committee
for approval
ET 24 Jan
59 Brought forward because of unavailability of DEQAS reviewers over the Christmas/New Year period.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
103
Phases, Deliverables and Timeline Responsible Dates
Final approval for IR received from internal evaluation
committee
ET 29 Jan
Sharing of final inception report with key stakeholders for
information
EM, WFP Ethiopia CO,
RB
29 January 2018
Phase 3 –Data collection
Observation visits to schools ET
Data collection ET 2 – 23 Feb
In-country Debriefing (s) ET, EM, WFP
Ethiopia CO,
23 Feb
Phase 4 - Analyse data and report
Draft evaluation report submitted ET 14 March
Revise draft ER based on feedback received by DE QS and EM ET 19-23 March
Submission of revised ER based on DE QS and EM QA N/A 23 March
Revise draft ER based on stakeholder comments received ET 28 March – 4 April
Submission of final revised ER ET 4 April
Sharing of final evaluation report with key stakeholders for
information
EM, WFP Ethiopia CO 4 April
Phase 5 Dissemination and follow-up
Prepare management response EM, WFP Ethiopia CO,
RB, ERG
April 2018
Share final evaluation report and management response
with OEV for publication
EM, WFP Ethiopia CO,
RB, ERG
April 2018
c. Support/Information required
5. It has been agreed that the CO will facilitate arrangements for appointments during
the field mission as well as logistical arrangements for site visits.
6. The CO has indicated that vehicles can be provided for the ET qualitative field work
and for the observations in a selection of schools. The ET may request the CO’s help with
flight reservations within Ethiopia, at the ET’s expense.
 Support and Information Requirements for the Evaluation
# Support/Information required Provider
1 Assistance with obtaining visas and travel permits for two international
evaluators (and for travel of one national evaluator)
CO
2 Schedule initial briefings, meetings, and appointments including: security and
administrative set-up with WFP, briefings and meetings with relevant CO units
and staff, as well as external partners,
CO
3 Schedule follow-up meetings as needed ET
4 Coordinate space and invitations for external and internal briefings CO
5 Schedule domestic flights for all team members if necessary – (ET to purchase) CO
ET responsible for
the cost
6 Provide office space for the evaluation team at WFP CO and Sub-offices CO
7 Provide vehicles and driver for transportation of ET during the observation
work in January and the in-depth interviewing in February
CO
8 Provide IT support as needed CO
9 Arrange and confirm accommodation in the field CO
ET responsible for
cost
10 Assist with fieldwork logistics at all sites CO
11 Provide names and contact information for individual stakeholders CO
12 Arrange de-briefing (invitations, venue, equipment etc.) - stakeholders to be
agreed between ET and CO
CO + ET
13 Support ET to resolve any additional information and documentation gaps CO
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
104
Annex F Team Roles and Responsibilities
Team members' expertise
1. Dr Muriel Visser (Team Leader) has over 25 years’ consultancy experience, with
particular expertise in policy analysis, programme design, evaluation and aid management,
as well as technical specialisms in gender analysis, education, HIV/AIDS and health. Muriel
is a highly experienced evaluator and team leader and has led large multi-donor country
programme evaluations for various bilateral and multilateral agencies (WFP, UNICEF,
UNAIDS, UNESCO, Irish Aid, DFID, DANIDA, EU,) using the OECD DAC criteria. Muriel is
currently leading the major evaluation of the 11th European Development Fund, the largest
of the EU’s Financing Instruments. She was also the team leader for an evaluation of a
baseline evaluation of School Feeding in Kenya She was also the team leader for the
evaluation of the inter-agency initiative REACH (Renewed Efforts Against Child Hunger),
which included global level engagement and analysis, as well as eight in-depth country case
studies (WFP,2015) , and was a member of the evaluation team undertaking the
Independent Comprehensive Evaluation of the SUN Movement (2014), with specific focus
on governance and supporting the evaluation methodology. Muriel has considerable
experience of evaluating WFP school feeding programmes: in 2016, she led the team
undertaking the school feeding programme evaluation in Sao Tome and she also led the
team undertaking an evaluation of WFP’s development operation in Ivory Coast, which
included a focus on WFP’s school feeding and take-home ration interventions and capacity
development. Muriel’s experience in the education sector include as an evaluator of
UNICEF’s upstream work in basic education and gender equality, designing and testing of a
practical tool for country stakeholders to support the mainstreaming of gender into
Education Sector Plans for UNGEI, and researching for the design of a pilot project for
testing demand side intervention in primary education in Mozambique. Muriel has
considerable expertise in statistical analysis, with a PhD in Health Communication with a
minor in Measurement and Statistics from the University of Florida (2004). Muriel has
worked across Africa (in more than 20 countries) and has strong regional experience from
Eritrea, Kenya and Uganda.
2. Dr Denis Alder (Senior Evaluator) is a specialist statistician and data analyst with
more than 40 years’ experience of supporting research in international development
through supporting/leading survey design, statistical modelling and data analysis for
monitoring and assessment of indicators. His knowledge and expertise includes use of R
statistical modelling, SQL database design, GIS, design of sample surveys including
stratified, cluster, multi-stage, multi-phase models, linear and nonlinear models and
regression and Bayesian methods. Denis previously worked as a World Bank Technical
Specialist, monitoring, evaluating and reporting for a large export sector development
project and was Head of Statistics for the British Potato Council where he developed
innovative surveys for crop yields, farm gate market prices, GIS mapping of crops and yield;
and planning and undertaking food production surveys, market surveys, household income,
and usage of household fuels. Most recently he has supported the National Information
Platform for Nutrition (NiPN) initiative, conducting a review of the NiPN anthropometry
data quality toolkit.
3. Rita Bhatia (Senior Evaluator) has over 25 years of extensive experience in
managing nutrition and public health programmes, working in humanitarian and
development operations. She has worked with UN agencies and NGOs in the areas of public
nutrition, school feeding, education, health, food security, protection and HIV AIDS. The
emphasis of her work has been on conceptualization, policy and strategy and programme
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
105
planning, management and evaluation. Ms. Bhatia has held a number of key positions with
lead on public nutrition and health. This culminated in her work as a Senior Regional
Programme Advisor- for the World Food Programme (WFP) in Thailand, with the primary
responsibility of managing public health and nutrition programmes in Asia and the Pacific.
Previously, she has worked with WFP in its global office in Italy and the United Nations
(UNHCR) in Africa and its global office in Switzerland. During her assignments at the
global offices she provided technical and programme support on strategic health, nutrition
and school feeding and capacity development programmes. Rita was a core team member of
the Mokoro team which undertook three simultaneous midterm evaluations of WFP USDA
McGovern-Dole funded school feeding programmes in Bangladesh, Laos and Nepal (2016).
4. Gadissa Bultosa (Senior Evaluator) is a highly qualified Social Statistician and
evaluator with over 34 years of experience in socio-economic and baseline surveys;
feasibility studies; data management; rapid appraisal methods; project/programme design,
implementation and management; impact assessment, monitoring and evaluation; and
social accountability instruments and processes. An expert in programme monitoring and
evaluation, Gadissa recently supported the evaluation of Finland’s country strategies and
country strategy modality as a team member and senior evaluator of the Ethiopia evaluation
team (2015-16). He was also the deputy team leader, statistician and M&E expert for a
Programme Level Monitoring & Evaluation for the Climate High Level Investment
Programme (CHIP), and for the Strategic Climate Institutions Programme (SCIP) (DFID,
2012-16). Gadissa’s relevant sectoral expertise includes conducting a midterm review of a
Pastoral Community Development Project in the Somali, Afar, Oromia and SNNP regions of
Ethiopia (2011). Gadissa has also supported numerous food security assignments, including
the Western Ethiopia Integrated Environment and Food Security Development Programme,
for which he was a Programme Adviser (2011). He was also a Socio-Economic/M&E expert
for the evaluation of the Mena-Sibu Integrated Food Security Project, and Project Design
for Mena-Sibu Environmental Rehabilitation and Food Security Project (2011). An
Ethiopian national with extensive country experience, Gadissa brings extensive regional
know how to the team, and experience of working with government agencies as well as
donor agencies and international organizations.
5. Doe-e Berhanu (Research Coordinator) is a sustainable development professional
with 11 years’ experience in the areas of gender analysis, agriculture, market based
livelihoods and food security. An Ethiopian national, Doe-e has broad and varied experience
of supporting development programmes from across the country in a variety of research
coordination roles. Her experienced is complemented by extensive international study and
work experience in countries including the USA and South Korea. Doe-e has contributed to
impact evaluations, reviews and the development of results-based M&E frameworks,
including a review of Ethiopia’s large-scale commercial agricultural policy, institutional
arrangements and implementation to identify challenges and policy intervention areas for
the subsector. Doe-e has worked with international organisations including WFP and
Oxfam. She supported the WFP as a Programme Officer in Juba, South Sudan between
2012-14, providing support to programme planning and management for the South Sudan
National Strategic Food Reserve (NSFR) Special Operations. She held a position as regional
Communications and Information Officer for Oxfam America’s Horn of Africa office in
Addis Ababa, a role that included providing strategic leadership for Oxfam America
communications relating to the Horn of Africa. Doe-e is a fluent English speaker with
native-proficiency in Amharic and intermediate proficiency in Afaan Oromo.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
106
Roles and responsibilities
 Evaluation team roles and responsibilities
Team member, Role and Profile Responsibilities/Coverage
Muriel Visser, Team leader • Team Leader with overall responsibility for all aspects of the
evaluation and the supervision and support of other team
members.
• Will be responsible for overall design, implementation, reporting
and timely delivery of the evaluation products.
• Principal liaison with WFP evaluation manager (EM), overall
direction of the evaluation team (ET).
• Technical areas of focus to include gender, health and education.
• Leads on GEEW at IR stage and ensures full mainstreaming
during the evaluation stage.
• Leads inception mission in Addis Ababa.
• Works to support the development and finalisation of data
collection instruments, sampling frame and design for the
evaluation groups.
• Leads the elaboration of the methodology and approach in the
inception phase, and the drafting of the Inception Report (IR).
• Leads main fieldwork mission to Ethiopia.
• Leads the preparation of the draft Evaluation Report (ER) and its
revision in response to feedback. Supports the finalisation of the
report through the DEQAS process.
• Production and delivery of the evaluation brief.
• Highly experienced aid￾effectiveness expert
• Significant experience of leading
teams to undertake large,
complex evaluations
• Specific expertise in the areas of
education and gender equality
and health
Rita Bhatia, Senior evaluator • Thematic responsibility for child nutrition and health dimensions
of school feeding; contributes to institutional and capacity
analysis, with special focus on the quality of M&E systems,
• Joins inception mission to Addis Ababa.
• Joins main evaluation mission in Ethiopia.
• Provides support to the design of survey instruments.
• Contributes to Inception Report and Evaluation Report.
• Qualified nutritionist with more
than 25 years’ experience of
nutrition programming,
evaluations, training and policy
development.
Denis Alder, Senior Evaluator • Thematic responsibility for the statistical and data management
aspects of the survey, including supporting the design of data
collection instruments, sampling strategy and evaluation
methodology. Support to data management, including providing
oversight of data cleaning process. Support to the analysis of
survey data. Joins briefings and inception workshop in Addis
Ababa
• Joins main evaluation mission in Ethiopia.
• Contributes to Inception Report and Evaluation Report.
• Joins inception mission
• Joins main evaluation mission in Ethiopia.
• Contributes to Inception Report and Evaluation Report.
• Senior statistician and data
analyst
• Expertise in range of tools such
as R statistical modelling, SQL
database design, GIS, design of
sample surveys including
stratified, cluster, multi-stage,
multi-phase models, linear and
nonlinear models and regression
and Bayesian methods.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
107
Team member, Role and Profile Responsibilities/Coverage
Gadissa Bultosa, National
consultant and evaluator • Brings thematic expertise in rural development, capacity
development, survey methodologies, survey implementation, and
statistical analysis.
• Participates in inception mission and detailed design of the SI
• Responsible for coordinating survey implementation, including
the recruitment and training of enumerators, sourcing translation
services as required, assisting in the procurement of survey tools
as required (such as electronic tablets), sourcing vehicles and
assisting in the coordination of local transport as required.
• Works with the lead statistician to oversee the implementation of
a statistically rigorous survey.
• Joins main evaluation mission and contributes to Evaluation
Report.
• Highly qualified social
statistician
• National consultant with
significant experience of both
Somali and Afar regions of
Ethiopia
• Considerable expertise in the
field of survey implementation
and the recruitment and training
of field enumerators.
Doe-e Berhanu Research
Coordinator • Ethiopia-based liaison with the evaluation management team in
Oxford. Assists in sourcing documents and data, managing the
team’s e-library, and setting up programmes for the main
inception and field-work missions.
• Under TL direction, undertakes literature review and data
analysis, and assists with scheduling of interviews and planning of
the field visit.
• Participates in inception and field missions, and assists liaison
between Mokoro and WFP ETHCO between missions.
• Provides coordination with WFP on data gathering including the
organising of interviews and planning of fieldwork.
• Contributes to the Inception Report and Evaluation Report, and
supports the Team Leader in compiling reports and responding to
comments from stakeholders and DEQAS. Joins the inception
mission meetings in Addis
• National consultant with
international experience and
experience inter alia in
sustainable livelihoods, capacity
building, gender analysis and
evaluation methodologies.
• Previous role with WFP in South
Sudan (2012-14)
Stephen Lister and Alistair
Hallam,
Quality Support Advisors
• The Quality Support Advisors review deliverables and advises on
the relevance, credibility and practicality of the evaluation’s
approach (at inception report stage) and of its findings,
conclusions and recommendations (at the evaluation report
stage). In particular, Stephen Lister provides guidance on DEQAS
compliance to the Team Leader, and Alistair Hallam reviews
impact evaluation methodology and delivery.
• More than 60 years of combined
expertise in international
development and humanitarian
settings
• Significant experience of
providing QS to WFP evaluations
Christine Fenning
Research Support • Provides guidance and assistance to the in-country Research
Coordinator as required, supporting key research processes and
ensuring compliance with DEQAS in all research processes
employed and templates used.
• Evaluator and research analyst
with highly-relevant recent
experience of evaluating WFP
School Feeding Programmes
Philip Lister and Jim Grabham
Assignment Editor; Logistics and
Research Assistance
• Under the guidance of the Team Leader and Researcher, provide
additional support including data and document assembly,
recording interviews; ensure logistical arrangements, and
assisting in the preparation of drafts.
• Proofreading, editing, and formatting of evaluation products to
WFP's high standards.
• Mokoro staff members with
decades of combined experience
in ensuring the quality of
Mokoro’s outputs and
assignment support
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
108
Annex G Detailed evaluation timeline
Phases, Deliverables and Timeline Responsible Agreed dates
Phase 1 - Preparation
Selection and recruitment of evaluation team WFP Ethiopia
CO, RB
Pre-contract meeting with winning company to ensure team availability WFP Ethiopia
CO, RB, ET
Signing of service contract WFP Ethiopia
CO, RB, ET
October 2017
Phase 2 - Inception
Virtual briefing, to allow for remote introduction to the team, preliminary discussions
regarding the inception phase, clarification of questions on the TOR, preparation of the
inception mission, and preliminary preparations for the inception phase
WFP Ethiopia
CO, RB, ERG,
ET, EM
October 2017
(following
contracting)
Preliminary activities to include review of available data, sourcing outstanding data,
drafting and finalising schedule for inception mission, and preliminary work on the
inception report including assessment of the feasibility of the control group, location of
schools etc.
ET, WFP
Ethiopia CO
October –
November
Selected other phone meetings with technical staff of WFP and the evaluation team
ahead of the evaluation inception mission
Week of 20
November
Introductory briefing core team (in country) by WFP office WFP Ethiopia
CO, RB, ERG,
ET, EM
27 Nov
In-country inception mission ET, WFP
Ethiopia CO,
EM
27-29 Nov
In-country internal team workshop ET 30 Nov
In-country follow-up preparatory work by sub-section of the Evaluation team on
evaluation tools, preparation of research team etc.
Submission of draft inception report (IR) to EM ET 15 Dec
Sharing of draft IR with outsourced quality support service (DE QS) and quality
assurance of draft IR by EM using the QC
EM, WFP
Ethiopia CO,
RB
15 – 22 Dec
Return of IR to ET with first round of DEQAS comments EM 22 Dec
Revise draft IR based on feedback received by DE QS and EM ET 22 Dec - 10 Jan
Submission of revised IR based on DE QS and EM QA ET 10 Jan
Circulate draft IR for review and comments to ERG, CO and RB EM, WFP
Ethiopia CO,
RB, ERG
10-17 Jan
Return of IR to ET with second round of DEQAS comments EM 17 Jan
Revise draft IR based on stakeholder comments received ET 17-24 Jan
Submits the final IR to the internal evaluation committee for approval ET 24 Jan
Final approval for IR received from internal evaluation committee ET 29 Jan
Sharing of final inception report with key stakeholders for information EM, WFP
Ethiopia CO,
RB
29 January 2018
Phase 3 –Data collection
Data collection ET Survey:
2 – 23 Feb
ET:
12 -23 Feb
In-country Debriefing (s) ET, EM, WFP
Ethiopia CO,
23 Feb
Phase 4 - Analyze data and report
Draft evaluation report submitted ET 14 March
Sharing of draft ER with outsourced quality support service (DE QS) and quality
assurance of draft ER by EM using the QC
EM, WFP
Ethiopia CO,
14-19 March
Return of ER to ET with first round of DEQAS comments N/A 19 March
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
109
Phases, Deliverables and Timeline Responsible Agreed dates
Revise draft ER based on feedback received by DE QS and EM ET 19-23 March
Submission of revised ER based on DE QS and EM QA N/A 23 March
Circulate draft ER for review and comments to ERG, RB and other stakeholders (list
key stakeholders)
EM, WFP
Ethiopia CO,
RB, ERG
23 – 28 March
Return of ER to ET with second round of DEQAS comments N/A 28 March
Revise draft ER based on stakeholder comments received ET 28 March – 4
April
Submission of final revised ER ET 4 April
Sharing of final evaluation report with key stakeholders for information EM, WFP
Ethiopia CO
4 April
Phase 5 Dissemination and follow-up
Prepare management response EM, WFP
Ethiopia CO,
RB, ERG
April 2018
Share final evaluation report and management response with OEV for
publication
EM, WFP
Ethiopia CO,
RB, ERG
April 2018
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
110
Annex H Detailed field work schedule
Activity Dates Team
member
Locations/
sites
Stakeholders
Field work for observation of school feeding – January
Travel to Afar region 4/1/18 DB Afar
Curtesy meeting at regional level 4/1/18 DB Afar Regional education bureau
WFP regional office
Observation visit school 1 4/1/18 DB Afar School director (contact
person for the school visit)
Observation visit school 2 5/1/18 DB Afar School director
Observation visit school 3 5/1/18 DB Afar School director
Return to Addis 6/1/18
Travel to Somali 15/1/18 DB Somali
Curtesy meeting at regional
level60
15/1/18 DB Somali Regional education bureau
WFP regional office
Observation visit school 1 15/1/18 DB Somali School director
Observation visit school 1 16/1/18 DB Somali School director
Observation visit school 2 16/1/18 DB Somali School director
Return to Addis 17/1/18 DB Somali
Survey Administration – February
Flight (from A/A to the respective
regions); contact entree
facilitators; accommodation; visit
to BoE & admin work
5/2/18 S161 (Afar); S2
& S3 (Somali)
Afar &
Somali
Bureau of Education (BoE)
of the respective regions
Training enumerators (at Semera
and Jigjiga/ Gode)
6/2/18 ST1 (Afar); ST2
& ST3 (Somali)
Afar &
Somali
BoE of the respective
regions
Travel; admin work; school level
interviews (with relevant teachers
& students: girls & boys); KIIs
7/2/18 –
24/2/18
ST1 Afar Education office of sample
Woredas & schools (in
Afar)
Travel; admin work; school level
interviews (with relevant teachers
& students: girls & boys); KIIs
7/2/18 –
24/2/18
ST2 & ST3 Somali Education office of sample
Woredas & schools (in
Afar)
Travel back to the respective
region’s capital
25/2/18 ST1 (Afar); ST2
& ST3 (Somali)
Afar &
Somali
Flight back to Addis Ababa 26/2/18 S1 (Afar); S2 &
S3 (Somali)
Completed questionnaire
submitted to B&M; data entry &
transmit to the Statistician
27/2/18 –
5/3/18
4 Encoders Afar &
Somali
B&M will design data entry
interface prior to survey
completion
In-depth interviews – February
Arrival team in Addis 5/2/18 MV and RB
Initial meeting with WFP on field
work arrangements
5/2/18
afternoon
MV, RB and
DB
Addis WFP Office
Travel from Addis to Somali 6/2/18 MV, RB, DB
60 This meeting will also be used by DB to prepare the field work in February by informing regional authorities
and WFP field offices of the procedure and requesting support in informing the schools and other authorities.
61 S1 = Supervisor 1; ST1 = Survey Team1; etc
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
111
Activity Dates Team
member
Locations/
sites
Stakeholders
Regional interviews Somali 6/2/18 MV, RB, DB Somali Regional education office
School 1 – Somali 7/2/18 MV, RB, DB Somali Woreda, Kebele officials,
school stakeholders, pupils,
parents
School 2 – Somali 8/2/18 DB + translator Somali Woreda, Kebele officials,
school stakeholders, pupils,
parents
School 3 – Somali 8/2/18 RB + translator Somali Woreda, Kebele officials,
school stakeholders, pupils,
parents
School 4 – Somali 8/2/18 MV +
translator
Somali Woreda, Kebele officials,
school stakeholders, pupils,
parents
Regional interviews continued 9/2/18 MV, RB, DB Somali NGOs
WFP Office
Return travel to Addis 9/2/18 MV, RB, DB
Travel to Afar 11/2/18
Regional interviews Afar 12/2/18 RB, DB Afar Regional education office
WFP field office
School 1 – Afar 13/2/18 RB + translator Afar Woreda, Kebele officials,
school stakeholders, pupils,
parents
School 2 –Afar 13/2/18 DB+ translator Afar Woreda, Kebele officials,
school stakeholders, pupils,
parents
School 3 – Afar 14/2/18 RB+ translator Afar Woreda, Kebele officials,
school stakeholders, pupils,
parents
School 4 - Afar 14/2/18 DB + translator Afar Woreda, Kebele officials,
school stakeholders, pupils,
parents
Return travel from Afar 14/2/18
National level interviews (in
parallel to Afar field work)
12 to
14/2/18
MV Addis
various
locations
MoE
WFP
Other stakeholders
Team work in Addis 15/2/18 MV, RB, DB WFP Office ET
Debriefing of preliminary
qualitative findings from field
work with CO
15/2/18 MV, RB, DB WFP Office WFP and other
stakeholders (if invited)
MV departs from Addis 15/2/18 MV
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
112
Annex K The McGovern-Dole Programme in Ethiopia
Introduction
1. This annex provides basic information about the McGovern-Dole-supported School
Feeding Programme (FFE-663-2013/026-00) in Ethiopia from 2014 to 2017. It includes
summaries of the programme’s design, its implementation, and performance data available
so far.
Context and scope
2. WFP introduced SFP in Ethiopia in 1994. The initial pilot targeted 250,000 children
in 40 primary schools in Amhara, Tigray, Afar and Oromia Regions, targeting chronically
food insecure districts, and was further expanded to SNNPR and Somali Region in 2002.
Between 1994 -2014, the programme was able to reach 605,540 students in 1,186 schools
throughout the six regions. Beginning in 2014, the programme narrowed the geographic
scope of its operations and has since been working in four regions (Afar, Somali, Oromia
and SNNPR), targeting 488,000 beneficiary students (WFP, n.d.-a).
3. Under the McGovern-Dole International Food for Education (FFE) and Child
Nutrition Programme, the USDA provided WFP Ethiopia’s SFP with a grant of
USD40,707,805 obligated against budget fiscal year 2009, 2011 and 2016 funds to cover
activities of the four year SFP until 2017. USDA signed the McGovern-Dole commitment
letter on November 09, 2012 and project implementation began in January 2014. The
planned McGovern-Dole grant of USD26,500,000 was revised twice. The first modification
was in March 2015 and it obligated an additional USD 2,209,835 against budget fiscal year
2011 funds by increasing the total programme value to USD28,709,835 (WFP, 2011e). The
second modification was done in July 2016 to extend the three-year programme into a four￾year programme and amend the budget by obligating an additional USD11,999,970 against
budget fiscal year 2016 funds, bringing the total programme value to USD40,709,850.
4. The McGovern-Dole grant under review accounts for 13.27 percent of the WFP
Ethiopia Country Programme (CP) 200253 (2012–2015) (WFP, 2011d) budget of
USD306,641,528. Other CP priorities include: i) increasing the capacity of Ethiopia’s
disaster risk management system; ii) enhancing natural resource management in food￾insecure communities and resilience to weather-related shocks; iii) facilitating access to
HIV care, treatment and support; and vi) promoting opportunities for livelihood
diversification and improved access to food markets.
5. The McGovern-Dole grant was designed to support WFP in providing school meals
in 583 schools for 200 days per year for the first three years (2014-2016) and 176 days in
2017. The McGovern-Dole SFP was designed to provide 200,591 primary school children
with daily school meals in 2014, increasing to 220,650 in 2015, 280,179 in 2016 and
289,000 in 2017 alongside capacity building activities that are aimed at supporting
increased dietary and health practices and improved student attendance. The capacity
building component also includes activities to support the capacity of the Government to
develop a National School Feeding Programme (NSFP).
6. The McGovern-Dole-supported SFP currently covers 590 schools (361 in Afar and
229 in Somali Regions), providing school meals in the April – September 2017 reporting
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
113
period to 292,249 children.62 This represents just under one third of the one million
primary school children in these two regions, and five percent of the total of 19 million
primary school children in the country. The McGovern-Dole-supported schools are located
in 32 woredas of Afar Region and 52 woredas of Somali Region (USDA, 2016a).
7. The programme provides children with about one third of their daily calorie intake.
It provides one hot meal per day (composed of 120 g of CSB, 6 g vegetable oil and 3 g salt)
for all targeted children, corresponding to 647 calories per day. In addition, a monthly THR
of two liters (8 liters per semester) of vegetable oil is provided to girl pupils as an incentive
to attend school and encourage parents to allow them to do so.
Objectives and activities
8. The purpose of McGovern-Dole FFE and Child Nutrition Programme interventions is
to ensure equal access is provided for boys and girls at primary schools, with a focus on
marginalized food-insecure areas and vulnerable children. Contrary to other McGovern￾Dole programmes, the intervention in Ethiopia does not target improving literacy or
numeracy levels but instead focuses on the two intermediary results of improved
attentiveness and improved student attendance. Presumably this is a reflection of the fact
that the challenges in the education system are still very much at the level of access rather
than quality for the regions concerned. Furthermore, the MoE’s General Education Quality
Improvement Programme (GEQIP) and other development partners are targeting
improving the quality of literacy and numeracy.
9. The McGovern-Dole grant proposal provides a broad overview of WFP’s previous
school feeding interventions in Ethiopia and the poverty and food insecurity contexts in
Afar and Somali Regions that justify the need for school feeding interventions. However, the
document does not clearly articulate the objectives of the programme and appears to be
more of a draft than a final document.
10. As stated in the USDA Modification II to Commitment Letter to WFP, the
McGovern-Dole SFP specifically aims to:
• Provide school meals to 200,591 primary school-age children in 583 schools for the
first year, 220,650 for the second year, 280,179 for the third year, and 289,000
primary school-age children in 590 schools for the fourth and last year of the
programme. 63
• Provide students in McGovern-Dole SFP target schools with water, sanitation and
hygiene (WASH), health and dietary interventions for each year of the proposed
programme (FY2014 – 17).
• Provide a monthly THR of two liters (8 liters per semester) of vegetable oil to all girls
in the targeted schools based on a minimum of 80 percent attendance rate as an
incentive to attend school and encourage parents to send their girls to school. The
targeted beneficiary girl students for the THRs are 90,243 for the first three years
and 128,783 for the fourth year of the programme.
62 However, the sex disaggregated school meals beneficiary figures of 131,530 girls and 160,755 boys do not
add up to the total number of school meals beneficiary figure of 292,249 children provided for the same
reporting period.
63 Information on the number of intervention schools obtained from WFP’s biannual progress reports to MGD
and field monitoring data.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
114
• Strengthen the Government’s financial and technical capacity to develop a National
School Feeding Programme.
11. The objectives of the McGovern-Dole-supported SFP are aligned to the McGovern￾Dole Strategic Objectives (SOs) 1 and 2. The activities of the McGovern-Dole-supported SFP
are summarized in Table 16 below and the project level results framework is provided at the
end of this annex.
 McGovern-Dole planned school feeding activities
Activity What/Who Partner
Adapt CHILD manuals
for pastoral communities
According to the first bi-annual progress report (01/04/2014 –
30/09/2014), the adaptation of CHILD manuals for pastoralist
communities was done in the previous McGovern-Dole SFP (01/10/2013
– 30/05/2014), not the McGovern-Dole SFP under review
Build/Rehabilitate
schools
Construction of separate latrines for boys and girls, kitchens and
canteens, storage facilities and underground water collection points
BoE,
BoUD
Develop school health
and nutrition planning
and management
capacity
Support to MoE in developing SHN training of trainers (ToT) manual
Provision of SHN training to school directors, teachers, woreda focal MoE
persons and supervisors from Regional BoE, BoH, BoA and Water
Bureaus
Promote health and
Hygiene clubs
Establish clubs and provide training on the concepts of health and
hygiene as well as mini media materials
Promote health and
nutrition education
Develop education information communication materials on nutrition,
hygiene and sanitation practices in schools to improve teachers' and
students' knowledge and practice.
Promote school gardens
Establishment of school gardens, provision of training on school gardens,
supply of seeds and agricultural tools to schools BoA
Pilot livestock production in selected schools
Provide energy saving
stoves School kitchen improvement
Provide financial and
technical support to the
regional education
management system
Develop an online database system to improve the SFP monitoring and
reporting capacity of Afar and Somali BoE
Provision of training on the online database system to each region's
Education Management Information System Directorate and SF focal
persons at the Woreda Education Offices
Technical staff (IT) assigned to support the bureaus
Provide financial and
technical support to the
National School Feeding
Programme
Deploy two technical staff to MoE to provide technical assistance at policy
strategy and programme management level. Deploy another two technical
staff to Somali and Afar to provide operational support for the SFP
implementation
Develop guidelines and training materials
Provide school meals Mid-morning snacks MoE
Provide teaching material
and equipment Provision of text books, science and geography teaching kits, first aid kits BoE
Provide technical support
to improve water and
sanitation facilities
Construction of latrines and water ponds BoE,
BoUD
Provide training on
CHILD planning
approach
Provision of training to school directors, teachers, PTA representatives,
health and agriculture extension workers
Provision of non-food
items to schools
Provision of kitchen shelves and pallets for proper food storage, cooking
pots, plates
Provision of motor bikes to support BoE's monitoring and supervision
activities
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
115
Activity What/Who Partner
Take-home rations Incentive to encourage girls' school enrolment and attendance by
providing 8 liters of cooking oil per semester
Training in food
preparation and storage
practices
Provide onsite training for on food preparation and storage handling
Develop guideline and training manual for cooks and provide training to
the cooks on food safety, personal and kitchen hygiene
Training: Food Handling
and Management for
government counterparts
Provision of food handling and management to school directors and
district level representatives
Source: Compiled from WFP's biannual progress reports to USDA
Planned outputs and outcomes
12. Table 18 and Table 17 below show the planned beneficiaries for the McGovern-Dole
school meals component and the food requirements as per the adjustment made in the
USDA Modification II to Commitment Letter to WFP (USDA, 2016b).
 Planned food requirements for the McGovern-Dole programme
Food item Planned food requirements (MT)
CSB 12,450
CSB+ 6,110
Vegetable oil 4,830
Source: USDA, 2016b
Implementation
Planned beneficiaries for the McGovern-Dole programme
Year Male Female Total
2014 0 0 200,591
2015 0 0 220,650
2016 0 0 280,179
2017 160,000 129,000 289,000
Source: USDA, 2016b
13. The UNDAF is the main programming instrument of the Delivering as One process
in Ethiopia, ensuring coherent, efficient and effective UN system operations that are fully
aligned with the country’s development priorities of the GTP. The Programme
Implementation Manual (PIM) developed in 2013 specifies one common set of procedures
and rules that govern the implementation of all UN assisted programmes in Ethiopia,
focusing on programme formulation, implementation arrangement, annual workplan
preparation, procurement, financial management and reporting, technical assistance, M&E
and auditing (Government of Ethiopia & UN, 2013). Hence, the implementation modalities
of the McGovern-Dole-funded SFP are governed by the common set of procedures.
14. WFP works in partnership with the Government to implement the McGovern-Dole
SFP. The key counterpart ministry for the programme is the MoE, which oversees the
implementation of the programme. The BoE for Afar and Somali Regions are responsible
for the overall management and implementation of the programme in their respective
regions, including handling food transportation from WFP warehouses to the schools,
conducting capacity development trainings in collaboration with WFP and monitoring of
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
116
programme activities. The Regional BoFED in Afar and Somali are responsible for
coordinating the financial planning, transactions and liquidation, including implementation
of the project activities at the regional level and conducting monitoring and review of
project implementation. The non-food activities that contribute to education, water,
sanitation and hygiene (WASH), health and dietary objectives are implemented by WFP in
collaboration with the BoEs. WFP also engages with each Region’s Bureau of Urban
Development (BoUD) on construction related activities such as improving water and
sanitation facilities in programme schools. The schools, through their Food Management
Committees (FMCs), manage the food and non-food items allocated to the schools, mobilize
resources from parents and the school administration for the payment of cooks and the
procurement of firewood and water for the school meals, manage the preparation and
serving of food to the students, manage the distribution THRs, and implement activities
related to the school gardens and school health and nutrition components.
15. Gender dimensions of the intervention. The McGovern-Dole SFP in Ethiopia
seeks to address gender parity and equity through the provision of school meals, THRs that
specifically target girls, and through capacity development at various levels (including
engagement of communities and community organizations), which includes attention to
gender issues. Additional measures that are part of the programme include construction of
separate latrines for boys and girls and provision of water and sanitation. These
improvements are implemented as part of community developed school improvement
plans, which also emphasize gender concerns. Progress against gender and equity indicators
is assessed through quantitative indicators, namely sex disaggregated enrolment rates,
attendance rates, and the GPI. Measurement of progress also takes place in internal WFP
narrative reporting.
Monitoring and evaluation
M&E planned for the McGovern-Dole operation
16. The USDA Commitment Letter to WFP (USDA, 2012b) provides a matrix for
monitoring the performance of the current (Fy2014-2017) McGovern-Dole-funded SFP.
However, the more comprehensive Evaluation Plan (EP) FY2012-2015 covers the last two
years of the previous McGovern-Dole intervention (2012-13) and only the first two years
(2014-15) of the current McGovern-Dole programme under evaluation (WFP, 2014d). The
EP was designed to assess the contribution of the McGovern-Dole funded activities towards
achieving equity in accessing primary education, improved school attendance for both boys
and girls, and narrowing the gender gap in primary education in Afar and Somali Regions.
USDA did not require an external evaluation or end-of-project report at the time of the
previous period, therefore, there was no evaluation conducted at the end of the previous
McGovern-Dole intervention and prior to the commencement of the current phase in 2014.
17. The 2012-15 EP was specifically meant to assess the programme’s effect on
increasing school enrolment and retention rates in the then targeted 583 primary schools by
using non-targeted schools in the two regions as a comparison group. The EP was also
meant to assess the effectiveness of the Children in Local Development (CHILD)
participatory planning tool in contributing to increased school enrolment and attendance
and the extent to which the McGovern-Dole funded activities have contributed to improved
health and dietary practices of school age children in Afar and Somali Regions. The
proposed M&E budget for the FY2012-15 McGovern-Dole programme was USD372,000
(WFP, 2014d).
18. The EP is composed of a baseline, mid-term and final evaluation. For the McGovern￾Dole school feeding programme under review, the outcomes achieved at the end of the
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
117
previous McGovern-Dole phase in FY 2011/12 were taken as baseline figures for continuing
activities, and WFP conducted a baseline survey in that same year for the new indicators
that were included in the current McGovern-Dole intervention (WFP, 2013a). According to
USDA, the requirement for a mid-term evaluation was waived at the time of agreement
negotiation and WFP submitted the UNDAF Ethiopia mid-term review report (Universalia,
2014) in lieu of a formal mid-term evaluation. However, the objectives of the UNDAF mid￾term review had a much larger scope meant to assess the relevance, effectiveness, efficiency
and sustainability of the UNDAF, not specifically the McGovern-Dole school feeding
programme. Hence, the UNDAF mid-term review report does not provide information on
the performance of the McGovern-Dole school feeding programme. As the EP has not been
updated to cover the entire duration of the current McGovern-Dole programme (2014-
2017), it is not clear if the evaluation that is under way for the current McGovern-Dole
programme is covered under the EP.
Past reviews and evaluations
19. As mentioned above, a mid-term review was not carried out for the current
McGovern-Dole intervention, and the UNDAF mid-term review that was submitted in lieu
of a programme review had a much broader scope and did not specifically review the
performance of the McGovern-Dole school feeding programme in Ethiopia. Furthermore, a
review of the baseline and performance targets set both in the McGovern-Dole grant
proposal and the USDA Commitment Letter to WFP for the McGovern-Dole school feeding
programme under review raises the question of how well the findings of the 2011/2012
baseline survey informed the design of the 2014-2017 programme.
Other available M&E sources
20. The Country Programme. The McGovern-Dole grant under review accounts for
13.27 percent of the WFP Ethiopia CP budget of USD306,641,528 (WFP, 2011d). While the
CP SPRs do not disaggregate between McGovern-Dole-funded activities (and beneficiaries)
and activities which are not funded through the McGovern-Dole grant, it will however be
important to review the contribution of the McGovern-Dole grant to WFP’s school feeding
activities and how that is reflected in the monitoring and reporting of the education
component of the CP.
21. Food for Education project monitoring. The Government is responsible for
regular monitoring of the school feeding programme, including reporting on the utilization
of food commodities and the number of beneficiaries. The Regional BoEs conduct
monitoring through their woreda offices and submit monthly monitoring reports to WFP of
its FfEP project as well as quarterly progress reports that include monitoring of programme
activity implementation and a breakdown of commodities received and distributed by
school, using standardized templates. The reports include sex disaggregated monitoring of
school enrolment, student attendance, total number of school days as well as feeding days
over the period, summary of financial contributions to implement interventions using
CHILD approach, and community contribution for the programme.
22. WFP CO is supposed to conduct monitoring visits to all target schools once a
semester.
Performance data
23. In all the three tables for outputs, outcomes and beneficiaries presented in the
following pages, blank cells mean that the biannual progress reports show no data or have
written “N/A” and zero means that zero was reported for that indicator in the period
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
118
concerned. The same is true for the targets set for each year of the McGovern-Dole school
feeding programme.
24. Outputs. Table 19 on page 124 shows the targets set for each of the planned
activities, and the outputs shown in WFP’s biannual progress reports to USDA. The
sequence of activities in Table 19 follows that in the USDA Modification II to Commitment
Letter to WFP and the targets are also as they are shown in the same modification letter
(USDA, 2016b), with comments indicating where changes in target figures have been made.
Some of WFP’s biannual progress reports show different targets than the ones set by both
the USDA Commitment Letter and its second modification and report on activity output for
a specific period which is higher than the target set for the entire year. There is also the
issue of inconsistency in what is reported in the biannual narrative report compared to what
is indicated in the spreadsheet report for the same reporting period. In addition, there is a
repetition of output indicators and possible double counting of construction activities, with
the indicator for latrine construction listed under the “build/rehabilitate schools” and
“provide technical support to improve WASH facilities” activities. Also, there appears to be
a case of double reporting, with one of the indicators (number of guidelines and training
materials developed) under the “provide financial and technical support to the NSFP”
activity reporting the same outputs as what was reported under the activities of “promote
health and hygiene clubs” and promote health and nutrition education.” Similarly, the
USDA Modification II to Commitment Letter to WFP added the “child health and nutrition
training” indicator while at the same time keeping the “school health and nutrition” training
indicator under the same activity, and reporting on both indicators. Furthermore, there is
the inclusion of an output indicator (number of CHILD manuals produced for pastoral
communities) for an activity that is reported to have been implemented during the previous
McGovern-Dole SFP (01/10/2013-30/05/2014). This is further complicated by the fact that
the reporting period for the last report of the pervious McGovern-Dole SFP and the first one
(01/10/2013-30/05/2014) of the current McGovern-Dole SFP under review are
overlapping.
25. Outcomes. Table 20 on page 132 shows the targets set at outcome level, and the
corresponding results shown in WFP’s biannual progress reports to USDA. Like Table 19, it
follows the sequence of results shown in the USDA Modification II to Commitment Letter to
WFP. Targets are as shown in the USDA Commitment Letter to WFP, unless otherwise
noted in Table 20. Results are drawn from WFP’s biannual progress reports to USDA.
Comments are provided where there are inconsistencies between the baseline figures
indicated in the baseline survey, the McGovern-Dole proposal, the USDA Commitment
Letter to WFP, USDA Modification II to Commitment Letter to WFP, and WFP’s biannual
progress reports to USDA. Furthermore, the comments on many of the indicators reveal a
number of adjustments, inconsistencies, apparent errors and variations in reporting. It is
also seen that some of the outcome indicators in fact show outputs, including number of
schools receiving school supplies and materials (McGovern-Dole 1.1.2), number of daily
school meals provided (McGovern-Dole 1.2.1.1), number of individuals benefiting from
USDA-funded interventions (McGovern-Dole 1.3), and percent increase in the number of
schools with CHILD planning team (McGovern-Dole 1.4.4).
26. Beneficiaries. Table 21 on page 136 shows the beneficiary targets as presented in
the USDA Modification II Commitment Letter to WFP, and the corresponding results
shown in WFP’s biannual progress reports to USDA. The comments on some of the
indicators outline a number of changes that have been made to the beneficiary figures as
well as variations from one reporting period to the next. Comments are also provided where
the sex disaggregated school meals beneficiary figures in WFP’s biannual progress reports
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
119
to USDA do not add up to the total number of school meals beneficiaries provided for the
same reporting period. There is also lack of clarity on the “social assistance beneficiaries” as
it seems to be the same albeit rephrased indicator for the school meals beneficiaries.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
120
 Outputs: targets and results
Activity Indicator
Targets64 Results65
FY 2014 FY 2015 FY 2016 FY 2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
1. Adapt CHILD
manuals for pastoral
communities
Number of CHILD manuals
produced for pastoral
communities
1 0 0 0 1 0
Comment
WFP's first biannual progress report (01/04/2014-30/09/2014) showed that this activity was accomplished during the previous reporting period (01/10/2013-30/05/2014).
This is further complicated by the fact that the two reporting periods for the previous McGovern-Dole SFP intervention and the current one are overlapping. Therefore, it is
not clear if this activity was indeed accomplished during the previous McGovern-Dole SFP intervention and not during the current McGovern-Dole SFP under review.
2. Build/Rehabilitate
schools
Number of educational
facilities (i.e. school buildings,
classrooms and latrines)
rehabilitated/constructed as a
result of USDA assistance
0 0 0 30 16 3
Number of educational
facilities (i.e. school buildings,
classrooms and latrines)
rehabilitated/constructed as a
result of USDA assistance
(kitchens, cook areas)
0 20 0 30 10 7 10 22 4 0 34
Comment
USDA Modification II Letter to WFP (USDA, July 25,2016) and WFP's biannual progress reports reworded the first indicator used in the proposal to McGovern-Dole, and
added a second indicator (row 2 above) for kitchens/cooking areas.
Modification II Letter's rewording of the first indicator is not clear, putting the building/rehabilitation of various educational facilities under the same indicator. WFP's
biannual progress reports later specified the first indicator (row 1 above) for latrines, however, the fifth biannual report (O1/04/2016-30/09/2016) showed "provision of
construction materials for the construction of kitchens" in the indicator that it had previously specified for latrines only.
Possible double counting of construction activities under the first indicator (row 1 above). The indicator for latrine construction and associated targets listed under both
"Build/Rehabilitate schools" and "Provide technical support to improve WASH facilities" activities.
64 Targets are as shown in Modification II to the USDA Commitment Letter (USDA, July 25, 2016).
65 Results are taken from WFP’s biannual progress reports to USDA.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
121
Activity Indicator
Targets64 Results65
FY 2014 FY 2015 FY 2016 FY 2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
3. Develop school
health and nutrition
planning and
management
capacity
Number of counterpart staffs
trained in school health and
nutrition
25 25 54 56 36 120 0 54 0 46 0
4. Promote health
and Hygiene clubs
Number of health and hygiene
clubs organized 0 40 82 50 42 45 42 45 116 25
5. Promote health
and nutrition
education
Number of individuals trained
in child health and nutrition as
a result of USDA assistance
0 0 0 156 143 534 136
Number of individuals trained
in child health and nutrition as
a result of USDA assistance
(Female)
0 0 0 57 21 196 62
Number of individuals trained
in child health and nutrition as
a result of USDA assistance
(Male)
0 0 0 99 122 338 74
Number of information
education communication
materials developed on
nutrition and hygiene practices
0 5 0 3 0 3 2
Number of teachers and
students trained in school
health and nutrition
25 25 0 100 25 65 150 60 143 309 100
Number of trainings and
advocacy meetings in school
health and nutrition
1 1 1 1 1 1 0 1 0 1 1
Comment USDA Modification II Letter to WFP (USDA, July 25, 2016) and WFP's biannual progress reports added four more indicators (rows 1 to 4 above) along with the targets.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
122
Activity Indicator
Targets64 Results65
FY 2014 FY 2015 FY 2016 FY 2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
6. Promote school
gardens
Number of school gardens
built and maintained 5 10 20 0 0 22 8 43 72
7. Provide energy
saving stoves
Number of energy saving
stoves provided 0 100 100 100 0 100 0 53 200 0
8. Provide financial
and technical support
to the regional
education
management system
Number of staff employed to
support project
implementation
2 2 2 0 2 2 2 2 2 2 2
Number of trainings provided
as technical support to the
regional education
management system
1 0 0 0 1 0 0 0 0 0
9. Provide financial
and technical support
to the National
School Feeding
Program
Number of guidelines and
training materials developed 0 0 0 4 2 2
Number of national school
feeding monitoring and
evaluation systems developed
0 0 0 1 0 0
Number of school feeding
plans developed 0 0 0 3 4 2
Number of technical advisors
and technical experts
supporting the National School
Feeding Program as a result of
USDA assistance
0 0 0 4 0 4
Comment
USDA Modification II to Commitment Letter to WFP (USDA, July 25, 2016) added this activity to be implemented in FY 2017 and set the indicators and targets.
For the first indicator (row 1 above), WFP's biannual progress reports repeat what has been reported on for "promote health and hygiene clubs" and "promote health and
nutrition education" activities.
10. Provide school
meals
Number of days mid-morning
snacks are provided 200 200 200 176 63 70 0 88 70 57 75
Number of metric tons
provided for school meals for
students
4,690 7,017 2,025 6,409 1,347 2,889 0 4,899.6 2,330 984 4,375
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
123
Activity Indicator
Targets64 Results65
FY 2014 FY 2015 FY 2016 FY 2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
Number of school-aged
children receiving daily school
meals (breakfast, snack, lunch)
as a result of USDA assistance
200,591 220,650 280,179 289,000 234,214 243,981 0 270,509 226,479 160,605 292,249
Number of daily school meals
(breakfast, snack, lunch)
provided to school-age
children as a result of USDA
assistance
0 0 0 50,864,000 21,504,317 6,708,907 23,274,354
Number of school-aged
children receiving daily school
meals (breakfast, snack, lunch)
as a result of USDA assistance
(Female)
0 0 0 129,000 101,174 72,296 131,530
Number of school-aged
children receiving daily school
meals (breakfast, snack, lunch)
as a result of USDA assistance
(Male)
0 0 0 160,000 125,570 88,309 160,755
Number of social assistance
beneficiaries participating in
productive safety nets as a
result of USDA assistance
0 0 0 289,000 160,605 292,249
Number of social assistance
beneficiaries participating in
productive safety nets as a
result of USDA assistance
(Female)
0 0 0 129,000 72,296 131,530
Number of social assistance
beneficiaries participating in
productive safety nets as a
result of USDA assistance
(Male)
0 0 0 160,000 88,309 160,755
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
124
Activity Indicator
Targets64 Results65
FY 2014 FY 2015 FY 2016 FY 2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
Number of students regularly
(80%) attending USDA
supported classrooms/schools
286,110 250,871
Number of students regularly
(80%) attending USDA
supported classrooms/schools
(Male)
158,400 137,901
Number of students regularly
(80%) attending USDA
supported classrooms/schools
(Female)
127,710 112,970
Number of students enrolled in
schools receiving USDA
assistance
289,000 280,179 262,698 292,249
Number of students enrolled in
schools receiving USDA
assistance (Male)
160,000 155,147 145,434 160,755
Number of students enrolled in
schools receiving USDA
assistance (Female)
129,000 125,032 117,264 131,530
Number of individuals
benefitting directly from
USDA-funded interventions
289,000 226,744 160,605 292,249
Number of individuals
benefitting directly from
USDA-funded interventions
(Male)
160,000 120,175 88,309 131,530
Number of individuals
benefitting directly from
USDA-funded interventions
(Female)
129,000 106,569 72,296 160,755
Comment The proposal to McGovern-Dole and the USDA Commitment Letter to WFP (USDA, Nov 9, 2012) had the first three indicators only (rows 1 to 3 above). USDA
Modification II Commitment Letter to WFP (USDA, July 25, 2016) added 15 more indicators and targets (rows 4 to 18 above).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
125
Activity Indicator
Targets64 Results65
FY 2014 FY 2015 FY 2016 FY 2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
In the McGovern-Dole proposal, the FY 2014 target for the second indicator (row 2 above) was 4,960MT but was presented as 4,690MT in the USDA Commitment Letter
(USDA, Nov 9, 2012). The figure (4,960MT) was an error in typing and the correct MT is what was presented in the USDA Commitment Letter (4,690).
The first WFP biannual report showed the activity output for indicator 3 as 234,119 students, which is higher than the target set for the whole FY 2014 (200,591 students).
WFP's progress report (01/04/2016 - 30/09/2016) listed the indicator for the number of school-aged children receiving school meals twice and provided differing figures
(226,479 & 226,744)
11. Provide teaching
material and
equipment
Number of schools receiving
school supplies and materials
as a result of USDA assistance
0 82 100 0 10 30 67 92 125
Comment USDA Modification II Letter to WFP (USDA, July 25, 2016) increased the target for FY 2016 from 82 to 100.
12. Provide technical
support to improve
water and sanitation
facilities
Number of latrines constructed 6 7 7 0 0 3 3 8 6 3
Number of schools supported
with technical support for
water and sanitation facilities
0 50 20 0 42 48 0 0 0
Number of water ponds
constructed 50 0 13 20 9 8 0 10
Number of schools using an
improved water source 267 366
Comment
USDA Modification II Letter to WFP (USDA, July 25, 2016) and WFP's biannual progress reports adjusted the target for the number of water ponds constructed (row 3
above) for FY2016 and set additional targets for FY2017.
USDA Modification II Letter to WFP (USDA, July 25, 2016) and WFP's biannual progress reports also adjusted the target for number of schools provided with WASH
technical support (row 2 above) for FY2016 to 20. The proposal to McGovern-Dole and the Commitment Letter (USDA, Nov 9, 2012) had 0 target for FY2016.
In addition, USDA Modification II Letter to WFP (USDA, July 25, 2016) and WFP's biannual progress reports added a fourth indicator (row 4 above).
Possible double counting with latrine construction. Refer to Comment under Activity 2.
13. Provide training
on CHILD planning
approach
Number of school directors,
PTA members, health and ag
extension workers trained on
460 0 0 0 60 80 160 120 159
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
126
Activity Indicator
Targets64 Results65
FY 2014 FY 2015 FY 2016 FY 2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
CHILD planning tool
14. Provision of non￾food items to schools
Number of target schools with
"standard" food prep and
storage equipment
82 0 373 400 369 436 413
Comment
USDA Modification II Letter to WFP (USDA, July 25,2016) adjusted the target number for FY2016 to 373 (there was no target set in the McGovern-Dole proposal and the
Commitment Letter) and set a new target of 400 for FY2017.
But WFP's progress report for the first reporting period showed very different targets for FY 2014 (244), FY2015 (270), & FY2016 (300) and reported an activity output for
the specific period which is higher than the target set for the entire year of FY2014.
15. Take home
rations
Number of individuals
receiving take-home rations as
a result of USDA assistance
0 0 0 128,783 369,531 0 131,490
Number of individuals
receiving take-home rations as
a result of USDA assistance
(Female)
0 0 0 128,783 369,531 0 131,490
Number of metric tons
provided for take-home rations 730 730 918 1,864 193.52 369.05 0 430.25 274 0 606
Number of take home rations
provided as a result of USDA
assistance
721,942 794,469 210,974 1,030,263 308,736 271,056 0 488,834 369,531 482,001
Comment
USDA Modification II Commitment Letter to WFP (USDA, July 25, 2016) and WFP's biannual progress reports added the first and second indicators (rows 1 and 2 above).
USDA Modification II Commitment Letter to WFP (USDA, July 25, 2016) and WFP's biannual progress reports adjusted the FY2016 target for number of MT provided for
THR from 730MT as was indicated in the McGovern-Dole proposal and Commitment Letter (USDA, Nov 9, 2012) to 918MT. USDA Modification II Commitment Letter
(USDA, July 25, 2016) also set a target of 1,864MT for FY2017.
The FY2016 target set for the number of THR provided (row 4 above) in the McGovern-Dole proposal and Commitment Letter (USDA, Nov 9, 2012) was 794,469 but was
reduced to 210,974 as per USDA Modification II Commitment Letter (USDA, July 25, 2016). The same Modification Letter II also set a target of 1,030,263 for FY2017.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
127
Activity Indicator
Targets64 Results65
FY 2014 FY 2015 FY 2016 FY 2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
16. Training in food
preparation and
storage practices
Number of school staff or PTA
members trained in Safe Food
Prep and Storage Practices
600 600 0 200 251 334 120 324 173 230 460
Comment USDA Modification II Commitment Letter to WFP (USDA, July 25, 2016) reduced the target number for FY 2016 from 600 as was indicated in the McGovern-Dole
proposal and USDA Commitment Letter (USDA, Nov 9, 2012) to 0. Modification II Letter (USDA, July 25, 2016) also added a target of 200 for FY2017.
17. Training: Food
Handling and
Management for
government
counterparts
Number of trainings provided
on Food Handling and
Management
1 1 2 4 1 1 0 1 0 1 0
Comment USDA Modification II Commitment Letter to WFP (USDA, July 25, 2016) adjusted the target for FY 2016 from 1 to 2 and also set an additional target of 4 trainings for
2017.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
128
 Outcomes: targets and results
Result Title & Description Performance Indicator
Baseline
survey
(2011/12)
Performance
Indicator
Taret66
Performance Indicator Results67
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/201768
MGD
RF1
Increased capacity
of government
institutions
Number of government staff in relevant
ministries/offices trained to monitor the
safety of food in school feeding
programs 0 55 11 0 54 54 0 46
Number of operational national school
feeding programs 0 1 0
Comment
The proposal to McGovern-Dole did not include this result and corresponding performance indicators. The USDA Commitment Letter to WFP (USDA, Nov 9, 2012)
was the first to include the activity and the first indicator (row 1 above) and the Modification II Commitment Letter (USDA, July 25, 2016) added the second indicator
(row 2 above).
The baseline survey didn't include the second indicator (row 2 above) and the USDA Modification II to Commitment Letter to WFP (USDA, July 25, 2016) added this
indicator and set a target of 1.
MGD
1.1.2
Better access to
school supplies
and materials
Number of schools receiving school
supplies and materials as a result of
USDA assistance 0 182 10 67 92 125
Comment USDA Modification II to Commitment Letter to WFP (USDA, July 25, 2016) put a target of 182 for the above indicator. However, all McGovern-Dole reports use 82
as the target.
MGD
1.2
Improved
attentiveness
Percent increase in number of students
in classrooms identified as attentive by
their teacher 95% 2% 2% 98% 98%
Comment The McGovern-Dole proposal showed a baseline of 97% and targeted a percentage increase of 3% to achieve 100% student attentiveness. USDA Modification II
Commitment Letter to WFP (USDA, July 25, 2016) erroneously indicated a 97% increase from the baseline figure of 95% for the above indicator when it should have
been a 2% increase to achieve 97% attentiveness.
66 As shown in the USDA Commitment Letter to WFP (USDA, Nov 9, 2012).
67 Drawn from WFP's biannual progress reports to USDA.
68 This biannual progress report's performance indicator page is dated as 2016 (01/04 - 30/09) instead of 2017 (possible copy-paste error).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
129
Result Title & Description Performance Indicator
Baseline
survey
(2011/12)
Performance
Indicator
Taret66
Performance Indicator Results67
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/201768
MGD
1.2.1.1
Increased access to
food (School
feeding)
Number of daily school meals
(breakfast, snack, lunch) provided to
school-age children as a result of
USDA assistance 0 132,791,200 14,744,482 16,232,311 0 23,804,729 21,504,317 6,708,908
Comment
The proposal to McGovern-Dole showed a baseline of 184,581 and performance target of 242,715. USDA Commitment Letter to WFP (USDA, Nov 9, 2012) presented
no baseline for the above indicator (blank cell).
Question on the validity of the baseline figure for the above indicator considering that the current McGovern-Dole SF activities did not start from ground zero but
instead continued on a previous McGovern-Dole-funded SFP.
USDA Modification II to Commitment Letter to WFP (USDA, July 25, 2016) showed a performance target of 50,864,000 for the above indicator.
MGD
1.3
Improved student
attendance
Number of individuals benefiting
directly from USDA-funded
interventions 201,437 244,039 234,214 243,981 90,850 270,670 214,624
Percent of boy students regularly
(80%) attending USDA supported
classrooms/schools 95% 99% 97% 83.8% 96%
Percent of girl students regularly (80%)
attending USDA supported
classrooms/schools 95% 99% 97.9% 95.3% 95%
Comment The USDA Commitment Letter to WFP (USDA, Nov 9, 2012) indicated no baseline figure (blank cell) for the first indicator (row 1 above) and set a performance
target of 244,039. The USDA Modification II Commitment Letter (USDA, July 25, 2016) showed a baseline figure of 0 with a performance target indicator of 310,856.
The McGovern-Dole proposal showed a baseline attendance rate (not sex disaggregated) of 98%.
IR
1.3.1
Increased
economic and
cultural incentives
(or decreased
disincentives)
Gender Parity Index in primary
education
0.72 1 0.89:1 0:82:1 0.82
Comment
The McGovern-Dole proposal did not include the Gender Parity Index as an indicator and instead showed a baseline figure of 80,841 girls and a performance target
of 99,309 girls.
USDA Commitment Letter to WFP (USDA, Nov 9, 2012) and WFP's biannual progress reports showed a different GPI baseline of 0.89 instead of the 0.72 indicated in
the baseline survey.
WFP's second biannual progress report (01/10/2014-31/03/2015) amended the GPI to 0.72:1
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
130
Result Title & Description Performance Indicator
Baseline
survey
(2011/12)
Performance
Indicator
Taret66
Performance Indicator Results67
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/201768
WFP's fifth and seventh biannual progress reports (01/04/2016-30/09/2016 and 01/04/2017-30/09/2017) showed GPI baseline of 0.89
MGD
SO2
Increased use of
health and dietary
practices
Percent of schools in target
communities that clean cooking and
eating equipment, consistent with
accepted standards, prior to use 69% 69% 62% 68% 62%
Comment
The McGovern-Dole proposal did not show the baseline for the above indicator, stating "baseline data will be established in January 2012), but showed a
performance target indicator of 90%.
USDA Commitment Letter to WFP (USDA, Nov 9, 2102) had 0 as the baseline and performance target of 69% (WFP's biannual progress reports showing these same
figures as well) while the USDA Modification II Commitment Letter (USDA, July 25, 2016) had a baseline figure of 69% and performance target of 100%.
MGD
1.4.4
Increased
engagement of
local organizations
and community
groups
Percent increase in the number of
schools with CHILD planning team in
place
0% 20% 70% 5% 10%
Comment The McGovern-Dole proposal showed a baseline figure of 6% and a performance target of 15% for the above indicator.
MGD
RF2
Improved
knowledge of
health and hygiene
practices
Percent increase in the number of
schools in target communities that
clean cooking and eating equipment,
consistent with accepted standards,
prior to use 19% 13% 18% 17%
Comment
The baseline did not measure for this indicator and it was not included in the McGovern-Dole proposal. However, the USDA Commitment Letter to WFP (USDA, Nov
9, 2012) and USDA Modification II Commitment Letter (USDA, July 25, 2016) had a baseline of 50%. The USDA Modification II Commitment Letter also has a
different performance target of 30 %. WFP's biannual progress reports use the 50% baseline and 19% performance target.
MGD
RF2
Increased
knowledge of safe
food prep and
storage practices
Percent increase in the number of
people at school, district and regional
level trained in food management and
handling 50% 20% 10% 20% 175
Comment McGovern-Dole proposal showed a baseline of 91% (684 out of a total of 751 individuals trained) for the above indicator and a performance target of 100% (751 out
of 751 individuals).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
131
Result Title & Description Performance Indicator
Baseline
survey
(2011/12)
Performance
Indicator
Taret66
Performance Indicator Results67
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/201768
Both the USDA Commitment Letter to WFP (USDA, Nov 9, 2012) and Modification II Commitment Letter (USDA, July 25, 2016) as well as WFP's biannual progress
reports showed 80% as the baseline and 20% increment for the performance target.
WFP's seventh biannual progress reports provided number of trained individuals rather than a percentage increase in the number of trainees.
MGD
RF2
Increased access to
clean water and
sanitation services
Percent increase in the number of target
schools with year round access to a
clean and safe water source 56% 8% 3% 38% 28%
Comment
The McGovern-Dole proposal had 0 as the baseline figure and 8% as the target. However, both the USDA Commitment Letter to WFP (USDA, Nov 9, 2012) and
Modification II Commitment Letter (USDA, July 25, 2016) had 35% as the baseline. Modification II Commitment Letter showed a different performance target (10%).
WFP's biannual progress reports use the baseline and performance targets (35% and 8% respectively) as indicated in the USDA Commitment Letter.
MGD
RF2
Increased access to
requisite food prep
and storage tools
and equipment
Percent increase in number of target
schools with 'standard' food prep and
storage equipment 38% 44% 13% 18% 14%
Comment
The McGovern-Dole proposal had 90% for the baseline and 10% increment as the performance target (100% of the targeted schools). On the other hand, both the
USDA Commitment Letter to WFP (USDA, Nov 9, 2012) and Modification II Commitment Letter (USDA, July 25, 2016) showed a baseline of 56% and an increment
of 44% as the performance target to achieve 100% coverage of the targeted schools. It is assumed the 100% performance target indicated in the Commitment Letter to
WFP is showing a 44% performance target to achieve 100% coverage of the targeted schools.
WFP's biannual reports use the baseline and performance target indicators (56% and 44% respectively) as stated in the Modification II Commitment Letter (USDA,
July 25, 2016).
IR
1.3.4
Increased student
enrollment
Percent increase in net enrollment rate
in school as a result of USDA
assistance in Afar 35% 50% 19% 20.49%
Percent increase in net enrollment rate
in school as a result of USDA
assistance in Somali 64% 40% 62% 34.64%
Comment
Differing baseline and performance target figures for both regions.
The McGovern-Dole proposal had baseline of 30% and 50% and performance target of 55% and 40% for Afar and Somali respectively.
USDA Commitment Letter to WFP (USDA, Nov 9, 2012) and Modification II Commitment Letter (USDA, July 25, 2016) showed a baseline figure of 35% and 50%
and performance target of 50% and 40% for Afar and Somali respectively. WFP's biannual progress reports also report on progress against these baseline and
performance target figures.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
132
 Beneficiaries: targets and results
Beneficiaries indicator
Targets69 Results70
FY
2014
FY
2015
FY
2016
FY
2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
1. Number of counterpart staff trained in school
health and nutrition (continuing) 25 25 54 56 36 120 0 54 0 46 0
Comment
For the above indicator, WFP's first, second, third and fifth biannual progress reports showed the targets for FY2016 as 25. USDA Modification
II to Commitment Letter (USDA, July 25, 2016) adjusted the FY2016 target to 54 and extended the three-year programme into a four-year
programme and set a target of 56 for FY2017.
2. Number of individuals trained in child health
and nutrition as a result of USDA assistance
(new) 0 0 0 156 143 534 136
3. Number of individuals trained in child health
and nutrition as a result of USDA assistance
(Female) 0 0 0 57 21 196 62
4. Number of individuals trained in child health
and nutrition as a result of USDA assistance
(Male) 0 0 0 99 122 338 74
5. Number of students and teachers trained in
school health and nutrition (continuing) 25 25 0 100 25 65 150 60 143 309 100
Comment
For the first three indicators (rows 1 to 3 above), WFP's fifth biannual progress report (01/04/2016-30/09/2016) reported on results but did not
indicate the targets
6. Number of staff employed to support project
implementation (continuing) 2 2 2 0 2 2 2 2 0 2 2
7. Number of technical advisors and technical
experts supporting the National School Feeding
Program as a result of USDA assistance (new) 0 0 0 4 0 4
8. Number of school-aged children receiving
daily school meals (breakfast, snack, lunch)
(continuing) 200,591 220,650 280,179 289,000 234,214 243,981 0 270,509 226,479 160,605 292,249
Comment
For the above indicator, WFP's first biannual report to McGovern-Dole (01/04/2014-30/09/2014) showed the target for FY2016 as 242,717 while
the second (01/10/2014-31/03/2015), third (01/04/2015-30/09/2015) and fifth (01/04/2016-30/09/2016) reports showed the target as 242,715.
WFP's fifth biannual progress report (01/04-30/09/2016) presented the above indicator twice and reported different results (226,479 and
226,744).
69 Targets are as shown in Modification II to the USDA Commitment Letter (USDA, July 25, 2016).
70 Drawn from WFP’s biannual progress reports to USDA.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
133
Beneficiaries indicator
Targets69 Results70
FY
2014
FY
2015
FY
2016
FY
2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
9. Number of school-aged children receiving
daily school meals (breakfast, snack, lunch) as a
result of USDA assistance (Female) (new) 0 0 0 129,000 101,174 72,296 131,530
10. Number of school-aged children receiving
daily school meals (breakfast, snack, lunch) as a
result of USDA assistance (Male) (new) 0 0 0 160,000 125,570 88,309 160,755
Comment
The sex disaggregated school meal beneficiary figures provided in the fifth (01/04/2016-30/09/2016) and seventh (01/04/2017-30/09/2017)
biannual progress reports do not add up to the total number of school meals beneficiaries provided for these same reporting periods. If the sex
disaggregated figures are correct, for the fifth and seventh reporting periods, the total number of SM beneficiaries should be 226,744 and
292,285 respectively instead of the reported 226,479 and 292,249.
11. Number of social assistance beneficiaries
participating in productive safety nets as a result
of USDA assistance (new) 0 0 0 289,000 160,605 292,249
12. Number of social assistance beneficiaries
participating in productive safety nets as a result
of USDA assistance (Female) 0 0 0 129,000 72,296 131,530
13. Number of social assistance beneficiaries
participating in productive safety nets as a result
of USDA assistance (Male) 0 0 0 160,000 88,309 160,755
Comment The "social assistance beneficiaries" seem to be the same students who are benefiting from the school meals, meaning the same indicator (with
different wording) is presented twice.
14. Number of students regularly (80%) attending
USDA supported classrooms/schools (new) 0 0 0 286,110 250,871
15. Number of students regularly (80%) attending
USDA supported classrooms/schools (Male) 0 0 0 158,400 137,901
16. Number of students regularly (80%) attending
USDA supported classrooms/schools (Female) 0 0 0 127,710 112,970
17. Number of students in schools receiving
USDA assistance (new) 0 0 0 289,000 280,179 262,698 292,249
18. Number of students in schools receiving
USDA assistance (Male) 0 0 0 160,000 155,147 145,434 160,755
19. Number of students in schools receiving
USDA assistance (Female) 0 0 0 129,000 125,032 117,264 131,530
20. Number of individuals benefitting directly
from USDA-funded interventions (new) 0 0 0 289,000 226,744 160,605 292,249
21. Number of individuals benefitting directly
from USDA-funded interventions (Male) 0 0 0 160,000 120,175 88,309 131,530
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
134
Beneficiaries indicator
Targets69 Results70
FY
2014
FY
2015
FY
2016
FY
2017
01/04/2014
-
30/09/2014
01/10/2014
-
31/03/2015
01/04/2015
-
30/09/2015
01/10/2015
-
31/03/2016
01/04/2016
-
30/09/2016
01/10/2016
-
31/03/2017
01/04/2017
-
30/09/2017
22. Number of individuals benefitting directly
from USDA-funded interventions (Female) 0 0 0 129,000 106,569 72,296 160,755
23. Number of school directors, PTA members,
health and ag extension workers trained on
CHILD planning tools (continuing) 460 0 0 0 60 80 160 120 159
24. Number of individuals receiving take-home
rations as a result of USDA assistance (new) 0 0 0 128,783 369,531 0 131,490
25. Number of individuals receiving take-home
rations as a result of USDA assistance (Female
students) 0 0 0 128,783 369,531 0 131,490
26. Number of school staff and PTA members
trained in Safe Food Prep and Storage Practices
(continuing) 600 600 0 200 251 334 120 324 173 230 460
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
135
Provide
teaching
materials
and
equipment
Teachers
trained by
Ministry of
Education
Increased Capacity of Government Institutions
Project – Level Result
Increased
knowledge of
Safe Food Prep
and Storage
Practices
Increased Access
to Requisite Food
Preparation and
storage tools and
Equipment
Increased access
to clean water
and sanitation
services
Improved
knowledge of
Health, Hygiene
Practices
Training on food
handling and
management for
government
counterparts
-Provision of
energy saving
stoves
-Promote health
and hygiene clubs
and school garden
-Provision of Non￾Food Items
Provide technical
support to
improve water
and sanitation
facilities - Promote health
and nutrition
Education
Increased use of Health and Dietary
Practices
Reduced
Health Related
Absences
Figure 5 Project level results framework
Key Result obtained
by Coordination
Result Obtained by
Ministry of Education
Increased Increased
Economic Access Student
and cultural Enrolment
incentive (or
Decreased
Disincentives )
Take Home Take Home
Rations Rations
Result obtained by
WFP
Improved
Attentiveness
Improved
student
Attendance
Provide
teaching
materials
and
equipment
Better
Access to
school
supplies and
materials
Teachers
trained by
Ministry of
Education
Increased
skills and
knowledge
of Teachers
Provide School
Meal
Increased
Access to Food
(School
Feeding)
Increased Capacity of Government Institutions Increased Engagement of local organizations and Community Groups
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
136
Annex L Other school feeding programmes in Ethiopia
Introduction
1. This annex provides background information on the genesis and expansion of school
feeding programmes in Ethiopia, including the pilot Home Grown School Feeding (HGSF)
programme and the Emergency School Feeding Programme (ESFP).
The genesis and expansion of school feeding in Ethiopia
2. School feeding in Ethiopia was started in 1994 with the support of WFP. The initial
pilot project covered 40 primary schools in Amhara, Tigray, Afar and Oromia Regions and
further expanded to SNNPR and Somali Region in 2002, targeting chronically food insecure
districts in these six of the country’s nine regional states.
3. Since 1996/97, the GoE has implemented five phases of its multi-year Education
Sector Development Programme (ESDP). One of the components of the successive ESDPs is
the school feeding programme the GoE has undertaken since the mid-1990s in partnership
with WFP. The third phase of the ESDP gave emphasis to expanding school meals to schools
in food insecure and vulnerable areas of the country, with a particular focus on pastoralist
areas and chronically food deficit highland districts with lower school enrolment and higher
gender disparity. School feeding was identified as a strategic instrument for increasing
enrolment and retention and increasing girls’ enrolment in programme areas with a view to
bringing about gender parity in school enrolment.
4. In 2004/5 the GoE and WFP introduced a new community-based effort by launching
a participatory planning tool called Children in Local Development (CHILD) to assist local
communities to utilise schools as development training centres and build the capacity of the
education sector at the local level to integrate children education in its development
programme. In 2007 and based on lessons learned CHILD was expanded and became the
framework for implementing school feeding.
5. The approach is aimed to improve the school premises, community ownership of
education and to make environmental improvements that support the school feeding
activity as well as improve the community’s awareness of environmental sustainability. The
CHILD approach focuses on: i) teaching basic organizational concepts of planning and
managing local development initiatives, ii) providing basic information on health, hygiene
and nutrition and iii) providing training and information on small-scale horticulture to
encourage the establishment of school gardens that could potentially supplement the school
meals (MTE (2009) WFP CP 2007-2011).
6. So far WFP has been the largest provider of school meals in the country. There are a
few local and international NGOs providing school meals to vulnerable urban and rural
school children but efforts of these organizations are independent and are not being
centrally coordinated.
Home Grown School Feeding (HGSF) Programme
7. There has been a growing interest to establish sustainable national school feeding
programme in Ethiopia, which resulted in collaboration between WFP and MoE to pilot a
Home Grown School Feeding (HGSF) programme in 37 schools in SNNPR in 2012 and later
expanding to an additional 50 schools. In 2014, the HGSF model was replicated in 18
primary schools in Oromia Region (SABER, 2015).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
137
8. Structured around WFP’s Purchase for Progress (P4P) programme, the HGSF links
the existing school feeding programme demand with local agricultural production through
the provision of locally produced food purchased from smallholder farmers. The way the
HGSF is structured is also intended to build the capacity of GoE to plan and manage
sustainable national school feeding programme in Ethiopia. This programme is primarily
supported by WFP with contribution from the regional governments of SNNPR and
Oromia.
9. The two HGSF programmes operating in SNNPR and Oromia currently target
139,000 students in 286 schools (WFP, 2017d).
Emergency School Feeding Programme (ESFP)
10. Ethiopia is currently experiencing its worst drought in 30 years as a result of the El
Niño crisis according to the UN, with levels of acute need across all humanitarian sectors
having already exceeded levels seen in the 2011 Horn of Africa drought. As per the GoE and
UN partners, the prolonged food insecurity, malnutrition and disrupted livelihoods have
greatly affected seven regional states and one city administration of the country (Afar,
Somali, Oromia, Amhara, Tigray, Harari, SNNPR and Dire Dawa). As per the rapid
assessment conducted by MoE and Regional BoE Cluster in August 2015, about 3 million
school children in these areas of the country are affected by the emergency (Evaluation
TOR, Annex A).
11. MoE developed an education in emergency response plan as an integral part of the
broader government-led response to the El Nino crisis to provide educational supplies,
WASH facilities and school feeding programme, psychosocial support and establishing
temporary learning spaces to prevent children in drought affected areas from risk of
dropping out of school. MoE required around USD53 million to implement the plan and
required the assistance of development partners. However, the expected funding support
didn’t materialize and MoE had to scale down the plan and focused its implementation on
the emergency school feeding (ESF) by allocating around ETB 570 million (approximately
USD27.5 million) of government budget. As per the education in emergency plan, 739,740
students were provided school supplies (notebooks, pens and pencils) for the 2015-16
school calendar year with US government funding. Out of the total number of students that
benefited from the provision of school supplies, 64,463 were in Afar and 147,308 were in
Somali Regions (Government of Ethiopia, 2015c).
12. The ESFP is set up with a framework similar to the HGSF programme, with linkages
to local cooperatives to provide the grains and legumes needed for the school meals. The per
child school meal ration provides approximately 650 kcal per day and the ingredients that
go into preparing the meals vary from one region to the next depending on what is locally
grown and the dietary preferences of the local population.
13. The emergency school feeding target beneficiary number of 739,740 that was
indicated in the education in emergency response plan for the 2015-16 school calendar year
had to be increased to 2.8 million as the drought conditions continued and additional
beneficiaries were included during the second semester of the school year (Government of
Ethiopia, 2015c). Out of the 6,742 schools targeted for the 2015-16 school year, 653 and 692
were in Afar and Somali Regions respectively. And out of the 2.8 million students that
received school meals, 93,904 and 168,085 were in Afar and Somali Regions respectively.
14. According the ESFP performance assessment that was carried out by MoE for the
2015/16 school calendar year, most of the regions lacked the technical know how to
implement the programme and required the support of MoE. Although figures are not
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
138
provided, the ESFP performance assessment indicates that students that had discontinued
classes due to the drought had returned to school and dropout rates had gone down as a
result of school meal provision. The assessment findings also raised some key challenges
pertaining to the quality and quantity of the meals provided, highlighting the fact that the
grains and legumes some of the cooperatives provide are of poor quality and that the meals
are not always served as per the recommended mix. Other challenges include delayed
delivery of food to schools, lack of proper feeding places, lack of proper training in hygiene
and food preparation, and low level of support and follow up of the ESFP at school level. To
address the hygiene and sanitation issue, MoE was able to provide training on food
preparation and handling, hygiene and sanitation and other issues related with the support
of WFP (Government of Ethiopia, 2015c).
Box 4 Evidence of the benefits of HGSF in Ethiopia
HGSF was initiated as a result of a recommendation made during a consultative workshop
organized to discuss the sustainability of school feeding programme in August 2011. The
recommendation centred around the need to collect evidence through a pilot programme for clear
understanding of the benefits the adoption of the HGSF approach, as a means to facilitate the
transition to government ownership. HGSF particular focus is to both increase children’s well￾being and promote local agricultural production and development by expanding an ongoing
market for small landholders.
Lessons from the HGSF programme included:
• Increased government ownership of the programme, which is demonstrated through
increased budget allocated for HGSF by regional governments,
• Served as a vehicle for introduction of new menu and diet diversity among school children
and community
• Improved operational capacities of government institutions such as the MoE: regional
bureaus of education
The pilot has showed evidence on key cost drivers of the HGSF programme and managed to reduce
the cost from $0.18 /child/day to $ 0.15/child/day. However, the cost reduction to the initially
planned amount was not materialized due to the utilization of high value commodities in the
menu.
The programme has created a foundation for better institutional market access for smallholder
farmers and motivated Farmers ’to engage in the production of diversified high value crops
productions. However, the proportion of market access created for surpluses commodities like
maize is still low compared to wheat.
15. ESFP is now in its third year and provision of school meals has continued in the
2016-17 school calendar year albeit on a reduced scale. MoE conducts an assessment twice a
year to determine if and where ESF should continue.
Ye Enat Weg
16. Ye Enat Weg is a charitable association initiated in July 2014 by H.E the former First
Lady of Ethiopia with the objective of improving the provision of food and school supplies
to impoverished school students in government primary schools across the city of Addis
Ababa.
17. In collaboration with the Addis Ababa City Administration Bureau of Women and
Children Affairs and the Addis Ababa Bureau of Education, Ye Enat Weg launched the
school feeding programme in February 2015, with funding from local and international
donors. The programme provides breakfast and lunch to students identified by the school
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
139
PTA as highly vulnerable and impoverished using a set of selection criteria, including those
whose families live in extreme poverty, orphaned or coming from single parent family
settings, students with HIV-AIDS, and children that have migrated to the city from rural
areas in search of a better life and are living on the streets. The programme also provides
these students with school supplies, clothing and shoes when resources allow.
18. In each school, five volunteer female teachers are selected as members of the
charitable association and supervise their school’s feeding programme. Local unemployed
women are trained and employed as cooks and follow a set menu to feed the children simple
yet nutritious meals at a reasonable cost (two meals at ETB 12 or USD 0.44 per child per
day).
19. In its first year of establishment, Ye Enat Weg’s school feeding programme provided
school meals to 5,106 students in 93 primary schools and continued expanding through
2016, reaching 20,069 students in 208 primary schools. Currently, Ye Enat Weg is
providing school meals to 20,135 students in 211 schools in all ten of Addis Ababa’s sub￾cities. The programme started out with 846 hired cooks and their number has gone up to
1,000 in 2017.
20. With the exception of about 100 students with HIV-AIDS who receive school meals
throughout the whole year, meals are provided to the remaining beneficiary students during
the 220 days in the school calendar.
21. The programme’s current annual budget is ETB 58,080,000 (USD2.1 million).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
140
Annex M Map
Map 1 Food for Education coverage in Somali and Afar region
Source: UN WFP/VAM Ethiopia, April 20, 2016
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
141
Annex N Gender
1. In Ethiopia, 80 percent of the population resides in rural areas and women provide
the majority of the agriculture labour in these communities. However, women’s access to
resources and community participation are usually mediated through men, either their
fathers or husbands, and their agricultural contributions often go largely unrecognized.
Additionally, when women have access to their own income, they are more likely than men
to spend it on the betterment of their families and successfully participate in village savings
or pay school fees for their children. (Gender Equality and Women’s Empowerment,
USAID, December 2017).
2. Ethiopia has seen remarkable economic growth in the last two decades, with figures
as high as 11 percent, making it the world’s fifth-fastest growing economy. The country has
also been widely hailed for achieving 6 of the Millennium Development Goals. However,
despite the rapid pace of economic development, the number of people living in poverty
remains high, estimated at 30 percent of the 99 million populations, and accordingly
Ethiopia ranks low in the Human Development Index at 174th out of 188 (UNDO, HDI,
2014).
3. Gender inequality in Ethiopia is deeply rooted in a patriarchal, conservative society,
where women are structurally and systematically disempowered. This is reflected in
numerous development indicators. Ethiopia entered the twenty-first century with extremely
low maternal and reproductive health indicator levels and Gender Based Violence- GBV
remains a persistent concern. The widespread and deep-rooted barriers to meaningful
gender equality and women’s empowerment (GEEW) hinder the country’s capacity to
achieve zero hunger. For this reason, it is imperative for WFP Ethiopia to consider
opportunities for and barriers to promoting GEEW in all activities, in the organizational
structure, and in partnerships developed and invested (WFP, 2016k).
4. Gender disparities remain vivid in Ethiopia, despite marked progress in recent years
(particularly in access to primary education). Ethiopia’s ranking in the 2016 Global Gender
Gap Report (109 out of 185 countries), though an improvement from the previous position,
as an example, among females 25 years and older, only 8 percent had at some secondary
education, compared to 18 percent of males of the same age group. The position and
empowerment of women and girls in society are hindered by negative attitudes
perpetuating inequality affecting all aspects of their lives. Although women’s political
representation has improved over the years, negative social perceptions about the
leadership ability of women, their low socio-economic status, low educational and skills
levels and lack of strong role models all contribute to women still being largely
underrepresented in decision-making positions. The burden of household chores and
inequitable access to higher education also limit women’s ability to enjoy the opportunities
and benefits of citizenship as men on an equal footing in the economic sphere.
5. While Ethiopia’s government has made significant strides over the recent years in
setting the policies in government both at federal and regional levels, a lot still needs to be
done in terms of uplifting the currently existing infrastructure, systems and procedures
from minimum requirements to what the government and development partners would like
to see in place. Specifically, there is need for systematic gender-mainstreaming,
harmonization and alignment of processes and systems, including M&E/MIS and data
analysis, which is the basis of evidence-based strategic planning.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
142
6. WFP corporate gender policy: WFP’s 2009 Gender Policy (WFP, 2009c) was in force
for the majority of the evaluation review period. This policy sought to mainstream gender
into WFP operations through the 2010–2011 Gender Policy Corporate Action Plan (WFP,
2009b) which specified commitment to gender across four dimensions: capacity
development; accountability; partnerships, advocacy and research; and operational
mainstreaming. While the Gender Policy of 2009 attempted to denote a shift from
“commitments to women” to a more comprehensive understanding of gender with an
examination of the interacting roles of both men and women, the subsequent Gender Policy
Evaluation (WFP, 2014b) found that it failed to develop a clear, comprehensive and shared
understanding of what gender means within WFP. It found that gender integration in WFP
programmes had largely been a bottom-up, country-led process, rather than one influenced
by a clear organisation-wide vision. While it found evidence of progress in identifying
gender-based needs and priorities in many programme areas, including nutrition, it noted
less evidence of WFP contributing to transformative changes in gender relations. Although
it found some good examples of gender-sensitive programming, it also found that capacity
development of WFP staff in gender had been inadequate and there was no shared
definition of what gender means for WFP; there was still a strong focus on enhancing
women’s engagement in programmes or specifically targeting women, so that while it found
strong evidence of increased inclusion of women and girls, this “results mainly from a
vulnerability rather than a gender lens”.
7. WFP’s latest Gender Policy 2015–2020 was adopted towards the end of the review
period. The new policy addresses previous weaknesses by reinforcing a gender, rather than
women-focused, approach, to establish four objectives: adapt food assistance to the
different needs of men and women, pursue equal participation, empower women and girls
in decision-making regarding their food security and nutrition and ensure the protection of
men and women.
8. Both WFP’s Strategic Plans, 2008–2013 (WFP, 2008a) and 2014–2017 (WFP, 2013),
also include clear commitments to gender equality. At regional level, an Africa Gender
Implementation Strategy has been developed which outlines the regional strategy to
operationalise the new gender policy within the specificities of the Africa context. Gender is
also mainstreamed in the Ethiopia UNDAF 2013-2017.
9. The East and Central Africa gender implementation strategy outlines WFP’s
corporate strategy to operationalise the Gender Policy 2015-2020 by focusing on four
objectives. i) Food assistance adapted to different needs: women, men, girls and boys
benefit from food assistance programmes and activities that are adapted to their different
needs and capacities. ii) Equal participation: women and men participate equally in the
design, implementation, monitoring and evaluation of gender-transformative food security
and nutrition programmes and policies. iii) Decision-making by women and girls: Women
and girls have increased power in decision-making regarding food security and nutrition in
households, communities and societies. iv) Gender and protection: Food assistance does no
harm to the safety, dignity and integrity of the women, men, girls and boys receiving it, and
is provided in ways that respect their rights. In addition, the strategy identifies 3 gender￾transformative regional priorities: 1. Evidence-based programmes: WFP COs to collect a
broad range of qualitative and quantitative information related to gender to be for
programme design and to deliver effective interventions and to better achieve the strategy’s
objectives. 2. Increased meaningful participation of affected populations: Even though
beneficiaries often participate in needs assessments through focus group discussions, this
participation is not always meaningful as it does not always influence programme design.
3. Gender and nutrition sensitive programmes can be a major opportunity to enhance the
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
143
nutrition impact of all WFP programmes, while enhancing gender equality. It also
identifies the most prominent gender issues in the region and priority actions to
mainstream gender in WFP programming, as well as the respective roles of WFP COs, the
RBB and HQ.
10. The WFP Ethiopia Gender Baseline Study identified achievements and lessons
learned, as well as challenges and gaps. These will be reflected to form the basis for the
Country Action Plan (WFP, 2016i).
McGovern-Dole USDA SF approach to gender
11. The USDA McGovern-Dole Programme documents include gender analysis related to
enrolment, attendance, drop out, gender parity and retention of boys and girls in pre￾primary school and primary school. The document also considered lessons learned from
past experiences and specifically noted that WFP had been able to address some of these
issues previously – notably by providing incentives of take-home rations for girls through
the school feeding programme, which resulted in a reduction of the gender gap. The major
outcome of the programme is “Equal access provided for boys and girls at primary school
with a focus on the marginalized food-insecure areas and vulnerable children”. And the
major outcome indicators for the programme were: the Percentage change in the
enrolment, Gender Parity. The gender parity and equity is addressed through School Meal,
Take-Home Ration (THR) and capacity development.
12. The Gender Parity Index during the baseline survey conducted in January 2013 was
0.72 and it reached to 0.85 in Afar and 0.78 in Somali region by 2017. At the end of funding
period, the Gender Parity Index will reach 1:1. WFP will manage to reach the target through
its continued support and engagement of communities and community organisations. The
net enrolment rate also increased from 63.7 percent in 2010/11 to 85 percent (2015/16) in
Somali region and from 35.4 percent to 56 percent in Afar region. Provision of facilities like
latrines, low cost canteens and provision of water and sanitation have also had an impact on
improving the quality of deliverables under the programme and have contributed to an
improved quality education in the two regions. The attendance rate for girls and boys on the
average was reported to be 96 percent and 85 percent which shows the THR helps girls to
regularly attend the class as compared to boys The progress in the two regions is
encouraging, particularly in Afar Region (WFP McGovern-Dole Narrative Report, April–
September 2017).
Gender issues and approach for this evaluation
13. The TOR for this evaluation require that GEEW should be mainstreamed
throughout. The evaluation matrix at Annex D responds to this requirement. It
acknowledges the necessity of checking on the programme’s coherence with national policy
on gender (EQ 1). EQ 5 asks whether the operation’s strategies were based on a sound
gender analysis that considered the distinct needs and participation of boys and girls (and
as appropriate within the context of the school meals programme, women and men), and
whether they have continued on that basis. Answers to EQs 6 and 7 on the attainment of
outputs and outcomes will be gender disaggregated. EQ 8 asks how adequately the
operation has addressed gender equality and protection issues. EQ 20 asks whether the
operation has made any difference to gender relations at any level thus far, and whether any
such change likely to be sustained after the programme is completed. In the course of these
enquiries, the ET will also explore the quality of women’s involvement in local school
feeding management and support committees; the continuing challenge of early marriage of
girls, typically terminating their education; the effect of girls’ burden of household labour on
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
144
their regular attendance at school; the problems older girls face in reaching often remote
secondary schools; and the status of women teachers.
14. In addition, the ET reviewed in depth the THR programme for girls by interviewing
the parents, teachers and the students.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
145
Annex O Equity
1. According to the UN Convention on the Rights of the Child (UNCRC), all children
deserve the chance to be happy and healthy, explore their world safely, and reach their full
potential.
2. Yet the rights of millions of children are blocked by deprivation and discrimination
based on factors beyond their control – their gender, ethnicity, socioeconomic status, place
of birth or whether they live with a disability, for example.
3. When children do not have a fair chance in life, significant inequalities emerge
between those who have the most and those who have the least. Those inequalities are
passed from generation to generation in a vicious circle that has significant economic,
political and social consequences – leading to an unequal and unfair world.
4. With smart investments and targeted actions, every child can have a fair chance in
life.
An abstract of UNICEF’s definition and programmatic focus on approaches to
equity
5. The pivotal 2010 report, Narrowing the Gaps to Meet the Goals, reminds the reader
that UNICEF’s mission is to prioritize the most disadvantaged children and why and how a
greater focus on equity will be adopted in UNICEF’s approach. While UNICEF is explicitly
concerned with the rights of children as described in the Convention on the Rights of the
Child, Narrowing the Gap also offers evidence that reaching the most disadvantaged
children is a cost-effective strategy for development overall and will make significantly
greater advances toward achieving the Millennium Development Goals (MDGs).
Determinants & Manifestations
6. UNICEF identifies discrimination based on gender, ethnicity, religion and disabilities
as well as geographical isolation, structural poverty, weak governance and the overlapping
nature of these deprivations among those living in the most extreme income poverty. These
deprivations are manifested in barriers to crucial social services such as health and
nutrition, education, housing, access to water and sanitation and information. All
perpetuate intergenerational poverty. They are also manifested in the increased chance of
exploitation and child protection issues as well as in decreasing the chance of being
registered at birth or surviving the first years of life.
Characteristics of UNICEF’s Equity Approach
7. For the most disadvantaged, these determinants commonly interact with one
another. UNICEF strives to achieve a multidisciplinary intervention approach. In designing
programs and policies UNICEF utilizes a methodology called “Monitoring Results for
Equity System” (MoRES). This tool monitors programs and policies to ensure that the
equity approach of reaching the most marginalized children is evidence-based and therefore
makes the expected impact. UNICEF stresses the importance of a holistic approach by
separate social service providers and sectors to meet the multi-dimensional needs. These
needs are to be defined and agreed upon based upon community demands, rather than from
the service provider’s perspective. In UNICEF’s view, this entails holding service providers
accountable and creating better access to information for the most disadvantaged
communities and, specifically, for children. Policy interventions, too, should reflect a
holistic, integrated approach to early childhood.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
146
8. A greater analysis of the determinants of inequity will allow better targeting of equity
programming that addresses the most substantial barriers faced by those living in extreme
poverty. That way, interventions can be adapted to supply services according to the demand
for them. According to UNICEF, “equitable budgetary allocations” and “adequate social
expenditures” (Knowledge for Action 5) are important factors in achieving results for
children. UNICEF’s studies show that a greater focus on the demand side of social services
for the most marginalized will ameliorate the “bottleneck” seen in countries where the social
determinants listed above inhibit access to service provision. This approach is “based on the
notion that certain bottlenecks and barriers prevent children and women from benefitting
from essential interventions and services” (Ibid 6). To this end, UNICEF names 10
additional, macro level determinants of inequity: legislation, policy, budget/expenditure,
management/coordination, accountability, availability of essential commodities, access to
adequately staffed services, facilities and information, financial access, social and cultural
practices and beliefs, continuity of use and quality (Ibid). These are manifested in barriers
related to delivery systems, capacity constraints, public policy and budgets (Ibid).
9. Practically, UNICEF identifies the elimination of user fees for social programs, and
geographic targeting of the poorest and most isolated communities through outreach. In
terms of service provision, UNICEF adopts a demand-based approach that may include
cash transfers and other social assistance mechanisms addressing the unique barriers faced
by the most marginalized. Two key programming goals are tailoring services to needs and
working holistically.
Target Demographic
10. UNICEF is the premier international organization implementing an equity approach
for children, which is aligned with their mission. Specifically, children in early childhood
(especially girls), women, families, and communities are targeted for expanding social
services, protection, and other self-identified needs (UNICEF and Equity 2). The Narrowing
the Gaps report notes that “excluded populations within countries generally have a larger
proportion of children than other groups owing to higher fertility rates” which amount to
higher mortality rates and higher rates of death due to treatable and preventable diseases
(Narrowing the Gaps 3).
Theory and why to implement this approach
11. While UNICEF has always aimed to reach the most disadvantaged children in the
world, UNICEF argues that a more focused approach on those children who live in poverty
and/or suffer from violence, exploitation or neglect is more important than ever because of
growing inequality in the world. UNICEF emphasizes that the global financial and economic
crises, coupled with rising food prices, has exacerbated the multi-faceted deprivations faced
by the most disadvantaged and excluded children.
Reference 1: ‘Narrowing the Gaps to Meet the Goals.’
Reference 2: ‘Progress for Children: Achieving the MDGs with Equity.’
Reference 3: ‘Knowledge for Action: Emerging experiences in Social and Economic Policy that
support equitable outcomes for children.’
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
147
Annex P School Health and Nutrition
1. Poor health and nutrition result in the loss of a considerable number of school days
annually. To protect an investment in efforts to increase access and improve the quality of
education, schools must help link students to essential health and nutrition services.
2. School age children in Ethiopia are affected by a wide range of health and nutrition
problems that limit their ability to benefit from education. Some of the common health
issues are parasitic infections, malaria, anaemia, trachoma, skin diseases, disabilities,
injuries, sexual and reproductive ill-health, and psychosocial and substance abuse. And the
common nutrition problems are poor diet diversity and inadequate food consumption
associated with high prevalence of malnutrition and micronutrient deficiencies such as
iodine and vitamin A, in most of the Ethiopian regions. Furthermore, there is a lack of
awareness on the effect of poor health and nutrition on children’s ability to learn.
3. To deliver a comprehensive school health and nutrition (SHN) intervention, a
National School Health and Nutrition Strategy (Government of Ethiopia, 2012) was
developed in October 2012 with technical and financial support from UNICEF and WFP.
4. The goal of the strategy is to improve access and education achievement of
schoolchildren through health and nutrition interventions in schools with the aim to
promote joint planning, design and implementation of sustainable and quality health and
nutrition interventions across the education sector. The mechanisms are put in place for
ownership and sustainability of SHN programmes (such as school feeding programmes and
nutrition interventions, cooking demonstrations and school gardens) thereby, increasing
access and completion rates by reducing dropout and absenteeism prevalent in chronically
food insecure areas of the country.
5. This National SHN Strategy adopts the FRESH (Focusing Resources for Effective
School Health) framework as its guiding principles for enabling effective coordination and
organization of SHN responses in the country. As part of the main strategic component,
three priority areas are being identified; i) safe and sanitary school environments; ii) skills￾based health and nutrition education; and iii) school-based health and nutrition services.
6. The Ministry of Education is the primary body responsible for providing SHN
interventions and thus, shall provide leadership and establish the policy framework for the
planning, coordination and implementation of SHN interventions. The Ministries of Health,
Education and Water Resources provide standard guidelines for ensuring that school
premises are clean, safe and functional for all including those with disabilities and special
needs. Hence, all educational institutions and organizations shall ensure compliance with
the building standards, public health rules and other relevant legislations and policies. By
providing clean and potable water and sanitation facilities, schools shall reinforce the health
and hygiene messages, and serve as demonstration places to both students and the wider
community. Separate latrines for girls and boys should be arranged, particularly for
adolescent girls.
7. School-based health and nutrition services highlighted in the strategy are: provision
of WASH, weekly iron supplementation, immunization, health check-up including eyes,
dental, and control of Intestinal Worms, Bilharzia, and other Parasitic diseases. The
deworming serves a preventive and treatment measure that results in immediate
improvement in child health and anaemia which leads to learning ability and attentiveness.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
148
Findings
8. Despite a strong national SHN strategy and commitment from MoE and MoH, the
implementation is very poor due lack of resources.
9. WFP provided technical and financial support to MoE on the development of SHN
strategy and developed a training manual and provided ToT at national level on SHN in all
regions. It was noted that the trainings are continued at the regional levels either through
the technical assistance or through the regional universities. In Afar region, RoE has
worked closely with Semera University to provide training on SHN to cluster leads, Health
officers, Directors, Woreda officials and teachers, cooks and PTA. The focus of the training
is safe food preparation, use of WASH facilities, provision of non-food items and energy
stoves.
10. WFP supported health services in the targeted schools through provision of first aid
kits, school health and nutrition clubs, trainings and materials support. It was found in
some schools that the school health and nutrition clubs are active.
11. During the schools visits, observations and interviews with various stakeholders, the
ET found that a component of the essential package –safe water and toilets – are not in
place. It was observed that separate toilets are built for girls and boys but not maintained
and not in use any more.
12. The biggest challenge is WASH in both Afar and Somali regions. Scarcity of water for
hand washing before eating and after use of toilet and washing of cups and spoons in water
is the biggest gap. These are the basic minimum for good health.
13. However, in a model school in Somali region – with the assistance of McGovern-Dole
– safe running piped water and clean latrines were available. It was reported that
insecticide is sprayed during the malaria season in the schools by the health department
14. The provision of deworming and weekly iron folic acid supplements is not in place.
15. WFP has provided technical and financial support in construction of water ponds –
birkas – in some 50 schools in 2016-2017.
16. In accordance with the national strategy, at the national level, a taskforce/technical
committee has been formed and is responsible for monitoring health and nutrition trends,
related changes in legislation, and providing technical advice to the SHN. This committee is
chaired by the Ministry of Health.
17. At the regional level in Somali region a taskforce was formed outlining terms of
reference for an SHN steering committee signed in 2015 by the Bureaus of Education,
Agriculture, Water, Health, Disaster Prevention and Preparedness and WFP for multi￾sectoral approaches as applicable in their areas. However, implementation of essential
services is a challenge due to limited financial resources.
Conclusions
• The national strategy on SHN is well documented and describes essential package
interventions and approaches.
• There is strong commitment at the highest level and excellent coordination between
health and education through the technical support provided under the McGovern￾Dole project.
• The school meal programme alone cannot be effective in addressing the health and
nutrition issues.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
149
Recommendations
• Multi-sectoral response and coordination is required by departments and
development partners to ensure effective school meal and education programmes.
• Engagement of partners in WASH is essential to have an impact through the SMP
programme on child health and nutrition, and to create awareness in schools to
prevent acute watery diarrhoea.
• Schools must administer regular mass de-worming campaigns based on the
prevalence and intensity of parasitic worms to include all school-age and out-of￾school children.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
150
Annex Q Nutrition
Introduction
1. This annex presents preliminary analysis by the evaluation team of nutrition issues.
It covers a general overview of the nutrition situation in Ethiopia; an analysis of the
nutritional value of the McGovern-Dole school meals; a review of international evidence of
the link between school feeding and educational performance; and a review of available
evidence on Take-Home Rations under the Girls Initiative Programme (GIP).
Nutrition, food security and geographic vulnerabilities
2. Since 2007, Ethiopia has achieved strong economic growth, making it one of the
highest performing economies in sub-Saharan Africa. Yet it remains one of the world’s least
developed countries, ranked 174 out of 187 in the 2011 UNDP Human Development Index
and 76 out of 70 in the 2012 Global Hunger Index. About 29 per cent of the population lives
below the national poverty line (IFPRI, 2017, quoting IFAD 2012).
3. While Ethiopia has enormous potential for agricultural development, only about 25
per cent of its arable land is cultivated (IFPRI, 2017, quoting IFAD 2012). Rain-fed
agriculture employs 80 percent of the country’s 82 million people, forming the basis of
Ethiopia’s economy (IFPRI, 2017, quoting WFP 2012). The vast majority of these farmers
are smallholders; about 12.7 million smallholders produce 95 percent of Ethiopia’s
agricultural GDP (IFPRI, 2017, quoting IFAD 2012). Household food security, particularly
for these smallholders, is determined by rainfall patterns, land degradation, climate change,
growing populations, low agricultural investments, and global market forces. The 2011
Horn of Africa drought left an estimated 4.5 million people in need of emergency food aid.
During the same period, cereal markets experienced a significant supply shock, causing
food prices to rise substantially. Due to improved rains in 2012 and sustained humanitarian
assistance, the overall food security situation has stabilized. However, the Humanitarian
Requirements Document issued by the government and humanitarian partners in
September 2012 estimates that 3.76 million people have required relief food assistance from
August to December 2012. The total net emergency food and non-food requirement
amounts to USD189,433,303 (IFPRI, 2017, quoting WFP 2012).
4. In addition to negative weather shocks, Ethiopia’s rural poor also lack access to basic
social services such as health care, schools, and safe drinking water. Households headed by
women are particularly vulnerable (http://www.foodsecurityportal.org/).
5. Despite these challenges, however, Ethiopia has made significant gains in education,
an expanded health extension system, and the fight against HIV/AIDS. The government’s
long-term strategy of Agricultural Development-led Industrialization continues to address
the country’s food insecurity and is complemented by Ethiopia’s Food Security Programme,
which includes the Productive Safety Net Programme, the Household Asset Building
Programme and others designed to ease households out of food insecurity.
6. Since 2003, the country has faced five serious droughts affecting millions of people,
the most recent of which unfolded over the course of 2015 and was compounded by the
global El Niño event. In July 2015, production assessments reported that up to 25 percent of
the harvest was lost at the national level, and in some regions this figure rose to 70 percent.
In the face of the worst drought in over 50 years, the scale of humanitarian needs over the
course of 2015 rose dramatically. In February 2015, the Government issued its official
Humanitarian Requirements Document (HRD), estimating that 2.9 million people required
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
151
emergency food assistance. By December 2015, this number had risen to 10.2 million -
almost a 250 percent increase (WFP, 2016j).
7. Malnutrition is a complex problem resulting from inadequate food consumption
because of poverty, low access to healthcare, poor sanitation, inadequate infant and young
child feeding practices, high levels of infectious disease and HIV.
8. Despite strong economic gains and a comprehensive policy framework for
development, the distribution of developmental gains remains uneven. While national
figures on nutrition are fairly promising, regional variation is quite pronounced. For
example, in Afar and Somali regions, global acute malnutrition (GAM) rates can be as high
as 30 percent, and stunting rates in Amhara and Afar are close to 60 percent (Central
Statistical Agency & The DHS Program, 2016).
9. According to the Ethiopian National Micronutrient Survey (ENMS) 2015, the highest
prevalence of anaemia was observed in preschool children 6 to 59 months of age, followed
by school age children 5 to 14 year of age and non-pregnant women age 15 to 49 years. As
per the WHO classifications in Ethiopia anaemia was a moderate public health problem in
children 6 to 59 months and 5 to 14 years of age, whereas a mild problem in non-pregnant
women
10. The national prevalence of vitamin A deficiency among school age children was
found 10.9 percent. Among the regions the prevalence of vitamin A deficiency of school age
children who live in Harari is the highest as compared to other region at a prevalence of
25.0 percent. And lowest deficiency was observed in Addis Ababa; almost all children in
this region were not at risk of Vitamin A deficiency during the survey period. According to
the ENMS 2015, highest prevalence of Vitamin A deficiency was observed in preschool
children 6 to 59 months of age, followed by school age children 5 to 14 year of age and non￾pregnant women age 15 to 49 years. As per the WHO classifications, Vitamin A deficiency
could be considered as mild for women of reproductive age and moderate public health
problem for children 6 to 59 months and 5 to 14 years of age.
11. The iodine content in iodized salt has to be monitored from production to
consumption level to ensure retention of adequate iodine. Salt measured by rapid test kit
indicated iodine status of the salt qualitatively. This study showed the national household
iodized salt coverage was some 90 percent. One out of ten household consumed non iodized
salt. Nationally, only one in six households had access for adequately iodized salt to meet
their daily iodine requirement. Iodine deficiency disorder is a severe public health problem
in Ethiopia (Government of Ethiopia, 2016e).
Analysis of nutritional value of meals
12. The school meal is comprised of micronutrient fortified corn soy blend (CSB+),
Vitamin A and D fortified vegetable oil provided by USDA in kind contribution and iodized
salt procured locally. This provides some 1/3rd of daily kcal requirements and over half of
protein and fat. In addition, this meal meets over 70 percent of requirements of Vitamins A,
D, C and iron and iodine.
13. Table 22 below reflects details nutritional composition and the requirements met.
This will depend if the child consumes this meal at school and does not share with others.
The information on acceptability and consumption will be assessed during the field work at
evaluation time.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
152
Nutrition Composition of School Meal per day per Child
Commodity Amount
(gm.)
Kcal protein
gm.
fat
gm.
Vit A
ug
Iron
mg
Iodine
Mg
Vit D
ug
Vit C
mg
CSB+ 120 451 18.3 9.6 666 11.3 048 7.2 121
Fortified oil 006 053 0 6.0 054 0 0 0.5 0
Iodised Salt 003 0 0 0 0 0 120 0 0
Total 129 504 18.3 16 720 11.3 168 7.7 121
Daily
requirements
- 1640 41 31 480 16 108 5.0 33
% requirements
met.
- 31 45 50 150 71 156 153 360
International evidence of the link between nutrition and educational
performance
14. It has been documented that there is a link between nutrition and educational
outcomes in particular with Iron and iodine. Iron deficiency affects more people than any
other health condition. It is a leading cause of anaemia which affects over 2 billion people
worldwide which is over 30 percentof the world’s population. It reduces work capacity and
impairs child’s physical and intellectual development and contributes to some 20 percent of
all maternal deaths. (WHO: http://www.who.int/nutrition/topics/en/).
15. Anaemia leads to 17 percent lower productivity in heavy manual and 5 percent lower
productivity in other manual and estimated 2.5 percent loss of earnings due to lower
cognitive skills (Horton, 2006).
16. Iodine deficiency is a lack of the trace element iodine, an essential nutrient in the
diet. It may result in a goitre, sometimes as an endemic goitre as well as cretinism due to
untreated congenital hypothyroidism, which results in developmental delays and other
health problems.
17. Chronic food shortage remains a serious obstacle to children’s physical and cognitive
development in many poor countries. Hunger diminishes children’s ability to concentrate
and to retain what they learn at school. School meals attempt to improve poor and credit￾constrained households’ investments in education by subsidising the cost of schooling by
reducing short-term hunger and improving nutrition. In poor countries, where school
enrolment is low, school meals can provide a strong incentive for poor households to send
their children to school and to support their education. School meals appear to be attractive
as they may not only increase school participation and reduce dropout, but they may also
improve learning and cognitive development (Poppe et al, 2017).
18. The cost of hunger study in Ethiopia also reflects that stunted children are at higher
risk of repeating grades in school and at higher risk for dropping out of school. Additional
instances of grade repetitions are costly to the education system and families. If a child
dropped out of school early and is working in non-manual labour, he/she may be less
productive. If she/he is working in manual labour he/she has reduced physical capacity and
may be less productive. People who are absent from the workforce due to undernutrition￾related child mortalities represent lost economic productivity.
Take-home rations as part of the Girls Initiative Programme (GIP)
19. In addition to the main school meal programme, the WFP launched ‘The Girls’
Initiative’ intervention in 2002 in food insecure pastoralist areas of four regional states
(Afar, Somali, Oromia and SNNPR). The initiative has the objective of encouraging girls’
education and narrowing the gender gap in pastoralist communities. The programme
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
153
provides eight litres of vegetable oil per semester (‘take-home rations’) conditional on 80
per cent girl’s attendance in addition to on-site school meals. In the first semester of 2010,
81,000 girls received take-home ration. The estimated cost of take-home rations is USD 8.1
per beneficiary girl (during the first semester of 2010).
20. WFP carried out an impact assessment in 2011 using qualitative and quantitative
methods. The assessment found that THR has positive effects on females’ participation in
education. It has been most successful in the chronically food insecure communities. WFP’s
assistance framework that links poverty and food insecurity with education is found to be
sound and relevant for addressing gender inequality. Though the programme had
attempted to enhance the capacity of stakeholders (WEO, PTA, school community), given
the high turnover of trained personnel and low capacity at woreda and school levels, this
was not achieved. THR was designed as a standalone programme with limited synergy with
other initiatives such as WASH, separate toilets for girls, nutrition and health intervention.
The baseline data was not exhaustive enough and did not include programme impact
indicators. It was also found that the programme had weak monitoring systems, particularly
from the partners. THR greatly increased girls’ enrolment, maintained school attendance,
prevented school dropout, and narrowed gender gaps in the target schools. The initiative
also motivated parents to send their daughters to school. The assessment further explored
factors that affect girl’s education and also the challenges in study areas. These include
pervasive chronic food insecurity and poverty, socio-cultural factors and school-related
factors that still a hindrance to girls’ schooling. The assessment also showed that there has
been late delivery of THR due to lengthy process of tendering for hiring transport service. It
was also felt that providing take-home rations to girls may lead to families’ withdrawal of
boys from school in favour of girls (WFP, 2011a).
21. The ET will review in depth the THR programme through intensive case study during
the field visits.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
154
Map 2 WFP Assisted Food for Education and Girls’ Initiative Programme
woredas, 2011
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
155
Annex R Case Study 1: The GoE Emergency School Feeding Programme
Purpose
1. This case study reviews the Government of Ethiopia’s (GoE) experience of
implementing its Emergency School Feeding Programme (ESFP). This programme is
government led and implemented in response to the drought in Ethiopia and first started in
2015/2016.
2. This case study is part of the evaluation of the McGovern-Dole SF programme in
Ethiopia. It was conducted to:
• Document how the ESF programme arose
• Reflect on the strengths as well as some of the continued challenges, in terms of
models, approaches, capacity and resourcing
• Review evidence of learning and influence from WFP’s school feeding work in
Ethiopia to date to the ESF model
• Reflect on ways forward for WFP and the GoE.
Context
3. The GoE Emergency School Feeding Programme emerged as a response to the
drought in Ethiopia. The drought is currently in its third year and the worst in 30 years as a
result of the El Nino effect. It has brought about prolonged food insecurity, malnutrition
and disrupted livelihoods in seven of the country’s nine regional states and one city
administration (Afar, Somali, Oromia, Amhara, Tigray, Harari, SNNPR and Dire Dawa).
Estimates suggest about 3 million school children have been affected by the emergency
(Annex A, Evaluation TOR). The effects of the drought have become progressively worse as
highlighted by FAO’s hotspot classification (FAO, 2017).
4. The drought has affected families and populations in various ways. It has directly
impacted on educational performance through an increase in educational drop-outs and
absenteeism, with students leaving school to assist their parents in search of food and
water, at times with whole families and communities moving away from the area where the
school is located.
5. In addition to direct impact on livelihoods, the drought has also affected parental
capacity to supply school materials (learning materials and uniforms), has produced
increased teacher absentee rates due to shortage of water and food, and has seen children
who would otherwise be in school, engage in income-generating activities to supplement
their families’ earnings.
6. Various reports highlight how the emergency situation has increased children’s’ level
of disturbance, hopelessness, fear and anxiety due to prevailing shortage of food and
water71. And there are indications that girls are particularly affected, with the drought
increasing the risk of early marriage and pregnancy as girls drop out of school or families
71 Disaster Prevention and Preparedness Bureau, Gu Emergency Needs Assessment (Non-Food), June 2017.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
156
resort to marrying off their children. Communal conflicts in some regions72 have further
exacerbated the situation.
An Emergency Response
7. In response to the drought the MoE developed an education in emergency response
plan in 2015.73 Key priorities in this plan include ensuring that children remain in school
through the provision of educational supplies, WASH facilities and school feeding, as well as
psychosocial support and establishing temporary learning spaces to prevent children in
drought affected areas dropping out of school.
8. The budget for the emergency response plan was set at USD53 million. However, the
expected funding support did not materialize. As a result, the MoE has had to scale down
the plan and focused its implementation on the emergency school feeding (ESF)
(Government of Ethiopia, 2015c), which has largely been implemented with the GoE own
funds.
Contours of the Emergency School Feeding Programme
9. The ESFP was first put in place in 2015. It is a government led, government funded
response to the drought.
10. The ESFP is managed by the MoE and funded from supplementary government
funds approved by the Ethiopian parliament. These funds do not come from the regular
education budget and are considered exceptional contributions to respond to an emergency
situation.
Box 5 Composition of ESFP meals per day
11. The ESFP consists of providing children in affected schools with one basic meal. All
commodities are locally procured within Ethiopia although usually in regions other than the
ones that are directly affected by the drought. Wheat flour and chickpea mix is processed.
Analysis of the composition of the meals shows that the meal provides 450 calories per day.
12. Government funding to the ESFP is reflected in Figure 6 below.
72 This is the case for example, in some of the woredas located in border areas between the Somali and Oromia
region.
73 The Education response is part of the broader government-led response to the El Nino crisis.
Wheat flour: 80 %
Chickpea flour: 20 % 120 g (this is Somali specific. See comment
below)
Oil:10 gr
Salt: 03 gr
Total number of calories: 450
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
157
Figure 6 GoE funding for the ESFP (2015-2018) in USD millions
Evidence of learning from WFP SF models
13. The emergency SF programme has included training of key staff (including cooks),
and a system of distribution and monitoring that is modelled on WFP's systems.
14. In contrast to WFP's McGovern-Dole SF programme the meal consists of grains and
legumes which are locally procured through cooperatives. Ingredients have varied from one
region to the next depending on what is locally grown and the dietary preferences of the
local population. In doing so the ESFP has built on WFP's experience in HGSF,
implemented in the SNNPR region (see Box 4 in Annex L).
15. ESFP is now in its third year and provision of school meals has continued in the
2017-18 school calendar year albeit on a reduced scale. MoE conducts an assessment twice a
year to determine if and where ESF should continue.
Performance of the ESFP
16. Evidence of the performance of the ESFP comes from an assessment by the MoE for
the 2015/16 school calendar year. Further evidence comes from this evaluation which
included the ESFP schools in a school based survey and conducted interviews in selected
schools and with education officials, as well as with key partners of the MoE. The following
results were recorded from the ESFP:
• Following the school meal response, anecdotal and survey evidences showed students
went back to school and attendance stabilized. Even students who had dropped out a
long time ago returned to school. Participation of pre-primary children increased
proportionately more than that at other levels. In Amhara and Tigray regions where
school feeding and water is provided, the dropout rate is almost nil. (MoE findings)
• A number of schools that had closed during the drought re-opened. The closure of
schools reduced from 400 to 158 in Somali region and from 137 to 45 Oromia region
after the provision of school feeding in the 2016/17 academic year. (MoE and evaluation
team findings)
• The ESFP also contributed to bringing back parents from migration. Receiving food in
the schools improved household food security through the provision of a meal that
functions as an income transfer to poor families and averted negative coping strategies.
(MoE findings)
• Teachers in drought affected area reported that up to 50 percent of the children were
sleepy in the class before the school meal intervention as they were coming to school
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
158
with empty stomach. Following the intervention the concentration and improve
attentiveness in the class was reported to have improved significantly. (MoE findings)
• Survey results showed that girls’ schooling was more affected than boys’ due to
increased household demands and dwindling finances which put girls at risk of early
marriages and pregnancy as well as increased burden of supporting household income
and survival. The ESF response therefore likely protected children from abuse and
increased vulnerability that occurs during migration/ displacement, with particular
benefits for girls. (MoE findings)
• Various anecdotal psychological and social benefits such as increased togetherness as
children eat together and increased security for parents who were worried about their
children were also reported (MoE findings, and also mentioned to ET in some
interviews)
• The multi-sectoral nature of the response (which involved various government
departments as well as donors) had a positive effect on increased coordination and
integration at all levels. (MoE and ET findings)
• Interviews by this evaluation team also suggest that the governments ownership and
commitment of SF in general increased through the ESFP. Various high-level persons in
government have become champions for SF, including the former First Lady of Ethiopia.
• There is some anecdotal evidence that the programme stimulated the local economy
through procurement of products on the local market. (MoE findings, and also
mentioned to the ET by some informants)
• Some of the decentralized government structures are now budgeting for school feeding.
In the case of ESFP the federal government delivers the emergency SF, and the local
government allocates a budget for non-food items. Parents also provide voluntary
activities – cooking, firewood, etc. (MoE and ET findings)
Challenges
17. Now in its third year, the ESFP has seen a good evolution. Nonetheless key
challenges remain to be addressed. These include:
• The emergency nature of the ESFP – funding comes from additional allocations by the
Government (on an emergency basis only). This means the additional funding has no
sustainable base.
• Costs for SF programmes are too high for many of the regions to consider funding this
from the regular budget, although some regions have also now started funding SF from
their own budget on a limited scale (this is the case for SNNPR and Oromia).
• Long lead times in terms of procurement and organization because of various
constraints related to internal capacity (of the MoE), slowness of government systems
(e.g. for procurement) and capacity of transporters and other sub-contractors, as well as
capacity of communities to transport food from the woredas centres to the schools. As a
result, the funds made available in the first semester only result in food being
distributed to schools in the second semester. This contradicts the emergency nature of
the intervention and potentially reduces the impact of SF.
• While the government has shown strong leadership in taking initiative to fund the ESF
and has been at the forefront of its organization, technical knowhow on how to
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
159
implement the programme falls short of what is needed. This puts a heavy burden on the
federal MoE staff and structures for support.
• Reports indicate that in spite of best efforts the quality and quantity of the meals
provided has not been consistent. The grains and legumes provided by some of the
cooperatives have been reported to be of poor quality and meals have not consistently
been served as per the recommended mix.
• The nutritional of value of the meals provided falls short of international
recommendations in terms of calories provided (450cal/day for ESFP compared to
650/day for the WFP model). The ingredients are also not fortified (WFP CSB and Oil
are fortified) and lack essential vitamins and minerals. In addition, the locally produced
mix is not comparable to CSB both in taste and quality. Combined with challenges of
quality (see previous point) this has contributed to some instances where children have
refused to eat the food.
• Conditions for school feeding are not adequate in a number of places. Many schools are
still reported to lack proper feeding places in schools, and face challenges in terms of
ensuring other basic conditions such as water, affecting proper hygiene and food
preparation. It should be noted that these issues are not unique to the ESFP but have
also affected the WFP SF.
• Contrary to the McGovern Dole SF programme which has the THR, the ESFP does not
include specific gender angle or focus and also does not include a capacity
building/training component as the focus is solely on food provision which is a challenge
as many regional offices do not have the requisite expertise.
WFP support to the ESFP
18. Overall there is strong recognition by key stakeholders of the role that WFP has made
in the establishment of the emergency SF programme. WFP contributed directly to the
design and implementation of the ESFP in a number of ways (Government of Ethiopia,
2017c), namely by providing:
• Technical support to the planning of the ESFP through mapping the food insecure areas
and estimating number of children in need of school feeding.74
• Support to the design of the intervention identifying the food requirements (preparing
food allocation table), and designing menu options based on local food availability in the
market.
• Technical guidance to the preparation of an emergency school feeding programme
implementation manual at federal level, which will be adopted by the regional and local
government. The manual includes many lessons from WFP's work in Ethiopia, and also
provides monitoring and reporting formats.
• Training of government in provision of food handling and management to focal persons
and school directors. WFP seconded technical staff – contracted with funds from the
McGovern Dole programme – played an important role in providing support to the
design and the training.
• Guidance and operational support to the ministry of education in the day to day
implementation process.
74 As part of WFPs involvement in the Ethiopia Education in Emergencies (EiE) cluster.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
160
• Technical assistance to the design of a survey tool to assess the outcome of the
emergency school feeding programme. WFP also took part in the survey with its
technical staff.
19. It should be noted that other partners were also critical to the rolling out of the ESFP.
UNICEF has been important partner given its global responsibility for education in
emergencies. In Ethiopia the education cluster is co-chaired by UNICEF and Save the
Children. UNICEF has provided support in the form of a data specialist, and has placed an
education cluster coordinator in the MoE.
Recommendations
20. To meet essential vitamins and minerals for HGSF, the GoE should explore options
for adding micronutrient powder at the school level in the cooked meal or fortify at the
source during processing. The option of adding during processing is preferable as it is more
efficient.
21. WFP should work with the staff of the MoE to identify ways of ensuring a more
speedy start-up of the ESFP by reviewing whether there are ways and means of reducing
lead times for contracting (of cooperatives and transporters) and reducing challenges in
delivery. The option of retainer contracts could be explored in this context.
22. WFP could consider working with the GoE to ensure that the ESFP includes a
stronger consideration of gender. This would include better organization of meal times to
ensure girls and boys have equal treatment at meal times.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
161
Annex S Case Study 2: Girls’ Initiative Programme – Take-Home Ration
1. This case study is part of the evaluation of the McGovern-Dole FFE programme in
Ethiopia, to document the evolution of THR, its performance reflecting strengths and some
of the challenges, in terms of model, approaches, capacity and resources. It also highlights
selected recommendations to move forward.
2. Background: Poverty is the major obstacle to girls’ education in Ethiopia, particularly
among girls from food insecure pastoralist areas. Food shortage and insufficient income
deter many parents from sending their daughters to school. The education statistics annual
abstract for 2000/2001 indicated that the overall enrolment of boys was about 67 percent
while only 47 percent for girls. There was no special incentive for girls in WFP’s SFP, and
the gender disparity in enrolments between boys and girls was very wide. Given this
disparity, WFP launched ‘The Girls’ Initiative Programme’ (GIP in 2002 in food insecure
pastoralist areas of four regional states based on 80 percent school attendance. (Afar,
Somali, Oromia and in SNNPR(2005) in partnership with Federal Ministry of Education
(MoE), Regional Bureaus of Education (RBoEs), Woreda Education Offices (WEO) as well
as School Communities. To demonstrate the impact of the project, WFP conducted the
study in 2011,75 after 8 years of implementing the project. The strengths, weaknesses and
challenges found by the study and its recommendations are highlighted below:
 Strengths, weaknesses and challenges
strength weakness/challenges
1. positive effects on girls’ participation in
education.
2. most successful in the chronically food
insecure communities.
3. greatly increased girls’ enrolment,
maintained attendance, prevented school
dropout, and narrowed gender gap
4. motivated parents to send their daughters
to school.
1. poor targeting criteria76
2. in some cases girls received the incentive
without attaining 80 percent attendance,
3. poor data recording, reporting and
monitoring
3. lack of participation of school
communities and the parents
4. lack of indicators at the initial stage to
measure the impact,
5. limited coordination and synergy with
other activities and programmes such as
wash, health, nutrition and education not
only on thr,
6. erratic delivery of oil due to long tender
process of hiring transport service
7. funding shortfall.77
recommendation:
wfp to continue its effort and scale up the programme in food insecure areas while addressing
gaps and challenges and lesson learnt from the implementation in 2002.
3. A paper published in May 2017 in the Journal of Development Studies, using data
from WFP Impact assessment study, highlights “THR are found to increase girls’
75Girls’ Initiative Impact Assessment, January 2011, Addis Ababa (WFP, 2011a) This study was conducted by a
local consultant.
76Targeting was often based on roster of girls in the 2nd semester of the previous academic year rather than on
enrolment in the new academic year.
77 Only 21 percent was resourced leading to a significant underachievement in meeting planned targets. To
address the resource shortfall, WFP negotiate the regional bureaus to prioritize the targeted schools, reduce
the number of feeding days and ration size.`
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
162
concentration remarkably. It also indicated that supplementing on-site school meals with
take-home rations can be beneficial for concentration, reading, writing and arithmetic
skills. The timing of the distribution of school meals was also found to play an important
role – serving food in the morning is more effective than at the end of day (Poppe et al,
2017.
4. Building on the findings and the recommendations from the impact assessment WFP
expanded the THR as part of the MDG FFE in food insecure pastoralist regions of Afar and
Somali in 2014. The programme addresses gender parity and equity through the provision
of school meals and a Take-Home Ration (THR) that specifically targets girls. The
programme provides 8 litres of fortified vegetable oil to all girls once a month who have
maintained at least 80 percent of school attendance. The cost of take-home ration is USD
10.68 per girl child per semester.
5. There are 131,529 girls both in Afar and Somali region with breakdown in Afar and
Somali region as shown in Table 24.
 FFE beneficiaries in Afar and Somali
FFE programme Afar Somali Total
Girl THR beneficiaries and percentage of
total beneficiaries
43,119 (46%) 88,410
(45%)
131,529
(45%)
On-site feeding beneficiaries
(boys and girls)
93,983 198,266 292,249
Source: WFP MDG FFE Ethiopia- September 2017 (WFP, n.d.-a)
Performance/ Findings78
6. The findings generally indicate that the current THR programme is appropriate/
relevant, and has been effective in addressing the objectives of the programme. The data
from the survey and the interviews with various stakeholders reflected THR has greatly
increased girls’ enrolment, maintained school attendance, prevented school dropout, and
narrowed gender gaps in the target schools. The initiative also motivated parents and the
communities to send their daughters to school. This has also led to a decrease in early
marriage and income transfer to food insecure households.
7. The effect of take-home ration has become evident in improving the gender gap. The
data from WFP shows the gender parity index has improved and the enrolment and
attendance and literacy have significantly gone up. The GPI for Afar Region is 0.9 and is
almost equivalent to the national GPI (0.91), while the GPI for Somali region is at 0.86
(Government of Ethiopia, 2016f).
8. THR has brought an effect on income transfer to parents and education benefits to
their daughters. Most parents stated that the programme has contributed to supplementing
household food income to cover the cost of learning materials, clothing for their school age
children and of course the diet diversity.
9. The survey data on programme status and grade completion rates shows
improvement for the girls is greater than for the boys, with the girls in the non-FFE schools
having higher dropout rates than for boys. In Afar completion rates are lower than in
78 The findings of the qualitative data are mainly from the visits and interviews from Jijiga. The ET, though it
visited Afar, could not interview many girls and parents as the schools were closed.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
163
Somali region for both FFE and non-FFE schools, and the non-completion rate for girls also
notably worse than in Somali. Table 25 reflects the data.
 Grade completion by gender for sampled FFE and non-FFE schools
(2009 Ethiopian calendar, Sep 2016 – Jul 2017)
In 95% confidence limits
Region FFE Schools Gender Enrolled Passed % Pass ±% Lower Upper
Afar Yes 24 Boys 2535 1997 79% 1.6% 77% 80%
Girls 2360 1760 75% 1.8% 73% 76%
No 6 Boys 1127 767 68% 2.7% 65% 71%
Girls 918 572 62% 3.1% 59% 65%
Somali Yes 30 Boys 11521 11032 96% 0.4% 95% 96%
Girls 10793 10422 97% 0.3% 96% 97%
No 30 Boys 6585 5662 86% 0.8% 85% 87%
Girls 4484 3787 84% 1.1% 83% 86%
10. Parents and Community attitude towards Girls’ Education. The parents in
Somali region said this was an excellent programme and very relevant and appropriate to
the Somali culture, where the girls stay at home to take care of household chores and get
married at an early age. This has dramatically changed since the inception of THR and SMP.
The enrolment and attendance for the girls has significantly increased.
11. As the THR is combined with school meal, this means that the effect is even bigger.
The community sees the benefits of both the SM and THR. The parents also cited – the
communities profit in two ways from the SMP/THR and how their children are getting
education and food in schools.
12. The fact that more and more girls are now enrolled in school has led to a decline in
traditional early marriage together with actions by Woreda officials against early marriage,
setting a minimum age for marriage (22 years for females and 25 years for males) to deter
the practice of early marriage.
13. THR has increased the duration of girls staying in schools and this would have
expected multiplier effects - longer stay delays the marriage age which in turns delays first
pregnancy and leads to healthy pregnancy and healthy children and better child care. In the
long run this will contribute to breaking the intergenerational cycle of under nutrition. 79
14. When community leaders were asked about the changes they have observed during
the years as the result of THR, the overwhelming majority of the respondents said “girls are
motivated to attend school regularly and has motivated parents to send daughters to
school and the gender gap in enrolment in schools has significantly narrowed. In addition
THR has contributed in providing income transfer to the families.
15. A majority of parents and the community said- Oil is an appropriate commodity and
is consumed and used in the family. It helps the poor families the most.
16. Impact of discontinuation of THR. When asked the girls and the parents of the
girls, what would be the impact if THR is not available- some of the parents and all the girls
79 Food Insecurity: Could School Supplements help break cycles of Intergenerational Transmission of
Inequalities? Journal of Pediatrics, Dec 2010
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
164
said we value education with or without the oil? The girls said ‘We would continue and even
go to high school and would like to be teachers ‘“learning is more important than oil.” They
said their families would also not forbid them from coming to school just because oil is no
longer provided.
17. However, some parents said the girls who are now in school may continue but new
enrolment and attendance may fall through and it will have a negative impact especially to
the poor families and some said that they will still continue to send their children (both boys
and girls) to school if the programme is ended saying, “we like getting the oil, but we’re
more interested in seeking education for our children.” One of the cooks interviewed said
she would not pull her two daughters out of school if the THR ends, I want my daughters to
be educated and get good jobs in the city. I want them to be like you, (pointing to the
interviewer), traveling around for work and writing well.” However, in the opinion of the
ET, respondents might have been reluctant to respond that they would discontinue.
18. According to the officials, parents and their children have become increasingly aware
of the benefits of education, so if the SFP/THR programme discontinues, the Woreda
doesn’t think that the students, including the girl students, will drop out. The fact that there
will be no more porridge and oil to take home may discourage new students from joining
schools, but it will not cause the ones that are already in schools to drop out. The school
enrolment will plateau. Therefore, it is important to continue with the programme in order
to encourage new students to join schools.
19. Some officials said, although the community in general has come to appreciate the
value of education, some families may choose to pull their children out of school,
particularly those who themselves are illiterate. If the THR is ended there will be a visible
impact on the participation of girls and overall enrolment will go down. The drought is also
a major factor that would compel families to pull their children out of school, if the SF
programme is ended. The SF programme is what is sustaining families during the drought.
20. According to BoE SFP focal persons, the very young students, especially those in
primary and first grades may drop out since they themselves have not yet developed an
appreciation for education, and for most, “the major motivation for coming to school is the
meal and the THR that is provided.”
21. Parents also expressed, the fact that there are no high schools in the area deter many
girls from pursuing education after graduating from elementary school. The nearest high
school are in towns which are more than 35 km away, and parents are very reluctant to send
their girls to live in the town unless they have family members there who are willing to take
the girls in and look after them.”
22. All the girls appreciate that they are given oil as THR when they attend school
regularly. They said they get one tin of oil (4 litres) but they had different answers to how
frequently they receive the THR. Some said once every month, while others said once every
two months. According the school director, each girl student that has attended class for no
less than 18 days in a month receives one tin of oil (4 litres) once every two months. The
THR is distributed once every two months instead of once a month so that one girl student
can take one tin of oil instead of dividing it into two, as it would have been the case if the
distribution happened once a month (2 litres per month). ET noted that some of the girls
were not aware of the amount of oil received. There were some confusing responses due
erratic supply/sharing of tins. However, as per WFP, Oil is distributed once a month.
(Personal communication with SFP, Unit).
23. Most parents but not all were aware of the amount given to each girl (8 litre) for a
semester. It was mentioned that there has been some delays. This was further confirmed by
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
165
WFP that due to pipeline break and logistics there has been disruptions. Parents, teachers
and girls did say that oil is mainly consumed with in the family and also bartered to buy
other household items and to diversify the family diet. It has an economic value and is used
as part of the pastoral diet.
24. The answers were consistent with the results of the quantitative survey as the girls
reported receiving 100 percent in FFE schools, whereas boys did not receive. Frequency was
reported 2-3 per semester, which is consistent with the protocol for THR. Most of the oil
(over 90 percent responded was used for cooking, with small proportion reporting being
sold/bartered.
25. The majority of the girls, parents and the officials did know that the oil provided by
USDA is of good quality and very healthy and some girls did say that oil has some vitamins
which makes them healthy but did not know which vitamins.
26. It was also felt by some that providing take-home rations to girls may lead to
families’ withdrawal of boys from school in favour of girls (Though boys did not complain
and said –“our sisters are bringing the oil and, come to school as well and we are happy for
them.” The girls also said the boys don’t complain that they don’t get the oil and said their
mothers use the oil to prepare food for everyone in the family, so it is not only the girls that
benefit from the THR but the whole family. The boys also agree that it is important that the
girls get oil for attending school regularly, saying, “the oil is not only for the girls but also for
the whole family and sometimes even gets shared with neighbours.” On balance and in spite
of questions probing this matter the ET did not find evidence of major negative impact on
boys’ attendance and schooling. Some girls said their families sometimes sells some of the
oil (but not all) to buy food for the family.
27. ET also noted that some of the schools visited in both regions are getting “double
benefits” from the THR, selling the oil containers and ploughing back the money into the
SFP (hygiene component) by purchasing hand-washing soap and dishwashing detergent.
28. It was also noted that Girls are not only receiving THR but are educated on gender
issues, health hygiene and nutrition which prepare them for making good decisions in life.
29. The senior government officials quoted- “GPI is the backbone of FFE”. It has
changed the lives of the girls-they are happier and have healthy faces- they are different
now. They have developed self-confidence. This is an eye opener of the parents, girls and
the community. And the eyes are now widely opened. “THR has “opened the eyes” of the
girls and their families to the value of education. The girls are now more confident to sit in
the same class with boys”. The ET also noted the heightened self confidence among girls
who were interviewed.
30. According to one of the school directors, the impact of THR has seen increased
enrolment and retention, and decreased absenteeism. And some girls who have graduated
from this school, have pursued higher education and are now engineers, teachers and
technicians as an example the Finance Manager of one of the woredas is a lady who
attended her primary education at this school.
Challenges
31. The interviews with various stakeholders showed that there has been late delivery of
THR due to lengthy process of tendering for hiring transport service. As an example the
food was not available in Somali region for one semester.
32. The ET noted the quality of oil containers poses some challenges in transport and
storage as many were leaking and oil became rancid. This issue was brought to the attention
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
166
of USDA by WFP and has been taken up and sturdy containers will be used in the next
phase.
33. The ET found on the reporting receipt forms for THR, in some schools girls use
thumb prints; this would appear to defeat the purpose of educating girls. In another school,
it was also found that in the signature column signatures/names of all the girls were written
in one handwriting while in another school no official record of THR distribution was kept,
which suggests there may be issues in terms of adequate monitoring/control in some
schools.
34. ET requested sex segregated data for each grade in primary schools. The data was not
easily available at BoE and WFP offices. However, the data results from the quantitative
survey provide very encouraging results.
Conclusions
35. The effectiveness of THR for girls has positively associated with high enrollment,
attendance and to address the parity gap between boys and girls.
36. THR not only benefits girls targeted in the programme, but also all children in the
household. This may be mainly due to value transfer to members of benefiting households.
THR has attempted to improve poor and credit-constrained households’ investments by
addressing chronic food shortage which remains a serious obstacle to children’s physical
and cognitive development.
37. In conclusion, both parent and pupils value the THR programme and would like this
to continue as this has created awareness on the value of education among the parents and
the community.
Recommendations and way forward
38. GIP should continue in some form (cash or in-kind) to encourage girls to go to
school, given its powerful impact on girls’ education and empowerment to break the cycle
on intergenerational gap.
39. WFP and development partners should advocate with the GoE to ensure that the
Government policies and strategy to include an incentive for girls’ education in food
insecure /pastoral societies using funds from the PSPN programme. A standalone
publication from the findings of this assessment to be used for advocacy.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
167
Annex T The Survey – Data Collection Tools and Methods
Objectives and sample size
1. The sample survey addressed the evaluation questions relating to impact and
effectiveness through a quantitative, evidence-based and statistically sound approach. At
the same time, it collected qualitative data from a significant number of key informants
involved in the sampling to be analysed for recurrent themes and observations in a quasi￾quantitative manner through word clouds, word and phrasal frequencies. The field
sampling also provided the framework for more discursive and purely qualitative KIIs with
school principals, and FGDs with PTAs during the fieldwork.
2. The sample size of 90 schools estimated in the project proposal was retained, as
being consistent with prior information to give 10 percent sampling error for 95 percent
confidence intervals on worst-case binomial estimates. This comprises 30 schools in Afar
region, and 60 schools in Somali region, the 1:2 split ratio being proportional to the number
of government primary schools in the region (573 Afar, 1207 Somali). The level of sampling
is also consistent with the resources available to, and the timeline of the evaluation.
3. In addition to the sample survey, data on educational indicators (enrolment, grade
completion, by gender) for all primary schools was provided by the MoE EMIS system for
the 4 evaluation years 2012/13 to 2015/16. This was not a sample, but a census
(100 percent coverage) of schools and therefore provided comprehensive information on
the impact of the programme on these basic educational indicators.
Sampling method and selection
4. The sample survey used pairs of schools, in and out of the programme, within the
same locality. The In-Programme (IP) schools were selected as a random sample within
each region. The nearest Out-of-programme (OP) school was then chosen as the sample
pair. This method requires prior information on the school locations and whether they are
IP or OP. This information is available as a 2013 shapefile provided by WFP CO giving
school locations, type (primary, ABE, private, secondary, etc.), grades taught and WFP
programme status. The consultants processed this data to remove unnecessary information
(schools outside Afar, Somali regions, schools other than government primaries teaching
first cycle, grades 1-4). The resulting distribution of potential sample schools is shown in
Figure 7 below, with IP as green dots, and OP as red dots.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
168
Figure 7 Distribution of In-Programme and Out-of-Programme Primary
Schools in Afar and Somali, 2013
● In-Programme; ● Out-of-Programme
5. Before proceeding with the sample selection, several difficulties had to be overcome.
WFP CO were able to provide a list of all schools in the FFE programme in Afar and Somali
in 2016. This comprised 361 schools in Afar and 229 schools in Somali (590 altogether).
However, these data were not georeferenced and could not be directly linked to the data in
the 2013 baseline. Georeferenced data was available from WFP monitoring activities in
2016, which covered a total of 179 schools in Afar and 79 in Somali, 258 in total. The school
positions were checked against those of the 2013 dataset and found to correlate closely
within the limits of expected measurement error (c. ±50 m). From this triangulation, it was
possible to establish the 2013 dataset as an effective baseline with respect to IP and OP
schools.
6. Sample selection was made from the 2013 data by downloading details from the GIS
file to Excel, including coordinates converted to UTM coordinates (expressed in metres East
and North, and therefore easy to work with to calculate inter-school distances). A random
number was assigned to each school, and the list then sorted on this number, within
regions. This gives a randomly ordered list. A VBA macro was written to find the nearest
OP school to each IP school, subject to a constraint of 30 km maximum. The first 15 IP
schools on the Afar list, with their paired OP school, constituted the Afar sample, and the
first 30 on the Somali list likewise the sample for that region. In all 45 IP-OP pairs were
selected (90 schools).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
169
Figure 8 Map of selected IP (●) and OP (●) sample school pairs
7. This first random sampling produced five problem selections in Somali. Four were
in border areas where there are currently security and access issues. One pair was very
isolated and would have required 3 days to sample. These problem selections were
reassigned to the next sequential pairs on the random list to produce a manageable
selection. This is shown in Figure 8 above. The selected schools are listed in Table 26
below.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
170
 List of schools to be sampled
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
171
8. By the time the fieldwork started in February 2018, further or different accessibility
issues might have arisen. The director of the field work was supplied with a list of randomly
ordered substitute schools, and could select the next IP-OP pair in the event that one of the
locations in Table 26 was inaccessible.
9. In Table 26, the Pool column refers to the sample pool. Samples could be substituted
by changing the pool number to 3, and then selecting the next available pair (i.e. the top
pair) in pool 2 (not shown here for reasons of space). The School ID is the ID used in the
database and on the SI to uniquely identify the school. The prefix A or S refers to the region
Afar or Somali, and the suffix a or b refers to the IP or OP member of the pair.
10. Geographical coordinates are given in UTM units or decimal degrees. The latter are
compatible with Google Maps, allowing this data to be easily uploaded to that system to
provide navigational assistance in the field. The zone, woreda, school code and name are all
taken from the 2013 dataset, and differ in some details from current WFP usage. School
names are generally phonetically recognisable but have a number of spelling differences.
However, correcting these is a long task that could not be done within the context of the
Inception Mission. This harmonisation was expected to be done once the EMIS dataset
(which was to provide definitive current definitions) was available.
Data collection procedures
11. The Survey Instrument is shown in Annex U. Each sample school was visited by a
team comprising a supervisor and 4 enumerators. One enumerator interviewed the school
head or senior staff member present and completed the school questionnaire (SQ) in the SI.
The other three enumerators selected one class each at random from each of grades 2-4 (it
being assumed that grade 1 children may often be too young to be able to answer some of
the questions). That is a total of 3 classes, one each from grades 2, 3 and 4. In each class, 2
boys and 2 girls were also selected at random for interview, and the Child questionnaire
(CQ) of the SI completed. At the end of the CQ were questions for the child's class teacher,
relative to attentiveness and last academic years’ mark. In both the SQ and CQs there were
a mixture of binomial (yes/no), multi-category choices, quantitative data such as enrolment
and completion figures, and open comment questions for textual analysis.
12. The Supervisor also conducted a Key Informant Interview with the school principal
or senior staff member present to obtain qualitative view regarding the effectiveness and
impact of school feeding, the benefits of infrastructure improvements and of capacity
building activities and any other relevant positive or negative factors associated with them.
These KIIs were conducted in all 90 schools.
13. A local facilitator assisted teams, and schools were pre-contacted to advise them of
visits. Letters of authority were obtained (from relevant sectors as well as WFP) prior to all
visits. All enumerators were from the respective regions and spoke Amharic, Afar and
Somali as necessary. Teams included both female and male enumerators; girls were
interviewed only by female enumerators. The forms shown below were translated into
Amharic, Afar, and Somali versions for field use. Text comments were translated back into
English for data entry.
14. Data collection was on paper forms. The use of tablets is these days commonplace
but there are significant disadvantages in this case. Entry of free-text data is slow and error
prone on small devices. Tabular data such as question SS1 in the SQ (see Annex U) are not
easy to arrange on small screen displays and are likewise error prone to enter. There is no
hard copy to reference during the error checking and cleaning phase, and devices are
subject to a number of risks not applicable to paper forms such as lack of power supply,
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
172
breakdown, damage or loss. However, paper forms impose an additional work phase for
data entry, and the data entry screens must be devised. Both of these were considered and
were included in the survey logistics.
15. It was estimated that each team would be able to complete 2 schools per day,
normally the IP-OP paired schools, which are relatively close (less than 30 km). Three
teams were deployed, able to complete 6 schools per day. 15 working days, or three
calendar weeks, were therefore required to complete the field work. Data collection was
expected to start at the beginning of the second semester of 2017-18 on Monday 5th
February 2018 and continue until Friday 24th February. Data entry and cleaning was to
commence the following week and not require more than 5 days, with statistical analysis
commencing in parallel and requiring a further week thereafter. Thereafter results were
available for the report writing phase.
16. The data was entered from paper forms using CSPRO census and survey processing
software, with SPSS for tabulation and basic analysis. This was done in Addis Ababa by the
national consultants, B&M consulting. The external consultant (Denis Alder/Valid
International) reviewed and validated the data using various R statistical software packages.
This included range checks, digit preference, outlier identification and similar methods to
identify anomalies to be checked as necessary against original paper forms to identify
correctable errors, with errors that could not be resolved being rejected from the sample set.
Analysis Considerations during inception
17. It was not clear whether direct comparisons of IP and OP schools as paired samples
would reveal significant differences, due to the complexity of variable factors. In particular,
the Emergency School Feeding (ESF) programme during 2015 and 2016 could have resulted
in some IP schools having had reduced rations, whilst some OP schools could have received
the ESF rations during this period. In addition, there were known to be other support
programmes for some schools. Neither for ESF nor for other, non-WFP programmes was
there sufficient prior information available for these to be factored into the design. The
pairing of schools mainly accounted for cultural and eco-agricultural variations that were to
be associated with geographic affinity.
18. The analysis approach therefore used general linear modelling (GLM) to consider
factors such as the presence-absence or level of infrastructure development, student-teacher
ratios, or level of alternative school meals support, such as ESF, in relation to the academic
indicators such as enrolment, promotion and at the child level, relative performance and
attentiveness scores.
19. The approach should allow for the leverage of particular factors, such as the quantity
and quality (food diversity) of food, or the presence/absence of particular infrastructure or
capacity building elements, in terms of educational performance to be assessed.
20. The Emergency School Feeding programme is a specific topic for review, and this
was covered in the qualitative analysis in terms of availability, sufficiency and evidence for
impact.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
173
Annex U Survey Instrument
1. The draft survey instrument is included on the following pages. This has two parts: a
School questionnaire (SQ) and a Child questionnaire (CQ). 12 child questionnaires are
completed for each school, with 4 children being selected from a class in each of grades 2-4.
Selection of classes within grades and of children is by a random process. Team supervisors
have already been trained in the use of this form (and their feedback has helped to improve
its design and language), and in the random selection process. Instructions on random
selection and a unique sheet of selection numbers will be included with the forms.
2. The survey instrument was presented to the WFP Country Office for review in a
workshop on 8th December 2017, and suggested amendments and improvements to the
initial draft have been incorporated in the version presented here.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
174
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
175
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
176
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
177
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
178
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
179
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
180
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
181
Annex V Stakeholder analysis
Introduction
1. Table 27 below provides a detailed stakeholder mapping. It distinguishes internal
from external stakeholders, and shows stakeholder interests in the McGovern-Dole
International FFE and Child Nutrition Programme and in the evaluation itself. The final
column identifies potential interviewees/informants (the ET is working with CO to obtain
appropriate names and contact information).
 Detailed stakeholder mapping
Stakeholder
Interest in the McGovern￾Dole International FFE and
Child Nutrition
Programme
Involvement in
Evaluation and likely
use
Who (specifically for
the Evaluation)
A. Internal (WFP) stakeholders
Country Office
(CO) Ethiopia
(and sub
offices)
The CO has commissioned this
evaluation and is also the
primary WFP internal
stakeholder of the evaluation.
The CO is accountable to
donors, beneficiaries and
partners for performance and
results. Interested in learning
from experience to inform
decision-making related to
project implementation. It has a
direct stake in the evaluation,
an interest in learning from
experience to inform decision￾making related to project
design, implementation, and/or
monitoring.
Interest in using the evaluation
findings in strategic planning
for the CO’s engagement with
the government in developing a
National School Feeding
Programme.
The key informant and
source of information. Also
primary user of (as well as
being affected by) the
evaluation findings and
recommendations to
inform programming.
Facilitate logistical
arrangements for in￾country mission;
Participate in briefings and
de-briefing missions.
Participated in inception
meetings.
Has established an
evaluation reference group
of WFP and external
stakeholders to review and
comment on the various
reports.
Senior Management;
Technical leads for school
feeding, nutrition, safety
nets; gender and M&E;
Former staff no longer in
position (where relevant)
Sub-office staff: field
monitors, relevant
technical staff and M&E
officers.
Sub offices responsible for local
planning and implementation
of SF.
WFP
Washington
Office
Responsible office for managing
communication with the USDA
FAD related to performance
management of the McGovern￾Dole Ethiopia grant; Has a
direct stake in understanding
the methodology and findings
of the evaluation.
Responsible to involve
USDA FAS in stakeholder
discussions and
communicate its comments
on deliverables.
Consulted at inception.
McGovern-Dole and
USDA liaison staff
Regional Responsible for oversight of the Informant and source of Selected members of
Bureau (RB) CO and providing technical regionally relevant Management/ Technical
Nairobi guidance and support. Interest
in an independent account of
information related to SF,
gender, safety nets and
Staff for school feeding,
Nutrition; M&E (as
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
182
Stakeholder
Interest in the McGovern￾Dole International FFE and
Child Nutrition
Programme
Involvement in
Evaluation and likely
use
Who (specifically for
the Evaluation)
operational performance and in
cross-country learning from the
evaluation.
various other technical
issues; Provide technical
oversight; Form part of the
external reference group;
Participate in debriefings
and meetings; Provide
comments on report.
relevant) and assigned
evaluation focal point
Participated and facilitated
inception mission.
Findings may inform
programming regionally
and in other countries.
WFP HQ Interest in lessons that emerge Potential source of Selected technical leads of
(including from evaluations, particularly information on WFP thematic units linked to
technical units) as they relate to WFP strategies, approaches, standards and thematic areas of
policies, thematic areas or success criteria, as well as relevance – M&E, School
delivery modality with wider corporate strategic Feeding, Capacity
relevance to WFP directions where these may Development, Safety Net
programming. influence
programming/operations
nutrition, gender, etc.) as
required.
Office of
Evaluation
(OEV)
Has a stake in ensuring the
decentralized evaluations
deliver quality, useful and
credible evaluations.
Potential source of
information through other
evaluations of relevance.
User of evaluation findings
and recommendations.
Where relevant, selected
members of OEV staff
who have been involved in
other school feeding or
McGovern-Dole related
evaluations.
WFP Executive
Board (EB)
Has an interest in being
informed about the
effectiveness of WFP
operations.
This evaluation will not be
presented to the EB but it
is a potential user of
evaluation findings
through corporate learning
processes/annual
syntheses.
Unlikely to be targeted
directly
B. External stakeholders
Ultimate
beneficiaries
Children are the ultimate
recipients of the McGovern￾Dole International FFE and
Child Nutrition Programme, as
well as a select group of parents
who directly benefit from the
McGovern-Dole Take-Home
Ration (THR). They have a
stake in WFP determining
whether its assistance is
appropriate and effective.
Key informants for the
evaluation will be
consulted during site visits
to determine the type of
support received, whether
it has been effective etc.
They are likely to be
affected directly or
indirectly by the
evaluation, but are unlikely
to directly engage in report
findings.
Mother and fathers of
students in targeted
schools, girls and boys
from different groups in
targeted schools;
Female students as
beneficiaries of THR.
Indirect
beneficiaries
School administrations,
teachers, other school staff,
parents and communities who
Will be consulted through
individual interviews and
focus group discussions, as
Administrators of schools,
male and female teachers,
male and female members
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
183
Stakeholder
Interest in the McGovern￾Dole International FFE and
Child Nutrition
Programme
Involvement in
Evaluation and likely
use
Who (specifically for
the Evaluation)
are recipients of various
training programs, equipment
support and other assistance.
appropriate. of Parent Teacher
Associations, Food
Management Committees,
CHILD Planning Teams;
Regional and District focal
persons
Government of
Ethiopia (GoE)
(Federal,
Regional and
District levels)
Have a direct interest in
knowing whether WFP
activities in the country are
aligned with their priorities,
harmonised with the action of
other partners, meet the
expected results, if capacity has
been built and what further
inputs might be needed in
future.
As the direct institutional
beneficiary, the Ministry of
Education (MoE) and the
Regional Education Bureaus of
Afar and Somali Regions are
most interested.
The Ministry of Finance and
Economic Development
(MoFED) leads the major
coordination mechanisms for
interventions under UNDAF.
Government of Ethiopia
(GoE) representatives
facilitate evaluation
mission(s) and participate
fully in the evaluation
process; has representation
in external evaluation
reference group.
Will be consulted through
individual interviews
and/or round table
discussions, as appropriate.
MoE consulted at
inception.
Relevant technical focal
points of the MoE (e.g.
Education Management
and Information System
(EMIS), Planning and
Resource Mobilization
Directorate), MoH, MoA,
MoFED, MoLSA.
At regional level, official of
REB, for Afar and Somali
region, plus officials of
other bureaus involved.
At district level, local
development officials and
technical staff of various
district level authorities
(education, WASH,
health) in selected
districts.
Issues related to handover and
sustainability are also of
interest to the Ministry of
Health (MoH), Ministry of
Agriculture (MoA) and Ministry
of Labour and Social Affairs
(MoLSA)
Donor (USDA) USDA funds the SF programme
in Ethiopia and they are the
primary stakeholders and users
of the evaluation. It has a
specific interest in ensuring
that operational performance
reflects USDA standards and
accountability requirements.
Potential source of
information related to
USDA standards and
accountability
requirements; Participate
in discussions of findings
and recommendations.
Where relevant, selected
members of USDA staff
who have been involved in
the Ethiopia McGovern￾Dole school feeding
intervention will be
consulted.
Consulted at inception.
Will be consulted through
selected individual
interviews with key
members and/or round
table discussions, as
appropriate
Other Aid
Agencies
Aid agencies supporting the
programme and/or working in
the same field, including Save
Key informants for the
evaluation both in terms of
national-level
Representatives at
national and district levels
from these agencies;
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
184
Stakeholder
Interest in the McGovern￾Dole International FFE and
Child Nutrition
Programme
Involvement in
Evaluation and likely
use
Who (specifically for
the Evaluation)
the Children and other strategic
partners under the Education
in Emergency Cluster.
priorities/focus/policy and
in terms of
implementation, including
technical aspects.
Relevant technical staff
members involved in SF
Potential users of the
evaluation findings for
strategic orientation and
wider programming.
Will be consulted through
selected individual
interviews and/or round
table discussions, as
appropriate.
Strategic
partners under
the UN Country
Team (UNCT)
The UNCT country team has an
important stake in the
effectiveness of the SF
programme and how it
contributes to the realisation of
the government developmental
objectives.
Informants through round
table discussion; potential
users of the evaluation
findings for wider
programming and policy
discussions.
Will be consulted through
round table discussions.
Key technical staff of these
agencies.
Reference Groups
2. Internal and external reference groups (WFP only and WFP with other stakeholders,
respectively) have been formed for the evaluation.
3. Their roles are in line with the guidance provided in the respective DEQAS Technical
Notes (WFP, 2016b, WFP, 2016c).
4. At the time of writing the draft IR, the ET was still waiting for the WFP Ethiopia CO
to provide the names of the members of both committees. For this reason the names have
not been included here.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
185
Annex W Approach to interviews and school level observations
Stakeholder interviews
1. Interviews will be a key source of information for the evaluation. They will be a
means to obtain more in-depth insight into issues that have been covered by the Survey
Instrument.
2. The interviews will also ensure that evidence drawn from other sources and the
document review are triangulated.
3. The ET will target a comprehensive range of stakeholders that fully represents all
significant institutional, policy and beneficiary interests. The stakeholder analysis will
inform the selection of interviewees at all levels (federal, regional and local).
4. The evaluation report will list all those individually interviewed (unless they ask for
their details to be withheld), along with their principal organisational affiliation (where
relevant) and their gender.
Interview process and note keeping
5. Key points to be covered in each of the interviews and focus group discussions with
various categories of informants are presented below. These guidelines will be flexibly used,
and not all interviews will cover all the points. The choice of the interview questions will be
left to the interviewer/facilitator and will be made in line with priority gaps, and the time
available.
6. Interviews will be confidential. Most interviews will be conducted on a one-to-one
basis. Reports will not quote informants by name and will not include direct quotes or
attribution without prior consent. Interviews at woreda, kebele and school level will be
done through a translator.
7. While the ET hopes that WFP personnel will accompany them to interview sites and
introduce them to interviewees, it will respectfully request them to leave once the
introductions have been made, in case this enables interviewees to speak more freely.
8. Interview notes will be written up, consolidated into an interview compendium and
shared among team members via the internal team-only e-library. To respect interviewee
confidentiality, the interview notes will be accessible only to team members. The
compendium of interview notes will facilitate analysis across all interviews and will enable
searches on key thematic terms. This will maximise the analytical potential of interviews
and the possibilities for triangulation.
Please note that these guidelines are not intended as questionnaires but rather
as generic questioning guides. Team members will use their judgment to focus
on areas which are likely to add most to the ET's existing knowledge, while
allowing interviewees and groups to highlight the issues of most importance to
them.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
186
DISCUSSION GUIDES
Discussion guide for WFP personnel
Introduction
• Introduction of team member(s) present
• Explain purpose of evaluation
• Explain confidentiality and next steps
Appropriateness
1. To what extent was the McGovern-Dole SF programme coherent with national policy, with 
the interventions of government, with the work of other development partners, and with
the needs of the population?
2. To what extent has the programme design reflect a strong focus on gender and equity?
How was this done?
Results and factors affecting results
3. How successful has the programme been in achieving its planned outputs?
4. To what extent have planned outcomes been attained?
5. How adequately has the operation addressed gender and equity issues? How was this
done?
6. To what extent has the operation built capacity of the GoE in managing national SF
efforts?
Factors affecting results
7. What accounts for the areas of success of the intervention? What internal and external
factors have positively affected the achievement of results?
8. What did not happen and why? What internal and external factors have negatively
affected the achievement of results?
(Note: follow-up questions can query internal WFP process, system and logistical
factors; monitoring and reporting; internal institutional and governance
arrangements; partnership and co-ordination; as well as external factors such as
national political and policy environment; domestic and external funding; etc.)
9. Were there unintended positive or negative results or outcomes? Please explain.
10. How likely is the GoE to continue to implement an effective SF programme following
WFP withdrawal? What are the key constraints?
General
11. Do you have recommendations about the design, implementation and continued
sustainability of the operation? With the benefit of hindsight what would you do
differently?
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
187
Discussion guide for Government of Ethiopia personnel (national level)
Introduction
• Introduction of team member(s) present
• Explain purpose of evaluation
• Explain confidentiality and next steps
Appropriateness
1. To what extent was the McGovern-Dole FFE and Child Nutrition programme coherent
with national policy, with the interventions of government, with the work of other
development partners, and with the needs of the population?
2. To what extent has the programme design reflect a strong focus on gender and equity?
How was this done?
Results and factors affecting results
3. How successful has the programme been in achieving its planned outputs?
4. To what extent have planned outcomes been attained?
5. How adequately has the operation addressed gender and equity issues? How was this
done?
6. To what extent has the operation built capacity of the GoE in managing national SF
efforts?
Factors affecting results
7. What accounts for the areas of success of the intervention? What internal and external
factors have positively affected the achievement of results?
8. What did not happen and why? What internal and external factors have negatively
affected the achievement of results?
(Note: follow-up questions can query internal WFP process, system and logistical
factors; monitoring and reporting; internal institutional and governance
arrangements; partnership and co-ordination; as well as external factors such as
national political and policy environment; domestic and external funding; etc.)
9. Were there unintended positive or negative results or outcomes? Please explain.
10. How likely is the GoE to continue to implement an effective SF programme following WFP
withdrawal? What are the key constraints?
11. What learning took place from the McGovern-Dole intervention that informed the
Government’s Emergency School Feeding Programme? What have been the strong and
weak points of the ESFP and what are the lessons learnt for future SF in Ethiopia?
General
12. How do you perceive WFP as a partner? Please explain.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
188
13. Do you have recommendations about the design, implementation and continued
sustainability of the operation? With the benefit of hindsight what would you do
differently?
Discussion guide for Government of Ethiopia personnel (local level)
Introduction
• Introduction of team member(s) present
• Explain purpose of evaluation
• Explain confidentiality and next steps
Background
1. Please explain briefly when and how the school feeding programme came to this area
and what partners have been involved?
Appropriateness and understanding of the intervention
2. How relevant was the SF programme to it beneficiaries when it started? Has it continued
to remain relevant?
3. What is CSB and what does it contain? Oil: What does it contain? What are the benefits
of adding these?
Results
4. To what extent has the programme produced results?
5. What difference has the programme made to the lives of beneficiaries, and to the
community? What has changed as a result of the programme?
6. Has the programme made a difference to the schooling of girls? And to that of boys? In
what way?
7. How effective have WFP’s capacity development activities been? Please explain.
8. Were there unintended positive or negative results or outcomes? Please explain.
Factors affecting results
9. What accounts for the areas of success of the intervention? (probe for WFP factors, and
for external factors, including for other social protection activities that may affect the
outcomes e.g. access to grants, etc.)
10. What activities were not implemented and why? (probe for internal and external factors
that may have negatively affected the achievement of results)
11. How do you perceive WFP as a partner? Please explain.
Sustainability
12. Are the changes that you mentioned took place as a result of the programme continued
after the programme is completed? Why or why not?
13. Do you have recommendations about enhancing the sustainability of the operation?
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
189
Discussion guide for USDA
Introduction
• Introduction of team member(s) present
• Explain purpose of evaluation
• Explain confidentiality and next steps
Appropriateness
1. To what extent was the McGovern-Dole FFE and Children Nutrition programme coherent 
with national policy, with the interventions of government, with the work of other
development partners, and with the needs of the population?
2. To what extent has the programme design reflect a strong focus on gender and equity?
How was this done?
Results and factors affecting results
3. How successful has the programme been in achieving its planned outputs?
4. To what extent have planned outcomes been attained?
5. How adequately has the operation addressed gender and equity issues? How was this
done?
6. To what extent has the operation built capacity of the GoE in managing national SF
efforts?
Factors affecting results
7. What accounts for the areas of success of the intervention? What internal and external
factors have positively affected the achievement of results?
8. What did not happen and why? What internal and external factors have negatively
affected the achievement of results?
(Note: follow-up questions can query internal WFP process, system and logistical
factors; monitoring and reporting; internal institutional and governance
arrangements; partnership and co-ordination; as well as external factors such as
national political and policy environment; domestic and external funding; etc.)
9. Were there unintended positive or negative results or outcomes? Please explain.
10. How likely is the GoE to continue to implement an effective SF programme following
WFP withdrawal? What are the key constraints?
General
11. Do you have recommendations about the design, implementation and continued
sustainability of the operation? With the benefit of hindsight what would you do
differently?
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
190
Discussion guide for school staff
Introduction
• Introduction of team member(s) present
• Explain purpose of evaluation
• Explain confidentiality and next steps
Background
1. Please explain briefly when and how the school feeding programme came to this area
and what partners have been involved.
Appropriateness and understanding of the intervention
2. How relevant was the SF programme to it beneficiaries when it started? Has it continued
to remain relevant?
3. What is CSB and what does it contain? Oil: What does it contain? What are the benefits
of adding these?
Results
4. To what extent has the programme produced results?
5. What difference has the programme made to the lives of beneficiaries, and to the
community? What has changed as a result of the programme?
6. Has the programme made a difference to the schooling of girls? And to that of boys? In
what way?
7. How effective have WFP’s capacity development activities been? Please explain.
8. Were there unintended positive or negative results or outcomes? Please explain.
Factors affecting results
9. What accounts for the areas of success of the intervention? (probe for WFP factors, and
for external factors, including for other social protection activities that may affect the
outcomes e.g. access to grants, etc.)
10. What activities were not implemented and why? (probe for internal and external factors
that may have negatively affected the achievement of results)
Sustainability
11. Are the changes that you mentioned took place as a result of the programme likely to
continue after the programme is completed? Why or why not?
General
12. What should have been done differently in terms of programme design, and
implementation?
13. How do you perceive WFP as a partner? Please explain.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
191
Discussion guide for school children
Approach
The ET will seek to interview small groups (between four and six) school children aged ten
or above. Girls and boys will be interviewed separately.
Introduction
Schools will have been asked to inform parents about the interviews and to seek their
consent. Children will be told they do not have to participate and that they may opt out of
the interview at any time.
Questions will be posed in simple personal terms (Do you like the food? Do you always eat
it? Do you eat before you come to school? What did you eat today before coming to school?
What did you eat yesterday after the school?
Introduction
• Introduction of team member(s) present
• Explain purpose of evaluation
• Explain confidentiality and next steps
Appropriateness
1. Do you think that the school feeding programme is needed at your school? Why?
2. Do you think the programme is providing the right kind of food?
3. Do you know children of school age who do not get school feeding? If so, why not?
Results
4. Do you like the food? Why or why not?
5. What is CSB and what does it contain?
6. Oil: What does it contain?
7. What do you think are the benefits of adding these elements to what you eat?
8. Are there any problems with the school feeding programme? If so, what are they?
9. What difference does the school feeding programme make to you?
10. Does the school feeding have a different effect for girls and boys?
11. Is the oil that girls get if they come to school important? In what way?
12. How could the school feeding programme be improved? How would you change the
school feeding programme if you could decide?
For the girls only:
13. Do you know of any girls who are no longer coming to school? What is the reason for
this?
14. Have you regularly received the take home ration of oil?
15. How is the ration used by your family?
16. How does it make you feel to receive this ration?
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
192
17. If you no longer receive the ration will this have an impact on whether you will be able to
come to school? Please explain.
For boys only:
18. Do you know boys who are no longer coming to school? What it the reason for this?
19. Are there days that you are not able to come to school? What are the reasons for this?
Discussion guide for PTA and community groups
Introduction
• Introduction of team member(s) present
• Explain purpose of evaluation
• Explain confidentiality and next steps
Background
1. Please explain briefly when and how the school feeding programme came to this local
school, and what the roles and contributions of the various stakeholders were and are.
Appropriateness and understanding of the intervention
2. How relevant was the SF programme to it beneficiaries when it started? Has it continued
to remain relevant?
3. What is CSB and what does it contain? Oil: What does it contain? What are the benefits
of adding these?
Results
4. What have been the results of the SF programme for the school? What have been the
results for the community?
5. What difference has the programme made to the beneficiaries, and to the community?
What has changed as a result of the programme?
6. Has the programme made a difference to the schooling of girls? And to that of boys? In
what way?
7. How effective have WFP’s capacity development activities been? Please explain.
8. Were there unintended positive or negative results or outcomes? Please explain.
Factors affecting results
9. What accounts for the areas of success of the intervention? (probe for WFP factors, and
for external factors, including for other social protection activities that may affect the
outcomes e.g. access to grants, etc.)
10. What activities were not implemented and why? (probe for internal and external factors
that may have negatively affected the achievement of results)
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
193
Sustainability
11. Are the changes that you mentioned took place as a result of the programme likely to
continue after the programme is completed? Why or why not?
General
12. What should have been done differently in terms of programme design, and
implementation? How would you change the school feeding programme if you could
decide?
Discussion guide for parents of families that benefit from THR
Introduction
• Introduction of team member(s) present
• Explain purpose of evaluation
• Explain confidentiality and next steps
Questions:
1. How many girls from your family go to school?
2. What did they receive in School in addition to a meal?
3. For how long and how much THR (Oil) have you received?
4. Is it regularly received?
5. How is the oil used by your family?
6. Has the programme made a difference to the schooling of girls? And to that of boys? In
what way?
7. If you no longer receive the THR, will you still send your (daughters) to school?
8. Do you what does the oil contain?
9. What other support does your family receive (in cash and in-kind)?
10.In what ways does this support help your family? (ask for details)
11. Does this support make it easier for you to send your children to school?
12. Does it make it easier for you to send your girl children to school?
Observation guide for school feeding
Storage of the food
1. How appropriate is the space in terms of:
• Ventilation
• Protection against rodents
• Are bags stacked
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
194
• Screens on windows
• Records and registers
• Security
• Other observations
2. Registration/record keeping at school level including attendance records
• How good is record keeping on food received and used?
• Is there are record of the composition of the meals on a daily basis?
• Is there a record of daily school attendance by children?
• Is there a record of store entries and use of food ?
• Is there are record of beneficiaries of the THR ?
Community contributions in kind
3. Are communities contributing:
• Food
• Firewood
• Water
• Local produce
• Through labour
• Other
4. How is this is organized?
5. Arrangements for food preparation Is there:
• Appropriate and sufficient space for all pupils
• Adequate hygiene
• Adequate organization
• Availability of water
• Availability of (sufficient) utensils
• Soap for cleaning
6. Are fuel saving stoves used?
7. Arrangements for meal times
• Appropriate space/organization
• Composition of meals
• Utensils for eating
• Drinking water
• Facility for hand washing
8. Are meals served in a way that provides equal portions and treatment to boys and girls ?
9. Are there latrines for boys and girls ?
10. Are the latrines :
• Clean
• Well maintained
• Appropriately placed
11. Reporting tools and processes (what are the tools, how are the used/understood, who is
responsible, frequency and quality of reporting compared to expected standards)
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
195
TEMPLATE FOR MAKING INTERVIEW NOTES
Date (e.g. 2018-01-06): Interview title
Include as many interviews as convenient in the same draft; when finalised, all interview
notes will be added to the Interview Compendium. Use the unshaded cells in the table
below.
Please use SF3 heading styles to make compiling the compendium easier.
General
Date: Location of Interview: Team Members Present:
Interviewee(s)
Name: Designation: organisation, job title Contacts:
Note taking
Name: Date completed: Recorded? Y/N
Background
Interviewee's general background
Topics
Record responses by topic with clear headings, not necessarily in chronological sequence
of discussion. Make clear when a direct quote is recorded. Add headings and sub-headings
as needed.
Data/documents provided/recommended
Seek full references for documents not already in evaluation e-library.
Other proposed follow-up
e.g. other interviewees recommended / proposals on consultation and dissemination
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
196
Annex X Results of EMIS data analysis
Introduction
1. Data was requested from the MoE EMIS system at the start of the inception mission,
in late November 2017. The data requested was, by school, for all Government primary
schools in Afar and Somali, the enrolment and repeat rates, for boys and girls separately, for
grades 1-4 separately, and for the school years commencing 2013 through to 2016 (2006 –
2009 Ethiopian Calendar).
2. Note that as the various data sets and school years are referred to in MoE statistical
publications and in most WFP CO internal lists and worksheets by Ethiopian dates, these
are used in the following descriptions. The corresponding academic years are shown in
Table 28 below:
 Ethiopian and Gregorian years
Ethiopian
Calendar (EC)80
Gregorian Calendar,
(GC) – Academic Year
2005 Sep 2012 – Jul 2013
2006 Sep 2013 – Jul 2014
2007 Sep 2014 – Jul 2015
2008 Sep 2015 – Jul 2016
2009 Sep 2016 – Jul 2017
2010 Sep 2017 – Jul 2018
3. Although the team at the time of preparing the Inception Report (IR) were informed
that there would be no problem in providing this data, in the end, the data received (in
January 2018) was more limited, and comprised for a single year, 2009 EC (2016/17),
enrolment and repeat rates, by boys and girls, for all grades aggregated, for all primary
schools in Afar and Somali81. This information also included data on numbers of teachers
and number of classrooms. It did not show which schools were in or out of the McGovern￾Dole FFE programme (IP or OP schools).
4. Table 29 below shows the files used here for the analysis. An additional potentially
useful workbook was provided by the WFP CO with 2005-2008 EC enrolment and dropout
numbers for boys and girls, by grades for 2005 and 2006, and school totals for 2007-2008.
Unfortunately, the data layout was problematic as the various data items
(enrolments/dropout rates) could not be linked across years or between schools with
certainty, so in the end it was not used.
80 The Ethiopian year runs from September to August in the Gregorian calendar and therefore fully includes
the academic year. The Ethiopian year starts on Sep 11th Gregorian (Sep 12th in leap years).
81 It was understood that MoE and EMIS were under considerable work pressure during December-January
due the need to complete the annual statistical publications for Government, which limited time available for
the WFP/ET work.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
197
 Data files contributing to EMIS analysis (Afar and Somali Regions)
Source Description of data set No. of Schools Symbolic
Name Afar 82 Somali Total
MoE
EMIS
For each primary school, 2009 EC, all grade combined,
school name, region, zone and woreda names,
enrolment and repeat numbers for boys and girls,
number of teachers (male and female), number of
classrooms, grades taught, school ID code.
677 1486 2163 EMIS09
WFP CO A list of schools in the FFE programme in 2009 EC,
including school name, region, zone and woreda, and
enrolment for boys and girls.
361 229 590 WFPSCH
ET/Annex
T
A list of paired schools in and out of the FFE
programme used as the basis for sample selection,
including school name, region, zone, woreda,
georeference in UTM and Latitude/Longitude, and
sample ID. This file was developed as described in
Annex T as a basis for sample selection from WFP
school monitoring data and a UNICEF-sponsored 2013
school mapping exercise.
232 194 426 SAMPLESCH
Problems matching school names and identifications
5. Combining the EMIS09 and WFPSCH data was a very slow process that took several
days of work. This is because there appears to be no unambiguous system of school
identification in use beyond the school and woreda name. These vary considerably in
spelling, with phonetic transcription issues to the Roman alphabet being very evident. To
match up school and woreda names, three strategies83 were used sequentially, as detailed
below:
• Soundex matching. The Soundex codes were calculated for woreda and school
names in the two files, and schools which matched based on this considered to be
valid matches. Soundex is not ideal for Ethiopian languages, as it is based on English
sound values and strongly weighted to the initial phoneme, but it appeared to
produce very few false positives. It however failed to match many schools where the
initial letter was modified. Soundex ignores vowels (except for the initial letter),
which seemed mostly to work well. About 50 percent of schools were matched by
this method.
• Jaro-Winkler (JW) distance. The JW score is based on number of transpositions,
insertions, deletions require to match two strings. It was found by trial and error
that allowing a maximum score of 0.25 (JW scores range from 0-1), when applied to
schools not matched by Soundex, gave about 20 percent additional matches.
• Finally, the remaining unmatched schools were examined manually with candidates
from the same woreda, and judgement used to select names likely to correspond.
6. Box 6 shows some examples of typical school name variations which were considered
based on the above processes to be matches. However, in the 590 schools listed as in the
programme by WFP, 170 could not be positively identified in the EMIS data. Overall,
81 percent of schools in Afar and 55 percent in Somali could be matched.
82 As used in R for analysis, and in this text for ease of reference.
83 See the R stringdist library documentation, page 20 for a brief description of the Soundex and Jaro-Winkler
algorithms. https://cran.r-project.org/web/packages/stringdist/stringdist.pdf.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
198
Box 6 Examples of school name variations
1. Baheta Bohoyita 9. badile Badle
2. GERBAH TENI Gerbahateni 10. afaburey Afeburie
3. Tiritiri
4. Sagantole
Tertera
Segentolie
11. weaytu
12. Igile
Waido
Egeli
5. Arafalale Arefelalie 13. Askabri bure Asikeriburi
6. Dudub Dudubi 14. Koka Bekoka
7. Wasero
8. Ongayoto
Wasiro
Hungaytu
15. Gawutoo Gawt
7. There are several possible explanations regarding the unmatched schools:
• They have not yet filed EMIS statistical returns for 2009 EC, and were therefore not
in the dataset provided. Considering this covers the period up to July 2017, this is
not improbable.
• They filed returns but used a name too different from that in the WFP data, and it
could not be matched.
• The status of the school has been confused with an ABE (Alternative Basic
Education) or other category of school.
8. The practical consequence of these problems is that a great deal of time (several
days), which should have been used for evaluation of the data, was lost on the simple
process of trying to marry up schools from different lists. The second consequence is that
in the analysis, when comparing in and out of programme schools, it will be noted that:
• In-Programme (IP) schools have been positively identified.
• Out-of-Programme (OP) schools are more weakly identified, and apart from the
possibility of their receiving food from non-McGovern-Dole FFE sources, such as the
HGSF programme, they could in fact be FFE schools that have been misclassified due
to the above issues.
Statistical comparison of IP and OP Schools in EMIS data for 2009 EC
9. Table 26 above shows the total number of schools included in the analysis. The
EMIS data included a number of schools which did not give any enrolment figures. These
were also excluded. Some schools listed on the WFP schedule of FFE schools could not be
positively identified in the EMIS data by name and woreda, and were therefore also
excluded.
10. Of the 590 schools listed by WFP as in the FFE, 411 could be identified in the EMIS
data set. Of the total of 2163 schools listed in the EMIS dataset, 2133 where included in this
study as providing complete enrolment data.
11. Table 30 below shows the enrolment and GPI for the schools included in Table 26
above from the EMIS data set. The GPI is more favourable in the FFE than non-FFE
schools. The difference is particularly marked in Somali, where FFE schools have an
overall GPI of 90 percent, whereas in the non-FFE schools, it is 69 percent.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
199
 Enrolment for Boys and Girls and Gender Parity Index, from EMIS
data
Boys
FFE Schools
Girls Total GPI Boys
Non-FFE Schools
Girls Total GPI
Afar 39431 34805 74236 88% 46802 36369 83171 78%
Somali 36618 32801 69419 90% 304042 211073 515115 69%
12. The figure for grade repeats is not supplied by many schools, and the summary
repeat data shown in Table 31 below should therefore be viewed with caution.
 Grade repeats (non-completion) for Boys and Girls, from EMIS
data
Boys Girls
FFE Schools
Total Boys% Girls% Boys
Non-FFE Schools
Girls Total Boys% Girls%
Afar 2342 2383 4725 5.9% 6.8% 2718 2331 5049 5.8% 6.4%
Somali 21 11 32 0.1% 0.0% 670 395 1065 0.2% 0.2%
13. Repeat rates appear to be slightly higher for girls than boys, but do not differ much
between FFE and non-FFE schools in Afar region. In Somali region, many schools did not
provide a figure for repeat rates. However, despite the small numbers, there is a clear
difference between FFE and non-FFE schools, with non-FFE schools having higher repeat
rates.
Conclusion
14. Although the EMIS data promised to be very useful and was emphasised in the
Inception Report as a source of data, in the end it was disappointing. Data was supplied
very late and was very difficult to use. The central problem is the lack of a unique
registration number for each school. Data cannot be linked between years and from various
sources because of considerable variation in the spelling of school and woreda names. This
is probably a factor in the difficulties MoE had in providing the data initially. It also
contributes to problems in data cleaning and error checking. It results in errors into the
query process through double counting or drop-out of records that cannot be matched up.
15. From the little that could be made of the data supplied, enhanced girls enrolment is
associated with the FFE. It also appears that grade repeat rates are lower in FFE schools in
Somali than for non-FFE schools, though there is little evidence of this effect in Afar.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
200
Annex Y Field Survey Results
Introduction
1. A field survey of 90 primary schools was undertaken during February 2018 in
accordance with the protocol and design set out in Annex T, and using the Survey
Instrument given in Annex U. During the field work, some substitutions were made
for schools that were inaccessible, wrongly classified or otherwise unsuitable,
according to the pre-planned protocol of re-selection from a randomized list to
maintain randomness and objectivity. The sampling design was a stratified random
sample (SRS) of school pairs, with the regions (Afar and Somali) as strata, and
sampling intensity proportional to primary school population. Each pair comprised
a school in the FFE programme (In-Programme, IP), and an Out-of-programme (OP)
school in the same vicinity. In the event, the prior information about the FFE status
of each school was not exact. Table 32 shows the final achieved distribution of the
sample.
 Planned and achieved sample distribution
(IP = In FFE programme, OP = Out of FFE programme)
Planned sample Achieved sample
Region IP OP Total IP OP Total
Afar 15 15 30 24 6 30
Somali 30 30 60 30 30 60
Total 45 45 90 54 36 90
% 50% 50% 100% 60% 40% 100%
2. Figure 9 shows the distribution of the sample by total enrolment. The data for
this chart is shown in Table 33. A map of the final sample of schools is shown in
Figure 10, together with the location of other, non-sampled primary schools, the
latter being based on 2013 data.
Figure 9 Distribution of sample schools by enrolment size
( In FFE programme,  out of FFE programme)
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
201
 Number of sample schools, by enrolment class
FFE up to 200- 400- 600- 800- 1000- 1200- 1400- 1600- 2600-
Region status 199 399 599 799 999 1199 1399 1599 1799 2799 Total
Afar IP 13 10 1 24
OP 5 1 6
Somali IP 1 10 4 5 2 4 1 2 1 30
OP 13 12 1 1 2 1 30
Total 32 32 6 5 3 4 3 3 1 1 90
Figure 10 Distribution of sample schools by enrolment size
Pale green area is Afar region, pale yellow is Somali region. Red squares are sampled, out-of-programme
primary schools. Purple circles are sampled, in-programme primary schools. Blue dots are other
government primary schools, not sampled. ABE and non-government primary schools are not shown.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
202
FFE Programme Status and Grade Completion Rates
3. Two objectives of the FFE programme are to improve attendance by girls, and
to improve overall school attendance. Question SS1 on the Survey Instrument
(Annex U) requested details of enrolment and completion for boys and girls
separately by grade for each school visited. The following tables summarise the
results from this data, relative to their status in the FFE programme.
 Grade completion by gender for sampled FFE and non-FFE
schools
(2009 Ethiopian calendar, Sep 2016 – Jul 2017)
In 95% confidence limits
Region FFE Schools Gender Enrolled Passed % Pass ±% Lower Upper
Afar Yes 24 Boys 2535 1997 79% 1.6% 77% 80%
Girls 2360 1760 75% 1.8% 73% 76%
No 6 Boys 1127 767 68% 2.7% 65% 71%
Girls 918 572 62% 3.1% 59% 65%
Somali Yes 30 Boys 11521 11032 96% 0.4% 95% 96%
Girls 10793 10422 97% 0.3% 96% 97%
No 30 Boys 6585 5662 86% 0.8% 85% 87%
Girls 4484 3787 84% 1.1% 83% 86%
 Statistical significance of performance differences associated
with gender and FFE status
based on data in Table 34
Region Comparison Difference
∆%
Probability
∆% due to
chance
Significant
difference?
Afar Boys - Girls, In FFE 4.2% 0.00051 ***
Boys - Girls, Not in FFE 5.7% 0.00654 **
FFE - non-FFE, Boys 10.7% < 0.00001 ***
FFE - non-FFE, Girls 12.3% < 0.00001 ***
Somali Boys - Girls, In FFE - 0.8% 0.00175 **
Boys - Girls, Not in FFE 1.5% 0.02561 *
FFE - non-FFE, Boys 9.8% < 0.00001 ***
FFE - non-FFE, Girls 12.1% < 0.00001 ***
Statistical significance: * ≥ 95%, ** ≥ 99%, *** ≥ 99.9% probability that observed difference not due to
chance.
4. Table 34 above shows the numbers of students who enrolled and completed
their grade year in 2009 (Ethiopian calendar, 2016/17 Gregorian calendar). The
confidence limits for the estimated grade completion rates from the sample are
shown. Table 35 above contrasts the main differences from Table 34 and shows the
statistical significance of the effects. Of note are the large differences in grade
completion between the FFE and non-FFE schools, with the FFE schools performing
much better. This difference is of the same magnitude in Afar and Somali (about
10 percent). The improvement for the girls is greater than for the boys, with the girls
in the non-FFE schools having higher dropout rates than for boys. In Afar
completion rates are lower than in Somali for both FFE and non-FFE schools, and
the non-completion rate for girls also notably worse than in Somali.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
The modal (most common) reply, with frequency % in parentheses, is shown, together with the next most common reply, except for FCS (CQ15), which shows percentages
above and below a threshold FCS of 35. See text for more details. Based on 1048 children interviewed, 2 boys, 2 girls each from grades 2-4 per school, school numbers per
Table 34 NA = Not Answered.
SI ref
Survey Instrument Questions
Question Choices
In
Girl
Afar Region
FFE Non-FFE
Boy Girl Boy
In
Girl
Somali Region
FFE Non-FFE
Boy Girl Boy
CQ6 How many days a week Never / 1-2 days / 3-4 Every day [88.0%] Every day [51.9%] Every day [79.4%] 3-4 days [52.9%] Every day [93.9%] Every day [93.9%] Every day [78.7%] Every day [76.9%]
do you come to school? days / Every day 3-4 days [7.2%] 3-4 days [43.7%] 3-4 days [20.6%] Every day [29.4%] 3-4 days [6.1%] 3-4 days [5.0%] 3-4 days [19.7%] 3-4 days [22.5%]
CQ7
Do you eat at home in
the morning before
coming to school?
Never / 1-2 days / 3-4
days / Every day
Every day [57.6%]
1-2 days [16.8%]
1-2 days [33.3%]
Never [26.7%]
Every day [52.9%]
3-4 days [20.6%]
Every day [55.9%]
Never [20.6%]
Every day [35.2%]
3-4 days [31.8%]
Every day [55.8%]
Never [29.3%]
3-4 days [43.8%]
Every day [37.1%]
Every day [65.4%]
3-4 days [20.3%]
CQ8 How often in a week do Never / 1-2 days / 3-4 Every day [69.6%] Every day [45.9%] Never [79.4%] Never [79.4%] Every day [93.3%] Every day [88.4%] Never [97.2%] Never [97.8%]
you eat in the school? days / Every day 1-2 days [20.0%] 3-4 days [43.0%] 3-4 days [8.8%] 1-2 days [11.8%] 3-4 days [6.1%] 3-4 days [5.5%] 3-4 days [1.7%] 3-4 days [1.6%]
CQ9
Do you eat in the
evening, after going Never / 1-2 days / 3-4 Every day [77.6%] 1-2 days [27.4%] Every day [94.1%] Every day [61.8%] Every day [81.6%] Every day [74.0%] Every day [74.2%] Every day [71.4%]
home? days / Every day 3-4 days [10.4%] Never [25.9%] 3-4 days [2.9%] 1-2 days [14.7%] 3-4 days [14.0%] 3-4 days [17.1%] 3-4 days [18.5%] 3-4 days [15.4%]
CQ10
Do you feel sleepy or
tired when you come to Not at all / A l ittle / Not at all [81.6%] Not at all [86.7%] A little [52.9%] A little [50.0%] Not at all [56.4%] Not at all [84.5%] Not at all [41.0%] Not at all [59.3%]
school? Quite tired / Very tired A little [16.0%] A little [9.6%] Not at all [41.2%] Quite tired [20.6%] A little [26.3%] A little [12.7%] A little [38.8%] A little [24.2%]
CQ11 Do you l ike eating the Yes / Not much / No Yes [98.4%] Yes [96.3%] No [64.7%] Yes [55.9%] Yes [95.5%] Yes [98.9%] NA [93.3%] NA [91.8%]
school food? No [1.6%] NA [3.0%] Yes [26.5%] No [26.5%] Not much [2.2%] No [1.1%] Yes [5.1%] Yes [4.9%]
CQ12 Is the food enough? Too much/ Enough / Not
quite enough / Too l ittle
Enough [92.8%]
Too much [7.2%]
Enough [50.4%]
Too much [44.4%]
Not quite enough
[47.1%]
Enough [23.5%]
Too much [41.2%]
Enough [29.4%]
Enough [53.6%]
Too much [41.9%]
Enough [91.7%]
Too much [7.2%]
NA [94.4%]
Enough [3.4%]
NA [94.0%]
Enough [3.8%]
CQ13 Do you feel satisfied
after eating? Yes / Not quite / No Yes [98.4%]
No [1.6%]
Yes [96.3%]
NA [3.0%]
No [52.9%]
Yes [32.4%]
Yes [58.8%]
No [23.5%]
Yes [94.4%]
Not quite [2.8%]
Yes [85.1%]
No [13.8%]
NA [94.4%]
Yes [5.6%]
NA [93.4%]
Yes [4.9%]
CQ14 Do you bring firewood Never / 1-2 days / 3-4 1-2 days [64.0%] 1-2 days [55.6%] Never [67.6%] Never [73.5%] Never [52.0%] Never [66.3%] NA [52.2%] Never [50.0%]
or water to school? days / Every day Every day [17.6%] Never [30.4%] Every day [17.6%] 1-2 days [17.6%] Every day [36.9%] Every day [14.9%] Never [47.8%] NA [47.8%]
CQ15 Food Consumption
Score
Frequency 8 food groups
over last 7 days, WFP
weightings
Below 35 [38.4%] Below 35 [55.6%] Below 35 [52.9%] Below 35 [50.0%] Below 35 [18.1%] Below 35 [13.3%] Below 35 [28.8%] Below 35 [21.4%]
Acceptable [61.6%] Acceptable [44.4%] Acceptable [47.1%] Acceptable [50.0%] Acceptable [81.9%] Acceptable [86.7%] Acceptable [71.2%] Acceptable [78.6%]
CQ17
Last semester, did you
get some oil to take
home?
Yes / No Yes [100.0%] No [96.3%]
Yes [3.7%]
No [79.4%]
Yes [20.6%] Yes [100.0%] No [98.9%]
Yes [1.1%]
CQ18 How often was that, in Once / 2-3 times / 2-3 times [93.6%] NA [96.3%] NA [79.4%] 2-3 times [82.1%] NA [98.9%]
the semester more often more often [3.2%] 2-3 times [3.0%] 2-3 times [20.6%] Once [16.2%] 2-3 times [1.1%]
CQ19
Do you know what your
family does with the
oil?
Don’t know / Cooks with
it / Sells or trades it
Cooks [95.2%] NA [96.3%] NA [79.4%] Cooks [87.7%] NA [98.9%]
Sells... [4.0%] Cooks [3.7%] Cooks [17.6%] Don't know [8.4%] Cooks [1.1%]
CT3 Teacher's assessment
of child's attentiveness
Very low ./ Below
average / Average /
Good / Very Good
Good [41.6%] Good [40.0%] Average [67.6%] Average [67.6%] Good [41.9%] Good [48.1%] Good [39.3%] Average [51.1%]
Average [38.4%] Average [37.8%] Good [29.4%] Good [20.6%] Very Good [24.6%] Average [30.9%] Average [33.7%] Good [23.6%]
207
Child Questionnaire Analysis
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
208
Child questionnaire analysis
5. The Child Questionnaire was applied in the schools according to the programme in
Annex T, using the Survey Instrument detailed in Annex U. The procedure was to select 2
girls and 2 boys at random from the school register in each of grades 2-4, who were then
interviewed outside the classroom by an experienced enumerator of the same gender. The
photographs in Figure 11 show this in process. In all, it was planned that 1080 children
should be interviewed (12 children each from 90 schools). In the end 1072 interviews were
conducted, and 1048 interviews used in the final dataset. Some data (24 children) were
omitted due to duplicate indexing issues, which could not be resolved within the time limits
available for analysis.
6. Table 36 above summarises the key results for each question. Except for the Food
Consumption Score (FCS), the modal (most frequent) response is shown first, with
percentage of responses, and then the second most frequent answer. For FCS, the
percentage above and below the borderline value of 35 is shown.
7. Reviewing these replies, interesting and consistent patterns emerge relative to FFE
and non-FFE schools. CQ6, attendance, is lower in Afar than Somali, and is consistent with
the completion data in Table 34 from the school register. In Somali, however, attendance at
non-FFE schools is notably lower.
8. Question CQ7, shows consistently for all categories that the children tend to eat at
home before coming to school in the morning, though the FFE figure for boys in Afar is low.
However, CQ8, regarding eating at school is very consistent with the FFE status, as
recorded from the school's own records. For Non-FFE schools, children rarely eat at school,
whilst for FFE schools, they report that they always eat at school. This is a striking
difference, suggesting that sources of school meals outside those provided by the FFE have
relatively little impact on meal provision.
9. CQ 9, regarding eating in the evening, shows the same pattern as morning meals,
with all categories except the boys in Afar FFE schools reporting that they generally eat at
home after school. For boys in Afar there appears to be dependence on the FFE as their
main food source where available.
10. CQ10 regarding feelings of tiredness or lassitude on arrival at school, are generally
answered 'Not at all' or 'A little'. However, the boys in the non-FFE schools in Afar report a
higher percentage who are 'quite tired'.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
209
11. CQ11-CQ13 concern the quality and quantity of school food available. For the FFE
schools, the responses are positive and clear, with food well liked and mostly enough, and
for some, too much. For non-FFE schools the replies are mixed and confusing, given that
they report in CQ8 that they never eat in school, so perhaps they are largely referring to
food outside school.
12. Regarding bringing firewood to school, mostly the children do not but in Afar, about
20 percent do so, though there is no consistent differentiation according to gender or FFE
status. In Somali, 37 percent of Girls and 15 percent of boys bring firewood to school in the
FFE schools, but none do so in the non-FFE schools.
13. The Food Consumption Score (CQ15) is calculated conventionally according to WFP
standards from the recall questions for food groups eaten in the last 7 days, as shown in the
Survey Instrument. In Table 36, the percentage with FCS below 35, classified as borderline
or low, are shown. In both Afar and Somali there are higher FCS scores for both boys and
girls for the FFE schools than the non-FFE schools. As with other indicators, nutritional
status appears to be somewhat better in Somali than in Afar.
14. Questions CQ17 – CQ19 concern the Take-Home Ration (THR), principally oil. The
answers are consistent with expectations: Girls in FFE schools report 100 percent received
the ration, whereas boys did not. Frequency was reported to be 2-3 times per semester,
again consistent with the protocol for the THR. Most of the oil (88-95 percent of replies)
was used for cooking, with a small proportion reporting it was sold or traded.
15. Question CT3, regarding the class teacher's perception of the child's attentiveness,
shows some small but consistent relationship to the FFE status. In Afar and Somali FFE
schools, Good is the most common category, whilst for non-FFE schools it is Average in
Afar and for Boys in Somali but remains Good for girls in Somali.
School facilities and capacity building
16. Table 37 below shows the results of the survey questions relating to school facilities.
The main observations can be summarised as follows:
Staff numbers and classrooms
17. Table 37 shows that schools in the FFE programme tend to have more classrooms
and teachers than the non-FFE schools (SF1, SF4). This may be a result of the targeting
policy for the preferential inclusion of larger schools in the programme. Across all
categories, there are a median of about 2 women teachers per school, whereas total staffing
has medians of 5 to 12 teachers depending on stratum. In Afar student staff ratios are
around 20, whereas in Somali they are higher, 50 in FFE schools, and 32 in the non-FFE
schools.
18. Cooks and assistants (SF2) have medians of 2 in the FFE schools and 1.5 in the non￾FFE schools in Afar, whereas in Somali there is a clear distinction between FFE and non￾FFE schools, with a median of 3 cooks/assistants in the FFE schools, and no cooking staff in
any of the non-FFE schools sampled. The catering staff are predominantly women.
19. Around 2 staff are designated as storeroom staff (SF3) in the FFE schools in both
regions. In Afar, non-FFE schools report 1.5 as the median of storekeepers, whereas in
Somali, the median figure is 0.5.
Libraries, stores, kitchens and eating areas
20. There is little correlation between FFE provision and libraries, but a clear regional
difference (SF5). In Afar more than one-third of schools report the presence of a library,
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
210
whereas in Somali it is only 10 percent for the FFE schools and 3 percent for the non-FFE
schools.
21. Food stores are associated with the FFE programme status (SF6). In Afar half the
FFE schools have stores; of those that do not, several note the use of a classroom for
storage. Non-FFE schools in both Afar and Somali do not in general have stores. In Somali,
90 percent of the FFE schools report having a store.
22. The distinction for kitchens is even more clear between FFE and non-FFE schools
(SF7). In Afar, 66 percent and in Somali 87 percent of FFE schools have kitchens, whereas
for non-FFE schools, the proportions are much lower, none in Afar and only 7 percent in
Somali.
 Analysis of Questions on School Facilities
For questions SF1-SF4, figures shown are lower quartile (median) upper quartile, units are staff or student numbers, or
rooms (SF4). For other questions, percentage of number of schools in sample are shown. Sample size is shown at the top
of the table.
SI ref
Question Replies
Sample size (schools)
Afar
In FFE
24
Non-FFE
6
Somali
In FFE
30
Non-FFE
30
SF1 How many teachers does Total 6.3 (9.5) 13.8 3.8 (5.0) 21.0 5.8 (12.0) 20.3 4.0 (6.5) 15.8
the school have?
Female 0.3 (2.0) 4.8 0.8 (2.0) 8.8 0.0 (2.0) 4.8 0.0 (1.5) 4.3
Student/Staff 13.9 (19.6) 28.0 12.7 (21.7) 26.1 31.0 (49.8) 92.5 21.0 (32.1) 56.0
Ratio
SF2 How many cooks and Total 1.0 (2.0) 2.0 0.0 (1.5) 2.5 2.0 (3.0) 4.3 0.0 (0.0) 0.0 assistants?
Female 0.3 (2.0) 2.0 0.0 (0.0) 2.5 2.0 (3.0) 4.0 0.0 (0.0) 0.0
SF3 How many storekeepers, Total 1.3 (2.0) 3.8 0.0 (1.5) 4.5 1.8 (2.0) 3.0 0.0 (0.5) 2.0
admin staff and
assistants? Female 0.0 (0.0) 1.0 0.0 (0.5) 1.3 0.0 (0.0) 0.0 0.0 (0.0) 0.0
SF4 How many classrooms 4.0 (6.5) 8.0 2.0 (3.5) 10.3 5.8 (8.0) 9.3 4.0 (5.0) 7.3
are there in the whole
school?
SF5 Is there a library? Yes/ No No 58.3% No 66.7% No 90.0% No 96.7%
Yes 41.7% Yes 33.3% Yes 10.0% Yes 3.3%
SF6 Is there a store for food? Yes/ No No 50.0% No 83.3% Yes 90.0% No 96.7%
Yes 50.0% Yes 16.7% No 10.0% Yes 3.3%
SF7 Is there a kitchen? Yes/ No Yes 66.7% No 100.0% Yes 86.7% No 93.3%
No 33.3% No 13.3% Yes 6.7%
SF8 Is there a covered eating Yes/ No No 83.3% No 100.0% No 66.7% No 96.7%
area or dining room for Yes 16.7% Yes 33.3% Yes 3.3%
the children?
SF9 What type of latrines None Concrete 79.2% None 50.0% Concrete 100.0% Concrete 83.3%
does the school have? Earth Pit None 20.8% Earth Pit 33.3% None 13.3%
Concrete Slab Concrete 16.7% Earth Pit 3.3%
Flush toilet
SF10 Are there separate Yes/ No Yes 75.0% 66.7% Yes 86.7% Yes 60.0%
latrines for boys and 25.0% Yes 16.7% No 13.3% No 30.0%
girls? No 16.7% 10.0%
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
211
SI ref
Question Replies
Afar
In FFE Non-FFE
Somali
In FFE Non-FFE
Sample size (schools) 24 6 30 30
SF11 What is the main water Containers Rotto 29.2% Pipe Water Birka 50.0% Rotto 30.0%
storage? Drum Other 20.8% 50.0% Rotto 23.3% Other 26.7%
Rotto Pipe Water Drum 33.3% Drum 13.3% Birka 20.0%
Birka 20.8% Containers Other 6.7% Pipe Water
PipeWater Drum 16.7% 16.7% Pipe Water 3.3% 13.3%
Other Birka 8.3% Containers 3.3% Drum 10.0%
Containers 4.2%
SF12 What is the water Hand-carry Borehole 33.3% Pipe Water Rain water Rain water
source? Tanker River 25.0% 33.3% 30.0% 26.7%
Rain water Pipe Water Hand-carry Pipe Water Other 20.0%
Stream or 20.8% 16.7% 20.0% Pipe Water
River Stream 12.5% Borehole 16.7% River 16.7% 16.7%
Borehole Hand-carry 8.3% Tanker 16.7% Borehole 13.3% Hand-carry
PipeWater River 16.7% Hand-carry 13.3%
Other 10.0% Borehole 10.0%
Other 6.7% River 10.0%
Tanker 3.3% Tanker 3.3%
SF13 What is the electricity None None 54.2% None 66.7% None 76.7% None 80.0%
supply? Generator Mains 29.2% Mains 33.3% Mains 16.7% Mains 13.3%
Solar Solar 16.7% Generator 6.7% Generator 6.7%
Mains
SF14 Were there any new or Classrooms 17% 17% 10% 13%
improved facilities added
during 2016-17 school Library 4% - - 3%
year? Storeroom 8% - 3% -
Kitchen 17% - 3% -
Eating area 4% - 3% -
Latrines 8% - 10% 3%
Water storage - - 7% -
Water Supply 8% - - -
Electricity - - - -
Other - - 3% 3%
Number of schools receiving 1 or more
improvement
42% 17% 23% 13%
SF16 Who supported these Government 8% - 10% 10%
improvements? Community 13% 17% 3% 3%
Private - - - -
WFP 4% - - -
UNICEF - - - -
SCF 4% - 7% -
Other
NGO/Project
25% - 7% -
23. Regarding covered eating areas for the children (SF8), a small proportion of the FFE
schools have them, whilst the non-FFE schools generally do not. In Afar 17 percent of
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
212
schools had a covered eating area, whilst in Somali the proportion was 33 percent, or 10
schools out of the 30 in the sample. On the other hand, in Afar none of the non-FFE schools
had such a facility, whilst in Somali, only 1 school out of 30 did.
Latrines
24. There was some difference in the quality of latrine facilities (SF9-10) between FFE
and non-FFE schools, particularly in Afar. Concrete slab latrines were the most common
type. No schools had flush toilets. In Afar region, the results of the survey show that
79 percent of FFE schools had concrete slab latrines, while 21 percent report no latrine
facilities at all. Where latrines were present, they were mostly separated between boys and
girls. 19 out of 24 schools in Afar had separate facilities, 1 did not, and 4 schools had no
latrines. In Somali region, all the FFE schools had concrete slab latrines, with 87 percent
having separate facilities for boys and girls. Of the non-FFE schools, 83 percent had
concrete slab latrines, 13 percent had no latrines, and 1 school (3 percent) had an earth pit
latrine. Of those with latrines, 30 percent did not have separate facilities for boys and girls.
Water supply and storage
25. Water supply and storage methods were diverse (SF11-12) and with no consistent
picture distinguishing FFE and non-FFE schools. Piped water supply was available in about
20 percent of schools, river or stream water in some 17 percent, boreholes and hand pumps
in a similar proportion, rain water harvest was reported in Somali province by around
30 percent of schools, but not mentioned in Afar region. The remainder relied on storage
in Rottos (plastic tanks), Birkas (concrete tanks), drums or smaller containers. Delivery by
tanker, animal or hand-carried containers was relied on by some 10-12 percent of schools
across both regions. Strikingly however, in Somali region, 6 out of 30 of the non-FFE
schools (20 percent) reported having no water source or storage at all, whereas all FFE
schools had some water supply and storage.
Electricity
26. In Afar region around two-thirds of schools had no electricity (SF13), whilst in
Somali region the proportion was a little higher, over three-quarters. The situation was
marginally worse for non-FFE schools in both regions. In Afar FFE schools, 29 percent
relied on mains electricity, and 17 percent on solar power. In Somali, no solar power was
reported but rather 2 schools each in the FFE and non-FFE category (7 percent) had
generators, whilst 17 percent of FFE schools and 13 percent of non-FFE schools had mains
power.
Facility improvements during 2009 EC (2016-17 GC)
27. A small proportion of schools reported improvements during the past year (SF14-16).
There was a clear difference between FFE and non-FFE schools in this respect. In Afar,
42 percent of FFE schools (10/24) had one or more improvement, whereas only 1/6
(17 percent) of non-FFE schools reported any addition. In Somali region, the relative
proportions where 23 percent FFE and 13 percent non-FFE schools reporting
improvements. For the non-FFE schools, improvements focussed on new classrooms. For
FFE schools, they were more diverse, including FFE-related areas such as kitchens,
storerooms, eating areas, latrines, water storage and supply.
28. The main support for the improvements was government or community, WFP being
directly cited in only 1 school in Afar region, SCF for 3 schools (1 Afar, 2 Somali, both in the
FFE programme), and a diversity of other NGOs and projects for the remainder (6 schools
in Afar, 2 in Somali, again both FFE schools). There does appear to be a clear sense that, for
whatever reason, the FFE schools are able to access a higher level of external support, and
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
213
have better facilities, than the non-FFE schools, although WFP is not directly reported by
the schools as the originator of this support.
Observations on food receipts and impacts for FFE schools
29. Table 38 below shows the summarised results for those sample schools in the FFE
programme (24 in Afar, 30 in Somali) relating to receipts of WFP rations during the 2009
EC (2016/17 GC) academic year, their perceived impact on attendance and attentiveness
and numbers of girls receiving take-home rations. As with the similar tables above, the left￾hand columns reproduce the questions in the SI, and the columns to the right summarise
the relative statistics by region.
30. In Table 38, question FE2 shows that all schools received their WFP supplies
between September and November 2010 EC (2017 GC).
31. Question FE3 regarding quantities received clearly varies with school enrolment but
shows that in general, salt was not provided, only CSB and oil. The oil was mostly sufficient
according to comments to supply the girls monthly with 2 litres each.
32. Question FE4 shows that for many schools which received supplies, small balances
remained at the time of the survey (start of second semester 2010 EC, February 2017 GC),
but no new supplies had been received at that point.
33. During the 2009 EC academic year (2016/17) in Afar, of the 24 sample schools, 18
had supplies available more than half the year, and 5 for 3-5 months. In Somali however
the situation was different, with 8 of the in-programme schools not receiving supplies at all,
and 2 having food for less than 2 months, 6 less than 5 months, and only 14 of the 30
sample schools having supplies for most of the year.
34. All the reports show strong impacts of the food on attendance and attentiveness. The
notes to the questions repeatedly emphasise the importance of the WFP rations in
encouraging attendance, which drops off when supplies are not available. It is reported as
a particularly important factor for girls’ attendance.
35. Take-home rations (THR) for girls were supplied in numbers consistent with the
enrolment registration for each school, indicating effectively 100 percent take-up of the
rations. It will be seen for Somali that the numbers receiving THR (11,106 girls) is slightly
higher than the recorded enrolment (10,793 girls). The statistical records of enrolment are
however imperfect, with some children not being registered at the start of the year, and
there perhaps being movement between schools.
36. The comments and notes from the interview respondents repeatedly emphasise the
great and critical importance of the WFP rations in encouraging attendance, particularly of
girls due to the THR, and also the decline in attendance that occurs when supplies are not
received.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
214
SI
ref
For Schools in FFE, food supplies, THR and impact on attendance
and attentiveness
Question Afar Somali Note
FE2 When did the school last
receive WFP food
commodities?
Oct 2010 (EC) –
21
Nov 2010 (EC) –
3
Sep 2010 (EC) – 3
Oct 2010 (EC) – 25
Nov 2010 (EC) – 2
For Sep-Nov,
2010 EC is 2017
GC
FE3 What were the quantities
received (Kg)?
CSB 2025 (2825) 3888 5131 (9538) 13425 Lower quartile
(Median) Upper
Quartile, in Kg
Oil 669 (1008) 1453 1951 (3348) 6167
Salt 0 (0) 0 0 (0) 0
FE4 How much is left now? CSB 0 (833) 1244 0 (250) 2513
Oil 30 (170) 462 0 (34) 392
Salt 0 (0) 0 0 (0) 0
FE5 During 2016-17 school
year, what was the total
time the WFP food was
available?
Always 9 12 Number of
schools 6-8 months 9 2
3-5 months 5 6
1-2 months 0 2
Never 0 8
FE6 What was your
impression of the effects
of the WFP School meals,
when available, on the
absenteeism for boys
and girls?
Boys Same 21%
Better 79%
Better 100% ‘Better’ signifies
lower
absenteeism,
Girls Better 100% Better 100% better attendance
FE7 What was your
impression of the effects
of the WFP School meals,
when available, on the
attentiveness for boys
and girls?
Boys Same 4%
Better 96%
Better 100%
Girls Better 100% Better 100%
FE8 How many girls received
the Take-Home Ration
(THR) of oil?
1923 / 2360 11106 / 10793 THR supplied /
Girls Enrolled
School meals support in the non-FFE schools
37. Table 39 below summarises the survey information for schools not in the FFE
programme regarding school meal sources. It will be seen that in Afar, all the non-FFE
schools received ESF supplies, which covered them for an average of 4 months. In Somali,
only 4 out of the 30 non-FFE sample schools received ESF, giving coverage for about the
same period on average.
38. School gardens are a rarity for non-FFE schools, being reported by only 2 schools,
with the produce being sold in one case, and use not recorded in the other. No schools
report support from the community with school meals, none receive HGSF rations, and
none have private sector support. One school in Somali region is adequately supported by
an NGO, though the name is not noted.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
215
39. In general, apart from the ESF, there is little or no provision of food in schools
outside the FFE programme.
 School meals information for non-FFE schools
SI
Ref Question Afar Somali Notes
SM1
Did you receive Emergency
School Feeding during the
2016-17 school year?
100% (6 / 6) 13% (4/30)
% of schools in
sample, and number
/ sample
SM2 If yes, for how many months
did this cover? 3 (4) 7 1 (4) 10
For schools receiving
ESF., min., mean and
Max. months
received.
SM3 Do you have a school garden? 0 / 6 2 / 30
Several FFE schools
have gardens. These
are not counted here,
see separate section.
SM4 If yes, what do you do with it? sell produce 1
not recorded 1
SM4 Does the community supply
food? 0 / 6 0 / 30
SM5
Are you part of the
Government Home-Grown
School Feeding (HGSF)
programme?
0 / 6 0 / 30
SM6
Does the private sector support
the school to supply or buy
food?
0 / 6 0 / 30
SM7
Are there other NGOs or
organizations who help to
supply or buy food?
0 / 6 1 / 30
SM9
Are the supplies your receive
from all these sources
sufficient to feed the children
adequately?
0 / 6 1 / 30
School Gardens
40. Table 39 shows that 2 of the non-FFE schools have gardens. However, overall 16
schools, 7 in Afar and 9 in Somali had gardens. Recorded uses included 3 for teaching, 4
were selling the produce, 4 were used to supplement school meals, and 2 are recorded as
not currently in use due to lack of water. Overall in Afar 29 percent of FFE schools, and
none of the non-FFE schools had gardens; in Somali region, 23 percent of FFE schools and
6 percent of non-FFE schools had gardens.
Capacity Building
41. Table 40 shows the results of the numbers attending workshops or short training
courses in the sample schools. It will be noted that the school numbers in the sample and
their sizes vary: These are unadjusted figures. However, the comparisons show that
capacity building is predominantly focussed on the FFE schools. There are some
exceptions, mainly associated with literacy and language training support predominantly
supported by USAID outside the WFP in Somali region (CB3, CB7). Male/Female ratios for
training are consistent with staff ratios (see Table 37, SF 1-4).
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
216
 Staff attending short courses or workshops in the sample schools
SI ref Category Total
Afar
FFE Non-FFE
Female Total Female Total
Somali
FFE Non-FFE
Female Total Female Sponsoring organisations
CB1 Water, Sanitation and Hygiene training
(WASH) 28 8 2 1 59 23 10 3
SCF (15), WFP (9), REB(9),
UNICEF, IRC, Kelem, etc. (12)
CB2 Nutrition, preparation of school meals,
recipes
34 18 0 0 73 70 0 0
WFP (41), REB(5), SCF(2), Kelem
(1)
CB3 CHILD kits, literacy training 4 1 0 0 15 6 20 7
WFP (3), SCF (2), Kelem etc. (6)
CB4 WFP FFE administration 26 9 0 2 46 9 1 0
WFP (47), REB (2)
CB5 Gender issues, support for girls 8 6 0 0 68 45 6 2
WFP (10), REB/GoE (6), UNICEF,
Kelem etc (5)
CB6 Schools health and nutrition 23 5 1 1 22 12 1 1
WFP + MoH/REB (16), SCF (4),
REB (3)
CB7
Other training, workshops or support (only
short courses, not formal higher education) 18 8 1 1 96 23 116 23
USAID (RTI, REAL TA) + REB (52),
Others (3)
42. The WFP together with the government regional education bureaus (REB) have a
major role in the areas capacity building for nutrition, preparation of school meals,
administration of the FFE, gender issues and support for girls, and school health and
nutrition, and outside the role of USAID in literacy and language training, appears to be the
main sponsor of capacity building workshops and short training courses. This naturally
applies only to those schools in the FFE, which largely accounts for the differential in
capacity building activities between FFE and non-FFE schools.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
217
Annex Z Assessment of progress against SABER framework
As part of its approach the evaluation team reviewed – on the basis of field work,
documentary evidence and interviews – whether progress has been made against key
indicators in the SABER framework. This assessment was not carried out using the
established SABER methodology but in the absence of an update against the 2015 SABER
status it was considered important to have an updated view, using a different (but
independent) methodology.
 Policy Goals and Status
Policy Goal Status 2015 Status *2018
1. Policy Framework
• There is recognition of the importance of SF
policies and it is already included in several
documents from various sectors. The
government is taking steps towards
developing a specific strategy for SF that may
inform a future SF policy.
• There is commitment at the highest level and
it may lead to endorsement of policy in the
future. WFP has played a major role to give
policy direction, in building the policy
framework, and in ensuring consultation
with the key management bodies.
Emerging
●●
Recognition of the
importance of SF has
increased further since
2015 with strong
support from senior
levels of the
government.
2. Financial Capacity
• At the national level, no budget
allocation/budget lines for SF. However, in
HGSF and ESFP budgets have been
allocated.
Latent
●
There has been
progress in this area as
some regional
governments are
continuing to allocate
regular funds to SF
and the GoE through
its federal budget has
made funds available
three years in a row
for emergency school
feeding.
3. Institutional Capacity and Coordination
• A national steering committee is not yet in
place. There is also no clear management and
accountability structure at the national,
regional or school levels.
• There is a technical working group on
education comprising donors and
development partners and an emerging
education cluster led by the GoE.
Latent
●
TA provided by WFP
has been embedded at
the federal and
regional levels and has
improved
coordination around
SF.
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
218
Policy Goal Status 2015 Status *2018
4. Design and Implementation
• There are no national M&E plans or specific
standards for SF. However, in two regions
piloting HGSF, WFP standards are adopted.
• Both HGSF/ESF have adopted the WFP
model and are considered to be of a standard
sustainable design, though scaling up may
prove to be a challenge.
Latent
●
There has been further
progress in this area
with the GoE adopting
WFP standards for the
roll out of emergency
SF.
5. Community Roles-Reaching Beyond Schools
• There is no national document that outlines
SF committees at the school level except the
guidelines / manuals supported by WFP.
Community involvement is not yet effective
in SF.
• CHILD tools/manual have been developed
and used, though not very effectively. But
communities in some regions/schools
provide NFI such as water, firewood and
cooks’ salaries.
Latent
●
While CHILD
continues to be rolled
out there have been
some challenges
related to the very
vulnerable nature of
communities which
have impeded their
participation in school
development
activities.
Source: SABER Ethiopia Report (2015) * and findings by the evaluation team (based on interviews)
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
219
Bibliography
1. In this Annex we provide bibliographical references for documents cited in the IR, plus
entries for other key documents from the ET's electronic library. We also list under
"additional documents sought", other documents we have heard of but have not yet
managed to obtain. In most cases we are actively seeking the continued assistance of the
EM and CO in filling the important gaps in our collection.
2. "Location" in the listing below refers to folder and document numbers in the evaluation
team's electronic library of documents that have been directly referenced in the IR.
3. The bibliography is a work-in-progress that will be continually updated as the evaluation
proceeds.
African Union
Commission et al,
2014
African Union
Commission et al,
2014
The Cost of Hunger in Africa: Social and Economic Impact of Child
Undernutrition in Egypt, Ethiopia, Swaziland and Uganda. African Union
Commission, NEPAD Planning and Coordinating Agency, UN Economic
Commission for Africa, and UN World Food Programme. Addis Ababa:
UNECA, 2014.
The Cost of Hunger in Ethiopia: Implications for the Growth and
Transformation of Ethiopia. African Union Commission, NEPAD Planning
and Coordinating Agency, UN Economic Commission for Africa, and UN
World Food Programme. Addis Ababa: UNECA, 2014.
14.10
14.11
Assefa, 2015 The impact of school feeding programme on students’ academic
performance: the case of selected elementary schools in Debre Libanos
Wereda, Oromia Region. A thesis submitted to the School of Psychology
Addis Ababa University. Ermias Assefa: Addis Ababa, June 2015.
16.7
Atem Consultancy
Service, 2012
Central Statistical
Agency & The
DHS Program,
2016
Discussion paper,
2013
Discussion paper,
2014
Ethiopia
Education Cluster,
2017
Study on Situation of Out of School Children (OOSC) in Ethiopia. Atem
Consultancy Service for the Ministry of Education and UNICEF Ethiopia
Country Office: Addis Ababa, July 2012.
Ethiopia – Demographic and Health Survey 2016. Central Statistical Agency
& The DHS Program, ICF: Addis Ababa, Ethiopia & Rockville, Maryland,
USA: July 2017.
Discussion notes on McGovern-Dole Capacity Building Implementation in
Afar region. 2013.
Discussion notes on the Regional Level Discussion on Field Visit Report of
McGovern-Dole Capacity Building Intervention. 17 April 2014.
Ethiopia Somali Region Education in Emergency Operational Response Plan
– 2017/18. Ethiopia Education Cluster: 2017.
16.6
2.6
7.1
7.3
8.1
FAO, 2017 Chronology of hotspot classification in Ethiopia – February 2015 – June
2017. Food and Agriculture Organization of the United Nations: 2017.
8.6
Short ref Full ref Location
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
220
Government of
Ethiopia &
Humanitarian
Partners, 2017
Government of
Ethiopia & UN,
2013
Government of
Ethiopia &
UNICEF, 2012
Government of
Ethiopia, 2010a
Government of
Ethiopia, 2010b
Government of
Ethiopia, 2012
Government of
Ethiopia, 2013a
Government of
Ethiopia, 2013b
Government of
Ethiopia, 2013c
Government of
Ethiopia, 2013d
Government of
Ethiopia, 2014
Government of
Ethiopia, 2015a
Government of
Ethiopia, 2015b
Government of
Ethiopia, 2015c
Ethiopia Humanitarian Requirements Document Mid-Year Review.
Government of Ethiopia & Humanitarian Partners: 2017.
Program Implementation Manual for United Nations Agencies assisted
programs in Ethiopia. Federal Democratic Republic of Ethiopia & United
Nations Agencies: Addis Ababa, December 2013.
Study on Situation of Out of School Children (OOSC) in Ethiopia. Ministry of
Education and UNICEF Ethiopia Country Office: Addis Ababa: July 1012.
Education Sector Development Program IV (ESDP IV). 2010/2011 –
2014/2015. 2003 EC – 2007 EC. Program Action Plan. Federal Ministry of
Education. Federal Democratic Republic of Ethiopia: Addis Ababa, 2010.
Growth and Transformation Plan 2010/11 – 2014/15: Volume II: Policy
Matrix. Ministry of Finance and Economic Development, Federal
Democratic Republic of Ethiopia: Addis Ababa: November 2010.
National School Health and Nutrition Strategy. Ministry of Education.
Federal Democratic Republic of Ethiopia: Addis Ababa: October 2012.
Growth and Transformation Plan. Annual Progress Report for F.Y. 2011/12.
Ministry of Finance and Economic Development. Ministry of Finance and
Economic Development. Federal Democratic Republic of Ethiopia: Addis
Ababa, March 2013.
National Nutrition Programme. June 2013 – June 2015. Government of the
Federal Democratic Republic of Ethiopia: Addis Ababa, 2013.
Social Protection Policy. Federal Democratic Republic of Ethiopia: Addis
Ababa, October 2013.
Education Statistics Annual Abstract. Ministry of Education. EMIS,
Planning and Resource Mobilization. Government of the Federal Democratic
Republic of Ethiopia: Addis Ababa, November 2013.
Growth and Transformation Plan. Annual Progress Report for F.Y. 2012/13.
Ministry of Finance and Economic Development. Federal Democratic
Republic of Ethiopia: Addis Ababa, February 2014.
Education Sector Development Programme V (ESDP V). 2008 – 2012 E.C.
2015/16 – 2019//20 G.C. Programme Action Plan. Federal Ministry of
Education. Federal Democratic Republic of Ethiopia: Addis Ababa, 2015.
Education Statistics Annual Abstract. 2006 E.C (2013/2014). Ministry of
Education. EMIS and ICT Directorate. Government of the Federal
Democratic Republic of Ethiopia: Addis Ababa, June 2015.
Plan to address the effects of El Niño caused drought on students’ school
attendance (Amharic). Ministry of Education, Government of the Federal
Democratic Republic of Ethiopia: Addis Ababa, September 2015.
8.3
1.12
16.6
1.1
1.7
1.10
1.3
1.8
1.13
2.1
1.4
1.2
2.2
8.4
Short ref Full ref Location
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
221
Government of
Ethiopia, 2016a
Government of
Ethiopia, 2016b
Government of
Ethiopia, 2016c
Government of
Ethiopia, 2016d
Government of
Ethiopia, 2016e
Government of
Ethiopia, 2016f
National Social Protection Strategy of Ethiopia. Federal Democratic
Republic of Ethiopia: Addis Ababa, January 2016.
Growth and Transformation Plan II (GTP II) 2015/16-2019/20). Volume I:
Main Text. National Planning Commission, Federal Democratic Republic of
Ethiopia: Addis Ababa: May 2016.
National Nutrition Program. 2016 - 2020. Government of the Federal
Democratic Republic of Ethiopia: Addis Ababa, 2016.
Education Statistics Annual Abstract. 2007 E.C (2014/2015). Ministry of
Education. EMIS and ICT Directorate. Government of the Federal
Democratic Republic of Ethiopia: Addis Ababa, June/July 2016.
Ethiopian National Micronutrient Survey Report. Ethiopian Public Health
Institute, Ministry of Health, Government of the Federal Democratic
Republic of Ethiopia: Addis Ababa, September 2016.
General Education Statistical Abstract. 2008 E.C (2015/16). Ministry of
Education. EMIS and ICT Directorate. Government of the Federal
Democratic Republic of Ethiopia: Addis Ababa, December 2016.
1.11
1.5
1.14
2.3
14.9
2.4
Government of
Ethiopia, 2017a
Government of
Ethiopia, 2017b
Government of
Ethiopia, 2017c
Government of
Ethiopia, n.d.-a
Government of
Ethiopia, n.d.-b
Second Growth and Transformation Plan (GTP II): Annual Progress report
for 2015/16. National Planning Commission, Federal Democratic Republic
of Ethiopia: Addis Ababa: April 2017.
National School Feeding Strategy. 3rd/final draft. Ministry of Education.
Federal Democratic Republic of Ethiopia: Addis Ababa: 30 September 2017.
Home Grown School Feeding National Consultative Workshop. Addis
Ababa: 2-3 May 2017 Proceeding Report. Ministry of Education: Addis
Ababa: 12 June 2017.
Primary School Age population. Government of the Federal Democratic
Republic of Ethiopia: Addis Ababa, no date.
Emergency School Feeding Performance Assessment – Part 1 (Amharic).
Ministry of Education, Government of the Federal Democratic Republic of
Ethiopia: Addis Ababa, no date.
1.6
1.9
16.2
2.5
8.5
Horton, 2006 The Economics of Food Fortification. Sue Horton. Journal of Nutrition, April
2006, vol. 136 no. 4, 1068-1071.
IFPRI, 2017 Food Security Portal, facilitated by IFPRI: Ethiopia section;
http://www.foodsecurityportal.org/ethiopia
Poppe et al, 2017 School Meals and Educational Outcomes in Rural Ethiopia. Poppe, Robert,
Frölich, Markus & Getinet Haile. The Journal of Development Studies, DOI:
10.1080/00220388.2017.1311405: 12 May 2017.
SABER, 2015 SABER Country Report: Ethiopia School Feeding. Systems Approach for
Better Education Results (SABER): 2015.
Stern et al, 2012 Broadening the Range of Designs and Methods for Impact Evaluation, DFID
Working Paper 38, Elliot Stern (Team Leader), Nicoletta Stame, John Mayne,
Kim Forss, Rick Davies, Barbara Befani, April 2012.
14.12
16.3
Short ref Full ref Location
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
222
TANGO, 2016 Operation Evaluation. Ethiopia Protracted Relief and Recovery Operation
200700 (2015-2018) Food Assistance for Eritrean, South Sudanese,
Sudanese and Somali Refugees: An evaluation of WFP’s current operation
and transition period. Evaluation Report. TANGO: June 2016.
21.1
The Oakland
Institute, 2013
Development Aid to Ethiopia. Overlooking Violence, Marginalization, and
Political Repression. The Oakland Institute: Oakland, CA, USA: 2013.
22.1
UN Women, 2014 Preliminary Gender Profile of Ethiopia. UN Women: Addis Ababa:
November 2014.
11.3
UNDAF, 2012-
2015
UNDAF, 2016-
2020
Ethiopia United Nations Development Assistance Framework 2012-2015.
United Nations Country Team: Addis Ababa, March 2011.
Ethiopia United Nations Development Assistance Framework 2016-2020
(Draft). United Nations Country Team: Addis Ababa, November 2015.
17.1
17.3
UNEG, 2008 UNEG Code of Conduct for Evaluation in the UN System. United Nations
Evaluation Group, March 2008.
UNEG, 2016 Norms and Standards for Evaluation. United Nations Evaluation Group,
June 2016
Universalia, 2014 UNDAF Ethiopia Mid-Term Review (MTR). Final Report Volume I.
Universalia: October 2014.
USDA, 2008a Agreement #FFE-663-2008/006-00 Ethiopia. USDA: Washington, DC, 10
June 2008.
USDA, 2008b Agreement #FFE-663-2009/006-00 Ethiopia. USDA: Washington, DC, 3
December 2008.
USDA, 2010 Agreement #FFE-663-2010/006-00 Ethiopia. USDA: Washington, DC, 18
June 2010.
USDA, 2011 Policy and Operational Guidance for Use of Results Frameworks and
Indicators. Food Assistance Division, Office of Capacity Building and
Development. United States Department of Agriculture: 15 July 2011.
USDA, 2012a 2013 Operational Plan for Agreement #FFE-663-2013/026-00 Ethiopia.
USDA: Washington, DC: 9 November 2012.
USDA, 2012b McGovern-Dole Commitment Letter for Agreement #FFE-663-2013/026-00
Ethiopia. USDA: Washington, DC: 9 November 2012.
USDA, 2013a Monitoring and Evaluation Policy. Food Assistance Division, Office of
Capacity Building and Development. United States Department of
Agriculture: 6 May 2013.
USDA, 2013b Second Year Operational Plan for Agreement #FFE-663-2013/026-00
Ethiopia. USDA: Washington, DC: 6 September 2013.
USDA, 2014 Food for Progress and McGovern-Dole Indicators and Definitions. Food
Assistance Division, Office of Capacity Building and Development. United
States Department of Agriculture: 15 July 2014.
17.2b
6.11
6.12
6.13
5.3
6.5
6.3
5.2
6.14
5.1
Short ref Full ref Location
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
223
Short ref Full ref Location
USDA, 2015a Modification I to Commitment Letter for Agreement #FFE-663-2013/026-00
Ethiopia. USDA: Washington, DC: 17 March 2015.
6.4
USDA, 2015b Third Year Operational Plan for Agreement #FFE-663-2013/026-00 6.6
Ethiopia. USDA: Washington, DC: 17 March 2015.
USDA, 2016a USDA’s School Feeding Program in Ethiopia is Making a Difference. Global
Agricultural Information Network (GAIN), United States Department of
Agriculture: Washington, DC, 5 February 2016.
21.2
USDA, 2016b Modification II to Commitment Letter for Agreement #FFE-663-2013/026-
00 Ethiopia. USDA: Washington, DC: 25 July 2016.
6.16
USDA, 2016c Fourth Year Operational Plan for Agreement #FFE-663-2013/026-00
Ethiopia. USDA: Washington, DC: 25 July 2016.
6.15
USDA, n.d.-a MGD RF 1: Literacy Results Framework and MGD RF 2: Health and Dietary
Practices Results Framework. USDA: n.d.
5.5
USDA, n.d.-b McGovern-Dole Results Framework. USDA, Washington, DC: n.d. 6.7
WFP, 2006 Country Programme – Ethiopia 10430.0 (2007-2011). World Food
Programme: Rome, 17 August 2006.
3.1
WFP, 2008a WFP Strategic Plan 2008–2013, World Food Programme 2008 4.6-8
WFP, 2008b Standard Project Report 2007 (1 January 2007 – 31 December 2011).
Ethiopia Country Programme (2012-2016) 10430.0. World Food
Programme (WFP): Rome, 2008.
9.43
WFP, 2008bc Ethiopia Resourcing Update. World Food Programme: 12 March 2008. 3.4
WFP, 2009a Standard Project Report 2009 (1 January 2007 – 31 December 2011). 9.44
Ethiopia Country Programme (2012-2016) 10430.0. World Food
Programme (WFP): Rome, 2009.
WFP, 2009b WFP Gender Policy Corporate Action Plan. Rome: WFP, 02 October 2009 4.3-7
WFP, 2009c WFP Gender Policy: Promoting Gender Equality and the Empowerment of
Women in Addressing Food and Nutrition Challenges. Rome: WFP, 10
February 2009.
4.3-1
WFP, 2010a Standard Project Report 2010 (1 January 2007 – 31 December 2011). 9.45
Ethiopia Country Programme (2012-2016) 10430.0. World Food
Programme (WFP): Rome, 2010.
WFP, 2010b School feeding and nutrition. Revised version 2010. World Food
Programme: September 2010.
4.3
WFP, 2011a Girls Initiative Impact Assessment. World Food Programme Ethiopia: Addis
Ababa: January 2011.
16.5
WFP, 2011b Standard Project Report 2011 (1 January 2007 – 31 December 2011).
Ethiopia Country Programme (2012-2016) 10430.0. World Food
Programme (WFP): Rome, 2011.
9.46
WFP, 2011c McGovern-Dole Online Application. World Food Programme: 2011. 6.1
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
224
Short ref Full ref Location
WFP, 2011d Country Programme – Ethiopia 200253 (2012-2015). World Food
Programme: Rome, 6 September 2011.
3.2
WFP, 2011e MGD project design: Strategic analysis. WFP Ethiopia, 2011 6.20
WFP, 2012a WFP Nutrition Policy. Rome: WFP, 13-15 February 2012. 4.4-1
WFP, 2012b Capacity Development Kit, WFP, May 2012. 4.5.1-1
WFP, 2012c Capacity Development Proposal for the United States Department of
Agriculture (USDA) for Capacity Development to Afar and Somali Regional
States Education Bureaus in support of School Feeding Program. WFP
Ethiopia: 18 June 2012.
6.8
WFP, 2013-2016 McGovern Dole Semi-Annual Reports, WFP Ethiopia, April 2014 – October
2017.
9.20 –
9.34
WFP, 2013a Baseline Survey for MGD supported School Feeding program in Afar and
Somali. WFP Ethiopia January 2013.
9.14
WFP, 2013b WFP Private-Sector Partnerships and Fundraising Strategy (2013-2017).
Rome: WFP, 3-6 June 2013.
4.6-3
WFP, 2013c Update of WFP’s Safety Nets Policy. The role of Food Assistance in Social 4.6-2
Protection. Rome: WFP, 4-8 June 2012.
WFP, 2013d WFP Strategic Plan (2014-2017). Rome: WFP, 3-6 June 2013. 4.7-3
WFP, 2013e Technical Note on Efficiency Analysis. Robrecht Renard and Stephen Lister,
WFP – OEV, 14 June 2013.
4.1.2-6
WFP, 2013f Revised School Feeding Policy. Rome: WFP, 4-7 November 2013. 4.2-1
WFP, 2014a WFP Strategic Results Framework (2014-2017) WFP, 1 January 2014. 4.6-5
WFP, 2014b Evaluation of WFP’s 2009 Gender Policy. WFP, January 2014. 4.3-5
WFP, 2014c Standard Project Report 2013 (1 January 2013 – 30 June 2016). Ethiopia
Country Programme (2012-2016) 200253. World Food Programme (WFP):
Rome, 2014.
9.49
WFP, 2014d WFP Ethiopia McGovern-Dole Evaluation Plan. WFP: 2014. 6.9
WFP, 2014e Guidance for the WFP Gender Marker. WFP Gender Office, 25 February
2014.
4.3-4
WFP, 2014f WFP Corporate Partnership Strategy (2014-2017). Rome: WFP, 3-6 June
2014.
4.6-6
WFP, 2014g Guidelines for Technical Assistance and Capacity Strengthening to End
Hunger. Rome: WFP, June 2014.
4.5.1-4
WFP, 2014h Innovations from the field: Gender mainstreaming from the ground up.
WFP, 11 July 2014.
4.3-3
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
225
Short ref Full ref Location
WFP, 2014i Capacity Gaps and Needs Assessment in Support of Projects to Strengthen
National Capacity to End Hunger. Complementary Guidelines: Series #1.
WFP, October 2014.
4.2.1-2
WFP, 2014j National Capacity Index (NCI) – Measuring Change in Capacity for Hunger
Governance in Support of Projects to Strengthen National Capacity to End
Hunger. Complementary Guidelines: Series #2. WFP, October 2014.
4.2.1-3
WFP, 2015a Training Manual for food preparation, food safety and hygiene. WFP
Ethiopia: Addis Ababa, January 2015.
14.7
WFP, 2015b Standard Project Report 2014 (1 January 2012 – 30 June 2016). Ethiopia 9.50
Country Programme (2012-2016) 200253. World Food Programme (WFP):
Rome, 2015.
WFP, 2015c Gender Policy (2015-2020). Rome: WFP, 25-28 May 2015. 4.3-2
WFP, 2016a Draft Theory of Change School Feeding. [not published] WFP, 11 April 2016 4.2-6
WFP, 2016b Technical Note, Evaluation Reference Group, WFP Office of Evaluation,
Decentralized Evaluation Quality Assurance System (DEQAS), March 2016
4.1.2-25
WFP, 2016c Technical Note, Evaluation Committee, WFP Office of Evaluation, 4.1.2-26
Decentralized Evaluation Quality Assurance System (DEQAS), April 2016
WFP, 2016d Technical Note: Integrating Gender in Decentralized Evaluations. WFP
Decentralized Evaluation Quality Assurance System (DEQAS), August 2016
4.1.2-28
WFP, 2016e Technical Note, Independence and Impartiality, WFP Office of Evaluation,
Decentralized Evaluation Quality Assurance System (DEQAS), March 2016
1.1.2-16
WFP, 2016f Decentralized Evaluation Quality Assurance System (DEQAS) Process Guide, 4.1-2
WFP Office of Evaluation, August 2016
WFP, 2016g Technical Note – Impact Evaluation, WFP Office of Evaluation, August 2016. 1.1.12–21
WFP, 2016h McGovern-Dole Capacity Development – Status and Gaps. Presentation for
Bureau of Education Management. WFP Area Office: October 2016.
7.2
WFP, 2016i Country Gender Action Plan (CGAP) 2017-2020. WFP Ethiopia: 2016. 11.1
WFP, 2016j Standard Project Report 2015 (1 January 2012 – 30 June 2017). Ethiopia
Country Programme (2012-2016) 200253. World Food Programme (WFP):
9.51
Rome, 2016.
WFP, 2016k Integrated Road Map: Positioning WFP for a changing world. World Food
Programme, 8 January 2016.
WFP, 2016k McGovern-Dole proposal FY17. WFP Bangladesh: 28 December 2017 2.9-18
WFP, 2017a WFP Ethiopia Country Brief. World Food Programme Ethiopia: Addis
Ababa, June 2017.
3.3
WFP, 2017b Terms of Reference – Final Evaluation of WFPs USDA McGovern-Dole
International Food for Education and Child Nutrition Programme’s Support
in Afar and Somali regions in Ethiopia 2013 – December 2017. WFP: 2017
19.1
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
226
WFP, 2017c Standard Project Report 2016 (1 January 2012 – 30 June 2017). Ethiopia
Country Programme (2012-2017) 200253. World Food Programme (WFP):
Rome, 2017
WFP, 2017d Ethiopia School Feeding. Ethiopia Investment Case. Cost-Benefit Analysis
Report. World Food Programme: Addis Ababa, June 2017.
WFP, 2017e Field Mission Report: Verification of reported diversion of FFE commodities
to drought-affected schools (Somali Region). Food diversion fact verification
report – Food for Education. WFP Programme Unit Addis Ababa and Jijiga
Area Office, March 2017
WFP, n.d.-a School Meals Program in Ethiopia - Presentation. World Food Programme
Ethiopia: n.d.
WFP, n.d.-b Strategic Analysis of School Feeding Programmes. World Food Programme:
n.d.
WFP, n.d.-c WFP Contribution in Emergency School Feeding Program Operation. WFP
Ethiopia: Addis Ababa, n.d.
WFP, n.d.-d Baseline Study Report on Gender Mainstreaming and Women’s
Empowerment in Ethiopia. WFP Ethiopia: n.d.
WFP, n.d.-e School Meals Program food preparation, food safety and hygiene guideline
for cooks. WFP Ethiopia: Addis Ababa, n.d.
WFP, n.d.-f School Health and Nutrition: A course for personnel involved in program
designing and implementation. Facilitator Guide. First Draft. WFP
Ethiopia: Addis Ababa, n.d.
9.52
16.4
9.41
3.5
6.2
8.2
11.2
14.2
14.8
World Bank
Group, 2015
Ethiopia Poverty Assessment. World Bank Group: January 2015. 22.2
World Economic
Forum, 2017
The Global Gender Gap Report 2017. World Economic Forum: Geneva,
Switzerland: 2017.
11.4
Short ref Full ref Location
227
List of Acronyms
ABE Alternative Basic Education
AU African Union
AICS L’Agenzia Italiana per la Cooperazione allo Sviluppo (Italian agency for cooperation and
development)
BoA Bureau of Agriculture
BoE Bureau of Education
BoH Bureau of Health
CHILD Children in Local Development
CO Country Office
CP Country Programme
CSB corn soya blend
CSB+ CSB with the addition of dried skimmed milk
CSPro Census and Survey Processing System
CQ Child questionnaire
DEQAS Decentralized Evaluation Quality Assurance System
EB Executive Board
EC Evaluation Committee / European Commission
EFA Education for All
EM Evaluation Manager
EMIS Educational Management Information System
EP Evaluation Plan
EQ Evaluation Question
EQAS Evaluation Quality Assurance System
ER Evaluation Report
ERG Evaluation Reference Group
ESDP Education Sector Development Programme
ESF Emergency School Feeding
ESFP Emergency School Feeding Programme
ET Evaluation team
FAO Food and Agriculture Organization
FCS Food Consumption Score
FFE Food For Education
FGD Focus group discussion
FGM/C Female Genital Mutilation/ Cutting
GDP Gross Domestic Product
GEEW Gender Equality and Women’s Empowerment
GEQIP General Education Quality Improvement Programme
GIP Girls Initiative Programme
GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit
GLM General Linear Modelling
GoE Government of Ethiopia
GPI Gender Parity Index
GTP Growth and Transformation Plan
HDI Human Development Index
HH Household
HGSF Home Grown School Feeding
HQ Headquarters
IP In programme
HQ headquarters
IEC Internal Evaluation Committee
IR Inception Report
228
McGovern-Dole school feeding in Afar and Somali Regions – final evaluation: Evaluation Report
IRC International Rescue Committee
KII Key Informant Interviews
KQ Key Question
LTA Long-Term Agreement
M&E Monitoring and Evaluation
MGD McGovern-Dole
MoE Ministry of Education
MoFED Ministry of Finance and Economic Development
MT Metric Tons
NER Net Enrolment Rate
NFI Non-food item
NGO Non-Governmental Organization
NSFP National School Feeding Programme
OEV Office of Evaluation
OP Out of FFE programme
PCI Project Concern International
PRF Project Results Framework
PRRO Protracted Relief and Recovery Operation
PSNP Productive Safety Net Programme
PTA Parent Teacher Association
QS Quality Support
RB Regional Bureau
REACH Renewed Efforts Against Child Hunger
SABER Systems Approach for Better Education Results
SCI Save the Children International
SF School Feeding
SFP School Feeding Programme
SHN School health and nutrition
SI Survey Instrument
SNNP Southern Nations Nationalities and People
SPR Standard Project Report
SPSS Statistical Package for Social Sciences
SQ Schools questionnaire
TA Technical Assistant / Assistance
THR Take-Home Ration
ToC Theory of Change
TOR Terms of Reference
ToT Training of trainers
UNCT United Nations Country Team
UNDAF United Nations Development Assistance Framework
UNDSS UN Department of Safety & Security
UNEG United Nations Evaluation Group
UNFPA United Nations Population Fund
UNICEF United Nations Children's Fund
UN-OCHA UN Office for the Coordination of Humanitarian Affairs
USD United States dollar
USDA United States Department of Agriculture
WASH Water, Sanitation and Hygiene
WFP World Food Programme
WFP Ethiopia
http://www1.wfp.org/countries/ethiopia
[Ethiopia, June 2018]