World Food Programme McGovern-Dole International Food for Education and Child Nutrition Program (FFE 699-2013/036-00-B) in Liberia. Covering September 2013-September 2016 Evaluation Report June 2017 WFP Liberia Evaluation Team: Kate Godden (Team Leader), Margaret Ferris Morris, Dexter D. Merchant, Sr., Nathan Horst Evaluation Manager Evans Binyason Prepared by Kate Godden, Team Leader Margaret Ferris Morris, Senior Evaluator Sophie, Dunn, Senior Evaluator Dexter Marchant, M&E Expert Nathan Horst, Data Specialist Decentralized Evaluation Acknowledgements We would like to give special thanks to the World Food Program and the Ministry of Education staff for their time, thoughtful input, and guidance to this review process. Without their help in sharing documents and providing insight into the history and evolution of the School Feeding Program, we would have missed valuable context. We would also like to thank all the in-country stakeholders who took the time to meet with the KonTerra team to share their thoughts and reflections about the School Feeding Programme. We would also like to give thanks to the Q & A Inc. services and their dedicated enumerators, who with their help made field data collection possible. Finally, thanks go to the schools, their staff and their children. Disclaimer The opinions expressed in this report are those of the Evaluation Team, and do not necessarily reflect those of the World Food Programme. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP of the opinions expressed. The designation employed and the presentation of material in maps do not imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. Table of Contents 1 Introduction ...........................................................................................1 1.1 Overview of the FFE WFP school feeding programme..................................................... 2 1.2 Context.............................................................................................................................. 4 1.3 Evaluation methodology and limitations ......................................................................... 6 2 Evaluation Findings ............................................................................... 9 2.1 Evaluation Question 1: How relevant/appropriate is the operation? .............................. 9 2.1.1 Appropriateness to needs ............................................................................................. 9 2.1.2 Coherence and alignment with WFP policies and with national policies on education, nutrition and child welfare ...................................................................................................... 11 2.1.3 Coherence and alignment with UNDAF and other humanitarian and development actors ......................................................................................................................................13 2.2 Evaluation Question 2: How effective is the operation? .................................................14 2.2.1 Capacity building results .............................................................................................14 2.2.2 SF outputs & other support to schools ........................................................................17 2.2.3 Promotion of basic education and develop human capital..........................................19 2.2.4 Community engagement results................................................................................. 22 2.2.5 Analysis of internal and external factors affecting the results ................................... 23 2.3 Evaluation Question 3: How efficient has the operation been? ..................................... 27 2.4 Evaluation Question 4: What are the impacts of the programme? ................................30 2.5 Evaluation Question 5: How sustainable is the operation?.............................................31 3 Conclusions and recommendations ..................................................... 36 3.1 Lessons learned and good practices ............................................................................... 39 3.2 Recommendations ..........................................................................................................40 iv List of Annexes [separate volume] Annex 1: Evaluation Terms of Reference ......................................................................... 4 Annex 2: Evaluation matrix ..................................20Error! Bookmark not defined. Annex 3: Evaluation mission timeline .................... Error! Bookmark not defined.6 Annex 4: Evaluation methodology in field modificationsError! Bookmark not defined.8 Annex 5: List of evaluation key informants ............ Error! Bookmark not defined.3 Annex 6: WFP McGovern Dole data collection instrumentsError! Bookmark not defined.5 Annex 7: Evaluation school sampling criteria ........ Error! Bookmark not defined.3 Annex 8: MGD indictors not measured throughout the programme by WFP or MoE ........................................................................... Error! Bookmark not defined.6 Annex 9: Changes in girl’s enrolment in primary school from 2005 to 2013 ....... Error! Bookmark not defined.8 Annex 10: Issues regarding the CO’s method of beneficiary countingError! Bookmark not defined. Annex 11: Additional data tables .............................. Error! Bookmark not defined.1 Annex 12: Liberia FY2013-16 Award LOP Performance ResultsError! Bookmark not defined.2 Annex 13: McGovern-Dole Outputs ........................ Error! Bookmark not defined.2 Annex 14: School site observations ......................... Error! Bookmark not defined.5 Annex 15: Documents reviewed .............................. Error! Bookmark not defined.7 Annex 16: Strategic Results Frameworks for McGovern Dole and WFP .............. Error! Bookmark not defined.1 Annex 17: Tables of Effectiveness Results ................ Error! Bookmark not defined. List of Acronyms v List of Tables [all contained in Annex 17] Table 1: WFP capacity building support to the National GovernmentError! Bookmark not defined. Table 2: WFP capacity building of school administratorsError! Bookmark not defined. Table 3: WFP capacity building of teachers ................... Error! Bookmark not defined. Table 4: Number of fuel efficient stoves provided to schools.............................................74 Table 5: WFP capacity building of school storekeepers and cooks75Error! Bookmark not defined. Table 6: Number of FFE beneficiaries (2013-2016) ......Error! Bookmark not defined. Table 7: Provision of in-school meals ............................ Error! Bookmark not defined. Table 8: Provision of GTHR .......................................76Error! Bookmark not defined. Table 9: Number of school gardens established ........76Error! Bookmark not defined. Table 10: Student enrolment indicators .................... 77Error! Bookmark not defined. Table 11: Enrolment statistics during 2015/16 and 2016/17 school years .............77Error! Bookmark not defined. Table 12: Student attendance – Grade 2, (2016/2017 school year)78Error! Bookmark not defined. Table 13: Gender parity indicators ............................. 78Error! Bookmark not defined. Table 14: WFP training for students on nutrition and health and hygiene practices ....7922 Table 15: Parent Teacher Associations ...................... 79Error! Bookmark not defined. Table 16: Food Management Committees ...................Error! Bookmark not defined.9 Table 17: Community awareness-raising events ........79Error! Bookmark not defined. Table 18: Value of SF support costs provided by WFP and MoE (2013-2016) ..... 80Error! Bookmark not defined. Table 19: Identified sources of food loss ................... 80Error! Bookmark not defined. List of Figures [all contained in Annex 17] Figure 1: Planned vs. actual achievement - in-school meal beneficiaries (2013/14 – 2016/17 school years) .......................................... 74Error! Bookmark not defined. Figure 2: Planned vs. actual achievement - provision of food commodities (MT) 75Error! Bookmark not defined. Figure 3: Primary School Enrolment Rates (2015 - 2016)76Error! Bookmark not defined. vi Figure 4: Change in enrolment between school years 2015/16 and 2016/17 school years .............................................................................. 77Error! Bookmark not defined. vii Maps Map 1: Schools visited by the ET and WFP operational area Source: WFP Liberia viii Map 2: Map of Liberia showing county chronic malnutrition rates (2013) Source: Ministry of Agriculture and WFP (2013) Liberia Comprehensive Food Security and Nutrition Survey. June 2013. Republic of Liberia. Map 3: Map of Liberia showing county poverty rates (2013) Source: Ministry of Agriculture and WFP (2013) Liberia Comprehensive Food Security and Nutrition Survey. June 2013. Republic of Liberia. ix Timeline of key events x Executive Summary S1. This report covers the final evaluation of the World Food Programme’s (WFP) implementation of McGovern Dole (MGD) funded Food for Education and Child Nutrition (FFE) in Liberia. The purpose of this evaluation is to assess the performance and results achieved, between September 2013 and September 2016, by the FFE activities implemented by the WFP Country Office (CO) under Component 1 of its Country Programme (CP) DEV 200395: School Feeding (SF). FFE was implemented by WFP in partnership with the Ministry of Education (MoE) with a $20 million grant. A no-cost budget revision was granted until July 2017. Implementation was significantly affected by the outbreak of Ebola Viral Disease (EVD) in 2014. S2. Key users of this evaluation include WFP (CO, Regional Bureau (RB), HQ), Government of Liberia (GoL) chiefly the Ministries of Education (MoE) and Gender, Child & Social Protection (MoGCWSP) and MGD. In addition, beneficiaries – school children, adolescent girls, male and female parents - and Mary’s Meals (MM) may also benefit. S3. The evaluation covers all school feeding activities supported through FFE which include school meals, Girl’s Take Home Rations (GTHR), developing school gardens and capacity development activities across 10 counties where the programme was implemented.1 It was conducted between February and June 2017 and the evaluation focuses on accountability (against intended results) and learning (for the continuance of the school feeding in Liberia). The evaluation mainstreamed gender and addressed the Evaluation questions from the Terms of Reference which are:  Evaluation Question 1: How relevant/appropriate is the operation?  Evaluation Question 2: How effective is the operation?  Evaluation Question 3: How efficient has the operation been?  Evaluation Question 4: What are the impacts of the programme?  Evaluation Question 5: How sustainable is the operation? S4. The Evaluation Team (ET) used a mixed method approach, with significant primary qualitative data collection to triangulate or replace the limited quantitative data available in country. Field visits were also conducted in 35 randomly selected schools. Key informant interviews were conducted with 150 stakeholders including WFP & MoE; 153 PTA members/teachers and 150 students were reached in FGDs. In each school, key education statistics were gathered and a School Observation Checklist was completed - looking at school records, kitchens, food storage facilities, water sources and gardens. S5. WFP provides food commodities as direct in-kind assistance to the targeted schools, while the MoE has overall responsibility for programme management, monitoring and reporting. WFP also provides capacity building support to the MoE. The School Principal has overall responsibility for the SF but Parent Teacher Associations and Food Management Committees support the SF by building appropriate kitchen and storage facilities, assisting in mobilizing students to attend and organise the provision of cooks and firewood. S6. WFP objectives of the programme include improving primary student enrolment (and the gender ratio) and attendance. The USDA/MGD results framework has the goal of improved literacy to be achieved by an array of 30 intermediary indicators including reduction of short term hunger, improving access to food and increased use of health and 1 Bomi, Gbarpolu, Grand Bassa, Grand Gedeh, Grand Kru, Maryland, Nimba, Rivercess, River Gee, Sinoe counties. xi dietary practices. WFP also aims to increase community awareness on the benefits of education, engage local organizations and community groups, and increase knowledge about safe food preparation, storage, good nutrition and health. S7. The evaluation used the internationally agreed OECD/DAC2 evaluation criteria of relevance, effectiveness, efficiency, impact, and sustainability. The key findings of the evaluation are as follows: How relevant/appropriate is the operation? S8. Overall, the ET finds that the SF programme is highly relevant to the Liberian context. The objectives align well with the national policy framework and with key WFP policies and strategies. Geographic and school selection criteria were clear; however, better adherence to the school selection criteria would have improved the 2016 scale-up. S9. Coverage of SF in counties is between 20-27 percent of national schools leading to a potentially negative pull factor of students into schools and increasing class sizes. S10. The ET find the choice of modalities of food assistance employed (in-school meal and GTHR) and the size of rations for both, to be appropriate for the context and culture. However, the 2016 scale up could have considered the GTHR to be an activity to utilise stockpiled commodities. Else gender sensitivity including sex disaggregation of data is used appropriately. How effective is the operation? S11. Overall, the ET find that some elements of the programme have been effective such as policy support, while others need some more strategic thought. For example, many of the CO’s capacity development trainings have been implemented as planned – or in excess of targets - but are not part of an agreed transition plan for programme handover to the MoE. No capacity development strategy was found. S12. The EVD highlighted some critical questions about the CO’s method of beneficiary counting for the provision of school meals as, despite schools being mostly shut during 2014/2015, the CO reported reaching 94 percent of annual target for SF. 41 percent of commodities were distributed. This calls into question the validity and reliability of the beneficiary output numbers. S13. Evaluation of effectiveness is difficult due to a lack of outcome monitoring data as only 18 of the 30 MGD indicators were tracked and school closures further limited data availability. However, all consulted stakeholders considered that the SF programme increases school attendance; the ETs data collection showed increased enrolment over previous two years.3 The ET finds that the GTHR has been effective in improving gender parity in primary education by creating an incentive for girls to regularly attend school. S14. Two key internal factors significantly influence the results of this programme. Monitoring by both MoE and WFP is irregular, although it is better in urban than rural areas. The lack of monitoring means that schools have made modifications to the programme. For example; SF was provided on average 4.3 times per week in sampled schools and fees were charged to cover associated costs. 2 Organization for Economic Cooperation and Development - Development Assistance Committee 3 Figure 4, Annex 17. xii S15. Secondly the partnership with MoE is an appropriate and crucial element of the programme to ensure cost-effectiveness and enable sustainability in the longer term. The MoE WFP relationship is good but currently MoE aren’t fulfilling all their obligations. S16. The major external factors are EVD outbreak that closed the schools and the lack of secure funding that stopped SF. The gender parity ratio fell after the EVD closure as girls returned more slowly to school. Poor roads can also hinder the delivery of commodities resulting in additional costs for ‘last mile’ delivery. S17. The ET also find that if WFP developed collaborations with other development and community actors for additional programme elements such as WASH, agriculture, or health related activities, it might improve the overall effectiveness of the programme. How efficient is the operation? S18. The commodity management and logistics systems of WFP have been efficient. The WFP logistics system resulted in provision of food to schools in a timely manner and a strong pipeline was maintained throughout the programme, with a good record for delivery. The CO also provided strong management and flexibility during the EVD outbreak, negotiating a transfer of food commodities to the emergency response programme in a timely fashion. S19. The GTHR offers higher levels of cost-effectiveness than the hot meal and has the advantage of the quantity of food provided is not constrained by the amount a child can eat in one sitting. A variety of sources reported it as effective in improving gender parity and the activity has been stopped for that reason in some urban districts. S20. SF programme coverage within counties is low at 20-28 percent, which reduces efficiency and cost-effectiveness via increased logistics. Low coverage per county also makes it harder to build capacity, knowledge and ownership by the County Education Offices. What are the impacts of the operation? S21. The ET find that the impacts of the programme are difficult to evaluate due to the short time frame shortened further by the EVD; the lack of impact (and outcome) level data; the lack of trend data. No specific impacts are stated in the CP200395 document. S22. The goal of USDA/ MGD in providing food assistance in schools is to improve literacy rates. However, literacy was not included in the baseline survey or any programme monitoring since, and the MoE does not collect this information. It is therefore not possible to evaluate any change in primary school literacy rates, nor the contribution of the SF programme to that change. The ET found that the SF is an important factor in increasing school enrolment and attendance; in offering an economic transfer to families and that it is likely that the SF programme contributes positively to household food security, particularly for the families of girls receiving GTHR. However, none of these outcomes are monitored. S23. The partnership between WFP and the Government of Liberia appears to be one of donor and recipient, rather than partners following an agreed timetable for handover. How sustainable is the operation? S24. Overall, the ET has concerns about the sustainability of the current SF programme. Although some milestones have been achieved in supporting the Government to take xiii ownership,4 the Government faces many challenges towards full ownership and implementation of the programme. S25. One of the MoE’s greatest constraints is their budget, which has no line item for school feeding, and SF also appears poorly represented in County Superintendents Development Plans. The GoL financial contributions to the programme fell over the evaluation period from 60 percent to 19 percent. These factors suggest the relationship is one of donor and recipient rather than partners with shared objectives. S26. Implementation of all the MoE’s assigned tasks, written out in Joint Plan of Action (JPAs), have not occurred even though they are re-stated year on year, and there are no apparent accountability mechanisms in place to address this. This situation is further hampered by the lack of a clear and agreed operational framework aimed at transitioning over to a government owned and implemented programme. In addition, a high MoE staff turnover is a further external challenge to the programme undermining WFP’s capacity development efforts resulting in little change, as trained staff leave the programme. Lessons learned and good practices S27. The ET commends the CO management for their flexibility and management of the use of the SF commodities during the EVD outbreak. The CO management implemented good practice in using in-country stocks for the EVD EMOP, while the SF programme was effectively on hold. This not only allowed for a timely provision of commodities to the EMOP but also prevented SF stock from expiring. This action was enabled by effective and prompt communication with and between the US donors and WFP supported by good logistic support. S28. Abridged Strategic Recommendations  Immediate start (within 6 months of completing this programme cycle) RECOMMENDATION 1: In collaboration with the MoE the CO should, with support from the RB as necessary, develop a clear capacity development strategy to support the handover of the SF programme to the Government of Liberia. RECOMMENDATION 2: In collaboration with the MoE the CO, with support from the RB as necessary, should develop a clear operational framework and timeline for the transition of the SF programme to the Government of Liberia. RECOMMENDATION 3: The CO, with support from RB as necessary, should review their capacity to implement the new SF Operational Framework and the new Capacity Development Strategy.  Immediate start, with a view to new collaborations during FY 2018 RECOMMENDATION 4: The CO, with support from RB as necessary, should strengthen collaborations and develop new partnerships with other development and community actors.  During the next phase of SF programming 4 Including the development and adoption of the School Feeding Policy (2013) and the School Feeding CoCR (2013). xiv Abridged Operational Recommendations RECOMMENDATION 5: The CO to consider expansion of the GTHR to all primary school grades in areas where there is a high gender disparity in primary school enrolment. RECOMMENDATION 6: The RB should provide ongoing support to the CO to develop a more transparent M&E system and ensure that monitoring and evaluation activities are appropriate, systematic and as per donor agreements. RECOMMENDATION 7: The CO should consider increasing the coverage of the programme within counties to improve multiple efficiencies including capacity building. RECOMMENDATION 8: The CO, with support from the RB as necessary, and in consultation with the MoE, should strengthen monitoring in the field. RECOMMENDATION 9: The CO, with support from the RB as necessary, and in consultation with the MoE, take steps to improve the provision of the in-school meal. 1 1 Introduction 1. The World Food Programme (WFP) Liberia Country Office (CO) commissioned this final activity evaluation of the McGovern Dole (MGD) funded Food for Education and Child Nutrition (FFE) in Liberia. The purpose of this evaluation is to assess the performance and results achieved, between September 2013 and September 2016, by the FFE activities implemented by the WFP Country Office (CO) under Component 1 of its Country Programme (CP) DEV 200395: School Feeding (SF). This final evaluation will inform future strategic and operational decision in Liberia, for both WFP and the Ministry of Education (MoE). 2. The evaluation will serve the dual and mutually reinforcing objectives of accountability (performance and results of the operation) and learning (the reasons why certain results occurred or not and lessons learned for the continuance of the school feeding in Liberia). It was designed to answer to the Evaluation questions outlined in the Terms of Reference (Annex 1), which were developed further in the Evaluation Matrix (Annex 2). The evaluation questions correspond to the OECD/ DAC criteria of relevance/ appropriateness, effectiveness, efficiency, sustainability and impact. The key evaluation questions are:  How appropriate/relevant is the operation?  How effective is the operation?  How efficient has the operation been?  What are the impacts of the programme?  How sustainable is the operation? 3. The main stakeholders concerned by the evaluation are WFP (Liberia CO, Regional Bureau (RB), Headquarters (HQ), Office of Evaluation (OEV) and Executive Board), McGovern Dole FAS/ USDA, Ministry of Education, Ministry of Gender, Child and Social Protection, FFE/ School Feeding (SF) donors in Liberia, school children and parents, school staff, NGOs working on SF and UN Country team. The primary users of this evaluation are expected to be:  The Liberia CO and the implementing partner the MoE. The evaluation will support decision-making related to programme planning, implementation and strategy.  Regional implementers of school feeding including Mary’s Meals.  MGD, as the major donor, will have a dual interest in performance and accountability.  The RB may use findings to provide strategic guidance and programme support.  WFP OEV/HQ may use evaluations for wider organizational learning and accountability 4. The evaluation covers all school feeding activities supported through the FFE programme which include school meals, Girl’s Take Home Ratios (GTHR), school gardens and capacity development activities across 10 counties where the programme was implemented: Bomi, Grant Bassa, Gbarpolu, River Cess, Sinoe, River Gee, Nimba, Grand Gedeh, Maryland, Grand Kru. At the time of this evaluation the WFP SF programme was ending for this phase and schools were using up their final stocks. 5. The evaluation was conducted in three phases; Inception Phase (February 2017 – March 2017), evaluation mission (22nd February -10th March 2017), additional data gathering 2 and reporting phase (March 2017 – June 2017). For further details on the evaluation timeline, see Annex 3. 6. The Evaluation Team (ET) was gender and culturally balanced, with an evaluation core team composed of two female international senior evaluators, including the Team Leader (TL) and a male national evaluator. In addition, a team of six Liberian enumerators participated in the primary data collection. The team covered technical experience in the fields of Education, School Feeding, Nutrition, Food Security, Gender and Capacity development. The WFP Evaluation Manager sat at CO and was responsible for quality assurance using WFP’s Decentralized Evaluation Quality Assurance System (DEQAS). 1.1 Overview of the FFE WFP school feeding programme 7. WFP has delivered school meals in Liberia since 1967 and the school feeding has been monitored and reported by the MoE since 2009. During this time, delivery was disrupted by 14 years of civil war (1989-2003) but the Government and partners (including WFP) reintroduced school meals in 2009. 8. The FFE programme is being implemented by the WFP Country Office (CO) in Liberia under Component 1 of Country Programme (CP) DEV 200395: School Feeding (SF). CP200395 was approved on 13th November 2013 and received $20m from MGD to run from September 2013 until September 2016. A no-cost budget revision was granted until July 2017. A mid-term evaluation of the current FFE programme was foreseen but had to be cancelled due to the Ebola epidemic of 2014. The prior MGD funded SF programme, 2009-2012, was not evaluated. 9. The outcome objectives of SF under CP200395 are to promote access to basic education and develop human capital through school meals; improve gender parity in primary schools through and to support the development of the pilot national Home-Grown School Meals Programme (HGSMP). The proscribed outputs to achieve this were delivery of WFP food and non-food items in sufficient quantity and quality to children and adolescent girls in targeted primary schools. The number of beneficiaries each year was planned to be 127, 000 school meals and 25,000 GTHRs for a family of five. The total quantity of food transferred under SF activity was planned to be 23, 198 MT for school meals and a further 4, 223 MT as GTHR.5 10. A SF results framework is included within the CP200395 logframe as Component 1. The WFP SF logframe falls under UNDAF 2013-2017 logframe, Outcome 3 Human development. It is presented in the Annexes alongside the USDA/FAS strategic results framework for FFE which presents a broader range of indicators. The WFP performance indicators employed relate to data reported by schools on enrolment and attendance (outcomes) plus disaggregated data (by age, gender, activity) on food assistance delivered to beneficiaries by schools (outputs). As such the logframe relies on the school’s capacity to monitor. Furthermore, there are no indicators to track the broader aspects of school gardens, capacity development or advocacy/system strengthening activities. 11. WFP provided hot meals in public and community primary and pre-primary schools initially in 10 counties of Liberia.6 Bomi County was in the programme until it was handed over to Mary’s Meals (MM) in 2015 at the request of the MoE. Nimba County then re-entered the WFP programme after being dropped under the previous MGD grant. 5 CP200395 6 Bomi, Grant Bassa, Gbarpolu, River Cess, Sinoe, River Gee, Nimba, Grand Gedeh, Maryland, Grand Kru 3 However, the final WFP distribution in Nimba stopped in December 2016 whilst other counties received food until February 2017. 12. The MoE is the only cooperating partner and is responsible for the management, monitoring and reporting7 of the FFE programme. The programme includes a significant component of capacity building for the MoE including support to policy frameworks, operational guidance, training for school staff and provision of equipment to schools. The current JPA between WFP and the MoE includes continued support to building the MoE management capacity; assisting the MoE in the development of a gradual handover plan, and to pilot and develop a HGSMP model. 13. Each school is expected to provide one meal for all primary students five days per week. The school pupils receive a meal designed to provide 30 percent of the recommended dietary allowance. In addition to providing a meal at school WFP provides a GTHR to girls from Grade 4 to Grade 6 to further encourage girl’s enrolment and attendance in four counties: Gbaropolu, Grand Bassa, Grand Kru and River Cess. The GTHR accounts for less than 5 percent of the in-school meal beneficiary numbers.  The in-school food ration is planned as 120g bulgur wheat, 35g pulses, 10g oil and 4g salt per child.  The GTHR is 111g bulgur wheat and 7g oil per day for 300 days a year. 14. Ebola viral disease (EVD). The Ebola epidemic of 2014 had a tremendous impact on WFP's ability to implement the programme. EVD began in April 2014 and by July schools had been ordered to close to combat spread of the disease. Schools did not reopen until mid-February 2015 but children returned slowly and the resumption of activities was chaotic. The MoE therefore decided to close schools for a second time to allow improvements to water and hygiene in schools; to introduce an incentive pack and to put all the schools on the same timeline for the next school year in September 2015. The slow restart to education, with consequent low levels of SF in 2015 resulted in an accumulation of food commodities at WFP. 15. At the time of the outbreak WFP had received two consignments of food aid and the warehouses were full. USAID/Food for Peace (FFP) authorised the use of the immediately available MGD food for the EVD EMOP 2007618 with agreement to later 'pay back' the school feeding commodities.9.This reimbursement happened as agreed and USAID/FFP repaid the food stocks transferred from SF and further MGD shipments were received. The high levels of food stocks required a ‘scale up’ of SF in 2016 to consume the foodstuffs prior to ‘best before’ dates. The SF scale-up expanded the SF programme from 595 schools to 815 schools in 2016. 16. WFP in Liberia. A UN Human Security Initiative Trust Fund is a joint project with FAO. It promotes food security and strengthens governance of local communities and women farmer groups. Activities include food assistance for assets (FFA) and support to community grain reserves (CGRs). The three-year project target 6,000 beneficiaries and is valued at US$425, 643. In addition, a four-year Japanese Bilateral Project 200541 uses FFA to support lowland rice cultivation. 7 JPAs 2013-2016 8 EMOP 200761 ran Aug-Nov 2014 9 Giving rice as the staple rather than bulgur wheat 4 1.2 Context 17. Economic and poverty: An estimated 63.8 percent of Liberians live below the poverty line10 of which 1.3 million live in extreme poverty.11 The highest incidence rate of poverty is recorded in Grand Kru, Maryland and River Gee all at 77 percent. Liberia was experiencing steady economic growth, with average GDP growth rate of 7.6 percent, until it was decimated by the Ebola outbreak in 2014 and it reduced to 0.7 percent. The World Bank predicted Liberia’s GDP growth to be 2.5 percent in 2016, and 5.3 percent by 2019, due to resumption of services and gold production after the Ebola crisis. 18. Food and Nutrition insecurity: Liberia is a low-income, food-deficit country and half of the population is food-insecure or highly vulnerable to food insecurity. Despite high potential productivity the agricultural sector faces major challenges including poor access to inputs and markets, inadequate rural roads, and limited extension work. The last Liberia Demographic and Health Survey (LDHS) shows that stunting is on the decline (42 percent in 2010 to 32 percent in 2013) and sex disaggregated data shows the usual pattern of boys having higher rates of malnutrition than girls.12 Only 28 percent children 6-23 months receive an adequately diversified diet and while national stunting rates have reduced they remain at ‘critical’ levels (WHO classification) in Grand Gedeh, Grand Bassa, River Gee and Nimba counties.13 19. Under-nutrition among girls 15-19 years old is 15 percent,14 a worrying trend that enables the intergenerational cycle of malnutrition. Likewise, data indicates a prevalence of anaemia among children, particularly pre-primary children of 60 percent.15 20. Education. The 14-years of civil conflict left the country devastated and the education system ruined: 80 percent of schools had been destroyed or damaged and there was a severe shortage of trained teachers. 21. Free and compulsory education for all was enshrined in law in 2002 and its enactment was significantly improved in 2009 by the Liberia Primary Education Recovery Program (LPERP). The Education Sector Plan 2010 (ESP) aimed to realise free primary school education for all by 2015 via a process of rehabilitation and construction of infrastructure combined with the training of primary school teachers. Although there are no school fees, primary education still involves some cost to parents: registrations fees, uniforms, PTA fees etc. The net enrolment ratio (NER) steadily increased until 2014 when the Ebola Viral Disease (EVD) outbreak occurred and schools were closed. 22. Currently several challenges remain including lack of qualified teachers and instructional materials. The MoE has set targets to address this: a pupil to teacher ratio of 40:1 and class repetition rate reduced from 7 percent to 2 percent by 2020. Due to children being unable to attend school during the civil war years, there is now a wide age range of children in primary education and the presence of adolescent girls and boys in Grade 1 is not uncommon. In 2013, UNICEF reported the proportion of out-of-school children of primary school age as 59 percent.16 While school enrolment rates increased over 2007- 10 PPP $1.25 a day http://hdr.undp.org/en/countries/profiles/LBR 11 http://data.worldbank.org/country/Liberia. Data 2007 12 Liberia DHS, 2013 for stunting and wasting MOHSW, LISGIS. 13 CFSNS 2010 and 2013. 14 Liberia Demographic and Health Survey, 2013 15 UNICEF (2012) https://www.unicef.org/appeals/files/Liberia_HA2013_26_dec.pdf 16 UNICEF (2015) State of the World Children (2013 data). 5 2013, and girl’s enrolment increased as well,17 the quality of education in Liberia is poor and the disparity between urban and rural schools is high. Education remains a major priority of the Liberian government, accounting for 13 percent of its national budget during 2015. 23. Liberia is slowly moving towards their Education for All Goal 5: “Eliminating gender disparities in primary and secondary education by 2005, and achieving gender equality in education by 2015, with a focus on ensuring girls’ full and equal access to and achievement in basic education of good quality.” 24. However marked differences exist between schools in all aspects of capacity. In deprived rural areas schools may have a single teacher - with one year training - on the MoE payroll, no text books and buildings in a dilapidated state. The education monitoring information system (EMIS) is not fully functioning and doesn’t collect basic data such as literacy rates. The National School Feeding Policy was produced with the support of WFP under the last MGD grant, and adopted during this programme in November 2013. 25. Gender. Liberia is traditionally a patriarchal society; only in 2001 was the first Ministry for Gender and Development created. In 2006 and 2011 a female President was democratically elected, providing inspiration to young girls for years to come. Even so, women remain at a disadvantage in Liberia since 90 percent of their employment is in the informal sector or in agriculture. In 2009, the first National Gender Policy was launched18 and in 2013, following political reshuffling the Ministry became the Ministry for Gender, Children and Social Protection (MGCSP). 26. The Gender Inequality Index (GII) value is poor at 0.649 and ranks Liberia as 177 of 188 countries in 2015.19 Gender disparity is a serious issue and girls face greater obstacles to school enrolment and are less likely to complete basic education. Early marriage of girls is commonplace in Liberia and was reported to the ET as a key reason for the high drop out of girls from formal education alongside pregnancy. 27. Literacy rates among women aged 15 to 49 is less than half (44.5 percent) the rates for men.20 The Gender Parity Index for school enrolment is 0.91 at the primary level and 0.81 at secondary level.21 In 2011, the Government of Liberia adopted a National Girls Education Policy to overcome barriers to female education and gender parity in enrolment has been reached in some urban areas. 28. In light of the above statistics the sustainable development goals (SDGs) are pertinent and the WFP school feeding is one tool that can contribute to their achievement. FFE is particularly relevant to SDG 2 – to end hunger – and SDG 4 – to ensure equitable education. 29. Other International Agencies. A range of UN agencies work in Liberia and food security is under pillar 2 of the UNDAF 2013-2017. UNICEF provides support to education – including provision of a return to school incentive post-Ebola – and improves schools water sources. FAO provides policy support including on marketing and agricultural diversification and strengthening national capacity. WHO played a 17 Ministry of Education (2015) Education for All 2015 National Review Report: Liberia. Republic of Liberia, Monrovia 18 http://www.genderindex.org/country/liberia 19 http://hdr.undp.org/en/composite/GII 20United Nations Development Assistance Framework 2013-2017 Liberia http://www.unliberia.org/doc/undaf20132017.pdf 21 UNICEF (2015) State of the World’s Children – 2013 data. 6 crucial role during the EVD outbreak and UN Women is active in the policy arena in Liberia. Mary’s Meals – a Scottish INGO – is another provider of school meals in Liberia. They work in 4 counties and feed 126,000 school children. 1.3 Evaluation methodology and limitations 30. The methodology was designed to be gender sensitive and the team ensured that the views and opinions of the most vulnerable, especially girls and women were adequately captured and incorporated in the analysis. Interviews were carried out in accordance with 2008 Ethical Guidelines for Evaluation of the United Nations Evaluation Group (UNEG), notably to ensure that key informants understood that their participation was voluntary and that confidentiality would be respected. In addition, steps were taken to ensure that men, women, boys and girls felt they were in a safe space where they could freely express their views and concerns without fear of reprisal. Consent to interview children was sought from the School Principal. 31. The ET comprised two women and one man; the ET requested a gender balanced team of six enumerators during inception but were given four men and two women. Time constraints prevented finding an additional woman and so the fieldwork teams were split to ensure each team had female representation. The sub-teams found no difficulty ensuring the voices of women were heard. 32. Evaluation questions were developed into an Evaluation Matrix as described above (Annex 2). To assess the performance of the FFE programme, the ET used both WFP output and outcome indicators determined in WFP’s Terms of Reference (ToR); Country Programme DEV 200395 (CP 200395) logframe and the WFP-MGD grant agreement which has a further 23 specific performance indicators and 31 results indicators. 33. The evaluation used a mixed methods and gender-responsive approach, which is described in full in Annex 4, and includes document gathering and review. National level data was not available to the ET and comparison to national data/non-programme schools wasn’t possible. Quantitative data from schools is viewed as open to political manipulation and this view was supported by the evaluation findings with significant differences between enrolment, attendance and head count data. To maintain credibility of the evaluation findings the ET gave extra focus on the primary data collected, including qualitative methods, and the use of triangulation across types of informant, across the ET and across different methods. 34. Data collection methods also included interviews with key stakeholders including the MoE at national and sub-national levels, WFP and MGD personnel by the ET. School personnel included Principals/Administrators, teachers, PTA groups, cooks and food stock keepers were interviewed by ET. The ET asked to have more women represented in these interviews if there was a gender imbalance, and information about gender balance for the PTAs and teachers was collected. Focus group discussions (FGDs) were conducted with school students (girls and boys separately) by enumerators and/or ET (dependent on gender requirements). It is of note that the ET found that in Liberia women were both well represented and vocal in discussions. 35. Enumerators also completed a School Site Observation Checklist looking at school records, kitchens, stoves, food storage facilities, water sources and gardens plus a table of key education statistics from schools and District/County Education Offices. The full list of key informants can be found in Annex 5. The primary data collection guides, observation check lists, quantitative education statistics forms, questionnaires and checklist can be found in Annex 6. Checklists/FGDs noted gender of the informant (s) in 7 positions of power and responsibility and all data was gender disaggregated as appropriate. 36. The selection of schools to be visited by the ET was done using the principles of randomness and representativeness using systematic sampling stratified by county. This process was carried out by the ET in consultation with the CO. The sampled schools varied widely in terms of infrastructure and organisation (Annex 7) and this typically, but not always, reflected the remoteness of locations. Once the list of schools was finalized, the ET split into three sub-teams to collect data. Data collection was done over an 8-day period, covering all 10 counties where the SF programme was implemented. The geographical spread of the sampled schools is presented in Map 1. All data on school children collected at the schools was disaggregated by gender. 37. In total 35 WFP schools were visited, 150 interviews conducted, 153 PTA members/teachers, and 150 students included in the FGDs. The ET also visited two former WFP schools in Bomi County for comparison. The ET team ensured that the gender ratio of students reached was 50:50. The ET worked independently of the CO during the field mission but were accompanied and guided by either the MoE District Focal Points (DFPs) or the County Co-ordinators (CCs). The MoE staff was aware of which schools had been selected with, on average, a day’s notice. The ET note that in some counties, the county staff supported the evaluation by conducting radio announcements the day before the evaluation in each area, however many schools were unprepared. 38. To enable data validity and factual accuracy team members regularly compared and analysed data collected. A briefing session was held at the end of field work for data cleaning, preliminary analysis and discussions between ET and enumerators. Steps for data analysis were laid out. ET members undertook initial data analysis and statistical tabulations in Monrovia with the rest of the school analysis being undertaken thereafter. Data analysis followed the Evaluation Matrix and requirements of the MGD and WFP evaluation requirements (Annex 2). This enabled triangulation and cross-referencing of information to mitigate data inadequacies or absences. WFP personnel were included in a debrief before the ET left Liberia (March 10th), and were also involved in a teleconference about the themes of the evaluation recommendations. 39. The Evaluation Matrix (Annex 2) enabled triangulation and cross-referencing of information to mitigate data inadequacies or absences and improve reliability. WFP personnel were included in a debrief before the ET left Liberia (March 10th), and were also involved in a teleconference about the themes of the evaluation recommendations. 40. Limitations. The ET note several limitations and constraints to the evaluation.  Limited time for the Inception Phase meant that background documents were still being sought and considered throughout the field mission. The Inception Report was submitted immediately prior to the field mission, QA was received at the end of the field mission and it was accepted end March pushing back the Reporting Phase and losing the sequencing.  The WFP Baseline Survey (April 2014) collected data on 31 MGD indicators however, the planned mid-term evaluation was cancelled due to the EVD outbreak. WFP/MoE was only able to track 14 of the 31 indicators in the MGD results framework; which was developed after the WFP/MGD grant agreement. This made it difficult for the ET to evaluate some components of the operation. There are several reasons for this:  WFP did not have the human resource capacity to measure all indicators, as 8 they had only one person in the M&E Unit for most of the MGD grant. For example, measurement of some indicators such as test scores, literacy rates take considerable time and resources.  The MGD results framework presents multiple multi-sector indicators, many of which are beyond the scope of MoE and WFP monitoring without significant training and resourcing.  Other indicators simply could not be measured because the school closures interrupted data collection.  Most data are collected by WFP, but some indicators require the assistance of the MoE either at school or county level. While baseline figures appear credible, it is difficult to assess the reliability of some of the indicators.  To mitigate for these and to improve data validity and reliability the ET enhanced triangulation and collected primary information at school level; however, many indicators remain unmeasured (Annex 8). These factors should be considered when reading the following section on the effectiveness of the operation.  School closures due to EVD meant that there is data for only one full academic year (2016). As such, the ET was unable to utilize trend analysis as anticipated.  WFP data was pooled at the Country Programme level until 2016 when the new COMET system was introduced. As such, data relating specifically to SF had to be done manually extracted by the CO. This was time consuming for the CO and open to error.  The new COMET system is not transparent, as some data is auto-generated by algorithms produced by WFPs Performance Management and Monitoring Division in Rome. For example, the GTHR beneficiary data (the girl and her family) can be presented two ways, as either four or five times the number of girls in the programme. It is not clear which factor is applied.  Overall the ET felt concern over the quality of the data collected from schools. At times the information was already written out on a single sheet of paper and with no apparent back up such as an office or ledgers; on other occasions, it was written out from memory before the ET. As a result, greater weight is given in this evaluation to qualitative methods.  The CO’s Standard Programme Report (SPR) for 2016 was only made available to the ET after their field mission. The SPR noted that WFP is the Chair of the Monitoring and Evaluation Technical Working Group of the United Nations Development Assistance Framework (UNDAF). If this had been known while the ET was in Liberia, it would have enabled a new line of enquiry. 9 2 Evaluation Findings 41. In assessing the appropriateness of the SF Programme, the evaluators examined the extent to which the objectives, targeting, choice of activities, and modalities of transfer were appropriate to the needs of the population, aligned with government and WFP corporate policies and strategies, and coherent with the work of other development actors. 2.1 Evaluation Question 1: How relevant/appropriate is the operation? 42. The objectives22 of the SF programme are to:  Promote access to basic education and develop human capital through school meals;  Improve gender parity in primary schools through GTHR; and  Develop a national HGSM programme. 43. In summary the ET found that the objectives are all relevant for Liberia struggling to rebuild its education sector after conflict (and after the EVD outbreak) and where gender equity is poor. Supporting the development of a HGSM programme is also appropriate as it strengthens the government’s ability to implement national programmes. 2.1.1 Appropriateness to needs 44. SF is globally recognised as an effective safety net for poor communities who are food insecure in a variety of circumstances including crisis, post-conflict, recovery situations such as post-EVD, as well as in chronic long-term development settings23, such as Liberia. SF mitigates household expenditure, freeing up resources for other productive financial activities.24 In areas where school enrolment and/or attendance is low, SF also offers an incentive for households to send their children to school and invest in education, thus contributes to breaking the poverty trap.25 SF also frees up parental time, enabling work activity, and under a gender lens in Liberia this means women’s time. The ET therefore finds that the implementation of a SF programme in Liberia is appropriate. 45. In addition, restoration of the education sector is a high Liberian government priority as it is critical to its future.26 Significant progress has already been made; in 2016, the national budget allocated US$86.2 million of a total US$ 602.4 million (14 percent) to education.27 However, KII reported that education receives only 7 percent of the national budget of which 98 percent is used on payment of salaries. Support from external donors is therefore highly appropriate. 46. Targeting and coverage of SF programme: The WFP SF programme covered 10 counties, out of a total of 15 counties in the country (Map 1). In addition, four counties are supported by Mary’s Meals, 28 an INGO independent of WFP. 22 Draft Liberia Country Programme (2013-2017) 23 The World Bank (2011) World Development Report – Conflict, security and development. Washington, D.C 24 Bundy, D. et al (2009) Rethinking School Feeding: Social safety nets, child development and the education sector. Directions in Development- Human Development. World Bank and World Food Programme. 25 Molinas, L. & Regnault de la Mothe, M. (2009) The multiple impacts of school feeding: a new approach for reaching sustainability. In: WFP (2010) Revolution: From food aid to food assistance: Thematic Areas, Chapter 14, p217-230. 26 Liberia Poverty Reduction Strategy 2008-2012 Final Report 27 https://www.mfdp.gov.lr/index.php/media-center/press-releases/370-president-sirleaf-approves-fy2016-2017- national-budget 28 A Scottish NGO 10 47. WFPs geographic targeting for the SF programme follows guidance from the VAM assessments on areas of highest food insecurity. Counties are also targeted based on low education indicators, areas requiring the most development assistance, and areas with the highest gender disparity in education. The ET found that there was some correlation between the latest food security and nutrition assessment29 (2013) and the selection of targeted counties at the time of project design.30 Within the targeted geographic locations, WFP selects a certain number of schools to include in its SF programme each year, with a focus on rural schools. Selection of schools is based on the level of primary enrolment, and again, the gender disparity. 48. The WFP SF programme is intended for public and community primary schools and pre￾primary schools. Private schools are not supposed to be included. The ET also found at least 14 private schools receiving food through the programme, including those run by the Socfin Group, a group of Belgian commercial owners of the LAC rubber plantation. Further investigation revealed that the private schools receiving WFP food were mainly newly ‘scaled up’ schools recruited to help utilise excess stocks caused by the EVD disruptions. However, many of these new schools had also been served by WFP in the past, casting doubt on the rigor of application of WFP Liberia’s school selection strategy. 49. Since 2013, the SF programme has seen turnover of schools, with each school not necessarily receiving continuous service. Whilst the ET understands that funding variations may necessitate reducing or increasing the number of schools, ultimately the food security/safety net function of SF is better served by regular, continuous service. In addition, the ET note that the MGD SF is implemented under the 2014-2017 Strategic Plan, and as such, is framed as a response to shocks, rebuild livelihoods and build lasting resilience31 all of which are undermined with discontinuous programming. Capacity building efforts are also undermined by stop-start programming. Aside from the post￾EVD scale up described earlier, the reasons for the difference in annual numbers are not clear to the ET. 50. Prior to the 2016 SF scale-up the WFP SF programme was implemented in about 20 percent of the public schools (in food insecure areas) in Liberia, and post-scale up, about 27 percent of all national schools (public, private, religious, community).32 51. Some key informants suggested that WFP would do better to increase county-level coverage (i.e. more schools per county) rather than the number of countries reached - a view that is supported by the ET. If WFP had a more focused approach to school targeting they would benefit from gains in terms of simpler logistics, a reduced ‘pull factor’ of SF33, minimized staff turnover and gain an ability to train more individuals. It would also contribute to the building of a community of practice by enabling SF staff in neighboring districts to exchange views and work more closely together. 52. 2016 Scale-up: In 2016, WFP scaled-up the SF programme with the dual objectives of encouraging children to return to school post-EVD and to utilize excess food stocks. The 29 Ministry of Agriculture and WFP (2010) The State of Food and Nutrition Security in Liberia: Comprehensive Food Security and Nutrition Survey. Republic of Liberia, Monrovia 30 Counties in Map 1 compared to Maps 2 & 3 31 WFP Strategic Plan 2014-2017 32 Based on data from the MoE and WFP school data. About 27 percent coverage of all schools by WFP in 2015-2016; in 2013-2015 approximately 20 percent coverage of public schools (both schools and # of beneficiaries fluctuated during those times). MoE statistics (2014), the total of 3,854 primary schools were recorded. Of this total, 806 are private schools, 396 Religious/Mission schools, 215 Community Schools and 2,375 are Public Schools. 33 Which leads to larger class sizes and subsequent poorer learning environment 11 ET find that the scale-up was appropriate given the circumstances, but it could have been more strategically implemented. As it was, many of the new schools were in Nimba county.34 Nimba had previously been dropped from the programme (in 2013) due to its higher levels of food security and MoE capacity.35 However, the calculation of the commodities required was inadequate so the new schools in Nimba were stopped in December 2016, whereas other counties received SF until February 2017. A more nuanced scale up plan would have enabled distributions to stop across the counties simultaneously. 53. Choice of modalities: WFP implements two modalities of assistance through this programme: provision of an in-school meal served around midday, and provision of a GTHR. Given the constraints faced by girls to complete education discussed earlier, the ET find it appropriate that WFP has included extra support for girls through this programme. However, as Liberia approaches gender parity for school enrolment36 - particularly in urban areas - careful application is required. To date WFP have been able to respond to these changing contextual needs - several MoE respondents noted that GTHR had been withdrawn in response to gender parity in primary school enrolment37. However, in more rural contexts it was visibly apparent and widely reported in interviews, that girls often drop out before Grade 4 when they would qualify for the GTHR. This raises the questions of whether a Grade 4 start is an appropriate starting point, at least in rural areas. 54. In addition, the timing of the in-school meal at midday is unlikely to improve student attentiveness as it is usually provided at the end of the school session. Afternoon classes are typically only provided in secondary schools. Given that anecdotal evidence suggests that children come to school without a meal, it would be appropriate to provide food earlier. 55. Choice of ration: The ET found that both the SF in-school ration, and the GTHR are appropriate for the needs of the beneficiaries. Bulgur wheat is a culturally appropriate choice, and providing a meal containing 30 percent of the RDA is appropriate given that children receive other meals at home. Whilst the prevalence of Vitamin A deficiency has been on a steady decline it is still a public health problem38 in Liberia so the delivery of Vitamin A fortified oil is appropriate. Bulgur wheat, unlike rice, also provides additional Vitamin A in the diet.39 Iodized salt is also appropriate to prevent iodine deficiency and associated morbidities.40 Produce from school gardens such as vegetables and fruit would be a useful addition to the school meals by improving dietary diversity, meeting local preferences and providing vitamins not found in WFP commodities such as vitamin C. 2.1.2 Coherence and alignment with WFP policies and with national policies on education, nutrition and child welfare 56. The Government of Liberia, MoGCWSP views the provision of food to hungry children in schools as an effective tool for promoting and improving educational outcomes in poor communities.41 WFP has specifically aligned the SF programme with national policy 34 195 out of 413 schools; Source CO spreadsheets School list pre- and post-scale up 35 KIIs WFP, Emergency Food Security Assessment 2015. 36 SPR 2016, KIIs 37 KIIs Buchanan and Grand Bassa 38 Global burden of disease 2016 39 Bulgur wheat provides 366 IU per 100 grams. USAID Commodity Reference Sheets. 40 Cretinism, goitre, cognitive impairments, low IQ 41 KIIs, Social Protection Policy & Strategy 12 strategies in gender, nutrition and food security and agriculture to maximise potential synergies. 57. The National Education Sector Plan (2010-2020) acknowledges the role that WFP has played, via feeding, to nutrition, maintaining school attendance and reducing drop out. The Plan also notes it as an area that the MoE should take on in the long run as they feel it has significant implications for attendance and dropout when feeding stops. 58. The National Girls Education Policy (April 2006) was designed to meet MDG 2 by providing free and compulsory primary schooling and reducing secondary school fees by 50 per cent. It complements school feeding and aims to achieve gender parity, to ensure education is free and enable girls to attend school. 59. The National School Feeding Policy (2013) was produced by the MoE following multiple agency involvement including WFP. The policy clearly lays out the modalities and implementation plan for school feeding as currently implemented. This was followed by the School Feeding Code of Conduct and Recognition (CoCR) in 2013.42 Key informant interviews with government confirmed that WFP had been actively involved with the MoE, Ministry of Agriculture (MoA) and MoGCSP. The MoA has partnered with WFP on the HGSMP since 2013. 60. The National Social Protection Policy and Strategy (2013) highlights that SF is the largest social protection programme in Liberia, accounting for 34 percent of all provision. It calls for international experience to improve the effectiveness of school feeding and to ensure improved ownership by the MoE and ensuring that the commitments of the new Education Reform Act43 (2011) are realized. The role of WFP in supporting the government in provision of school feeding is acknowledged. 61. In addition to the Government of Liberia policies and strategies mentioned above, the ET find that the SF programme is guided by several of WFP’s own corporate policies, and strategies. 62. WFP Strategic Plan (2014-2017)44: The SF programme contributes to Strategic Objective 4 – to reduce under-nutrition and break the intergenerational cycle of hunger. The Strategic Plan specifies that school feeding programmes contributes to this goal by providing quality food, and contributing to addressing micronutrient deficiencies. The second goal of the Strategic Plan is to increase access to education and health services. 63. The WFP Updated School Feeding Policy (2013)45 specifies that WFP will focus increasingly on helping countries to establish and maintain nationally owned programmes linked to local agricultural production and this is embedded in the future direction of the CO. Currently the CO is engaging in policy dialogue and provision of technical assistance in line with the policy. However there is not a clear hand-over strategy to national ownership. The CO is also following the policy by provision of GTHR in schools with a gender disparity of 15 percent or more between the sexes in upper primary grades (4-6th) to encourage girls to attend school. 42 http://allafrica.com/stories/201311250983.html, launched in November 2013. 43 Commits 14 percent of national budget to education 44WFP (2014) WFP Strategic Plan (2014-2017) https://www.wfp.org/about/strategic-plan 45 WFP (2013) WFP Updated School Feeding Policy https://www.wfp.org/content/school-feeding-policy 13 64. The WFP Gender Policy (2012)46 provides guidance to ensure that WFP’s programmes promote a gender equality and women’s empowerment. 65. In general, school feeding in Liberia is aligned with other significant WFP and national policies. It is also in line with, and contributing to SO 4 (“reduce undernutrition and break the intergenerational cycle of undernutrition”) of, the WFP Strategic Results Framework 2013-2017 designed to contribute to the MDGs. Moreover, the programme is in line with Zero Hunger, the second SDG. 2.1.3 Coherence and alignment with UNDAF and other humanitarian and development actors 66. The current United Nations Development Assistance Framework (UNDAF) for Liberia (2013-2017) recognizes the importance of gender equality and women’s empowerment. The UNDAF was developed in consultation with the Government of Liberia in line with the Agenda for Transformation (AfT) and the MDGs. The ET found SF to be aligned with UN partner agency and donor policies and priorities. The WFP SF falls under the Human Development pillar of the AfT. WFP also attend monthly education co-ordination meetings which ensure that they do not overlap operations with Mary’s Meals. 47 67. Aside from coordinating with Mary’s Meals, the ET found limited evidence of WFP building partnerships and maximizing synergies with other UN agencies for school feeding. Reports and conversations indicated that WFP were actively involved with USAID, UNMIL, INGOs and other international organisations during the EVD crisis. WFP provided food and logistical support to food-based interventions. However, for the ongoing SFP, the ET consulted UNICEF and FAO field staff who presented a mixed picture of WFP partnership behaviour with WFP not systematically coordinating and communicating on their activities. WFP are not mentioned in the Education Cluster post￾EVD assessment48 nor sharing any relevant trainings/monitoring with UNICEF. Key findings and conclusions Evaluation Question 1: How appropriate/relevant is the operation? Overall, the ET finds the SF programme is coherent with both government and WFP policies and strategies. In addition, the ET finds that the programme has been designed well with the input of the MoE and that good criteria for school selection are embedded in the SF policy. However, WFP needs to ensure that these criteria are strictly applied. The ET also find that the CO and MoE should reconsider the starting age for the GTHR as stakeholders acknowledge that many girls have already dropped out of school by fourth grade, particularly in rural areas.  SF is highly relevant to the Liberian context.  The programme is well aligned to the national policy framework and with key WFP policies and strategies but handover to national ownership needs enhancement.  A system strengthening approach to SF at national, county and district levels is 46 WFP (2009) Gender policy: promoting gender equality and the empowerment of women in addressing food and nutrition challenges. WFP/EB.1/2009/5-A/Rev.1 47 Mary’s Meals is the only other SF actor in Liberia, serving 135,000 children across 4 counties, each of which WFP has served in the past. 48 Education cluster, 2015. Assessment of the effect of Ebola on education in Liberia 14 highly appropriate in Liberia.  The ET found inconsistencies in WFP’s adherence to school selection criteria.  There has been a variance in number of schools served over the programme period, from 627 in 2013, down to 595, then up to 815 during the 2016 scale-up. The ET find that discontinuous service provision undermines the programme outcomes and capacity building efforts. Coverage ran at between 20-27 percent 2014-2016.  The ‘scale up’ of SF post-EVD outbreak could have been better planned and implemented to avoid the inclusion of private schools, so that all the scaled-up schools received commodities for the same period. 2.2 Evaluation Question 2: How effective is the operation? 68. This section of the report evaluates the SF programme 2013-2016 performance on the achievement of the planned results based on performance indicators jointly identified by WFP and USDA (Annex 17). Where possible, the performance at endline is compared to the baseline (2014) and/or WFP’s targets. Additional data tables and overall summary tables of the programme and evaluation results can be found in Annexes 11-14. 69. The outbreak of EVD had a tremendous impact on WFP's ability to both implement and monitor the programme. School closures meant that WFP only has education data for one academic year (2015/2016) during the evaluation period so trends cannot be assessed. 2.2.1 Capacity building results 70. As WFP supports the MoE to implement the programme capacity building is a large component of SF. Each year a JPA is drawn up between WFP and the MoE but these have not yet identified steps or milestone indicators for the transition of full ownership and funding to the GoL. To increase capacity of government institutions & improve the policy and regulatory framework 71. National level development of policies, strategies and operational frameworks for MoE49: WFP developed the School Feeding CoCR50 in the previous phase of SF but it was adopted this phase in November 2013. During this programme, the new School Feeding Policy was rolled out and the CoCR was operationalized, with key points followed by MoE and WFP. No additional national policies were produced. 72. In addition, two Standard Operating Procedures (SOPs) (from previous grant) were reviewed and updated this phase - one on management of bikes and vehicles and one on disbursement of funds to MoE. SOPs on school’s selection, assets management and funds disbursement to MoE remained enforced during the programme. The ET identified the need for additional SOPs, for instance on M&E and the reinforced application of current SOPs/WFP procedures. The ET determined that low levels of monitoring by MoE had enabled some schools to have poor adherence to the school feeding policy – this included but was not restricted to not feeding every day, sharing of school meals with Juniors/Secondary pupils and charging pupils for meals.51 Several WFP stakeholders stated poor levels of reporting, particularly narrative, from MoE to WFP and the ET 49 See Annex Effectiveness Findings, Table 1 50 The School Feeding CoCR is intended to monitor the use of schools supplies in the country and to ensure transparency and accountability in the management of the school feeding resources in Liberia. 51 These examples all observed during field mission by evaluation sub-teams and triangulated with KIIs/FGDs with different stakeholders on site. 15 found irregular monitoring of schools by the MoE – due to a combination of factors including high work load, poor maintenance of bikes, insufficient training. 73. The CO also planned to draft two school curriculums: one on nutrition and one on agriculture. At the time of the evaluation, the National Agricultural Curriculum (Grades 1-6) developed with the MoA was pending validation from the MoE, then awaiting funding for pre-testing by MoA. The National Nutrition Curriculum was being finalized with the Ministry of Health, with plans to finish it in the coming year although no publication date has been set. To increase skills and knowledge of school administrators 74. Developing human capacity (training): Each year, before school starts, WFP conducts training to schools on the SF program. WFP exceeded its target (1,300) in the first reporting period52 and continued to exceed the target each reporting period when school resumed in 2015/16. The total number of school administrators trained, including training of trainers recorded by WFP was 2,992 persons (230 percent achievement) (Table 2). To increase skills and knowledge of teachers 75. In addition, WFP conducted several training activities for teachers. Although these were interrupted by the EVD outbreak, once the schools reopened, WFP continued training teachers and school administrators. Training focused on how to run, manage and monitor the SF program, rather than on the topics agreed with MGD: increasing knowledge on health and nutrition topics or efforts to influence behaviours. 76. Some teachers involved in the school administration and PTAs who received annual WFP training on running and managing a school feeding program, also received additional training on taking attendance, hand washing behaviours and nutrition. However, this was a short, annual, one-day training and a more extensive coverage of health and nutrition topics is required.53 When asked, “is the PTA involved in nutrition and health promotion?”, answers were mixed or limited, with hand washing encouragement mentioned most frequently. This behaviour was strongly emphasized during and after the EVD outbreak. 77. Table 3, Annex 17 shows that WFP exceeded its target of training 1,950 teachers by reaching 4,637 teachers. This figure is the total persons trained, some which are counted twice or more if they received annual training. It also includes the training of additional teachers and record keepers during the program scale up in May 2016. The numbers do not factor in the Training of Trainers (ToT), which they encourage of those who receive WFP annual training. WFP were challenged by high staff turnover within the MoE which led to increased training needs. 78. The endline target was 630 persons trained per year in health and nutrition. The EVD impacted and training only got underway, as a priority for re-opening schools, during school year 2015/2016. WFP consequently trained a total of 4,925 persons in health and nutrition, however some people were trained annually so it’s not unique individuals. 52 1347 school administrators trained between November 2013 – May 2014 53 Based on responses from FGD 16 To increase knowledge and use of health and dietary practices (safe food preparation and storage) 79. WFP provided training to school storekeepers on the proper storage of food (off the ground54), and to school cooks on safe food preparation. WFP planned to conduct 5 trainings in 2013 but completed 3. Two additional trainings were then conducted in 2016. The ET found that all these trainings were carried out in just one cluster of schools per district and county. 80. Food preparation: The ET found that 40 percent of the cooks in the schools visited had been trained by WFP in safe food preparation. 60 percent said they had not been trained, though some may have received training from a ToT and responded “No” because it was not formal training. Using the School Site Observation Checklist, the ET found 63 percent of school kitchens to be clean and orderly. More than a third of the kitchens visited were in poor condition, including some with no walls. WFP also intended to provide 100 energy efficient stoves to schools each year and despite the EVD outbreak they have achieved their target (Annex 17, Table 4). The ET checked the school kitchens and found that less than half the schools visited (48 percent) had fuel efficient stoves and two of those were in poor condition. The other schools were cooking over an open fire. 81. The ET also noted that several schools lacked the necessary non-food items to efficiently run the SF programme. Children were seen sharing bowls and eating with their hands. 82. Food storage: WFP expects and trains schools to store their food off the ground. At baseline 65 percent of storekeepers had knowledge of safe food storage, but this indicator has not been tracked during the programme. However, the number of schools storing their food off the ground has increased from 68.8 percent to 98 percent over the course of the programme, indicating success (Annex 17, Table 5). Where it was possible to see the food storage practices (n=28),55 the ET found that 93 percent of schools stored food off the ground. 83. The ET used a school observation checklist to determine if food was safely and properly stored. The ET team found 32 of the 35 schools to have good or acceptable storage locations (91 percent). Food stock records were up-to-date, readable and complete in 31 schools (89 percent). The ET also confirmed with storekeepers if they had received training on care of the food store, and how many trainings they had received. 80 percent had received training (n=28/35), with a range of 1-9 trainings, with the average being 3.3 sessions. 84. Storekeepers mentioned the following challenges in carrying out their work:  Calculating requirements for hot meals  Storerooms can be far from the school or too small for the food stocks  They need spray to kill pests  Food supplies are sometimes short compared to the number of students  Lack of NFI’s (plates, bowls, utensils etc.) 85. WFP have not measured any indicators related to the cooks training on safe food preparation. The ET collected data on the status of school kitchen facilities and determined that 22 of the 35 were good or acceptable (63 percent) while the remaining 37 percent were either dirty, had no roof or no kitchen at all (i.e. three rocks to cook on). In total the ET Presence of a working water source was measured at baseline and repeated 54 WFP recommends that food be stored off the ground – either on pallets, benches or some other method. 55 Excluding most of the schools in Nimba County 17 by ET. Of 35 schools, 25 had good or acceptable water, 10 schools (29 percent) had no on￾site water source. This despite having an on-site water source being an entry criteria to be part of the SF programme. 86. Equipment support: In addition to the provision of energy efficient stoves mentioned above, the CO has supported the MoE with computers and motorcycles to assist the CCs and DFPs to monitor the programme. WFP also pays 50 percent of the salary of the MoE staff responsible for the programme. Overall, the ET find that WFP’s capacity building trainings have been implemented as planned or in some instances of training well in excess of targets. National policy development has been well supported however, the capacity support has focused on training on SF implementation modalities and not been found to be part of a transition plan to government ownership. As such they are unlikely to be effective in supporting handover to government in the foreseeable future. The ET found very little evidence of any handover or capacity development strategy that is being followed in this regard. Although the CO planned a SABER assessment on the capacity building needs of the government during this phase of programming, this has not been done. The ET find that this would be an important step in understanding the barriers to handover, and enable a clear handover strategy, including a capacity building component to be developed. DFPs need to work closely with WFP to ensure equitable distribution and timely replenishment of NFIs. 2.2.2 SF outputs & other support to schools 87. The SF program is commonly referred to as the largest social safety net in the country.56 At the start of project, WFP provided food for 90,720 beneficiaries (Annex 17, Table 6). WFP supported 234,000 beneficiaries during the 2012 school year under the previous phase and planned to decrease to 127,000 beneficiaries per year for 2013-2016, when commodities arrived. Unfortunately, the arrival of commodities was soon followed by the EVD outbreak and schools were closed. The CO demonstrated flexibility and good management, negotiating a transfer of food commodities to the regional emergency response programme (EMOP 200761) in a timely fashion. In addition to the direct beneficiaries, the SF programme has indirectly assisted more than a quarter of a million people.57 88. Despite the school closures and very limited programming during the 2014/15 school year, the CO reports feeding 118,902 children that year (94 percent of the annual target) (Annex 17, Figure 1). This despite only providing meals on 63 percent of school days and in 594 (60 percent) of target schools to September 2015.58 This raises questions about how the CO is counting their beneficiaries (discussed later in report). 89. In the 2014/15 school year when schools were mostly closed, WFP reported providing only 22 percent of the commodities originally planned. This is understandable given the school closures, but it does not tally with almost reaching their beneficiary targets described in Table 6 and Figure 1, Annex 17. The ET find that the significant discrepancy 56 WFP, MoE documentation, KIIs. 57 Household members (n=5) of girls provided with GTHR. 58 SPR 2015 18 between the low level of commodity distribution while more than doubling their targeted beneficiary number is due to an inappropriate method of beneficiary counting. The CO doesn’t have a standard documented method for beneficiary counting59 and the process used isn’t transparent. 90. Over the full programme, WFP has supplied 15,208 MT of food commodities, half of what was originally planned (Annex 17, Figure 2). Commodity data is not available by school year, as it was delivered for the Country Programme in total, and the SF commodity data had to be manually extracted for the ET. 91. Overall, the CO reports almost reaching the total beneficiary target, and exceeding the target of the number of girls receiving in-school meals (Annex 17, Table 7). The CO has fallen slightly short of its target of the number of boys, and in the total quantity of commodities provided for school meals. The large scale up of SF in 2016 led to the improvements in total beneficiary targets. 92. WFP experienced no major pipeline breaks during the programme, in fact the scale-up in 2016 was largely done to utilize excess food before the expiration date. The ET did not find significant reports of food spoilage. Although some schools mentioned delays in food delivery, this did not appear to be a significant issue. The most significant food issue found by the ET relates to food loss. These are described in more detail in Section 2.3, regarding the efficiency of the operation. 93. The ET found that instead of providing meals each school day as per agreement,60 the schools visited provided meals, on average 4.3 days per week. The reasons provided by the schools for not providing a meal five days a week mainly included cooks being sick or absent for a wide variety of reasons including them not getting paid, routinely not cooking on Fridays, or no firewood at school. Prior to the civil war (pre-2003), cooks were paid by the MoE, and received a certain status. Currently, they are not paid or compensated for their work by WFP or MoE, and are considered volunteers. Most school communities have worked fees to provide some compensation to cooks and to purchase firewood. To increase economic or cultural incentives (or decrease disincentives) 94. In addition to the in-school meal, WFP supported the provision of GTHRs to 8,052 girls (Annex 17, Table 10). This is well short of the original 18,588 planned as this component of the project was not scaled up during 2016. The GTHR was not scaled up because the gender disparity was decreasing. This is confirmed by the 2016 SPR, which states that gender disparity was no longer an issue in many of the targeted schools. Establishment of school gardens 95. In addition to the food assistance provided by USDA/MGD, WFP has supported targeted schools to establish school gardens by providing seeds and tools. Over the course of this programme, WFP have supported the establishment of 285 gardens (Annex 17, Table 9).61 The gardens were intended to provide vegetables to add to the hot school meal, and provide an opportunity for agricultural education. 59 CO acknowledge the need to document the method used for institutional memory in WFP Guidance for Accurate and Consistent Beneficiary Counting. (received 9/6/17) 60 Twenty days per month 61 By November 2013, 100 school gardens had been initiated. The group 4H were then contracted in to work with MoA from June 2013-Jan 31, 2014 to increase the number of gardens. The contract was not renewed over the school closure period. 19 96. The ET saw 13 gardens in the 35 schools (37 percent) visited, and determined that all gardens were in good or acceptable condition, although it was noted that four of the gardens had only a small number of plants. FGD participants explained that it was not yet the right time for planting, however a visit with the MoA in Monrovia indicated that it was actually, the best time to plant. WFP has not monitored the use of the school gardens, so the ET asked the school cooks if they used food from the garden in the school meals. Five (out of 13) responded that they did not. The most common ingredients were (in descending order) potato leaves, cassava leaves and pepper, with one response each for potato, cassava, okra, watergrass, corn, beans and bitter balls. Overall, the ET has identified some critical questions about the CO’s method of beneficiary counting for the provision of school meals. The ET find it highly unlikely that the CO managed to reach 94 percent of their beneficiary targets with only 41 percent of the planned tonnage of food. The CO doesn’t have a transparent documented method for beneficiary counting. Schools are not feeding each school day. The ET found an average figure of 4.3 school meals are provided each week. Fees are also charged by the school community to cover associated costs of SF. In addition, the ET found that the CO has been unable to meet their GTHR targets because of the school closures, and this gender sensitive activity doesn’t appear to have been considered as an option for scale up. The ET found limited evidence that the WFP supported gardens are being utilized by the schools. 2.2.3 Promotion of basic education and develop human capital To increase student enrolment 97. A key aim of the WFP-MGD program is to increase primary school enrolment and attendance, particularly of girls. The WFP baseline found 41 percent of children in schools’ catchment area to be enrolled, slightly lower than the 44 percent NER.62 However, neither the CO nor the MoE have monitored this indicator over the course of the programme. 98. National statistics on girl’s enrolment show a steady increase from 2007 until 2011 when a little over 300,000 students were attending primary school (Annex 9). To continue to promote attendance of girls in the upper primary grades girls are given take home rations. 99. To evaluate if the SF programme has increased student enrolment, the ET obtained enrolment data during their school visits and from the DEO and CEO offices for two school years 2015-2016 and 2016-2017. The ET found that primary school enrolment increased by 19 percent for both girls and boys over this two year period (Annex 17, Figure 3). However, it is difficult to attribute this increase solely to the SF programme although it is likely to be the largest factor contributing to the increase. Other factors include more students returning to school after the Ebola crisis, and to a small extent, population growth. This increase exceeds the WFP targets of 11 percent, and 12 percent increase for boys and girls respectively (Annex 17, Table 11). 62 UNICEF 2010/2011 20 100. To further investigate enrolment rates, the ET compared enrolment statistics for all the WFP SF schools, with the schools visited in Nimba County, where SF ceased in December 2016. Table 11 shows that schools in Nimba did not have increasing enrolment rates as noted in Figure 4, rather their enrolment rates dropped (despite having school feeding at that time). Even when Nimba is taken out of the equation from all schools, it lowers the percent increase in enrolment. The question is whether the fact that SF stopped in December 2016 impacted the enrolment rate for the 2016-2017 school year. The inference is yes, as there were no other known notable factors or events in Nimba that would have impacted enrolment rates so significantly. 101. The Nimba case points to a miscalculation by WFP of needs and/or an over ambitious scale-up. Had fewer schools in Nimba been involved in the scale-up, all schools could have had food until February 2017 like the other 8 counties. 102. The ET found that schools often inflate their enrolment figures, as verified by WFP in discussions with sub-offices, Monrovia. This is understandable given that the school budgets for primary education are determined by the number of pupils enrolled. Schools then provide these same numbers for WFP to receive food commodities. WFP and DFP hold joint headcounts annually with schools to determine attendance numbers and adjust the commodity figures either upwards or downwards. To improve student attendance 103. The baseline survey found that 66.9 percent of all students (66.4 percent of boys and 67.9 percent of girls) attend school at least 90 percent of the regularly scheduled school year. The programme set attendance targets of 90 percent (93 percent for boys, 87 percent for girls). However, due to the multi-year disruption of schooling, no assessment of attendance was made by WFP since the baseline. 104. As a result, the ET collected attendance data and compared the figures with a physical headcount in two grades (Grades 2 and 4) (Annex 17, Table 12). Findings indicate significant discrepancies between headcount and attendance data of 6 percent for girls and 16 percent for boys. The highest gaps were in Nimba County - 34 percent boys and 30 percent girls in Grade 4. 105. The ET concludes that either attendance records are regularly overinflated as with enrolment rates, possibly to receive more funding from MoE and/or food from WFP, or many children leave the school during the day for whatever reason, boys in greater percentage than girls. It is also important to note that school feeding in Nimba had stopped in December 2016 and so there is less incentive for children to stay in school for the whole school day. 106. All key informants agree that the presence of school meals helps to get children into school and helps to keep them there, increasing both enrolment and attendance and reducing absenteeism. To reduce hunger and improve attentiveness 107. The MGD performance results include reducing short-term hunger, and improving attentiveness. However, the CO has not included either of these in the baseline survey, nor have they been tracked during the programme therefore the ET cannot evaluate whether the school meals are resulting in these changes. 108. Relevant global literature indicates that students concentrate better in class when they are not hungry, and that food helps people concentrate and improve their cognitive 21 functioning.63 However, this is only possible when meals are provided early in the school day. A midday meal, as provided in Liberia will not help concentration in morning classes, but might be an incentive for children to stay in school to get the meal. 109. To try and obtain some data on these indicators, the ET conducted FGDs with teachers and administrators in each school visited and asked, “what do you see as the impact of school meals on students?” Some of the most common unprompted responses included: reduces hunger, helps them to concentrate, perform better, and they enjoy school. Some teachers mentioned that students attending the afternoon session sometimes leave school after receiving the in-school meal, missing some afternoon classes. Interviews with teachers and PTA members frequently made mention that children are hungry and less attentive when they do not have meals. To improve gender parity in education 110. In the 2010/11 school year, 47 percent of girls enrolled for lower primary, and 45 percent for upper primary.64 However, in 2014 the EVD crisis set back the entire school system impacting most school statistic collection and MoE goals for education. Table 13, Annex 17 shows that in January 2013, the gender parity ratio was 0.85, 0.84 by the programme baseline. By 2016 the figure had decreased substantially, to 0.47. This can only partially be explained by the lower than planned distribution of GTHRs: 35 percent of target due to the EVD crisis. 111. The randomized sample of schools visited by the ET yielded two schools providing the GTHR.65 This made it impossible to compare gender ratios in those schools with non￾GTHR schools. Although both the visited schools reported that the GTHRs diminished the gaps between boys and girls, there is insufficient data to validate the finding. Reports were received of GTHR being phased out in some urban areas approaching gender parity such as Buchanan; and in some cases, girls even exceeded boys in attendance. 112. It was clear to the ET that the SF programme was having positive results in terms of girl’s enrolment and girls being retained in school. This was due, at least in part, to a wish to receive the GTHR in the upper primary grades. The ET also found some negative gender results of the SF programme. Anecdotal reports66 were received that if school cooks are absent from work, girl students may be asked to leave class and cook the school meal. Girl students cooking meals is against WFP policy and students should not be distracted from learning. To reduce health related absences 113. This indicator is difficult to track, requiring interviews with students, alongside tracking daily attendance rates with teachers. The ET found that many teachers do not record attendance in books, some teachers were in possession of attendance books but many gave the evaluation team records written out on a sheet of paper. According to WFP, there were initial discussions with the MoE and the Liberia Teacher Training Programme as to how to meaningfully track this indicator. However, measurement was interrupted by EVD and no baseline figure was captured. 63 Including Department of Health and Human Services (2014) Health and academic achievement. National Centre for Chronic Disease Prevention and Health Promotion. Division of Population Health, CDC, USA; and Adolphus, K., Lawton, C. & Dye, L (2013) The effects of breakfast on behaviour and academic performance in children and adolescents. Front Hum Neurosci.. 2013; 7: 425. 64 Education for All 2015 National Review Report: Liberia (May 2015) http://unesdoc.unesco.org/images/0023/002332/233218e.pdf 65 This is expected as it is a small element of the programme 66 One report in FGD with girls plus noted in some monitoring reports. 22 To improve student knowledge on nutrition, and health and hygiene practices 114. WFP planned to provide students with training on nutrition, and on health and hygiene practices. However, the school closures meant that they have been unable to reach their targets (Table 14, Annex 17), and monitoring of whether the training resulted in improvement of student knowledge has not been measured. Overall, the ET finds it difficult to ascertain whether the SF programme has resulted in increased participation in education.  The ET found that schools regularly inflate their enrolment and attendance figures, so these indicators are unreliable indicators of success. In addition, it is difficult to attribute changes to the SF programme, although the programme is likely to contribute to changes. However, all consulted stakeholders consider that SF increases school attendance. The ET find that head counts should be done more regularly, at least once a term, as students sometimes migrate from one school to another  The ET also find that providing the in-school meal at midday will only improve attentiveness, behaviour and concentration for afternoon classes. However, this indicator is difficult to measure.  Training for students has not been conducted as planned. No monitoring of whether training resulted in any improvement in knowledge. 2.2.4 Community engagement results To increase engagement of local organizations and community groups 115. WFP has engaged the local community in a few ways during this programme: via the established Parent Teacher Associations (PTAs) and by forming Food Management Committees (FMCs). WFP also planned to conduct awareness raising campaigns on the importance of education. 116. PTAs: The baseline survey shows that 92.4 percent of schools had a functional PTA at commencement (Table 15, Annex 17) and that about one third of parents (29 percent) were part of the PTA. WFP set a target to increase that to 75 percent however this indicator was not monitored. The ET interviewed PTAs in each school but did not collect this specific indicator due to time and measuring constraints. 117. WFP supports PTA involvement in school activities but the programme end target was for 75 percent of parental engagement (Table 15). It remains unclear why the final target was lower than the baseline. During school closure, PTA activity was not tracked, but by 2016, 85 percent of schools again had active PTAs. 118. It was clear to the ET that the support provided to the SF programme through the activities of the PTAs are critical to the running of the SF programs. During the 2016 scale-up, the PTA proved invaluable by providing condiments, firewood, encouragement to the children to attend school and monitoring of the programme. PTA members also monitored the food arriving at the school stores, and counting the days when children were not fed. PTA members support the school by raising funds, organizing the building of classrooms and administration buildings, repairing schools, building fences, contributing to the care of gardens and provision of hot meal condiments, assisting cooks, bringing firewood, and encouraging children to go to school. The ET found that in the sampled schools, men slightly outnumbered women on the PTAs but the ET observed that women actively partook in FGDs. Only one member of a school PTA would receive the annual WFP training on the SF program. 23 119. In addition, WFP planned to provide training on community mobilization, and to conduct two public outreach events each year on household health. However, WFP has not reached their target on these indicators and the number of public events were never monitored (Table 15). The ET is however, aware of community outreach activities on hand washing and household health practices that were held through the Emergency Operation (EMOP200761) during the time of the EVD outbreak. 120. Establishing FMCs: WFP established FMCs in schools within the PTA. These committees comprise teachers, storekeepers and parents, and were responsible for managing the SF programme. Most members of the FMC were the same active persons from the PTAs and to a large extent, it appears that their functions overlap. The ET found that most FMCs were newly formed during the 2016 scale-up. This is one of the reasons why WFP has not been able to meet its target for training on commodity management, as more than 80 percent of those trained were only trained in 2016. To increase community understanding on the benefits of education 121. Over the three-year implementation period WFP planned to conduct three community awareness campaigns on the benefits of education. Only one was done (first quarter 2014). However, during FGDs with women, it was frequently mentioned that women felt secure sending their children to school where they knew they would get educated as well as receive food, freeing them up to go off to work. Overall, the ET find the WFP’s community engagement activities have largely not been implemented as planned. The PTAs in sampled schools were very active in schools and represented a key resource. The CO has established FMCs in schools since the scale-up in 2016, and they have similar membership as the long established PTAs. The PTA and FMC have overlapping roles in SF. Other community activities such as raising awareness of the importance of education, and training the PTAs in community mobilization have commenced but the CO has not reached their targets in these activities. 2.2.5 Analysis of internal and external factors affecting the results Internal 122. Partnership with MoE: The SF programme is run jointly by MoE and WFP, with MoE staff benefitting from WFP support at all levels if involved in SF. The JPAs are established each year between MoE-WFP however the ET found that the written responsibilities of MoE don’t reflect what happens on the ground. 123. The positions of CC and DFP were introduced by the MoE in 2011 specifically for the SF Programme.67 The DFPs have direct contact with schools and are responsible for programme monitoring and support. DFPs are supported by WFP in their operational districts. Neither the CC nor the DFP roles are dedicated titles within the MoE and the roles are taken by MoE staff – teachers, administrators, who are re-assigned to DFP or CC status. As such, inconsistent programme funding may lead to staff being redeployed by the MoE. 67 There were also regional coordinators position created but they were later dropped. 24 124. DFPs are located in WFP sub offices when possible as county facilities are poor, others remain within County and District MoE offices when there is no WFP sub-office. Whilst hosting offers the potential of on-the-job training, it also compromises the roles of the CEOs and enhances the image of a ‘donor driven’ programme, rather than one being handed over to the government. 125. Currently the WFP incentives to MoE staff expires in June 2017 and this is likely to affect68 the capacity building aspects of the programme. Although WFP has worked to improve and maintain the effectiveness of the SF and build the skills of the MoE there remains limited capacity of the MoE to manage, monitor and report on SF as described earlier. 126. MoE programme monitoring: The CCs and DFPs are largely responsible for the SF programme monitoring. However, due to the scale of this function and the limited time capacity of the role, WFP is still providing significant ongoing support, including writing MoE reports. WFP staff reported varied levels of proficiency of the MoE staff and that most reporting is undertaken by WFP staff. 127. The main area of monitoring where the MoE seems to be taking action is when cases of diversion are brought to their attention. The SF CoCR outlines that six-consequence actions should be taken, and these appear to be done, although it is difficult to ascertain how consistently. CEO and DEO officers demonstrated letters of warning to principals and school administrators, as did DFP’s and WFP Monrovia and sub-offices. The ET were told stories of diversion and how principals had to pay back the food or the school excluded from the programme. 128. WFP programme monitoring: The ET note the dedication and determination of the M&E unit to improve the effectiveness of the programme monitoring. The team worked hard to provide required information to the ET but at times struggled against poor institutional memory and poor record-keeping. Reports are not systematically compiled and it is acknowledged that the M&E system is not fully compliant with WFP corporate M&E reporting. One example came when the ET asked to see the waybills for one school while in WFP’s sub-office in Grand Gedeh. Neither the WFP sub-office personnel nor the DFP could provide any waybills or receipts for food deliveries.69 The ET found that either this was a phantom school or the filing system was not in order. The ET got the impression that the M&E systems adopted had been largely personality driven rather than because of corporate guidance. 129. Historically the CO’s M&E Unit has had limited human resources and funding70 and this remains the case. WFP has insufficient human resources to undertake adequate monitoring of its SF programme and relies on the MoE CCs and DFPs.71 WFP was only able to track about half the MGD indicators in the initial grant agreement and several of them are beyond the scope of both WFP and MoE. The beneficiary feedback system (phone-based) planned for the next school year (2017/18) will potentially add value. However additional WFP M&E support is required, particularly at district and community levels. 68 The last food distribution was February 2017 69 Six persons, including the ET, looked through DFP and WFP records without finding Tuglor School food receipts (waybills) for the last food delivery, although they were on the list to receive food. 70 WFP Liberia Mid-year 2016 M&E report 71 JPAs 2014, 2015 & 2016 25 130. Pre-existing monitoring challenges were exacerbated during the 2016 scale-up72. For example, in Grand Gedeh there were 89 schools after scale up73 (41 before) and three DFP’s who were required to carry out monthly visits in each school. For this to be done, 30 schools would have to be visited in 30 days of the month, to cover the county. When the ET visited Boetown, a two-hour drive from Zwedru, it was noted that the last visit from the DFP was seven months prior. In the same school the team noted that the Storekeeper struggled with calculations of rations and would have benefited from refresher technical assistance by either a DFP or WFP visit after the school year had started. In another school visited by the ET, the DFP had to step in to correct the GTHR distribution at a scale-up school, as only 10 percent of the oil ration was being provided. 131. In addition, to extract only the SF beneficiary numbers from within the overall Country Programme beneficiary numbers was challenging and had to be done manually by M&E staff. The new COMET system, introduced in 2016, should be able to produce the component data more readily. Likewise, SPRING, another innovation that produced the 2016 SPR, also seems advantageous. 132. The MGD logframe & indicators: The MGD performance indicators include many that rely on MoE data such as student attentiveness, changes in literacy rates, and changes to enrolment and attendance rates. Data collection therefore necessitates MoE active involvement in programme monitoring. 133. Furthermore, the ET found the output indicators do not capture the breadth of the programme and contain no indicators to track the use of school gardens, capacity development or advocacy/system strengthening activities. Similarly, the outcome indicators are all educational indicators, which fail to capture other aspects of the programme such as the important safety net function of the programme, and the contribution it makes to household food security. 134. Method of beneficiary counting: The ET found several flaws in the CO’s methodology of beneficiary counting, resulting in very significant over-representation of beneficiaries. On investigation, no documented method of beneficiary counting was found but was described as follows. The CO takes the highest monthly count from the 9 counties over the course of any one year, and this figure is reported as the annual total number of beneficiaries. This method can sometimes be accurate, but it can also either over- or under-represent the true picture. For example:  WFP feed 10 children each month for 12 months, the composite count fed = 10.  WFP feed 10 children for only one month in a year, the composite count fed = 10.  WFP feed 120 children, 10 different children each month, the composite count fed = 10 135. The ET believe that in the case of the SF programme, this method has significantly over￾represented the beneficiary numbers achieved. As previously noted, the ET found that the CO reported almost reaching their beneficiary targets in 2014 school year. This is clearly not possible, given that schools were mostly closed. The ET therefore finds that although this method might be suited to a fluctuating population it is not appropriate in a 72 Nimba had 150 schools prior to scale up 345 after scale up in 2016. 73 WFP Monrovia office had a file with different numbers for scale up, namely 76 for Grand Gedeh. Numbers also differed for Nimba County. Field offices had more up-to-date data. This made a difference on the team’s random calculations. 26 stable situation such as Liberia. Some graphic examples of the differences in beneficiary counts can be found in Annex 10. 136. PTAs: PTAs are critical entities that add tremendous value to the SF program. PTAs have shown strong commitment to school performance. In FGDs, they outlined activities they were involved in to support the schools evidenced by them building classrooms/warehouses and supporting volunteer teachers with additional PTA fees, maintaining school gardens, and encouraging school attendance. 137. Relationships with other development actors: The ET find that the CO currently under-utilizes the potential synergies and opportunities for partnership, coordination or collaboration with other development actors.74 Relationships with NGOs and with other UN agencies such as UNICEF and FAO are limited. The MoE clearly needs additional support and the ET find that WFP has lost an opportunity to support complementary activities by other development actors. External 138. EVD outbreak: The main external factor that has influenced the results of the programme was the EVD outbreak. The EVD outbreak resulted in necessary but prolonged school closures. WFP was left with a surplus of SF food commodities and were flexible enough to utilize the stock in other programmes. When schools re-opened, WFP scaled up their SF programme to encourage children to return to school, and to use excess food items. Programme monitoring was greatly affected during the outbreak, with many indicators unable to be monitored. 139. Road infrastructure: Liberia has 66,000 miles of roads75, and of these less than 7 percent are paved. This is a considerable logistical constraint for food delivery, and on monitoring of school feeding among other things. Even with motorcycles for monitoring, road and bridge conditions can make it impossible to get to schools during the rainy season. Roads can be in a very poor condition and particularly so in the rainy season when they can be unpassable. The ET came across one or two case reports of when the path/road to the school site was very bad and the school authorities and/or PTAs agreed to receive the food at another location. Foodstuffs were then transported on foot/bike by the community to the school sites. In these cases, a proportion of food items are used as payment for its transfer which also contributes to shortage in ration size. In one instance, a fee of 4 out of 10 sacks of food was reported. Key findings and conclusions Evaluation Question 2: How effective is the operation? Overall, the ET has found it difficult to determine the effectiveness of the SF programme, due to insufficient outcome monitoring. WFP’s capacity building activities have been implemented as planned. However, the planned activities do not appear effective enough to support a transition to a fully government owned and implemented programme in the foreseeable future. The CO’s method of beneficiary counting for the provision of school meals significantly overestimates the outputs concerning the provision of the in-school 74 CO reported MoE as sole partner in Liberia. The ET had to persevere to have discussions with UNICEF/FAO who reported limited engagement. Field mission didn’t yield any joint activities/training with UNICEF. 75 https://blog.usaid.gov/2014/05/liberias-road-miles-and-miles-to-recovery/ 27 meals. There is no documented beneficiary counting methodology. The ET was also unable to ascertain whether the provision of school meals contributed to an increase in enrolment and attendance because of the lack of reliable programme monitoring data in this regard. The provision of GTHR appears to be effective at creating an incentive for girls to enrol in school, but needs to be available to earlier grades, particularly in rural areas, to effectively prevent drop out. The ET found little evidence that WFP supported gardens were being used effectively, either as a means of increasing school meal diversity, or as a tool for nutrition or agricultural education. WFP has put significant time into training school personnel in SF implementation but the ET find the WFP’s community engagement activities have largely not been implemented as planned. Internal factors that influence the programme include the JPAs which don’t appear to be enforced, the high work load of DFPs and the beneficiary counting method that inflates output figures. External factors include the EVD outbreak that closed schools, poor road infrastructure and limited high level political support including donor funding. Lastly, the ET finds that WFP has under-utilized potential synergies and collaborations with other development actors. WFP could improve their coordination, and strategize with other agencies to optimize the support provided to the MoE for the SF programme. This might include shared monitoring visits with FAO or UNICEF, shared trainings on nutrition/hygiene, and water and sanitation support. 2.3 Evaluation Question 3: How efficient has the operation been? 140. In addition to the performance indicators agreed upon by USDA and WFP described above, the ET collected information to evaluate the functioning and efficiency of the programme. 141. Cost efficiency of the operation: The SF programme is implemented with a budget of USD$20 million from USDA and the Government of Liberia provides a contribution determined on an annual basis to cover 50 percent of the staff salaries, 50 percent of the fuel and mobile phone charge costs. WFP also supports 50 percent of the salaries of the key MoE SF personnel. The contribution of the MoE was 60 percent of the support costs in 2013 school year, decreasing to approximately 19 percent during the 2015 and 2016 school years (Table 18, Annex 17). The contribution of the MoE is low and despite the JPA they generally only cover salaries, leaving WFP to pay the fuel, phone and other costs. 142. In-school meals vs. GTHR: The hot meals have higher costs compared with the GTHR due to associated transport, storage and processing. It has been estimated76 that these costs account for about 30 percent of total program costs. The GTHR, when appropriately targeted, has been highly effective contributing to gender parity some locations and as such offers higher levels of cost-effectiveness. Furthermore, the GTHR is that it is not constrained by the amount of food a single child would usually eat in a single 76 Grosh, M., del Ninno, C., Tesliuo, E. and Ouerghi, A. (2008) For Protection and Promotion: The design and implementation of Safety Nets, World Bank, Washington D.C 28 sitting and so it can provide a longer and larger food transfer for the entire family and motivation. 143. Food losses: WFP provides a quantity of food to each school based on enrolment and attendance numbers. WFP bases school meals on overall calories, protein and fat content per child per meal. However, the ET found several sources of food loss/leakage, some of which have a relatively minor effect on the programme, and some with more significant effects. 144. The more significant losses result in a reduced quantity of food being provided to children each meal, mainly due to additional people eating the meal or food not being provided each day. Additional food is also prepared on a regular basis because the cook is not aware of the daily attendance figures when she starts the cooking process. (Table 19, Annex 17). In addition, the ET learned of multiple cases of diversion77 by school personnel, that the MoE had uncovered during monitoring visits. The scale of the diversion and food loss is unknown to the ET. 145. Food procurement: WFP can provide food for both the in-school meal and the GTHR at much lower prices than could be achieved by purchase in the local markets. While WFP has not conducted a market study, the ET found that some families said the cost of a school meal if purchased would be between 25-50LDs per day (US$0.25-0.50). For families with three or more children, this could mean a saving of 150 USD or more over the school year. This savings figure would increase if GTHR values were included. 146. SF fees for parents: The school meal is not typically a free service to children. The school communities, including the PTAs, establish a fee to cover expenses such as provision of fire wood,78 condiments and spices and critically payments to the cooks. In one school, PTA groups explained that parents do not like to send their children out to collect wood but prefer to buy it. The fee structure varies between schools but 10LD/meal (US$0.10) was often cited. 147. Payment for the cooks: Currently, the role of school cook is voluntary. The lack of regular, predictable payments for the school cooks plays a part in reducing the number of meals provided in schools each week. Prior to the war, school cooks were on the payroll of the MoE and this conferred both income and status. Ensuring the cooks receive a fair and regular payment again would improve the implementation of the programme, helping to ensure that meals are cooked daily, and prevent students preparing meals. 148. School infrastructure and equipment: The ET found numerous examples of minor inefficiencies that when combined make the in-school meal preparation much less efficient than needed. For example, several schools do not have functioning weighing scales, making accurate use of the required ration per child impossible. In addition, the pulses need to be soaked for several hours before being cooked (i.e. before the actual attendance is known). This usually means an overuse of pulses compared to the ration. The absence of a common measure for the pupils’ rations and the fact that children use food containers of different sizes also means that rations distributed are not equal. Overall, the ET find that the preparation and distribution of the in-school meal could be significantly improved. 77 Theft 78 Reports received that parents prefer to pay than to send children out to collect firewood. 29 149. Programme monitoring: A significant contributor to poor efficiency of the programme is the lack of adequate programme monitoring. Without adequate monitoring, schools are finding ways to make the programme more efficient at individual school level, which does not always result in overall efficiency, and at times, results in food losses as described above. 150. Aside from the food losses mentioned above, the ET found multiple examples of modifications made by school communities. Each of these reduced the effectiveness of the programme but increased the efficiency to the school. These include but are not limited to:  Not cooking each school day  The distribution of only 10 percent of oil in the GTHR,  Sharing of food with afternoon pupils/siblings,  Feeding of staff,  A wide range of charges for the meals and  The using of girl pupils as cooks. 151. Food deliveries: The overall efficiency of the programme can be compromised by the ‘last mile’ delivery of food to school sites. Despite agreements with haulage contractors being in place, the ET found some instances of SF commodities not being transported the ‘last mile’ to school when roads are in poor condition. This can result in schools having to organize collection of food from another location with the cost usually paid from the food commodities, reducing the rations for the children. Improved monitoring of delivery, perhaps by PTAs, can ensure that transport agreements are honored. WFP has acted, when aware of the situation, by enforcing established contractor agreements. Key findings and conclusions Evaluation Question 3: How efficient has the operation been? Overall, the ET finds that, for the most part, the SF programme is delivered efficiently. The MoE financial contribution has fallen over the course of the grant and in 2017 was 18.8 percent. The roles and responsibilities outlined in JPAs – frequency of monitoring, reporting, payments for fuel - have not been honoured for a variety of reasons. Insufficient monitoring of schools has enabled them to make modifications to SF including not feeding each school day, introducing fees to cover associated costs of the programme including compensation for school cooks, to purchase wood for fuel and additional spices. The ET identified multiple sources of food loss/leakage, including food not being provided each day, food being shared with older children, diversion by school personnel, food used as payment for ‘last mile’ delivery when roads are bad. GTHR offers higher levels of cost-effectiveness than the hot meal and has the added advantage of the quantity of food provided is not constrained by the amount a child can eat in one sitting. FGDs revealed that SF offers financial savings for parent in excess of the fees charged for the meal. 30 2.4 Evaluation Question 4: What are the impacts of the programme? 152. As with effectiveness, the ET find that the impacts of the programme are difficult to evaluate due to the short time frame being further shortened by the EVD outbreak; the lack of impact (and outcome) level data and no specific impacts are stated in the CP200395. However, SF falls under the WFP SO4 Reduce chronic hunger and undernutrition. SF contributes to a reduction short term hunger and provides cost￾savings for parents which in the longer term may contribute to improved food security. 153. The ET finds that the SF programme clearly plays a significant role in encouraging parents to enroll their children in school and to send them to school regularly. This is shown by an improvement in the average net enrolment rates over 2015 and 201679 and is consistently reported from parents, teachers, and other stakeholders. Eventually, this should have a long-term effect on children’s education if the quality of teaching is adequate. 154. Ultimately, the goal of MGD in providing food assistance to the schools is to improve literacy rates. However, literacy rates were not included in the baseline survey or any programme monitoring since, and the MoE does not collect this information. It is therefore not possible for the ET to assess, nor the contribution of the SF makes to it. Likewise, there is no measurement of promotion rates, or school completion rates which might demonstrate a positive impact on human capital. 155. WFP has made significant efforts to train school administrators, teachers, other school personnel and PTA members in a range of topics. However, there is limited evidence of data collection to assess whether training has resulted in improved knowledge, behaviour changes or new skills. This severely limits the evaluability of the impact of training. 156. The partnership between WFP and the Government of Liberia appears to be one of donor and recipient, rather than a true partnership following an agreed timetable plan to transition the SF programme to one fully funded and implemented by the Government. Indeed, there is no framework transition, the MoE reports not seeing a transition plan and the GoL financial contribution has significantly fallen since 2013. 157. The ET finds that it is likely that the SF programme contributes to improving the food security of both the children and their households. This is especially true of the households of the girls receiving the GTHR. FGDs revealed that PTA’s felt that school feeding was an important source of food for the families and that the children go to school and often get their first meal of the day there. As such, the tailored nutrient content of the meals provided by WFP, combined with the high levels of food security and poverty identified by the VAM Unit, the ET feels that these results are likely. 158. The meal also provides a value transfer to the household in terms of money saved by them not having to provide the in-school meal at home. The GTHR contributes as well, as the household receives a direct, in-kind food transfer. However, there have been no household food security surveys over the course of the programme so it is not possible to definitively assess the impact of the GTHR or the in-school ration on household food security. Through FGDs with parents, the ET found that over the course of a school year, the savings to each household from the in-school meals would be as much as 9000LD per 79 2015, 2016 the only years with data. Figure 4, Annex 17. 31 child per school year, resulting in a significant annual savings on food for a family of 45,000 LD (about 450USD).80 159. In addition, the presence of the in-school meal, allows parents, especially mothers, to know their children are safe and fed at school, freeing them up to spend their days on other activities including income production rather than food preparation. However, there is no quantitative evidence to this effect. 160. The ET identified two potentially negative impacts of the programme: the ‘pull factor’ of SF increasing class sizes, and thereby lowering the quality of the teaching environment, and the loss of income security of the cooks. The long hours that cooks spend at the schools in a voluntary position means they lose time to look for other employment opportunities. This ultimately affects their household income and food security. Key findings and conclusions Evaluation Question 4: What are the impacts of the programme? The ET find that the impacts of the programme are difficult to evaluate due to the short time frame, lack of trend data and lack of impact indicators. The USDA/MGD FFE aims to improve literacy however no data is available on this indicator in Liberia. Likewise, there is no measurement of promotion rates, or school completion rates to demonstrate a positive impact on human capital. However, all evidence suggests that school feeding increases school enrolment and attendance. In addition, the fieldwork suggests that SF increases enrolment and offers an important economic transfer to families and as such it is likely that the SF contributes positively to household food security, particularly for the families of girls receiving GTHR. The partnership between WFP and the Government of Liberia appears to be one of donor and recipient, rather than partners following an agreed timetable for handover. 2.5 Evaluation Question 5: How sustainable is the operation? 161. To answer this evaluation question, the ET investigated factors relating to sustainability of the SF program and the steps taken towards transition to national ownership. 162. To what extent is Liberia taking ownership of the programme? Some milestones have been achieved in supporting the Government to take ownership of the programme. Under the previous MGD grant (2009-2012), the School Feeding Policy and the School Feeding CoCR were developed. Under the current grant, WFP has supported the formal adoption and roll out of the policy and the operationalization of the CoCR. The policy clearly states that “School Feeding is a vital safety net for the most vulnerable and food insecure households accounting for a third of national social protection spending”. However, its importance is not reflected in the government’s development plans and budgets. 163. The ET note that there is a good relationship between WFP and MoE, characterized by mutual respect and trust. WFP has implemented an approach of working alongside the MoE and this capacity development/implementation model is appreciated by the MoE. However, this only points to a sustainability if there is an agreed capacity development 80 For an average family size 5 as per the calculations for GTHR indirect beneficiaries 32 strategy and a shared understanding of a transition plan for the government to take over the programme. This is not currently the case. 164. The roles of the MoE in managing, monitoring and reporting on the SF programme have long been established, at both national and county levels, and are clearly articulated in the annual JPAs. However, implementation of all the MoE’s tasks has not occurred. SF is not currently part of the County Superintendents Development Plan and is not budgeted for at county level. MoE informant interviews stated that there is no line item for school feeding in the MoE budget and the MoE financial contribution to SF has fallen to 19 percent during school years 2015 and 2016 (Table 18). 165. In 2011, the DFP role was established and the MoE assigned 27 staff to support the CC in school feeding. The introduction of the DFP role, has contributed to improved monitoring of the programme by the MoE. The 2016 JPA shows that WFP supports 24 DFPs, 9 CCs and 3 Regional Coordinators on the SF programme; and a Director, Data Analyst, Cashier and 2 drivers in the Monrovia School Feeding Unit. Whilst the MoE covers half of the salary of CCs and DFPs, these posts do not have designated roles with functional titles and are technically still in their original posts (typically as teachers or administrative assistants). As such they can be repositioned at any time. WFP works with the DFPs to establish school monitoring schedules but reaching those monitoring targets has been a challenge due to weak county-level support (finances and transport), staff turnover and insufficient staff. 166. Evaluation interviews point to slow handover and limited ownership by the MoE. The reasons are multiple: lack of high level political commitment81, the complete change-over of MoE personnel82 in 2016, and no budget line for school feeding. The government lacks any plan to run the SF programme should donor-funding cease. Some stakeholders noted that the incumbent Minister, in post since April 2016, is not open to taking on SF responsibilities and he was not available to speak with the ET. 167. Although WFP has conducted many capacity building trainings for schools, CEOs, CCs, DEOs, DFPs, there has been significant staff turnover since 2013 including the entire staff of School Feeding at the MoE being replaced in June 2016. In addition, the Minister for Education has changed twice over the programme period. For schools, this has been a major challenge, with many WFP-trained Principals/Registrars moving on and/or being absent during the evaluation mission and leaving a gap in SF programme knowledge. The ET found that ‘good’ SF schools tended to have long standing staff members, based in the community, who could also be trained up in SF modalities. 168. The WFP SF team report an increase in the quality of reporting since 2016 but that many challenges remain. To date training has focused on the processes and modalities of SF but needs remain at all levels for technical analysis. 169. Capacity building, incentives for school feeding CCs and DFPs, policies developments, coordination with other education state actors are some of the collaborations that enable transition towards government ownership. However, to date, SF has not yet been embedded in the national financing framework, nor planned for future budgeting processes. The ET also found that SF is typically not part of County Superintendents Development Plans, and not budgeted for nor mentioned in co-ordination meetings. SF is firmly within the sole remit of the School Feeding Unit CCs and DFPs. 81 KIIs 82 Outlined above 33 170. The ET noted that in Nimba county, where SF had stopped 3 months ago, some private and church schools had continued feeding pupils using resources from the local school community. They were able to garner support from the communities or churches to carry on feeding, bring resources (food, condiments, wood, labour). 171. However, political interest was noted in the Home-Grown School Meals Programme (HGSMP) which is part of the MoA ‘Global Agriculture and Food Security Program’ (GAFSP). 172. Championing transition of SF within the MoE: The MoE, along with other ministries, are in a state of change as the Government of Liberia is mid-way through a decentralization programme. WFP has been successful in advocating for the movement of the School Feeding Unit from the Bureau of Administration to the Bureau of Instruction in 2015 which brought the DFPs and CCs under line management of the respective CEO.83 173. Overall, the ET find that WFP has made some efforts to establish a system strengthening approach of the MoE, via its work and training of MoE staff at central and county levels. But the ET found varying levels of engagement from MoE staff at national and county levels of government. Interest and ownership of the programme is seen principally in WFP supported MoE staff. 174. Capacity building activities: The annual JPAs indicate that WFP provided support that improves “institutional arrangements and reinforce Government capacity by providing support to MoE’s decentralization drive through institutional and individual capacity development assistance at central, county, district, school and community levels”. To accomplish this, WFP has conducts capacity building activities, however there is no documented action plan of how or when the handover of the programme to the MoE will occur. The MoE also stated they had not seen any plans for transition to national ownership. WFP have many activities focusing on individual training but not set within a transition plan. More discussion on the capacity development activities and findings can be found in Section 2.2.1 ahead 175. What is the national readiness to implement the programme? Overall, the ET has concerns about the sustainability of the current SF programme without a coherent capacity development strategy. Key informant interviews indicated that the MoE sees the SF programme as ‘donor driven’ and that the programme is ‘almost closed’. However, capacity development of this type is a longer-term commitment of 7-10 years and variable ‘political will’ will be an ongoing part of the process. The ET found that there is no handover strategy or documented plan for the MoE but on a positive note MoE requested continued working together and production of a transition plan. Political buy-in remains pivotal and it is of note that the next Presidential election is in October 2017. 176. The ET note that the CO has been planning a SABER assessment for some time84 which should determine the potential barriers to government handover and help develop a roadmap for government ownership. 177. WFP has planned to implement its National Capacity Index85 or similar but has yet to do so. Table 1 (Annex 17) presents the achievement of WFP’s national capacity development actions against the MGD output indicators and planned WFP outputs. 83 KIIs 84 SPRs 2015 & 2016 85 National Capacity Index (NCI) is a composite index developed by WFP to measure the change in capacity because of investment in hunger governance. See WFP (2014) National Capacity index (NCi) – Measuring Change in Capacity for 34 Box 1: The Pilot Home Grown School Meals Programme (HGSMP) One promising aspect of the SF programme is the HGSMP being piloted by WFP in 12 schools in Nimba County. This is done in partnership with the MoA and supported by the MoE. The ET visited one school at the request of the CO, during which children stated that they preferred the HGSMP meal to the traditional in-school meal because it more closely mirrored the foods they eat at home. Each day there is a different meal with specific calculations based on protein, energy and fat content. Much of the non-staple food came from an extensive school garden. In other countries, WFP engages in HGSMPs86 to increase the sustainability of school feeding programmes, by ensuring government ownership and procuring food from local markets to supporting local farm systems. It focuses on linking school feeding with local small-scale farmer production by creating an ongoing market for small landholders. The United Nations 2005 World Summit recommended “the expansion of local school meal programmes, using home-grown foods where possible” as one of the “quick impact initiatives” to achieve the MDGs, especially for rural areas facing the dual challenge of high chronic malnutrition and low agricultural productivity. 87 WFP plans to scale-up the HGSMP depending on future evaluations of its relative success in Liberia. Among the many challenges of enlarging the program is the sophistication of its running and management (more complicated daily ration calculations, ordering and ensuring delivery of foods from local farms or coops) and the establishing of multiple small - medium scale farmer markets. Development of markets to replace the present scale of SF could take many years. Poor road infrastructure continues to be a major challenge. A HGSMP has been met with great enthusiasm from all sources interviewed - the schools (there is already a positive spread effect of knowledge of the program), the MoA, MoE, WFP, potential donors and international organizations. The ET note that WFP is currently seeking funding for another phase of the HGSMP pilot, under the GAFSP.88 The funding proposal was launched in January 2017. Key findings and conclusions Evaluation Question 5: How sustainable is the operation? There are concerns over the sustainability of the current SF programme. Although some milestones have been achieved such as the development and adoption of the School Feeding Policy (2013) and the School Feeding CoCR (2013), the Government faces many challenges towards full ownership and implementation of the programme. On paper the MoE is responsible for the management, monitoring and reporting for the SF however in reality not all tasks are being implemented putting additional burden on hunger governance in support of projects to strengthen national capacity to end hunger. Complementary Guidelines Series #2. Country Capacity Strengthening Unit. 86 WFP (circa 2008) Home-Grown School Feeding: A Framework to Link School Feeding with Local Agricultural Production 87 World Summit Outcome, 2005; UN Millennium Project, 2005a 88 Global Agriculture and Food Security Program (GAFSP), Multi-sectorial agriculture project: Linking agriculture, nutrition and education through an integrated Home-Grown School Feeding (HGSF). January 2017. 35 WFP. This has not been acknowledged by decision makers and the tasks are simply restated in the subsequent JPA. WFP and MoE have not developed a clear and agreed operational framework aimed at transitioning to full government ownership and implementation. There is no line item for school feeding in the national MoE budget, and SF appears poorly represented in County Superintendents Development Plans. High MoE staff turnover has been challenging to the programme, with WFP’s capacity development efforts resulting in little change, as trained staff regularly leave the programme. In Nimba County some private and church schools have set up their own school feeding. The pilot HGSMP has been met with positive acceptance by schools and students, and has shown success when implemented in other countries where WFP operates. WFP plans to scale-up the model in 2017 but faces hurdles of maintaining political will, complicated ration calculations, poor road conditions, and need to scale up farm production of foods. The HGSMP is led by the MoA, and supported by the MoE. 36 3 Conclusions and recommendations 178. Based on the findings presented in the previous section, an overall assessment that responds to the evaluation questions is provided below. How relevant/appropriate is the operation? 179. Overall, the ET finds that the SF programme is highly relevant to the Liberian context. The objectives align well with the national policy framework and with key WFP policies and strategies. Geographic and school selection criteria were clear, however, better adherence to the school selection criteria might have improved the 2016 scale-up. 180. WFP can enhance their partnering by working with other agencies to co-ordinate activities of shared interest. There are opportunities for shared trainings, monitoring visits and learning when working with local communities and the MoE. 181. The ET finds that the choice of modalities employed (in-school meal and GTHR) and the size of rations for both, to be appropriate for the context and culture. The programme has also been implemented with appropriate sensitivity to gender considerations. 182. The ET also finds that the system strengthening approach at national, county and district levels is highly appropriate in Liberia. Ownership of SF by MoE is fragile and ongoing support is required to embed SF in national and the county development plans. 183. The ET finds that it was appropriate for the CO to scale-up their programme post-EVD outbreak to encourage children to return to school. Still, the scale-up could have been better planned and implemented to avoid the inclusion of private schools and so all the scaled-up schools received commodities for the same period. After scale up Nimba County had the highest number of schools despite being previously dropped from the programme in 2015 due to high levels of food security and stronger government capacity. The new schools received food for 9 months only before being dropped from the programme in December 2016. Overall, the scale-up appears to be more food driven than need driven, as the CO was looking to ensure that excess food stocks were used up before the ‘best-before dates’. Other options for the use of the commodities do not appear to have been considered. How effective is the operation? 184. The implementation of the SF programme was significantly affected by the outbreak of EVD in July 2014. Schools were closed for almost an entire school year, and programme monitoring was restricted. Despite this challenge, the CO has managed to implement many of the planned activities and reach many of its programme targets. 185. Overall, the ET find that some elements of the programme have been effective, while others need some more strategic thought. For example, many of the CO’s capacity development activities have been implemented as planned, but the activities have not been effective in supporting a transition to a fully government owned and implemented programme in the foreseeable future. The ET found very little evidence of any handover or capacity development strategy that is being followed in this regard. Likewise, although the CO has been planning to conduct a SABER assessment for some years, this has not yet been done. This is an important step in understanding the barriers to handover, and to enable a clear operational framework, capacity development strategy and handover strategy to be developed. 186. The ET has identified some critical questions about the CO’s method of beneficiary counting for the provision of school meals that make it difficult to ascertain an accurate 37 picture of the implementation of the programme. There is also no correlation between the number of beneficiaries served and the volume of commodities provided. The CO reports meeting 94 percent of its beneficiary targets overall, including 94 percent of its 2014/15 beneficiary target despite schools being closed for almost that entire school year, and with only 41 percent of the planned commodities distributed. The ET doesn’t find this credible. 187. Over the course of the programme there has been a variance in number of schools served, with each school not necessarily receiving continuous service. The ET find that discontinuous service provision undermines the effectiveness and the capacity building efforts of the programme. 188. The ET has found it difficult to conclusively determine whether the SF programme has resulted in increased participation in education. The ET found that schools regularly inflate their enrolment and attendance figures, so these indicators are unreliable measures of success. In addition, while all consulted stakeholders consider that the SF programme increases school attendance, it is difficult to say what proportion of the increase can be attributed to the SF programme, although the ET agrees that it does contribute. The ET finds that the GTHR has been effective in contributing to improved gender parity in primary education by creating an incentive for girls to enroll in school. There has not been any data collection regarding literacy, so it is not possible to evaluate the role of SF in achieving that outcome. 189. Likewise, for many of the other elements of the programme, the ET has found it difficult to evaluate the effectiveness, largely because of the lack of outcome monitoring. The ET found multiple examples of activities being conducted as planned, but little evidence of outcomes achieved from them. For example, school gardens have been established as planned, but few schools are using the gardens to conduct agricultural education, or using the produce to improve the school meals. Similarly, the CO has completed multiple training activities for school administrators, teachers, and FMC members, but aside from schools storing their food off the ground, there has been no measurement of whether the training has resulted in improved knowledge, new skills or any other change. 190. The ET found that providing the in-school meals at midday is effective at keeping the children in school, but ineffective for improving student attentiveness as per the stated outcomes of the programme, except for the few primary school students with afternoon classes. 191. Several internal and external factors have contributed to WFP achieving the results of this programme. Most significant is the partnership with the MoE, the EVD outbreak and the insufficient programme monitoring. The ET also find that if WFP developed collaborations with other development and community actors for additional programme elements such as WASH, agriculture, or health related activities, it might improve the overall effectiveness of the programme. How efficient is the operation? 192. As with effectiveness, the ET finds that there are some elements of the SF programme that work efficiently, while other elements need to be more systematic to be efficient. The SF programme is implemented with a budget of USD$20 million from USDA and the Government of Liberia provides a contribution determined on an annual basis to cover 50 percent of the staff salaries, and 50 percent of the fuel and mobile phone charge costs. WFP supports the remaining 50 percent of the salaries of the key MoE SF personnel. In the last two school years, the contribution to the SF by the MoE has been 19 percent to 38 the total costs accrued by covering the salaries. Despite the JPA’s, the MoE has left WFP to pay the fuel, phone and other costs. 193. The ET find that the commodity management and logistics systems of WFP has been efficient. The WFP logistics system resulted in provision of food to schools in a timely manner and a strong pipeline was maintained throughout the programme, with a good record for delivery. The CO also provided strong management and flexibility during the EVD outbreak, negotiating a transfer of food commodities to the emergency response programme in a timely fashion. However, the ET find that there has been little monitoring of the endpoint of food deliveries, either by WFP or by the MoE, which has resulted in transporters not always delivering commodities directly to the schools despite written agreements. This had forced schools to pay additional costs for the last part of the journey, and this had often been paid in food commodities, reducing the overall efficiency of the food delivery. WFP acknowledges this and has acted – by enforcing established contractor agreement. 194. SF programme coverage within counties is low at 20-30 percent which decreases the cost-effectiveness via increased logistics. This also creates a stronger “pull factor’ to SF schools, which has the potential to increase class sizes and reduce the quality of education. Low coverage per county also makes it harder to build capacity, knowledge and ownership by the County Education Office. 195. Management and monitoring by both WFP and MoE is irregular, although it is better in urban than rural areas. The lack of monitoring means that schools are not systematically implementing the programme as per agreements. This has resulted in schools taking steps to increase the efficiency of the programme including introducing fees to cover associated costs of the programme involving compensation for school cooks, purchase of wood for fuel and additional spices. It might therefore be prudent to consider a form of ‘results based management’ in the next round of JPA. What are the impacts of the operation? 196. The ET find that the impacts of the programme are difficult to evaluate, again partly because of the insufficient outcome monitoring by WFP and MoE and partly because of the significant disruption of the EVD outbreak on the programme. 197. Although there is a large capacity development component within the programme, the impact of this is limited due to the excessive staff turnover in 2016. In addition, the capacity development component lacks strategic direction, accountability, monitoring or clear milestones. 198. Ultimately, the goal of MGD in providing food assistance to the schools is to improve literacy rates. However, literacy rates were not included in the baseline survey or any programme monitoring since, and the MoE does not collect this information. It is therefore not possible for the ET to establish whether there has been any change in primary school literacy rates over the course of the programme, nor the contribution of the SF programme to that change. Likewise, there is no measurement of promotion rates, or school completion rates which might demonstrate a positive impact on human capital. 199. The ET found that the SF programme is an important attraction to increase school enrolment and attendance, and it offers an economic transfer to families in the form of household savings. In addition, it is likely that the SF programme results contribute positively to household food security, particularly for the families of girls receiving GTHR. However, none of these outcomes are monitored. How sustainable is the operation? 39 200. Overall, the ET has concerns about the sustainability of the current SF programme. Although some milestones have been achieved in supporting the Government to take ownership,89 the Government faces many challenges towards full ownership and implementation of the programme. The ET felt that the partnership between WFP and the Ministry of Education is closer to being one of donor and recipient, rather than partners with a goal of transitioning the SF programme to one fully funded and implemented by the Government. 201. The pilot HGSMP has been met with positive acceptance by schools and students, and has shown success when implemented in other countries where WFP operates. The pilot HGSMP has similarities with the WFP model90 for sustainable school feeding but faces similar hurdles of maintaining political will, complicated ration calculations, poor road conditions, and need to scale up farm production of foods. 202. One of the MoE’s greatest constraints is their budget, which has no line item for school feeding, and SF appears poorly represented in County Superintendents Development Plans. In the last two school years, the contribution of the MoE to the SF implementation has been less than 20 percent, with little motivation to increase the value. This despite the roles of the MoE in supporting the SF programme being established at both national and county levels and clearly articulated in the annual JPAs. 203. In addition, implementation of all the MoE’s assigned tasks have not occurred even though they are re-stated year on year, and there are no accountability mechanisms in place to address this. This is further hampered by the lack of a clear and agreed operational framework aimed at transitioning over to a government owned and implemented programme. These problems have been exacerbated by a high MoE staff turnover, which has been a challenge to the programme. This has undermined WFP’s capacity development efforts resulting in little change, as trained staff leave the programme. 3.1 Lessons learned and good practices 204. The ET would like to recognize the good practice utilized by the CO management for their flexibility and management of SF food commodities during the EVD outbreak. Good communication and timely action between the CO and USDA/MGD enabled the commodities to be transferred to the new EVD EMOP, while the SF programme was effectively on hold. This allowed for a timely provision of commodities to the EMOP from the warehouses – which were full - and prevented SF stock from expiring. It is of note that these actions occurred at a time of great uncertainty and personal risk and when the infectious diseases trajectory was unknown. 205. The ET also commend the USAID FFP programme for the prompt reimbursement of the loaned stock which enabled the SF programme to start-up in a timely manner when schools were officially re-opened. In future, the ET feel that the CO could benefit from similar exchanges when needing to utilize excess commodities, by redirecting them to other programmes, then being reimbursed when programme implementation requires it. 89 Including the development and adoption of the School Feeding Policy (2013) and the School Feeding CoCR (2013). 90 WFP (circa 2008) Home-Grown School Feeding: A Framework to Link School Feeding with Local Agricultural Production 40 3.2 Recommendations 206. Based on the findings and conclusions of this evaluation, the recommendations of the evaluation team are outlined below. The recommendations are listed in priority order under two categories: strategic and operational recommendations. Each recommendation outlines who is responsible for acting, as well as the recommended timeline. 207. The ET is aware that although this phase of SF programming is ending in July 2017, proposals for further funding have been developed and submitted. The recommendations therefore relate to future SF programming in Liberia. Strategic Recommendations  Immediate start (Start within 6 months of completing this programme cycle) RECOMMENDATION 1: In collaboration with the MoE the CO should, with support from the RB as necessary, develop a clear capacity development strategy to support the handover of the SF programme to the Government of Liberia. 208. The ET recommend that the first step in the development of the strategy is to undertake a comprehensive SABER assessment to identify the key barriers and opportunities for handover. The Capacity Development Strategy should then include appropriate activities to address the identified issues. The Strategy should also ensure that actions are given appropriate timeframes, with responsible entities named, and clear accountability frameworks developed. 209. Further, the ET find that the Capacity Development Strategy should include national, county, and school level capacity development activities.  National Level: Due consideration should be given to the inclusion of high-level advocacy, participatory technical studies, information-sharing mechanisms, stakeholder consultations and workshops, joint assessments and inter-agency coordination. The School Feeding Unit at the MoE should be fully engaged with this process and participate in the process of developing clear and concrete accountability mechanisms, including annual milestones.  County Level: Advocate that SF is included in the County Superintendents Development Plans, and ensure that appropriate county level personnel have specific roles in the programme, and are trained as such.  School Level: Engagement of project implementers (teachers, administrators) in capacity building activities at the school level to strengthen record-keeping and filing practices. Provide the necessary tools to do so (e.g. ledger books, booklets). 210. The final strategy should be regularly monitored and reviewed. The ET recommend using a comprehensive indicator such as the National Capacity Index or similar, to measure changes in government capacity. RECOMMENDATION 2: In collaboration with the MoE the CO, with support from the RB as necessary, should develop a clear operational framework and timeline for the transition of the SF programme to the Government of Liberia. 41 211. To implement effective capacity development activities, the CO and MoE must agree as to the timeline of the programme handover to the MoE, and develop milestones to the achievement of the goal within that timeline. Currently, the capacity building activities focus on training, rather than on high level advocacy to enhance and maintain the political will and any other systemic requirements to implement SF. Advocacy at all levels for government, and inclusion of SF in its national and county development plans will be critical to a successful transition to a government owned programme. 212. The ET therefore recommend that a clear operational framework is developed alongside the capacity development strategy. Both documents should clearly state the goal of handover and the steps required to achieve it. The ET also recommends the following to be done during the development of the framework:  The CO should review the policy & legal framework and associated strategies regarding education and SF in Liberia to assess the enabling environment; this should also consider the role of civil society.  Within the operational framework, due consideration should be given to the inclusion of partnerships with national and regional stakeholders to help ensure that ownership of these initiatives is in the hands of institutional partners, and the government.  The operational framework should include regular reviews (at least quarterly) against milestones, and updates to the JPA’s to ensure that they reflect the situation on the ground.  The JPAs should also include activities at the organizational level – these could be development of a transition/handover plan; information sharing mechanisms; joint assessment of MoE implementation capacity; introduction of results based management; joint evaluations – to complement the institutional policy support and field level training activities. RECOMMENDATION 3: The CO, with support from RB as necessary, should review their capacity to implement the new SF Operational Framework and the new Capacity Development Strategy. 213. In parallel to the development of the framework and strategy mentioned above, the ET recommend the CO review its own human resource and capacity development needs. This should include an appraisal of whether the CO currently has the skills and profiles to undertake the necessary capacity development of the MoE, and implement the required community development activities. 214. The CO should also ensure that there is sufficient monitoring capacity to enable WFP and MoE to adhere to donor requirements for monitoring of program performance indicators, not only outputs but also programme outcomes. 215. The capacity review process should enable the development of staff profiles – including the ‘soft’ skills for capacity development - needed within WFP CO. These profiles should then be used for the recruitment of new staff and the training up of current staff. These skills include negotiation, communication, facilitation, information synthesis, coaching and engagement skills (for national capacity building), as well as community engagement and mobilization skills.  Immediate start, with a view to new collaborations during FY 2018 RECOMMENDATION 4: The CO, with support from RB as necessary, should 42 strengthen collaborations and develop new partnerships with other development and community actors. 216. The ET recommend that the CO expand their partnerships for the SF programme and look for opportunities to coordinate and collaborate with other development and community actors to add value to SF activities. 217. Partnerships91 should focus on maximizing synergies with other UN agencies (e.g. UNICEF for equipment, training and WASH, FAO with school gardens, WHO for health). The CO can use these partnerships to conduct joint sensitization, monitoring, training and other activities with the schools and community on the relevant shared issues of their programs. 218. The CO should also look to the model of private and church schools who are able to garner support from the communities or churches to carry on feeding, bring resources (food, condiments, wood, labour), and bring these lessons learned to future SF programming with the aim to reduce donor dependency and encourage schools (community and public) to increase community participation.  During the next phase of SF programming Operational Recommendations RECOMMENDATION 5: The CO to consider expansion of the GTHR to all primary school grades in areas where there is a high gender disparity in primary school enrolment. 219. GTHR is a small element of the programme averaging around 10 percent of the beneficiaries. The ET found the GTHR to be effective, more flexible and to offer more cost efficiencies than SF. It is also apparent that girls commonly dropped out of schooling before Grade 4, so before reaching programme eligibility. This action would necessarily double the size of female beneficiaries to the benefit of programme gender sensitivity. RECOMMENDATION 6: The RB must provide ongoing support to the CO to develop a more transparent M&E system and ensure that monitoring and evaluation activities are appropriate, systematic and as per donor agreements. 220. The ET recommend that the RB provide the CO support to develop a more transparent M&E system for the next phase of programme. Support from the RB should include the following:  Review the CO’s methodology used for beneficiary counting to ensure reports reflect actual beneficiary numbers and tally with the commodities distributed. The ET recommend that RB provide support on how this is done in other countries, with similar contexts.  Support the CO to develop a more robust M&E system including documenting methods used & their rationale, systematic reporting and filing. Reporting should be systematic and transparent. RECOMMENDATION 7: The CO should consider increasing the coverage of 91 Global Humanitarian Principles of Partnership – equity, transparency, results-orientated approach, responsibility and complementarity/mutuality. 43 the programme within counties to improve multiple efficiencies including capacity building. 221. County level coverage of schools receiving SF is low. This increases logistic costs, monitoring efforts, and reduces the ability of SF staff in neighboring districts to build a community of practice. The low coverage also means that the schools implementing the SF programme create a significant “pull factor” leading to larger class sizes, and potentially reducing educational outcomes due to lower quality of teaching. 222. Increasing county level coverage rates would minimise these negative effects and improve efficiencies for logistics, training needs and enable information sharing, knowledge and ownership by the County Education Office. This would involve reducing the number of counties or districts served by WFP. RECOMMENDATION 8: The CO, with support from the RB as necessary, and in consultation with the MoE, should strengthen monitoring in the field. 223. The ET have identified several areas for improvement of the SF programme monitoring. The ET recommend that the CO and the MoE review their JPAs to include the following monitoring activities:  The CO to continue to advocate and request MoE to provide funds at the county level to support programme monitoring.  The CO and MoE to increase joint WFP-DFP monitoring visits to strengthen capacity of DFPs. These monitoring visits should be used for capacity strengthening of the PTAs/FMCs. This recommendation may also require the number of WFP monitors to be increased, to at least one per county, to support the MoE to carry out their monitoring duties.  The CO and MoE should consider using the PTAs as additional programme monitoring entities and enable them to report or contact WFP directly if needed. PTAs are in a good position, and motivated, to regularly monitor school activities, such as the number of days’ hot meals are provided, check the delivery of food stocks to the schools, view ration calculation and daily food preparation.  The CO should consider making commodity delivery conditional upon reconciled records (both SFP and School records).  The CO to ensure the new phone-based beneficiary feedback system is anonymous and that all information is recorded, actioned appropriately with gender sensitivity and in a timely manner.  The CO need to agree monitoring roles with DFPs/PTAs to ensure that food is delivered directly to the schools to enhance effectiveness. WFP to regularly monitor and continue to enforce agreements with transporters. RECOMMENDATION 9: The CO, with support from the RB as necessary, and in consultation with the MoE, should take steps to improve the provision of the in-school meal 224. This evaluation has highlighted several inefficiencies in the implementation of the in￾school meal, and the ET recommend that the processes be reviewed and revised during the next phase of programming to ensure that schools provide school meals, every school day. 225. The ET recommend the CO and the MoE work together to ensure the following are done on a regular basis throughout the next phase of programming: 44  The CO should ensure that multiple people receive SF training in each school to avert future staff turnover crisis. Including long standing school staff and the PTA would help.  Provide technical assistance to storekeepers to improve the accuracy of the ration calculations. This should include some visual aids to ensure correct rations are prepared.  Ensure cooks are provided with adequate financial or other incentives for their work.