Decentralized Evaluation Quality Assurance System (DEQAS) Decentralized Evaluation Final evaluation of the World Food Program USDA/McGovern-Dole International Food for Education and Child Nutrition Program (FFE 615- 2013/041/00) in Kenya 2014-2016 Evaluation Report October 2017 Commissioned by the WFP Kenya Country Office Evaluation Managers: Jennifer Nyanjui and Beatrice Mwongela Evaluation Team: Analysis and reporting (Independent Consultants) Sophia Dunn, Team Leader John Otsola, Research Specialist Design and data collection (TNS) Stephen Aloo, Team Leader & Director, East & Southern Africa, Kantar Public Bernard Nyauchi, Project Manager & Research Manager Imelda Ochari, Assistant Project Manager & Research Executive Decentralized evaluation for evidence-based decision making ii Acknowledgements This evaluation report was developed through wide consultation and collation of data from different partners and stakeholder of the School Meals Program. The evaluators would like to acknowledge and appreciate the contributions provided by the WFP Kenya Team, the Government of Kenya through the Ministry of Education, the county government officials, and other stakeholders for their contribution towards the evaluation. Most importantly, the evaluation team would like to acknowledge all the participants in the evaluation process – households, teachers, cooks, and other key informants. Without your input, this evaluation would not have been possible. Disclaimer The opinions expressed in this report are those of the Evaluation Team, and do not necessarily reflect those of the World Food Programme. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov iii Table of Contents 1 Introduction ................................................................................................ 1 1.1 Overview of the Evaluation Subject........................................................ 1 1.2 Context..................................................................................................4 1.3 Summary of findings from the Mid-Term SMP Evaluation..................... 7 1.4 Evaluation Methodology & Limitations..................................................8 2 Evaluation Findings....................................................................................11 2.1 Evaluation Question 1: How appropriate is the SMP? ...........................11 2.1.1 Appropriateness to needs..................................................................................... 11 2.1.2 Alignment with national policies and strategies ................................................12 2.1.3 Alignment with WFP corporate strategies, policies and normative guidance ...13 2.1.4 Alignment with other development partners .....................................................14 2.1.5 Summary of findings – Evaluation Question 1 ...................................................15 2.2 Evaluation Question 2: What are the results of the School Meals Program?...................................................................................................... 16 2.2.1 Thematic area: School performance....................................................................16 2.2.2 Thematic area: Provision of school meals ...........................................................21 2.2.3 Thematic area: Participation in education ......................................................... 27 2.2.4 Thematic area: Food utilization and food safety.................................................31 2.2.5 Thematic area: Increased national capacity....................................................... 36 2.2.6 Thematic area: Efficiency of program delivery ...................................................41 2.2.7 Summary of findings – Evaluation Question 2 .................................................. 44 2.3 Evaluation Question 3: How and why did the School Meals Program produce the results? ..................................................................................... 47 2.3.1 Internal issues affecting the implementation of SMP....................................... 47 2.3.2 External issues affecting the implementation of SMP ......................................48 2.3.3 Summary of findings – Evaluation Question 3 .................................................. 50 2.4 Evaluation Question 4: How sustainable is the School Meals Program? 50 2.4.1 National Government Commitment................................................................... 50 2.4.2 National Government Contribution ....................................................................51 2.4.3 Summary of findings – Evaluation Question 4.................................................. 52 3 Conclusions ............................................................................................... 53 4 Recommendations.....................................................................................56 5 Bibliography ..............................................................................................59 6 Annexes .....................................................................................................62 iv List of Annexes Annex 1: Evaluation Terms of Reference 62 Annex 2: Counties where the USDA-supported SMP is implemented 72 Annex 3: UWEZO 2014 report - school performance rankings for the targeted SMP targeted counties 73 Annex 4: Recommendations of the SMP Mid-Term Evaluation 74 Annex 5: Evaluation criteria 77 Annex 6: Evaluation Matrix 78 Annex 7: List of key informants 83 Annex 8: Quantitative data collection tools 90 Annex 9: Qualitative data collection tools 142 Annex 10: List of schools selected for field visits 150 Annex 11: Detailed methodology 153 Annex 12: Secondary data on education outcomes in the SMP counties 158 Annex 13: Household survey findings re commuity needs 160 Annex 14: Other development actors working in the SMP locations 161 Annex 15: Additional UWEZO education data (2015) 162 Annex 16: Additional analysis re attentiveness in class 163 Annex 17: Activity indicator – Number of boys and girls receiving school meals (2014-2016) 164 Annex 18: Additional analysis re parental reporting of meals before and after school 165 Annex 19: Activity indicator – Total quantity of commodities provided to SMP as a result of USDA assistance 166 Annex 20: Parental reporting of whether there were times that meals were not provided at school, by county 167 Annex 21: Additional analysis re school attendance (students) 168 Annex 22: Additional analysis re school enrolment 170 Annex 23: Percentage of households able to name benefits of education, by county 172 Annex 24: Additional analysis re WFP provided training on food preparation and storage practices 173 Annex 25: Additional analysis re food preparation and hygiene 175 Annex 26: Additional data re sanitation (toilet) facilities in surveyed schools 176 Annex 27: Activity indicator – Capacity building: local, regional and national level 177 Annex 28: Additional data re reported security threats by county 178 Annex 29: Additional data re locations of households reporting knowledge of the feedback hotline 179 Annex 30: Updated PMP 180 Annex 31: Additional data - Significance test results 187 v List of Tables Table 1: Summary of recommendations - Mid-Term Evaluation of SMP (2015) 7 Table 2: Summary of key results - Evaluation Question 1 15 Table 3: Literacy and numeracy rates during SMP implementation 17 Table 4: Activity indicators: Promotion of teacher attendance 18 Table 5: Results indicator: More consistent teacher attendance 19 Table 6: Results indicator: Improved attentiveness 20 Table 7: Percent of students in classrooms identified as inattentive by their teachers 20 Table 8: Results indicator: Increased access to food (school feeding) 22 Table 9: Percent of students in target schools consuming daily meals at school by county 22 Table 10: Results indicator: Reduced short term hunger 23 Table 11: Percent of students in target schools who regularly consume a meal before and during the school day (as reported by school children) 24 Table 12: Parents reporting of how children ate lunch if meal not provided at school 26 Table 13: Results indicator: Improved student attendance 27 Table 14: Results indicator: Improved student attendance (completion) 29 Table 15: Net enrolment rates 2014-2016 29 Table 16: Results indicator: Increased student enrolment 30 Table 17: Activity indicator: Raising awareness on the importance of education 31 Table 18: Results indicator: Increased community understanding on the benefits of education 31 Table 19: Results indicator: Increased knowledge of safe food preparation and storage practices 33 Table 20: Results indicator: Increased access to requisite food preparation and storage tools and equipment 33 Table 21: Results indicator: Increased capacity of government institutions 37 Table 22: Results indicator: Improved policy and regulatory framework 37 Table 23: Results indicator: Increased engagement of local organizations and community groups 38 Table 24: Household reporting of monies requested for school meals 38 Table 25: WFP cross cutting indicators - gender 39 Table 26: Security threats identified at WFP SMP sites 39 Table 27: Household reporting of possible ways they can make complaints 40 Table 28: Planned vs. Actual commodities distributed (2015) 42 Table 29: Hypothetical cost comparison of paper versus technology enabled reporting 43 Table 30: Summary of key results - Evaluation Question 2 44 vi List of Figures Figure 1: Percentage of illiterate and innumerate children in the SMP targeted counties (2015) 17 Figure 2: Reasons for pupil’s inattentiveness in class 20 Figure 3: Total number of students receiving school meals 21 Figure 4: Parental reduction in food portions for school children on weekdays compared to weekends 24 Figure 5: Total quantity of commodities provided to SMP as a result of USDA assistance 25 Figure 6: Proportion of school days that SMP Meals were provided 25 Figure 7: Primary School Attendance by County 27 Figure 8: Reasons for school absence, by county 28 Figure 9: Number of trainings provided on food preparation and storage techniques 32 Figure 10: Number of teachers trained in food preparation and storage practices 32 Figure 11: Attributes of the SMP school cooks and kitchens 34 Figure 12: Attributes of SMP food stores 35 Figure 13: Number of HGSMP manuals distributed to MoE by WFP 36 Figure 14: Number of MoE officers benefiting from HGSMP manual distribution 36 Figure 15: Proportion of support to non-food inputs, provided by parents 38 Figure 16: Key informant views on SMP implementation efficiency 41 Figure 17: Flow of SMP food provision 42 Figure 18: Government contributions to SMP (2014-2016) 51 vii List of Acronyms ASAL Arid and semi-arid lands CO (WFP) Country Office CP Country Program DEO District Education Office DEQAS (WFP) Decentralized Evaluation and Quality Assurance Standards DFID (UK) Department of International Development EB Executive Board (WFP) ECD Early Childhood Development EOP End of Project FFE Food for Education FTC Feed the Children Kenya GPE Global Partnership for Education HGSMP Home Grown School Meals Program HQ (WFP) Headquarters IPC Integrated Food Security Classification System KCPE Kenya Certificate of Primary Education KES (KSh) Kenyan Shilling (currency) KESSP Kenya Education Support Sector Program LESS Logistics Execution Support System MDG Millennium Development Goal MGD McGovern-Dole MoE Ministry of Education MoH Ministry of Health MT Metric Tonnes MTR Mid-term Review mVAM Mobile Vulnerability, Analysis and Mapping NESP National Education Sector Program NSNP National Safety Net Program OEV Office of Evaluation (WFP) PCD Partnership for Child Development PTA Parents-Teachers Association QASO Quality Assurance and Standards Officer RB (WFP) Regional Bureau SCDE Sub-County Director of Education SDG Sustainable Development Goal SMC School Management Committee SMP School Meals Program SNV Netherlands Development Organization SO Strategic Objective SOP Standard Operating Procedure SUN The “Scaling up Nutrition” Movement TOR Terms of Reference UK United Kingdom UN United Nations UNDAF United Nations Development Assistance Framework UNICEF United Nations Children’s Fund USD United States Dollar (currency) USDA United States Department of Agriculture WaSH Water, sanitation and hygiene WFP (United Nations) World Food Programme viii Map 1: Map of Kenya showing location of targeted SMP districts Source: WFP VAM Unit, Kenya Acronyms on map: • SMP: School Meals Program • HGSMP: Home Grown School Meals Program (national government) • WFP-CTS: World Food Programme – Cash transfer to schools ix Map 2: Integrated Food Security Phase Classification Map – February 2014 Source: IPC Kenya (2014)1 1 http://www.ipcinfo.org/ipcinfo-countries/ipcinfo-eastern-middle-africa/Kenya x Map 3: Integrated Food Security Phase Classification map and forecast – June 2016 Source: IPC Kenya (2016)2 2 http://www.ipcinfo.org/ipcinfo-countries/ipcinfo-eastern-middle-africa/Kenya xi Executive Summary 1. This report is for the final evaluation of the World Food Programme (WFP) McGovern-Dole International Food for Education and Child Nutrition Programme (FFE 615-2013/041/00) in Kenya. The School Meals Program (SMP) is implemented by WFP in 1,766 schools in ten targeted arid counties and the unplanned settlements of Nairobi, Kenya. The evaluation covers the period from program design phase (2013) to the time of the evaluation (September 2016). The purpose of the evaluation was to assess and report on the performance and results achieved through support from the United States Department of Agriculture (USDA) under the authority of McGovern-Dole Food for Education and Child Nutrition Program. 2. This evaluation was conducted between 8th June and 5th September 2016. It was designed to compare achieved results against the baseline survey (May-June 2014) and a mid-term review (May 2015) and uses the internationally agreed criteria of relevance, effectiveness, efficiency, sustainability and impact. The evaluation focuses on accountability (against intended results) and learning (for the continuance of the school feeding in Kenya). The evaluation used a mixed-methods approach and triangulated information from different methods and sources to enhance the reliability of findings. The evaluation was designed to answer four key questions: • Evaluation Question 1: How appropriate is the School Meals Program? • Evaluation Question 2: What are the results of the SMP? • Evaluation Question 3: How and why did WFP achieve these results? • Evaluation Question 4: How sustainable is the SMP? 3. The main expected users of the evaluation report are WFP, the Government of Kenya, the donors (USDA, Canada/DFATD, Australia, Russia, and private donors), and other stakeholders in decision-making for program implementation and/or design. This includes the WFP Kenya Country Office (CO), Regional Bureau (RB) which provides strategic guidance, program support, and oversight, WFP Headquarters in Rome (HQ) for wider organizational learning and accountability and its Office of Evaluation (OEV) to feed into evaluation syntheses as well as for annual reporting to the Executive Board. The key Government of Kenya line ministries that will be interested in the evaluation findingsare the Ministry of Education, Ministry of Agriculture, Livestock and Fisheries, Ministry of Health, and Treasury, including relevant Ministries at county level. County and Sub￾County Education Officers, School Management Committees are also key, as they are involved in program implementation and policy support. 4. WFP and the Government of Kenya have been supporting school meals in Kenya since 1980.3 Traditionally, this has been done through in-kind food assistance, with food procured largely from outside Kenya. To pursue greater national ownership and sustainability of the program, the Government of Kenya started implementing a national Home-Grown School Meals Program (HGSMP) in 2009. HGSMP is now implemented in semi-arid counties that were previously served by WFP. While the Government and WFP work to gradually expand the coverage of the HGSMP, WFP continues to support children in all public schools in the arid lands and in targeted schools in the informal settlements of Nairobi, where food insecurity continues to be widespread, and education indicators are below the national average. 3 Langinger, N. (2011) School Feeding Programs in Kenya: Transitioning to a Home-grown Approach. Stanford Journal of International Relations. Fall 2011. xii 5. The objectives of the SMP include improving primary student enrolment, attendance, literacy and attentiveness, reducing short term hunger and improving access to food for school children. The project also aims to enhance teacher attendance, spread awareness on the benefits of education among the community, engage local organizations and community groups, increase knowledge about safe food preparation and storage and provide equipment for this purpose. Gender issues have been mainstreamed through the program including an emphasis on the need for gender balance in schools Board of Management, and improving sanitation facilities in schools to provide a more gender￾sensitive school environment that encouraged girls to consistently attend school, even during menstruation. Finally, to ensure sustainability, the objectives include building government capacity and improving the policy and regulatory framework in support of child health and nutrition. Methodology 6. Qualitative key informant interviews were conducted with 160 people with either local, county or national level roles in the program. Key informants included WFP, MoE and other government representatives, donors, national partners, and parents. Field visits were also conducted in 79 randomly selected schools. In each school, the evaluation team completed quantitative surveys with Head Teachers, School Cooks, Storekeepers, parents, and school children. Observations on the quality of school facilities were also conducted in each of the visited schools, including the kitchen, food preparation and storage areas. Key Findings 7. The key findings of the evaluation team are summarised below, structured according to the four evaluation questions. Evaluation question 1: How appropriate is the SMP? 8. The long-term objective of the SMP is to promote universal primary education of socio-economically disadvantaged and nutritionally vulnerable children, especially girls, within the primary schools in targeted areas. The relevance and importance of this objective was confirmed by key informant interviews with multiple Government officials at all levels: national, county and district. At the time of the SMP design, all ten targeted counties were behind the national average education figures – enrolment, attendance and literacy and numeracy, indicating that educational support programs were appropriate. 9. The SMP aligns well with Government of Kenya policies and strategies, with WFP’s own corporate guidance, and aligns well with the work of other development actors. WFP also coordinates with others to ensure that their targeted communities receive complementary assistance that WFP cannot provide. Overall, this EoP evaluation found the design of the SMP to be appropriate to the education, food security, and gendercontext, and coherent to the policy framework of the Government of Kenya as well as WFP corporate guidance. Evaluation Question 2: What are the results of the SMP? 10. The evaluation finds that overall, the SMP has been well implemented and effective at meeting program results and targets. The SMP has been very effective at improving the capacity of the MoE to implement SMP and transition to HGSMP. WFP has contributed to Kenya’s education policy direction, supporting the development of multiple government policies and strategies. WFP has also provided numerous trainings for teachers and MoE officials, and supported the distribution of HGSMP and National School Health Guidelines. xiii 11. The provision of school meals has been implemented as planned, with both targets and actual delivery decreasing annually, as counties transition to the government’s HGSMP. The evaluation found that only half (53.4%) the surveyed children eat a meal at home before coming to school, highlighting the importance of the school meal in reducing short term hunger. Parents were asked if and how the children ate lunch when school meals were not provided. Although many children would go home for lunch and return to school, 30 percent would not eat lunch, and more than ten percent of children (12.5%) would go home for lunch but then not return to school. This indicates the role that the SMP plays not only in food security, but in keeping children in school all day. 12. The presence of SMP has also contributed to improving school enrolment, and has been critical for improving student attentiveness in class. Although inattentiveness has numerous causes, teachers identified hunger as the primary reason. Attendance rates have been high (>80%) since baseline, with no significant change over the course of this period of implementation. All school level key informants reported that when food is not available, student attendance reduces, so there is clearly a link between the SMP and attendance. This makes it even more important that meals are available every school day. The evaluation found that school meals were provided only 65-70% of school days over the course of the program in part due to funding constraints from other (non-USDA) donors and pipeline delays, as well as occasional insufficient firewood and water provision by the communities. This is likely to have reduced potential education outcomes. 13. Despite the role of SMP in improving enrolment, attentiveness of students and keeping them in school all day, the intended school performance outcome – reading comprehension/literacy - has not improved. The evaluation found that literacy is poor throughout Kenya, and that the Government is taking steps to address this. 14. The capacity development support that WFP has provided to train school personnel in appropriate food storage and food preparation practices has been effective, with 82% of schools now having dedicated kitchen facilities and 80% having dedicated food storage facilities. These results are also testament to the effective model of parental engagement employed by the program, as these facilities are supported in part by parental contributions. WFP’s participatory approach has also improved gender equality in SMP management with most schools now having equal representation of men and women on the School Management Committee (SMC). 15. The evaluation found that some counties performed poorly on multiple indicators including school attendance, children eating meals before school, the percentage of days that school meals are provided, attentiveness of students, and the presence of energy efficient stoves. These counties - Baringo, West Pokot, Turkana and Marsabit - will need tailored support depending on their results, to bring them up to the standard of the other counties. Evaluation Question 3: How and why did WFP achieve these results? 16. Despite considerable challenges due to the harsh terrain in the arid counties, the WFP system of delivering food to schools was efficient. WFP has provided high quality food commodities, delivered to schools in a timely manner with no complaints, and minimal loss of food during transportation. 17. A comprehensive monitoring system has been utilized by WFP and the findings and analysis from it has contributed to the efficient implementation of the program, and changes in implementation as required. One area identified for improvement was poor supervision by MoE staff to the more remote parts of some counties. This was identified xiv in multiple areas including Baringo, Samburu, and Garissa counties. The evaluation recognizes that WFP and MoE are working to improve joint monitoring activities so this issue may be resolved soon. 18. Although useful for project monitoring, teachers found the paper based system cumbersome and time consuming, and the evaluation found it to be expensive. The evaluation also found that the components of the program implemented by the Sub￾County Director of Education (SCDE), including delivery of food to schools introduced inefficiencies in terms of delays and extra costs for Head Teachers. Evaluation Question 4: How sustainable is the SMP? 19. Overall, the evaluation found that the partnership between WFP and the Government of Kenya is strong and the agreement for transitioning the SMP to the government’s HGSMP is holding. With ongoing support over the next phase of the program, the government key informants felt that they would be able to take over all the school feeding in the country, resulting in a sustainable program. The three arid counties that have successfully transitioned to the HGSMP during this phase of programming are testament that a full handover of the program will eventually be possible. 20. At a policy level, the National Government’s commitment to the SMP is demonstrated by its numerous policies and strategies that include school feeding. These policies, as well as the transition strategy for the handover of all WFP SMP schools to the government-led HGSMP require that the national government allocate resources for the SMP. Sustainability of the SMP can only be achieved through the Government ensuring allocations commensurate with requirements, increasing year on year to keep up with market prices and increasing student numbers. Overall Conclusion 21. The evaluation has found that overall, the SMP is relevant and coherent with government priorities. There are also many components of the program that have been effectively implemented. Given that there is a handover strategy to the government in place, and holding, the evaluation largely recommends a continuation of the program in its current form so that a full handover to the government will be possible. Recommendations 22. The findings and conclusions of this evaluation led the evaluation team to make the following recommendations: Overall program strategy Recommendation 1: WFP should continue to implement SMP in the ASAL counties while supporting the handover of counties to HGSMP as per the current transition plan. The support recommended for continuation is as follows: • WFP should continue to support the MoE capacity development. This includes supporting schools to understand the requirements for implementing SMP through a cash-based modality. This is coherent with the government’s HGSMP approach. • WFP should continue to collaborate with other development actors such as World Vision to support interventions that contribute to increased school attendance, especially for girls. • WFP should continue to support the MoE to actively seek partnerships for the provision of school infrastructure that WFP currently provides support for, including kitchen facilities, energy efficient stoves, food storage, and water. • WFP should continue to invest in public awareness campaigns to encourage school attendance and improve parents understanding on the benefits of education. This xv should be done together with local authorities, church groups or other well-respected parties. Priority 1: To be actioned within the next six months Recommendation 2: In recognition that WFP Kenya has been successfully implementing mVAM4 in the arid counties, this evaluation recommends that the WFP CO support the MoE to digitize their School Meals reporting system. This would improve the timeliness of data delivery and support improved implementation. Use of electronic reporting that utilizes mobile network for instant uploading of data, in place of the paper based form (SMP 6, 7 and 8) could be piloted in one county and if successful scaled up to cover the entire program. Integration with mVAM monitoring system could also be considered. The evaluation team recommends that WFP pilot digital monitoring in some or all of the SMP locations, either as a stand-alone monitoring strategy or combined with SMS, or paper-based monitoring. Results for the pilot should then be used to identify challenges before upscaling to areas where connectivity, security or other concerns makes it possible. Recommendation 3: As per the Mid-Term Evaluation findings, this evaluation recommends that WFP Kenya and its partners carry out community-level sensitization on the threats to pupils’ safety. This is due to the high level of threats still reported by parents in relation to the commute to school. SFP stakeholders should also increase awareness on these topics during the program implementation. Strategic partnerships with agencies focusing on Child Protection (i.e. UNICEF, Plan International, Save the Children etc.) would be an added value in helping reinforcing synergies and complementarity with the SFP. Recommendation 4: WFP Kenya, with support from the Regional Bureau as needed, should implement additional awareness campaigns on the presence and purpose of programme feedback mechanisms, especially the new feedback hotline. The evaluation found that less than half the surveyed households were aware of any feedback mechanism, and only 14 households mentioned the possibility of using the hotline. Recommendation 5: WFP Kenya should ensure that all cooks and storekeepers in SMP schools provided with training on safe food preparation and appropriate food storage. More work is needed to ensure that all school kitchens have a valid health certificate, and that pest/insect control measures are adequately carried out, as the evaluation found evidence of presence of rodents or presence of insects in almost half the stores. Recommendation 6: WFP Kenya should assess why Baringo, West Pokot, Turkana and Marsabit counties have consistently achieved poorer results on several indicators than other counties. The specific indicators include school attendance, children eating meals before school, the percentage of days that school meals are provided, attentiveness of students, and the presence of energy efficient stoves. Each county has different areas of weakness, so WFP Kenya should ensure that each county receive tailored, additional support on the areas where they are weak, to bring them up to the standard of the other 4 This project was launched in 2013, and uses mobile technology to track food security trends in real time, providing high frequency data that supports humanitarian decision-making. In order to achieve high performance, data collection methods are tailored to the needs of the country in which the project operates. xvi counties, and ensure that they will eventually be able to successfully transition to the HGSMP. Priority 2: To be actioned within the next one year Recommendation 7: WFP Kenya, with support from the Regional Bureau as needed, should conduct a gender and protection assessment to identify and contextualise issues related to school feeding in the ASALs. The assessment should provide recommendations for implementing targeted measures to improve gender parity in school participation as latest figures show that girls enrolment in all the targeted counties is still below 50%, while boys enrolment is 51-63%.5 Priority 2: To be actioned on an ongoing basis until full handover of the SMP to the HGSMP Recommendation 8: Based on the financial challenges of the MoE, this evaluation recommends that WFP Kenya continue to support the Government of Kenya to find solutions to improve the management of financial resources for the SMP. This should include the following: • Helping to identify ways to advocate for, and allocate a sufficient, and annually indexed budget to the HGSMP. The budget should include sufficient funds for procurement, transportation, storage and preparation of school meals each school day. • Helping to identify ways to ensure existing SMP funding is not reallocated, while helping to secure additional funds. • Providing technical support to improve the efficiency of financial allocations to the schools. 5 MoEYS/WFP (2017) School Meals Programme – Verification of school enrolment in arid counties. Report, September 2017. 1 1 Introduction 23. This report is for the final evaluation of the World Food Programme (WFP) Mc Govern-Dole (MGD) International Food for Education and Child Nutrition Programme (FFE 615-2013/041/00) in Kenya. The School Meals Program (SMP) is implemented by WFP in 1,766 schools in ten targeted arid counties and the unplanned settlements of Nairobi, Kenya. The evaluation covers the period from program design phase (2013) to the time of the evaluation (September 2016). The purpose of the evaluation is to assess and report on the performance and results achieved through support from the United States Department of Agriculture (USDA) under the authority of McGovern-Dole Food for Education and Child Nutrition Program. WFP Kenya was awarded a total of US$20.2 million for the period 2013–2015 and more recently, a grant of US$28 million for a further five years (2016-2020). 24. The grant agreement between WFP and MGD/USDA incorporates 12 specific performance indicators and 21 results indicators against which performance of the program is measured. In the evaluation plan agreed between with USDA, WFP commits to conducting a final evaluation to measure performance of the program for accountability and learning purposes. The full Terms of Reference for the evaluation can be found in Annex 1. 25. It should be noted that the SMP is a multi-donor project costing almost US$45.3 million over the 2014-2016 period, and the USDA contribution is 47% of the total funding received. Therefore, although this is the evaluation of the use of USDA-MGD funds, it is difficult to attribute results to specific donor contributions. The report should therefore be seen as the result of multiple donor efforts. Other program donors include the Governments of Australia, Canada, Germany, Japan, and the Russian Federation, as well as private donors. 1.1 Overview of the Evaluation Subject 26. The subject of this evaluation is the School Meals Program (SMP) implemented by WFP in 1,766 schools in ten targeted arid counties of Kenya (Map 1) and the unplanned settlements of Nairobi. Annex 2 provides a table showing the counties and the number of schools SMP is implemented in. WFP planned to provide school meals to 815,000 school children in the first year (2014) reducing annually to 450,000 in 2016 as counties transitioned to the Home-Grown School Meals Programme (HGSMP). 27. The evaluation covers the period from program design phase (2013), to the time of the evaluation (September 2016). The agreement between USDA and WFP was signed in September 2013 but the food commodities (mainly bulgur wheat) only arrived between February–March 2014. Distribution to schools could therefore not commence until Term 2, 2014. The project end date was September 2016 and has been implemented with no changes to the initial agreement. 28. To ensure effective implementation of the project in schools, WFP works in partnership with the Government of Kenya’s Ministry of Education (MoE) both at national and county levels. Other partners include UNICEF, UNDAF thematic working groups, the Education Sector Development Partners, The World Bank, The Kenya Ministry of Agriculture, Livestock and Fisheries, The Ministry of Health; The National Treasury including relevant Ministries at county level, Feed the Children Kenya (FTC), Partnership for Child Development (PCD) and the Netherlands Development Organization (SNV). 2 29. The objectives of the SMP include improving primary student enrolment, attendance, literacy and attentiveness, reducing short term hunger and guaranteeing access to food for school children. The project also aims to enhance teacher attendance, increase community awareness on the benefits of education, engage parents and local businesses in SMP implementation, increase knowledge about safe food preparation and storage and provide equipment for this purpose. Finally, to ensure sustainability, the objectives include building government capacity and improving the policy and regulatory framework to support school feeding, and child health and nutrition in general. 30. WFP provides food commodities as direct in-kind assistance to the targeted schools. Over the evaluation period, three counties have transitioned to the national government HGSMP: Isiolo in 2014, Samburu in 2015, and Tana River in 2016. Transitioning schools to HGSMP involves a period of changeover as schools move from in-kind assistance to cash transfers. WFP supports this process by providing a period of transitional cash transfer. Under the HGSMP, WFP no longer has any role in the provision of school meals. Food assistance is provided through a direct cash transfer from the national government to each school. Schools are then responsible for the tendering, procurement and storage of food commodities and preparation of the school meals. 31. In total, the SMP has utilized a total of 52,794 MT of food from various donors over the 2014-2016 period, with USDA providing 19,260 MT (36.5% of total). The food commodities provided by MGD/WFP only partly cover the requirements for providing a hot lunch to primary school children each school day, other donors make up the difference.6 In addition, other donors supported pre-primary children to receive a daily ration of 40 grams per child per day of Corn Soy Blend (CSB) as a mid-morning snack until January 2016. WFP’s provision of CSB was then discontinued in line with the transition strategy where county governments are responsible for pre-schools and the national government for primary school children. The meals are designed to provide 30 percent of the recommended daily energy intake. • Mid-morning snack: 40 grams of SuperCereal (corn soya blend with micronutrients added), prepared as porridge • Lunch: 150 grams of cereals, 40 grams of pulses and 5 grams of vegetable oil. WFP uses funds from other donors to also provide 3 grams of iodized salt per child in all targeted schools. 32. Given the remoteness and long distances to the schools, WFP and the MoE pre￾position the food in government warehouses at Sub-County Director of Education (SCDE) level during the school holidays to ensure its availability when the schools open. The SCDEs then moves food to schools based on enrolment figures, number of feeding days and ration sizes. Head teachers and members of the School Board of Managements (BOMs) receive the food at the schools and ensure it is appropriately stored. 33. The SMP requires all targeted primary schools to have store rooms, kitchens and cooking utensils. The School Meals Committees (SMCs) are responsible for mobilizing parents to contribute payment to provide a salary for the cooks, build appropriate kitchen and storage facilities, provide firewood, water, and cooking and provide serving utensils. Although the MGD support does not provide financial resources for food preparation or storage facilities, WFP is able to support the efforts of the MoE and SMCs to improve 6 Other donors include the Governments of Australia, Canada, Germany, Japan, and the Russian Federation, as well as private donors. 3 their infrastructure though other complementary funding mechanisms. In addition to the provision of food for the school meals, and support to kitchen and storage infrastructure, WFP implements activities under the following areas: • Community sensitization on the importance and benefits of education • Promote teacher attendance • Training on safe food preparation and appropriate food storage practices • Capacity building activities for the MoE to ensure they have the capacity to implement the HGSMP in the arid counties. WFP also supports national policy development on school feeding and related topics. 34. Gender issues have been mainstreamed through the program. Specific design features of the program include an emphasis on the need for gender balance in schools Board of Management, and improving sanitation facilities in schools to provide a more gender-sensitive school environment that encourages girls to consistently attend school, including during menstruation. WFP also trained all project committee members on the transition to HGSMP, and consistently collect sex disaggregated data during monitoring. 35. It should be noted that one of the MGD objectives is to improve literacy in primary school pupils. The evaluation recognizes that the SMP is not the only factor that affects this outcome, as education outcomes fall largely under the responsibility of the MoE. However, activities implemented under SMP are designed to directly contribute to improved attendance and student attentiveness. As a result, achievement of WFP’s intermediate results contributes to improved literacy. The evaluation has relied on secondary data – particularly Twaweza’s Uwezo7 literacy and numeracy reports, to determine if literacy performance is changing. 36. The main expected users of the evaluation report are WFP, the Government of Kenya, the donors (USDA, the Governments of Australia, Canada, Germany, Japan, and the Russian Federation, as well as private donors), and other stakeholders in decision￾making for program implementation and/or design. For WFP, this includes the Kenya Country Office (CO), the Regional Bureau (RB) which provides strategic guidance, program support, and oversight, WFP Headquarters in Rome (HQ) for wider organizational learning and accountability and its Office of Evaluation (OEV) to feed into evaluation syntheses as well as for annual reporting to the Executive Board. For the CO, the evaluation findings will be used to guide future program implementation and as an advocacy tool to strengthen and support the government to maintain their commitment to their HGSMP. 37. The key Government of Kenya line Ministries that will be interested in the evaluation findings are the MoE, Ministry of Agriculture, Livestock and Fisheries, Ministry of Health, and Treasury including relevant Ministries at county level. County and Sub￾County Education Officers, School Management Committees are also key, as they are involved in program implementation and policy support. The Government of Kenya has a direct interest at both county and national levels in knowing whether WFP activities in the country are aligned with its priorities, harmonised with the action of other partners and meet the expected results. 7 Twaweza is a non-government organization working on enabling children to learn, citizens to exercise agency, and governments to be more open and responsive to addressing literacy and numeracy. Twaweza implements the Uwezo Initiative: a five-year program to improve competencies in literacy and numeracy among children aged 6-16 years old in Kenya, Tanzania and Uganda. http://www.twaweza.org/go/what-is-twaweza. 4 38. CO management will be responsible to respond to the evaluation recommendations by providing actions that will be taken to address eachrecommendation and estimated timelines for taking those actions. The CO should also ensure that the final evaluation report is shared with all relevant stakeholders. The final evaluation report will also be published on the WFP public website, and findings will be disseminated and lessons incorporated into other relevant lesson sharing systems. 1.2 Context 39. Kenya ranks 145 out of 187 on the Human Development Index.8 The country has a population of 46.5 million, approximately 74 percent of whom live in rural areas and rely almost totally on agriculture.9 It also has the largest, most diversified economy in East Africa, with a Gross Income per Capita of USD 2,762 per annum.10 Most of the land area of Kenya (over 80%) is classified as Arid and Semi-Arid Land (ASAL), and these areas are home to more than 4 million people.11 40. In September 2014, The World Bank reclassified Kenya's economy as lower- middle income. However, poverty, food insecurity, under-nutrition and income inequality remain high12 and nearly half the country's population live below the poverty line or are unable to meet their daily nutritional requirements. The most severe poverty isexperienced in the arid and peri-urban settlements.13 41. Kenya is a food-deficit country, relying on imports to meet the gap between food production and food requirements. Although more than 75 percent of Kenyan households produce some of their food, most Kenyans rely on markets for some or all their food needs.14 A large part of household food security is therefore determined by household income. In times of crisis such as lean seasons or droughts, poor households may be forced to resort to negative coping mechanisms to access food. These include withdrawing children from school (to reduce expenditure) and selling productive assets (to increase household income).15 These coping strategies come at the long-term detriment of the household as keeping children in school, or holding onto the productive assets would have positive long-term impacts. Most of the SMP counties are regularly classified as “stressed” on the Integrated Food Security Phase Classification (IPC). In 2014, the food security in parts of Turkana and Marsabit counties deteriorated and were classified as IPC category 3: crisis (Map 2). As a result of ongoing food security concerns in the ASALs, there are several other development agencies working in the same counties where the SMP is implemented. 42. Food insecurity contributes to malnutrition, particularly in children. At the national level, 35 percent of children under five years are stunted, 16 percent are underweight, and 7 percent are wasted.16Micronutrient deficiencies are also high, particularly iron deficiency anaemia, and vitamin A and zinc deficiencies, with the highest rates being among young children. Micronutrient deficiencies are largely a result of lack of dietary diversity, and low intake of fruit and vegetables. The stunting, wasting 8 Based on 2014 data. United Nations Development Program (2015) Human Development Report Office. http://report.hdr.undp.org 9 World Bank (2016) http://data.worldbank.org/indicator/SP.RUR.TOTL.ZS & Unknown author/date. Kenya: Situation analysis to transform nutrition. http://pubs.iied.org/pdfs/G02742.pdf 10 United Nations Development Program (2015) Human Development Report Office. http://report.hdr.undp.org 11 Key statistics on the dry lands of Kenya, Uganda and Ethiopia, REGLAP Secretariat, October 2012 12 UNICEF (2014) UNICEF Annual Report 2014 – Kenya. 13 United Nations Development Program (2015) Human Development Report Office. 14 Feed the Future: https://www.feedthefuture.gov/country/kenya 15 United Nations Development Program (2014) “Human Development Report 2015” 16 Kimani, E (2014) The nutrition paradox in Kenya: What are we doing? Global Nutrition Report. 5 and iron deficiency anaemia prevalence rates have remained high in Kenya over the last twenty years. Arid lands of Kenya 43. The USDA supported SMP is largely implemented in the arid counties of Kenya. These are concentrated points of vulnerability and poverty in the country. The arid counties are characterised by lack of water, poor infrastructure, poor network coverage, and increased risk of insecurity. They are also a considerable distance from Nairobi and other large urban centres. 44. The dominant production system in the arid counties is pastoralism, and nomadic pastoralists are a majority or significant minority in these counties. There are significant inequalities between the arid lands and the rest of Kenya in access to services and investment, not primarily due to the region’s ecology, but due to a long history of political marginalization. The Government of Kenya is now seeking to redress some of those long￾standing issues through their Vision 2030 Development Strategy. 17 45. Within Kenya’s Vision 2030 is a social protection objective to invest in vulnerable groups. The Hunger Safety Net Programme (HSNP) focuses cash-support to drought￾affected people in the arid and semi-arid lands (ASALs) as part of the national Ending Drought Emergencies (EDE) framework that sits within the National Disaster Management Authority (NDMA). Education 46. Since the turn of the century, there have been significant changes in the Kenyan national educational policy. The Government introduced free primary education in 2003, and by 2006 school enrolment had increased from 6.1 million children to 7.6 million, and the net enrolment rate increased from 77 percent in 2002 to 87 percent in 2006.18 The Kenya Education Sector Support Program (KESSP) 2005-201019 then provided the blue print for a comprehensive development program in education, including school feeding, health, and nutrition programs. The government also joined the Global Partnership of Education (GPE)20 and established national programs such as Tusome21 and Uwezo, to directly address literacy and numeracy. Since then, net enrolment figures for primary school and pre-school have continued to increase.22 47. At the time of the SMP design (2013), the national net primary enrolment was 83.5% for boys, and 84.5% for girls,23 with national primary school completion figures of 96.1%. Despite these high national figures, in the ASAL counties, primary school enrolment rates were lower than the national average. In 2013, the net enrolment in the ASALs was 40 percent with 35 percent completion. Completion rates had also been declining due to economic hardship and food insecurity.24 It is also estimated that there are nearly a million children of primary age who are not in school, concentrated largely in 17 Republic of Kenya (2011) Vision 2030 Development Strategy for Northern Kenya and other Arid Lands. August 2011. 18 WFP CP Kenya Approved Project document 19 Government of Kenya (2005) Kenya Education Sector Support Program (KESSP) 2005-2010. Delivering quality education and training to all Kenyans. Ministry of Education,25 July 2005. 20 GPE is a multi-stakeholder partnership and funding platform established in 2002 that aims to strengthen education systems in developing countries in order to dramatically increase the number of children who are in school and learning. 21 In 2015, the U.S. Agency for International Development (USAID) joined with the UK Department for International Development (DFID) to fund a program known as the Tusome Early Grade Reading Activity. Tusome is designed to dramatically improve primary literacy outcomes for approximately 7 million Kenyan children in grades 1–3. 22 Uwezo (2014) Literacy and numeracy across East Africa: Are our children learning? 23 UNICEF (2013) State of the World’s Children. 2008-2012 data 24United Nations Development Program (2014), “Human Development Report 2015” 6 the arid and semi-arid districts as well as in the informal settlements in large urban centres including Nairobi, Mombasa and Kisumu.25 48. In the last decade, Kenya has covered significant ground in ensuring gender parity in both primary and secondary education. Nationally, the ratio of boys and girls sitting national examinations is almost at par. In 2016, the ratio of boys to girls sitting theKenya Certificate of Secondary Education (KCSE) stood at 53% boys, 47% girls. At the primary level, the figures are even more impressive; in 2015, almost equal number of boys and girls sat the Kenya Certificate of Primary Education (KCPE) exams – 50.4% girls, 50.6 % boys. However, there are gender disparities according to geography – with the ASALs having lower percentage of girls sitting the national examinations. In 2015, girls accounted for only 29% of KCSE candidates in Garissa County, and 28% in Wajir County. The figures are more or less the same in other ASAL counties.26 49. School performance in Kenya is below international standards, and even lower in the ASALs. Although Kenyan children outperformed Ugandan and Tanzanian children, only 68% of Kenyan school children passed the standard numeracy test, 78% the literacy test, and 64% combined.27 In the ASALs these figures are worse. The Uwezo reports from the 2009-2014 show all the targeted SMP counties in the bottom 25% of rankings, with Mandera West and Central districts ranked the lowest in the whole country (155 and 154 respectively) (Annex 3). The exception is Nairobi which is the highest-ranking county in terms of literacy, numeracy and school achievement. Similarly, although at a national level, gender parity in enrolment has been achieved (albeit with regional disparities), in the arid counties, gender parity still remains elusive due to cultural practices, traditional values and poverty. Children from poor households also consistently show lower levels of learning. School feeding 50. WFP and the Government of Kenya have been supporting school meals in Kenya since 1980.28 Traditionally, this has been done through in-kind food assistance, with food procured largely from outside Kenya. WFP currently provides in-kind food assistance in all the USDA supported primary schools in the arid northern counties of Kenya, as well in the informal settlements in Nairobi. WFPs support to the arid areas reduces each year, as WFP hands over schools to the government’s HGSMP. 51. To pursue greater national ownership and sustainability of the program, MoE established the HGSMP. Since 2009, the Government of Kenya has implemented the national HGSMP in semi-arid lands that were previously served by WFP. The HGSMP provides funding directly to schools that then purchase food locally, creating a market for agricultural producers and traders. After all semi-arid counties had been handed over, in 2014 WFP started handing over school feeding in arid counties beginning with Isiolo County. While the Government and WFP work to gradually expand the coverage of the HGSMP, WFP continues to support children in public schools that are yet to be handed over and in targeted schools in the informal settlements of Nairobi, where food insecurity continues to be widespread, and education indicators are below the national average. 25 United Nations Development Program (2014), “Human Development Report 2015” 26 Education Development Trust (2016) https://www.educationdevelopmenttrust.com/en-GB/news-and-blogs/news/2016/barriers￾to-girls-education-kenya 27 Uwezo (2014) Literacy and numeracy across East Africa: Are our children learning? 28 Langinger, N. (2011) School Feeding Programs in Kenya: Transitioning to a Home-grown Approach. Stanford Journal of International Relations. Fall 2011. Program’s support (2013-2015) in Kenya from September 2013 to December 2014. Final report. October 2015 7 Gender 52. Gender equality and empowerment is a major social issue in Kenya. Women are underrepresented in decision-making positions, and they have less access to education, land, and employment. Those living in rural areas spend long hours collecting water and firewood, leaving them with little time to earn money or engage in other productive activities. Only 29 percent of people earning a formal wage throughout the country are women, leaving a huge percentage of women to work in the informal sector without any federal support. The effect is severe— although nearly 40 percent of households are run solely by women and because of a lack of fair income, nearly all these homes suffer from poverty or extreme poverty.29 53. Kenya’s Gini coefficient30 is 0.477, meaning that approximately half the people in the country control most of the resources. An example of this inequality is the Kenyan agriculture industry. Agriculture creates over 80 percent of Kenya's jobs and 60 percent of income.31 Currently, women in Kenya do the vast majority of agricultural work and produce/market the majority of food, yet they earn only a fraction of the income generated and own a nominal percentage of assets. Girls are also raised to contribute the household from an early age, including collecting water and firewood. 1.3 Summary of findings from the Mid-Term SMP Evaluation 54. This EoP Evaluation follows a baseline survey (May-June 2014) and a Mid-Term Evaluation (May 2015) conducted by other evaluation teams. The Mid-Term Evaluation32 mainly focused on identifying process level changes and did not focus on the achievement of all program outcomes. The Mid-Term Evaluation provided a generally positive review of the program and made recommendations in six thematic areas (Table 1). Each of these recommendations has been accepted and actioned by the WFP Kenya management. This EoP evaluation has noted throughout the report of whether the identified recommendations were adequately addressed. More details on the recommendations from the Mid-Term Evaluation can be found in Annex 4. Table 1: Summary of recommendations - Mid-Term Evaluation of SMP (2015) Thematic area Key Recommendation Immediate and short-term recommendations Food access Consider providing SMP meal one or two hours earlier to address short term hunger Threats to children’s safety Carry out community level sensitization on the threats to pupil’s safety. National Capacity WFP and GoK should consider implementing a national, independent entity to manage SFP. GoK should ring-fence the SFP budget line to ensure allocation of funds. Food distribution Provide a unified scoop measure to all supported schools. School Continue to synergise and support activities that improve the quality of 29 FSD International: http://www.fsdinternational.org/devsubject/womensempowerment/kenya 30 The Gini coefficient measures the extent to which the distribution of income or consumption expenditure among individuals or households within an economy deviates from a perfectly equal distribution. Equal distribution receives a score of zero, while the most unequal distribution (one person having all resources) is represented by a score of one. 31 FSD International: http://www.fsdinternational.org/devsubject/womensempowerment/kenya 32 Braidotti, F (2015) A Mid-Term Evaluation of WFP’s USDA McGovern-Dole International Food for Education and Child Nutrition Program’s support (2013-2015) in Kenya from September 2013 to December 2014. Final report. October 2015 8 performance education and teaching. Medium to long-term recommendations Government coordination Strengthen county level school committees Source: Compiled by evaluation team from the Mid-Term Evaluation report33 1.4 Evaluation Methodology & Limitations 55. The evaluation was designed to answer four key questions: • Evaluation Question 1: How appropriate was the School Meals Program? • Evaluation Question 2: What are the results of the SMP? • Evaluation Question 3: How and why did WFP achieve these results? • Evaluation Question 4: How sustainable is the SMP? 56. The evaluation was designed to compare achieved results against the baseline survey34 and the mid-term evaluation35 and uses the internationally agreed evaluation criteria of relevance, effectiveness, impact, efficiency, and sustainability (Annex 5). The evaluation focuses on accountability (against intended results) and learning (for the continuance of the school feeding in Kenya). 57. To ensure validity and reliability of data, the evaluation questionnaires were designed using an evaluation matrix (Annex 6). This helped ensured that all aspects of the ToR were included. The evaluation also used an independent team of enumerators, who collected data from a random sample of SMP schools. The evaluation used a mixed￾methods approach and triangulated information from different methods and sources to enhance the reliability of findings. 58. Strict ethical considerations were used to inform the evaluation methodology. All enumerators underwent training on research ethics offered by the National Institute of Health prior to conducting the surveys. Participation was voluntary and participants were informed about the risks and benefits of participating in the research, confidentiality and use of the data. All information received from the interviews was anonymous so it could not be referred to directly. It was also made clear to respondents that there would be no personally identifiable information collected. The evaluation adopted additional procedures for obtaining consent for school-aged children to be interviewed. Head Teachers signed a guardian consent form that students could participate in the evaluation. Additional consent was obtained from class teachers, and the evaluation was explained to children before starting each interview. As above, the children were told that participation was voluntary, and that all results would be anonymous. School children were randomly sampled across the classes with equal numbers of boys and girls participating. Girls were then interviewed by female enumerators, and boys interviewed by male enumerators. 33 Braidotti, F (2015) A Mid-Term Evaluation of WFP’s USDA McGovern-Dole International Food for Education and Child Nutrition Program’s support (2013-2015) in Kenya from September 2013 to December 2014. Final report. October 2015 34 WFP (2014) Baseline survey for the USDA McGovern-Dole International Food for Education and Child Nutrition Program’s Support (2013-2015) to WFP Kenya Country Program. 35Braidotti, F (2015) A Mid-Term Evaluation of WFP’s USDA McGovern-Dole International Food for Education and Child Nutrition 9 59. The key data collection methods were as follows: • Field visits were conducted in 79 randomly selected schools. In each school, the following activities were conducted: quantitative survey with teachers (n=552), quantitative survey of School Cooks (n=87), quantitative survey of Storekeepers (n=85), quantitative survey of households/parents (n=1,124), quantitative survey of school children (n=1,119)36, as well as observation of school facilities including kitchen, food preparation and storage areas. • Qualitative key informant interviews were conducted by a team of twenty-seven enumerators (12 female and 15 male Key informant interviews were done with 159 people at national, county or local roles in the program. These included WFP, MoE and other government representatives, donors, national partners, as well as parents and school children. Key informants were purposively selected based on the role they played in the program. The full list of key informants can be found in Annex 7. 60. The quantitative and qualitative data collection instruments can be found in Annexes 8 and 9 respectively. The full list of schools selected for visits can be found in Annex 10. All evaluation findings were disaggregated by gender and where possible, the evaluation has reflected any gender differences in the results. Analysis of the quantitative data was done using SPSS, STATA and Microsoft Excel software. All results are reported against the Performance Monitoring Plan agreed by MGD and WFP in their grant agreement. Evaluation challenges and timeline 61. This evaluation has faced several challenges that have resulted in the timing for the final reporting being significantly later than the initial plan. 62. The evaluation was originally conducted between 8th June and 5th September 2016 by TNS RMS East Africa, and managed by WFP Kenya. Data collection was carried out as per the detailed methodology and sampling methodology in Annex 11. However, once data had been collected, the original evaluation team faced internal staff performance issues which affected delivery of a quality report. In January 2017, TNS therefore hired an Independent Consultant to revise the report. This process was managed by TNS and the report revision was done based on the information provided by TNS personnel and the survey enumerators. The report was finalized in March 2017. 63. During WFP’s report review process however, it was noted that there was inadequate qualitative data in the report and that some quantitative data had not been included. This limited the data triangulation and resulted in inconsistent findings across related variables. As a result, WFP Kenya and USDA agreed to have WFP directly contract Independent team leader and a Research Specialist/Statistics Expert in August 2017. The duo was tasked to check and verify the reliability of the quantitative data, collect additional qualitative data and revise the report accordingly. Additional data was also provided by TNS and WFP Kenya. 64. The final drafting of the evaluation report included a period of data review where all datasets were re-assessed, cleaned and tested for reliability. Once complete, additional analyses of the data were carried out including One-Sample Wilcoxon Signed Rank Test, One-Sample Test, One-Sample Kolmogorov-Smirnov Test, Spearman's rank- order correlation, multiple regression analyses, and sign test for one median was 36 552 boys and 567 girls 10 conducted on the data. This process was managed by a WFP evaluation manager and both the internal committee and reference group member involved appropriately. Quality assurance was done by the WFP Evaluation Manager using WFP’s corporate Decentralized Evaluation Quality Assurance System (DEQAS). Limitations of the evaluation 65. The limitations of the evaluation were as follows: • Measurement of the literacy indicators has been affected by a change in approach taken by Uwezo in 2015. This limited the evaluation’s ability to draw comparison to baseline and midline surveys that relied on yearly outcomes. Also, the 2016 Uwezo data used different literacy and numeracy indicators to previous years, making direct comparison impossible. Data was also not collected in Samburu North due to security concerns, and Garissa and Marsabit data had to be removed due to quality challenges. • The evaluation questionnaires did not contain adequate gender specific variables to enable comprehensive discussion of the differences in results for boys and girls. • The ability to compute and model variance is dependent on comparison of endline data to the baseline quantitative data sets. Given that there was no comparison group of schools at baseline, the evaluation can only assess performance using before-and- after comparisons using cross-sectional approaches of statistically representative samples and qualitative methods. The data sets also do not have homogeneous identifiers across the three windows of data collection that can facilitate of a merged data set. • The input of other actors in the same locations makes it impossible to attribute changes specifically to the WFP SMP without robust a priori variables to control for the confounding. • The baseline, midline and endline surveys were conducted in random (and different) samples of schools, therefore the cohort is not identical. This makes it difficult to make direct comparisons at the school or student level. 11 2 Evaluation Findings 2.1 Evaluation Question 1: How appropriate is the SMP? 66. In assessing the appropriateness of the SMP, the evaluators examined the extent to which the objectives, targeting, choice of activities, and modalities of transfer were appropriate to the needs of the population, aligned with government and WFP corporate policies and strategies, and coherent with the work of other development actors. 2.1.1 Appropriateness to needs 67. The long-term objective of the SMP is to promote universal primary education of socio-economically disadvantaged and nutritionally vulnerable children, especially girls, within the primary schools in targeted areas. The relevance and importance of this objective was confirmed during key informant interviews with Government officials at all levels: national, county and district. 68. At the time of the SMP design, all ten targeted arid counties had amongst the lowest educational outcomes in Kenya (Annex 12). The targeted counties education achievements: enrolment, attendance, completion and school performance, were also below the national average, indicating that educational support programs were appropriate. Although Nairobi County has the highest national educational ranking, the SMP supported only the informal settlements due to high food insecurity in the area. In addition, despite improvements in food security in Kenya, the population in the ASALs, and particularly in the arid lands remains largely food insecure compared with the rest of the country.37 The inclusion of the ten targeted counties for SMP was therefore appropriate, as the program could also support household food security. 69. The evaluation found that WFP and the Government of Kenya have a strong transition strategy in place, to handover all the WFP counties to the HGSMP. This includes a phase of capacity building and ensuring infrastructure and procedures are in place for a cash-based approach. USDA’s McGovern Dole program provides for in-kind commodity contributions and monetary assistance for the implementation of comprehensive school meals projects. Provision of cash-based assistance in the year before each school transitions to HGSMP is then provided by other donors. The evaluation finds that this is appropriate, and notes that WFP Kenya has successfully implemented three transitions during this program phase. 70. In addition to the capacity building support to the government, the SMP includes activities that are designed to address specific areas of concern in the targeted areas: poor teacher attendance, student short-term hunger and the resulting lack of attentiveness in class, poor enrolment and attendance rates. WFP has also supported the improvement of sanitation facilities in schools to provide a more gender-sensitive environment. All these activities are highly appropriate to the context. 71. Through key informant interviews and focus group discussions, teachers, parents, and other community stakeholders explained the needs of and challenges faced by their communities. The challenges reported most frequently were hunger, and food insecurity (Annex 13). Key informants also emphasized poor health care and water shortages as occurring frequently with poor health care having the most negative impact. Other issues 37 Analysis of Integrated Food Security Phase Classification (IPC) data shows that since 2013 all the counties have, at a minimum been classified as “stressed.” The north-western parts of Marsabit County, and parts of Turkana were classified as “Crisis” in 2013/2014. The most recent IPC map (Map 3) shows the forecast for all the SMP counties is ‘stressed”. 12 included gender exclusion, unemployment, malnutrition and illiteracy. Issues around education were not mentioned by parents. Although the SMP does not have an explicit food security objective, it does address short-term hunger of school children. Global evidence indicates that SMP functions as a safety net, particularly for poor households and supports household food security. The evaluation team therefore find that the SMP appropriately meets some of the priority needs of the targeted communities. 2.1.2 Alignment with national policies and strategies 72. At the time of SMP design, the Government of Kenya had multiple policies that were relevant to the SMP implementation. These include the National School Health Policy (2009), the National Food and Nutrition Security Policy (2011), the National Social Protection Policy (2011) and the Basic Education Act (2013). In addition, the right to nutrition is included in the Constitution of Kenya (2010). The Constitution expressly guarantees all Kenyans their economic, social and cultural rights including basic rights to health, education, food and decent livelihoods.38 • National School Health Policy (2009)39: Currently being revised, this policy recommends provision of balanced school meals in all Kenyan schools. This policy was developed jointly by MoE and Ministry of Health. The policy recognizes children’sright to health and nutrition services, water and sanitation, and to education as guiding principles. This is a highly relevant government policy as it places food,nutrition security and free education, among the key priorities for the Government, in the Medium-Term Plan 2013-2017. • National School Health, Nutrition and Meals Program Strategy (draft 2011)40: This strategy aims to bring the existing policies and guidelines a step closer to implementation. This strategy was revised in 2016 but was not yet endorsed at the time of the evaluation. The 2016 draft highlights the need to mainstream SMP into national policies and frameworks, and recognizes the need for capacity building initiatives.41 • Food and Nutrition Security Policy (2011)42: Recognizes that school meals contribute to reduced hunger, which helps children to concentrate on their studies, and addresses specific micronutrient deficiencies in school age children, particularly iodine and iron, which directly affect cognition and can affect school performance. In this policy, the Government commits to enhance school meals program by, among other things, establishing standards and regulations for school meals initiatives that cover storage, preparation, handling and quantity of food served to students; improve the quality of and expand the school meals program to include pre-schools and boarding schools in collaboration with local communities. • Kenya National Food and Nutrition Security Policy (2012)43: Expresses the Government’s commitment to the eradication of hunger, and has a component addressing school meals and health and nutrition education in schools. The policy aims to improve nutrition, ensure that adequate food is accessible and affordable, andprotect vulnerable populations through safety nets, linked to long-term development. 38 Government of Kenya (2010) The Constitution of Kenya – Chapter 4: The Bill of Rights. http://www.wipo.int/wipolex/en/text.jsp?file_id=207673 39 Government of Kenya (2009). National School Health Policy. 40 Government of Kenya (2011). National School Health Strategy Implementation Plan (2011-2015). Ministry of Public Health & Sanitation, and the Ministry of Education. 41 Ibid. 42 Government of Kenya (2011). Food and Nutrition Security Policy. 43 Ibid. 13 • Social-Protection Policy (2012)44: Recognizes school meals as an important safety net for school children and their families. SMP contributes to the wider goal of poverty alleviation and promoting quality education.45 • The National Nutrition Action Plan (2012-2017)46 :The seventh Strategic Objective is to “promote appropriate nutrition for school children and adolescents.” 73. All the above policies and strategies require that the Government of Kenya set aside resources for school meals. This demonstrates that SMP is a priority for the government. The Government has also committed to adopting a holistic approach to ensure the provision of water and sanitation in schools, immunize and de-worm children regularly.47 2.1.3 Alignment with WFP corporate strategies, policies and normative guidance 74. In addition to the Government of Kenya policies and strategies mentioned above, the SMP is guided by several of WFP’s own corporate policies and strategies. • WFP Strategic Plan (2014-2017)48: The SMP contributes to Strategic Objective 4 – to reduce under-nutrition and break the intergenerational cycle of hunger. The Strategic Plan specifies that school feeding programs contribute to this goal by providing quality food, and contributing to addressing micronutrient deficiencies. The second goal of the Strategic Plan is to increase access to education and health services. • The WFP Updated School Feeding Policy (2013)49 specifies that WFP will focus increasingly on helping countries establish and maintain nationally owned programs linked to local agricultural production. The policy states that WFP will implement school feeding programs with clear hand-over strategies, where appropriate, and will engage in policy dialogue and provide technical assistance. WFP should also assess the cost-effectiveness of school feeding models, and ensure that school feeding programs address micronutrient deficiencies among schoolchildren. The primary delivery mechanism will continue to be through multi-fortified foods where foods with high micronutrient contents are not readily available or are unaffordable. • The WFP Gender Policy (2012)50 provides guidance to ensure that WFP’s programs promote a gender equality and women’s empowerment. 75. In addition to these key policies and strategies, a World Bank and WFP paper51 highlights that the transition to sustainable national programs depend on mainstreaming school feeding into national policies and plans, especially education sector plans. The research aligns with WFPs systematic handover of the SMP to the Government’s HGSMP. 44Government of Kenya (2011) Kenya National Social Protection Policy. Ministry of Gender, Children and Social Development. 45 Social Safety Nets; A collection of services provided by the state or other institutions such as friendly societies including welfare, un employment benefit, universal health care, homeless shelters, and sometimes subsidized services such as public transport which prevent individuals from falling into poverty beyond a certain level. 46Government of Kenya (2012) The National Nutrition Action Plan (2012-2017). Ministry of Public Health and Sanitation 47 Government of Kenya (2016) School nutrition and meals strategy for Kenya. Ministry of Education, Ministry of Health, and the Ministry of Agriculture, Livestock & Fisheries. Draft – February 2016. 48WFP (2014) WFP Strategic Plan (2014-2017) https://www.wfp.org/about/strategic-plan 49 WFP (2013) WFP Updated School Feeding Policy https://www.wfp.org/content/school-feeding-policy 50WFP (2009) Gender policy: promoting gender equality and the empowerment of women in addressing food and nutrition challenges. WFP/EB.1/2009/5-A/Rev.1 51 Bundy, D (2009) “Rethinking School Feeding: Social Safety Nets, Child Development, and the Education Sector 14 2.1.4 Alignment with other development partners 76. The SMP aligns closely with the work of other development actors working in the ASALs. WFP coordinates or partners with UNICEF, Feed the Children Kenya (FTC), Partnership for Child Development (PCD) and the Netherlands Development Organization (SNV) to ensure that the targeted counties have water, good nutrition, and livelihood opportunities not directly supported through WFP’s SMP intervention. • SNV supports the HGSMP in the areas of procurement and governance.It also supports farmers in accessing school markets, ensuring that procurement procedures are farmer-friendly and the community is engaged. SNV is also one of the members of the national level Technical School Feeding Committee, along with WFP and other partners. • PCD is also part of the Technical School Feeding Committee, playing an advisory role and working with WFP and other partners to provide capacity building and support to the program. PCD was also a strong partner in the development of the National School Health Policy. • FTC is the only cooperating partner used in the informal settlements in Nairobi, and WFP uses non-USDA funds52 for this collaboration. WFP also used FTC warehouses for food storage before distribution in Nairobi in the early stages of the program being evaluated, but changed Nairobi schools to cash in 2015 thus eliminating the need for the facilities. • UNICEF is active within both the Education and Water, Sanitation and Health (WaSH) sectors. Within the Education sector, UNICEF works in close collaboration with the government at policy level and works in synergy with WFP within the UNDAF framework. UNICEF also works with the government to update the current national curriculum, to improve the quality of teaching and pupils’ learning experience. WFP is supporting this process and providing inputs to the review of the national curriculum. UNICEF also aims to increase enrolment, through awareness campaigns sensitizing communities about the importance of education and increasingliteracy. 77. In addition to the major partners, WFP collaborates or coordinates with other agencies in WFP’s SMP operational areas. In particular, WFP attends and collaborates with Education Donor Coordination Partners and the Education in Emergencies working group. Each SMP county also has different development partners and faith-based organization working in the same or similar geographic locations. These include World Vision, the Kenya Red Cross, and various Catholic Diocese, who implement a range of activities that complement the SMP (Annex 14) including agriculture, construction of school gardens, provision of water, and provision of educational items such as books and pens. Evidence Action53 also works in many of the targeted SMP counties providing de- worming support to schools. 78. The SMP program was also originally aligned to the global Millennium Development Goals (MDG) 1 “Eradicate extreme Poverty and Hunger” and 2 “Achieve universal primary education”. With the development of the Sustainable Development Goals (SDG) in 2015, the SMP now aligns to SDGs 2: Zero hunger, SDG 4: Quality education, and SDG 5: Gender equality. 52 Funding for FTC partnership is from the Australian Government. 53 Evidence Action implements the Deworm the World Initiative, scaling up school-based deworming programs to improve children’s health, education, and long-term development. 15 2.1.5 Summary of findings – Evaluation Question 1 79. Table 2 provides a short summary of the main evaluation findings relevant to the first evaluation question – How appropriate is the School Meals Program. The key findings are colour-coded, as they are throughout the report for ease of reading and clarity. Findings key: Achievement of objective Within 10% of achieving target Objective or target has not been met Table 2: Summary of key results - Evaluation Question 1 Program elements Appropriateness Objectives The objectives of the SMP to improve educational achievement are coherent with national policies. The objectives are strongly aligned with recent WFP strategies, policies and guidance. Targeting of intervention area The targeted arid counties all have low education outcomes, and chronic food security. The unplanned settlements in Nairobi have high food insecurity. Choice of modality WFP provides direct in-kind food assistance and in counties preparing for transition to HGSMP, cash. The Government provides food assistance through cash transfers so it is appropriate to transition to cash before handover. WFP and the MoE are jointly implementing a transition strategy through which all the targeted SMP counties will become part of HGSMP. Choice of activities The core activity of school meals provision increased school enrolment and addressed short term hunger. Training to improve teacher attendance, better food storage and transition to HGSMP all address identified needs. Alignment with national policies The SMP is coherent with key Government of Kenya policies and strategies. The SMP is also implemented closely with MoE and other ministries. Alignment with WFP strategies The SMP is coherent with key WFP corporate policies. Alignment with other development actors The activities under the SMP complement other interventions with no duplication. Aligned with partner UN agencies and donor policies The SMP objective is coherent with the partner UN agencies and other donor policies that are embedded within the SDGs. Summary of key findings Evaluation question 1 - How appropriate is the SMP? The evaluation finds the design of the SMP to be appropriate. • The SMP targets counties with poor education outcomes, and chronic food insecurity. • The SMP aligns well with Government of Kenya policies and strategies, and with WFP’s own corporate guidance. • Provision of food through direct in-kind assistance is appropriate in the targeted counties despite the government providing food assistance in their HGSMP through cash transfers. The evaluation team understands that ongoing in-kind assistance is temporary while the SMP counties transition to cash transfers (through the HGSMP). • The SMP aligns with other development actors, and WFP coordinates to ensure that their targeted communities receive other needed assistance that WFP cannot directly provide. 16 2.2 Evaluation Question 2: What are the results of the School Meals Program? 80. The long-term objective of the SMP is to promote universal primary education of socio-economically disadvantaged and nutritionally vulnerable children, especially girls, within the primary schools in targeted areas. • To contribute to Feed the Future • To improve literacy of school-age children • To increase access to food (school feeding) • To achieve more consistent teacher attendance • To improve student attentiveness • To reduce short term hunger • To increase student enrolment • To increase knowledge of safe food preparation and storage practices • To increase access to requisite food preparation and storage tools and equipment • To increase the use of health and dietary practices (food storage) • To increase capacity of government institutions • To improve policy and regulatory framework • To increase engagement of local organizations and community groups • To increase community understanding on the benefits of education. 81. This section evaluates the SMP 2014-2016 program performance on the achievement of these results based on performance indicators jointly identified by WFP and USDA. Where possible, the performance at the end of project is compared to the baseline and midline. 2.2.1 Thematic area: School performance 2.2.1.1.1 MGD Strategic Objective 1: Improved literacy of school aged children 82. Within school performance, the first MGD strategic objective for the SMP is to improve literacy rates among school-aged children. WFP contributes to this objective through the provision of school meals, and activities to improve teacher attendance. There are no explicit literacy-based activities within the programme. 83. Although neither WFP nor the mid-term or EoP evaluation have collected primary data on literacy or numeracy, there is sufficient secondary data for evaluation purposes available from Twaweza’s annual Uwezo reports. However, as noted earlier, in 2016 Uwezo reports changed format and the report no longer provides county level data on thetwo indicators used at baseline.54 This makes it impossible to assess whether there has been any change in literacy and numeracy rates in the SMP county schools since baseline.The closest indicators are Pupils in Grade 3 who can do Class 2 work, by subject (Maths and Kiswahili). 84. Table 3 shows the literacy and numeracy rates per Uwezo reports in 2014 (baseline) and 2016 (EoP). Although the low rate of literacy and numeracy in the targetedSMP counties underscores the relevance of the SMP in selecting the counties as intervention areas, there is currently no relationship between school feeding and changes in literacy rates. Without the inclusion of specific literacy-related activities within the 54 Pupils in Class 2 who can read and understand a grade level text, and Pupils in Class 3 who can do Class 2 work. Rather, they have used a combined indicator of literacy and numeracy. 17 programme, school feeding cannot improve school performance as the responsibility for achieving education outcomes lies largely with the Ministry of Education (MoE). Nationally, literacy and numeracy rates have shown little progress over the course of the SMP, and the Government of Kenya is engaging in several initiatives to improve this. Table 3: Literacy and numeracy rates during SMP implementation Baseline (2014) EoP Kenya Average (2016) Literacy Pupils in Class 2 who can read and understand a grade level text (Kiswahili)55 48.8 46.156 Numeracy Pupils who can do Class 2 work. Ranges between 33.8% (Samburu) and 62.6% (Baringo) 47.157 85. Overall, both the Uwezo 2015 and 2016 reports indicate that most of the targeted SMP counties are yet to attain the national average in numeracy and literacy (Figure 1).58 This is not surprising given that Uwezo assessment results indicate that literacy and numeracy rates have not changed significantly since 2009. The 2016 Uwezo report also indicates that all the targeted arid counties are still below the Kenyan average (30%) of the percentage of children in Class 3 who can do Class 2 work59. For this indicator, the SMP counties range from 24.5% in Marsabit, to 9.9% in Wajir (Annex 15). Figure 1: Percentage of illiterate and innumerate60 children in the SMP targeted counties (2015) Source: UWEZO 201561 Note: Nairobi figures include all schools not just the unplanned settlements. 55 Data taken from Uwezo reports, not collected directly from the baseline or EoP Evaluation 56 Uwezo (2016) Are our children learning (2016)? Uwezo Kenya sixth learning assessment report. December 2016. Pupils in Grade 3 who can do Class 2 work, by subject – Kiswahili (p.11) 57 Ibid. Pupils in Grade 3 who can do Class 2 work, by subject – Maths (p.11) 58 Note that the Nairobi data includes all schools, not just the unplanned settlements. 59 Defined as the proportion of school children aged 10-16 years who cannot read letters 60 Illiterate and innumerate defined as a child who cannot read any letters, or recognize any numbers. School readiness indicators. Uwezo (2015): Are Our Children Learning? The State of Education in Kenya in 2015 and Beyond. Nairobi: Twaweza East Africa. p.40 61 Uwezo (2015): Are Our Children Learning? The State of Education in Kenya in 2015 and Beyond. Nairobi: Twaweza East Africa Illiterate Innumerate 30 25 20 15 10 5 0 National Nairobi Tana Baringo Isiolo Marsabit Garissa West Mandera Samburu Wajir Turkana Average River Pokot Percentage of children 18 2.2.1.1.2 MGD SO 1.1: More consistent teacher attendance 86. The lack of improvement in literacy and numeracy rates, highlights the need for qualified and experienced teachers for achieving good educational outcomes. Research has shown that teacher certification and academic qualifications are particularly important in determining student performance.62 Researchers have also found that teacher’s qualifications, experience and amount of education and knowledge are significantly associated with students’ achievements.63 Qualified teachers are therefore a necessary precondition for good student performance. 87. Key informant interviews indicate that in many parts of Kenya, and particularly in the ASALs, it is difficult to attract qualified teachers. This is due to several factors including low payment for their work, insecurity in some locations, as well as long working hours and high levels of responsibility. In many parts of the county, pupil- teacher ratios are high64, which further adds pressure on the teachers. The 2016 Uwezo report shows that both the counties with the worst teacher to classroom (stream) ratio were SMP counties: Mandera and Garissa with only 6 teachers per school of 10 classrooms (streams).65 In addition, most of the targeted SMP counties are considered to be hardship posts due to the distance from Nairobi and services in general. As a result, teachers assigned to schools in the targeted counties are sometimes unwilling to move to the area, especially when accommodation is not provided. 88. WFP provides technical support to the MoE to enhance the capacity of the County Directors of Education and Quality Assurance and Standards Officers (QASOs) in targeted districts to monitor and supervise teachers. UNICEF and other stakeholders alsosupport the MoE to engage with communities through School Meals Committees (SMCs) and Parent-Teacher Associations (PTAs) to increase their participation in the hiring and supervision of teachers. WFP also conducted training for MoE officers on how to promote teacher attendance. Table 4 shows that over the course of the SMP, WFP conducted 42 trainings (almost double the target), attended by 584 MoE officials, achieving 97% of their target of number of officers trained. Table 4: Activity indicators: Promotion of teacher attendance Indicator Target Achievement Total 2014 2015 2016 Number of MoE officers trained in promoting consistent teacher attendance 600 200 295 89 584 Number of trainings in promoting teacher attendance conducted for MoE Officers 23 12 15 15 42 89. Results from the EoP school-based survey showed that 88% of teachers were present in school on the day of the EoP survey. This is the same figure found at midline 62 For example: Hopkins, D. (1997) powerful learning, powerful school. London Chapman; Guthric J.W (1970) A survey of school effectiveness. In do Teachers make a difference? Washington D.C Department of health, Education and Welfare 63 Goe, L (2008) Teacher quality and student achievement: Making the most of recent research. TQ Research and Policy Brief. National Comprehensive Centre for Teacher Quality. Washington DC. 64 Average number of pupils per teacher (MoE, 2014): Garissa 46.5, Marsabit 37.4, Nairobi 36.2, Tana River 36.6, Turkana 71.7, West Pokot 37.2. 65 Uwezo 2016 19 in 2015, and consistent with the Uwezo 2016 findings (87.8%)66. The EoP evaluation found the mean attendance for the teachers is 89.7%67 of scheduled school days in a year. Based on the one sample test with a test value of 90, this is 0.3 percentage point below the target of 90%. However, the difference is statistically insignificant (p>0.05). In this regard, the target was attained (Table 5). Table 5: Results indicator: More consistent teacher attendance Indicator Target Baseline MTR EoP Percent of teachers in target schools who attend and teach school at least 90% of scheduled school days per year 90 51 74.8 89.7 90. It should be noted that during the project period, Kenya experienced several formal teachers strikes with teachers protesting the conditions under which they are expected to work. For example, during Term Three68 2015, a national teacher's strike over a pay dispute interrupted school for 25 days.69 2.2.1.1.3 MGD SO 1.2: Improved attentiveness 91. In order to meet the MGD Strategic Objective of improving school performance, the SMP aimed to reduce short-term hunger by providing a meal at school. This was expected to improve children’s attention in class, enabling them to better concentrate on their studies. Literature confirms that hunger is a key cause of inattentiveness.70 92. The baseline and midline measured student attentiveness solely through an interview with teachers. At EoP, the same process was used. However, since this is subjective per each teacher’s meaning of ‘inattention”, the findings were highly skewed71. Although teachers reported a mean percentage of inattentive students of 21%, which is consistent with both baseline and midline findings (Table 6), the skewness of the data means it is more appropriate to use the median. The median result was 15%, indicating a positive change from the baseline and midline figures. In addition, the EoP evaluation collected data to enable the compilation of a more objective composite indicator of attentiveness, created from other variables.72 When the composite indicator is used, the proportion of inattentive students reduces to 7.1%. This is a significant regression from the baseline73 (Annex 16). 66 Uwezo (2016) Are our children learning (2016)? Uwezo Kenya sixth learning assessment report. December 2016. 67 Test value + Mean Difference of a one sample test 68 September – December 2015 69 WFP Kenya 2015 Standard Program Report 70 Hincks Dellcrest Centre - The Child with Attention Problems - The Inattentive Child. https://www.hincksdellcrest.org/ABC/Teacher-Resource/The-Child-with-Attention-Problems/The-Inattentive-Child.aspx 71 Skewness = 1.5; Standard Deviation = 19.7; Kolmogorov-Smirnov Test p = 0.000 72 Composite indicator created from results of the following variables: student asks questions or contributes to discussions in class, student seeks advice from academic staff, student makes class presentation, student uses library resources in school, student comes to class having completed readings or assignments, student keeps up to date with studies, student works with other students on projects during class, student works with other students outside class to prepare assignments, and student is attentive in class. 73 Variance = 84.5%, r = 1 20 Table 6: Results indicator: Improved attentiveness Target Baseline MTR EoP Percent of students in classrooms identified as inattentive by their teachers ≤20 20 20.1 15 93. When the attentiveness data is disaggregated by county (Table 7), it is clear that all counties except Baringo have achieved the target of having ≤20 percent of students inattentive in class. Table 7: Percent of students in classrooms identified as inattentive by their teachers County N Median Nairobi 68 5 Isiolo 47 14 Turkana 57 15 West Pokot 38 15 Marsabit 61 15 Garissa 49 16 Tana River 128 20 Samburu 62 20 Baringo 42 30 94. Although inattentiveness has numerous causes, teachers identified hunger as the primary reason (cited by 38% of teachers) (Figure 2). Other causes include tiredness from domestic or income-generating work (22%), the perceived unimportance of education, sickness, or reasons other than those mentioned in Figure 2 (22%). In counties where parents reported that food was sometimes not provided at school, teachers were more likely to report that hunger was the main cause of inattention. In those schools, lack of school meals account for 70.7% of the variance in student inattentiveness (r2 = 0.707, p < 0.01). This indicates that school meals play a critical role in attentiveness of students. Figure 2: Reasons for pupil’s inattentiveness in class Hunger Sickness Other (specify) Tiredness from domestic or income-generating work Perceived unimportance of primary education 22% 38% 11% 7% 22% 21 95. To improve attention in class, the EoP evaluation agrees with the recommendation of the Mid-Term Evaluation, that is important to provide children with the school meal as early as possible in the school day. This would mean the SMP has a contribution to improved cognitive function. The evaluation team recognises that WFP has had discussions with the MoE on this issue but have encountered difficulties in adopting the recommendation. This is because schools operate on a national timetable, and meal times cannot be changed only for some schools. At a minimum, this means the programme should ensure meals are ready at the designated meal times. 2.2.2 Thematic area: Provision of school meals 96. WFP planned to provide school meals to 815,000 school children in the first year (2014) reducing annually to 450,000 in 2016 as counties transitioned to the HGSMP. Figure 3 shows that WFP have reduced their beneficiaries over the three years as planned. In 2016, WFP reached 430,409 children, a decrease of 46% from 2014 figures Annex 17.74,75 Figure 3: Total number of students receiving school meals Source: WFP Kenya, program monitoring data 97. The median number of beneficiaries benefiting directly from USDA-funded interventions is 763,490 which is within the target (Table 8).76 In addition to the direct beneficiaries of the SMP (school children), the program has provided some benefits, indirectly, to the children’s family members. These benefits include the value transfer element of the program, saving households from providing food for their children, as well 74 2014 actual = 796,116; 2015 actual = 586,100; 2016 actual = 430,409. Total = 1,812, 625 children 75 Number of students receiving school meals as a result of USDA assistance: Baseline 767,108; MTR 753,139, EoP 430,409 76 There is a slow (p =0.102) regression of the total individuals benefiting directly from USDA-funded interventions. The variance change is within the set target given that p > 0.05. Boys Girls Total 1,000,000 800,000 600,000 400,000 200,000 0 Planned Actual Planned Actual Planned Actual 2014 2015 2016 Number of students Summary of results on school performance: • The targeted arid SMP counties were all below the national average of literacy and numeracy at the start of the program period, and remain so in 2016. Nationally, literacy and numeracy rates have shown little progress over the course of the SMP. • School feeding alone cannot improve school performance and that the responsibility for achieving education outcomes falls largely under the Ministry of Education (MoE). • Teachers reported that hunger was the main cause of inattention in class (38%) although the measurement of “inattentiveness” is a subjective measure per each teacher’s opinion. When a more objective, composite indicator of the result of children’s attention is used, the number of inattentive children falls considerably, and shows significant improvement since baseline. Baringo County was the only county that has not achieved the ≤20 percent target. Regression analysis shows that school meals play a critical role in attentiveness of students. 22 as the unintended benefit of children bringing food home for their siblings (described in more detail ahead). The median number of beneficiaries benefiting indirectly from USDA￾funded interventions is 777,373 which is also within the target of <1,135,467.77 The total number of indirect beneficiaries at the time of the EoP was 527,98478 (Table 8). In total, this means that almost 1.3 million79 people benefitted from USDA support in 2016. Table 8: Results indicator: Increased access to food (school feeding) Target BL MTR EoP Number of total individuals benefiting directly from USDA-funded interventions <1,020,483 767,108 - 763,490 Number of total individuals benefiting indirectly from USDA-funded interventions <1,135,467 536,758 - 527,984 Percent of students in target schools consuming daily meals at school 100 100 100 91.5 98. The baseline and midline surveys both found the percentage of students who reported eating a meal at school the day before the survey was 100 percent. At EoP, the evaluation found a figure of 91.5 percent (Table 8).80 Table 9 shows that while no county has met the 100 percent target, most counties, except for West Pokot and Turkana are within 10 percent. The p-value of Levene’s Test81 (p>0.05) indicates that both girls and boys received meals, as the variance in meal consumption between girls and boys is statistically insignificant.82 It is also noteworthy that all three counties that transitioned to HGSMP during the evaluation period (Isiolo, Samburu and Tana River) are all among those counties within 10 percent of the target. This is a good indication that the counties transitioned after appropriate training, and procedures were put in place. Table 9: Percent of students consuming daily meals at school by county County Percentage of students who consumed a meal at school the day before the survey Samburu 98.5 Baringo 97.0 Marsabit 96.7 Nairobi 94.4 Isiolo 93.7 Garissa 93.2 Tana River 92.8 West Pokot 80.9 Turkana 72.0 Total 91.5 77 There is a slow (p =0.180) regression of the total individuals benefiting indirectly from USDA-funded interventions. The variance change is within the set target given that p > 0.05. 78 Calculated using method applied during the baseline and midline survey: Interviews with parents determined the average number of children per HH going to school. HH average size is will be computed say X. Number of HHs= direct beneficiaries/the number of children per HH going to school. Number of indirect beneficiaries = number of HHs * (X- number of children per HH going to school). 79 763,490 + 527,984 = 1,291,474 80 Primary data collected from questionnaire with school children. 81 Levene’s Test of Equality of Variance 82 Percentage of students reporting eating a meal at school on the day before the survey: Girls 91%, Boys 92%. 23 99. Key informant interviews indicate that in most schools the school meals are consumed by additional beneficiaries, including teachers and other school personnel, children who are not in school, and the younger children from Early Childhood Development (ECD). Children were also observed by the evaluation team keeping part of their ration in order to take it home to share it with other household members. “… the [WFP] policy is that they do not allow anyone else who is not a student to eat the food. But if you go there … the teachers give us many challenges and say when this food gets to school and when it’s time to store it, the locals come and they want the food. This is one of the challenges that teachers face.” Baringo key informant interviews 2.2.2.1.1 MGD Strategic Objective 1.2.1: Reduced short term hunger 100. To assess short term hunger, WFP has monitored the proportion of students consuming a meal before attending school, and the proportion consuming a meal in school. 101. Eating a meal before school: It is clear that hunger is an issue among school children, with only half the students (53.4%) reporting regularly (every day of the last five days) consuming a meal at home before attending school (Table 10). If the definition of “regularly” was reduced to “at least three of the last five school days” this figure increases considerably, to 76 percent. Table 10: Results indicator: Reduced short term hunger Target BL MTR EoP Percent of students in target schools who regularly consume a meal before the school day 80 41 59.1 53.4 Percent of students in target schools who regularly consume a meal during the school day 100 70 68.1 66.9 102. In the 3+ days definition of regularly, the p-value from Levene’s Test (p>0.05) indicates that the variance in meal consumption between girls and boys is statistically insignificant. However, the 5-day definition shows a statistical difference, with girls more likely to be eating breakfast, than boys.83, 84 103. The finding that only half (53.4%) the surveyed students reported eating a meal before school every day, highlights the importance of the SMP as a food security intervention, particularly in the arid areas. It may also reflect a lack of parental understanding on the importance of breakfast. Despite not achieving the targets on either of these indicators, both indicators show a great improvement from the baseline. 104. Eating a meal during school: Table 10 shows that 66.9 percent of children reported regularly eating a meal at school during the school day. This result is not statistically different from the 70 percent reported at baseline (variance = 2.3; Standard Deviation = 1.5; Skewness < 1). This is consistent with the finding that school meals were provided 65 percent of the time (described ahead in Figure 6). This is due to the inconsistent pipeline caused by insufficient resources. 105. The EoP evaluation found that within the targeted counties, the lowest proportions of children consuming food before school were found in Baringo, Turkana and Marsabit. This is consistent with the high levels of food insecurity experienced in the 83 Consuming meal during school: 5 days a week = Girls – 68.1%, Boys – 65.8%. 3 or more days a week = Girls – 88.7%, Boys – 87.3% 84 Consuming meal before school: 5 days a week = Girls – 55.7%, Boys – 51.1%. 3 or more days a week = Girls – 77.1%, Boys – 74.8% 24 region at the time. Marsabit, Baringo and West Pokot had the lowest proportions of students reporting consuming a meal during the school day. Samburu is the only county at EoP that met the targets (or within 10%) for both children eating a meal before, and during the school day regardless of the definition of “regularly” (Table 11). Table 11: Percent of students in target schools who regularly consume a meal before and during the school day (as reported by school children) Before school During school County Meal consumed every day of the last 5 days Meal consumed at least 3 days out of last 5 Meal consumed every day of the last 5 days Meal consumed at least 3 days out of last 5 Samburu 87.7 97.7 90 100 Isiolo 73.9 88.7 81.7 95.1 Nairobi 68.5 81.5 87 97.2 Garissa 66 93.2 72.8 89.3 Tana River 56.3 82.6 67.1 95.8 West Pokot 50 76.6 62.8 84.0 Baringo 26.1 61.9 38.8 75.4 Turkana 25.4 31.4 81.4 88.1 Marsabit 25.2 67.5 22.8 64.2 Average 53.4 76.0 66.9 88.0 106. To verify the above data re percentage of children eating a meal before school, as reported by the school children, parents were also asked if their children ate a meal before school. The evaluation found no significant difference between what the school children reported, and what their parents reported (Annex 18). 107. Eating a meal after school: Parents were also asked if a meal was provided to school children after school. The evaluation found that 66 percent of parents provided a meal to their children every school day (5 days a week), and 87 percent at least three days a week (Annex 18). In addition, parents were asked if they reduced the portion of food provided to their school children during the weekdays because of the school meals program. Overall, 29% of parents reported that they reduced the portion of food on the weekdays (school days) compared to the weekends. Most of these households reported reducing the food portion served to their school children, by 25 percent (Figure 4). Figure 4: Parental reduction in food portions for school children on weekdays compared to weekends 25% 50% 75% Linear (25%) Linear (50%) Linear (75%) Ba ri n g o Ga riss a C o u n t y C o u n t y N a ir o b i S am b u r u T a n a Ri v e r T u r k a n a We st P o k o t C o u n t y C o u n t y C o u n t y C o u n t y C o u n t y C o u n t y 50.0% 47.1% 2.9% 67.7% 25.8% 6.5% 78.0% 19.5% 2.4% 18.9% 56.8% 24.3% 46.8% 46.8% 6.4% 68.4% 23.7% 7.9% 78.9% 21.1% 0.0% 68.4% 31.6% 0.0% 75.0% 25.0% 0.0% 25 2.2.2.1.2 MGD 1.2.1.1: Increased access to food (school feeding) 108. To provide the school meals, WFP has delivered 52,794 MT of food commodities to targeted SMP schools, including 19,260 MT from USDA. Although USDA has provided all that commodities that were included in the agreement, overall the project was under￾resourced, and provided 71.5% of the targeted volume (Figure 5). Commodities included wheat, oil, pulses, maize, rice, SuperCereal, micronutrient powder, and iodized salt (Annex 19). The reduction in commodity provision over the course of the programme is due to the gradual handover of counties to the Government’s HGSMP. Figure 5: Total quantity of commodities provided to SMP from all sources Source: WFP monitoring data 109. The evaluation found that overall, WFP had delivered high quality food, including minimal loss of food during transportation. Complaints relating to food contamination because of poor food storage were few, and Public Health Officers reported only one incident in Langata (Nairobi) where maize was confiscated due to contamination. No other incidents were reported in Langata after the school started using wooden pallets. 110. Although WFP delivered food to the SCDE storage, the evaluation found that based on school register data, school meals were provided only 65% of school days in 2016. This is a lower figure, but not statistically different from the Mid-Term Evaluation (Figure 6). Findings that school meals were not provided each day is consistent with the household interviews (Annex 20) findings that 37 percent of parents reported that there were times when meals were not provided at school. This figure was even higher in Baringo at 61.4%. Similarly, the number of students reporting regularly (every day of the last five school days) consuming a meal during the school day, was 66.9 percent. However, using the 3+ day definition of “regularly”, the figure increases considerably, to 88 percent. This figure is more consistent with the figure in Table 5 of 91.5% reporting eating a meal at school the day before the survey. From this, it seems that many schools are providing meals 3-4 days per week, rather than on every school day. Figure 6: Proportion of school days that SMP Meals were provided Source: Evaluation team - Collation of data from SMP 6 form, EOPE 2016 111. Pipeline breaks as a result of under-resourcing, contributed to the low proportion of days when meals were provided in school. For example, in Term 2, 2015 (May– Planned Achieved 80,000 60,000 40,000 20,000 0 2014 2015 2016 Total Metric tonnes (MT) Endline Midline Baseline 0% 20% 40% Endline, 65% Midline , 68% Baseline, 70% 60% 80% 100% 26 August), meals were provided on 50 out of 70 school days. Similarly, in Term 3, 2015 (September to November), resources were available to feed children for 40 out for 55 school days. Key informants also mentioned other factors that reduced the number of days when schools served meals including delays in receiving food from the SCDE stores, insufficient firewood and insufficient water. 112. A similar situation was noted by the mid-term evaluation. When SuperCereal was being provided for pre-primary children (2014) through other donors it was only available on 38% of school days. The main reason for this is explained by the lower quantity of CSB received85 by WFP (due to insufficient funding) compared to the quantityplanned. To compensate for the low volume of food delivered, some schools reduced the size of rations per child. Some schools reduced even further to accommodate additional rations for the pre-primary school children and children from the neighbourhood who were not in school during meal times. 113. Parents were asked if and how the children ate lunch when school meals were not provided. Almost half the surveyed parents (42.9%) reported that children would go home for lunch and return to school (Table 12). However, 30 percent would not eat lunch, and more than ten percent of children (12.5%) would go home for lunch but then not return to school. It is clear that the program therefore also plays a role in school children’s food security. However, as 29% of parents reported reducing the portion of food served to their children after school because they had received the school meal, it reduces the benefit of the school meals as a food security intervention. Table 12: Parents reporting of how children ate lunch if meal not provided at school Frequency Percent Child brought own food and eat at school 19 6.8 Gave cash to child to buy lunch 10 3.6 Child came home for lunch and then went back to school 120 42.9 Child remained home and eat at home 35 12.5 No lunch 84 30.0 Other (Specify) 12 4.3 Total 280 100 85 WFP received 591 MT of CSB but received and distributed only 388 MT (66 percent of the planned quantity). 86 2014 actual = 796,116; 2015 actual = 586,100; 2016 actual = 430,409. Total = 1,812, 625 children Summary of results on the provision of school meals • Over the three-year program implementation, WFP has provided meals to almost 2 million children.86 • The evaluation found that school meals have been provided 65-70% of school days over the course of the program, in part due to funding constraints and pipeline delays as well as insufficient firewood and water provision by the communities. • Around half the students consume a meal before attending school, so the SMP is a significant contribution to the food security of the children. However, if parents reduce meals after school, the food security contribution of SMP reduces. Parents also reported that if school meals were not provided at school, 30 percent of children would not eat lunch, and 12.5 percent of children would go home for lunch but then not return to school. 27 2.2.3 Thematic area: Participation in education 114. One of the major objectives of the SMP is to increase enrolment rates in primary school. This was later revised to include multiple indicators of participation in education including increased enrolment, stabilized attendance, and improved completion rates. 2.2.3.1.1 MGD SO 1.3: Improved student attendance 115. In Kenya, the national net attendance in primary education is 87 percent87, which is consistent with the EoP evaluation findings (86.4%)88. Since baseline, there has not been a significant change in this indicator (Table 13). The target of 95 percent of children regularly attendance primary school has therefore not been achieved, although the EoP result is within 10 percent of the target. Table 13: Results indicator: Improved student attendance Target Baseline MTR EoP TOTAL students regularly attending USDA supported classrooms/schools 95 85.6 88 86.4 Source: Assessment of school registers EOPE, 2016 116. A county level assessment by the evaluation team, found that pupil attendance changed over the SMP implementation period in almost all the targeted counties (Figure 7) albeit with inconsistent results. Overall, only three counties: Baringo, Garissa and West Pokot improved since the baseline. Baringo and Garissa had the lowest attendance at baseline. Figure 7: Primary School Attendance by County Source: Evaluation team - Assessment of school registers 117. Key informant interviews indicate that the presence of the SMP plays a significant role in participation in education. Teachers believe that school attendance would drop considerably if meals were no longer provided. They also reported that school meals have improved retention rates although that is not one of the program results indicators so could not be verified. (The school feeding program) “is very, very important. If the food is not there, this is not a matter of saying probably; the children would not come to school. Here the children look for the smoke (from the 87 UNICEF – State of the World’s Children – 2015: 2014 data 88 Computation based on the following figures: Total registered students = 26,234, attrition rate (2008-2015) = 5.7%  adjusted denominator = 24,739. Total attendance = 21374. Completion rate = attendance/enrolment (21,374/24739*100) = 86.4%. 2014 2015 2016 100% 80% 60% 40% 20% 0% Baringo Garissa Mandera Marsabit Tana River Turkana Wajir West Pokot Nairobi Primary school attendance rate 28 kitchen) when they are at their homes. If they don’t see the smoke, then they would not come to school. If they see the smoke, they just come. That is how important the SFP is to education in this county! Actually, if you don’t feed children in this county for one year, then you can as well say goodbye to education in this county.” Samburu key informant interviews 118. Household interviews indicate multiple reasons for student absence, with family events and sickness accounting for 99.7% of absenteeism (r2 = 0.997, p < 0.001) (Annex 21). In Nairobi, Isiolo and Marsabit, school closures (because of the teacher’s strike) was also a significant reason (Figure 8). Key informant interviews confirm that in some locations, student absence increases during particular times of the year, as children engage in household tasks such as working on the farm or livestock migration (mainly in Baringo, Samburu and Tana River). Insecurity in some locations is also a concern (Baringo, Turkana, Garissa and Tana River) or cultural activities such as ceremonies (Baringo, Tana River and Samburu). Figure 8: Reasons for school absence, by county 119. There are several other development actors implementing complementary projects in the SMP targeted counties to improve school attendance, particularly for girls. The MoE together with World Vision implement the “Wasichana Tusome” initiative that aimsto keep girls in school throughout the month by providing sanitary pads. Research indicates that menstrual hygiene management among school-going girls reduced absenteeism.89 Welthungerhilfe is also implementing a water project that brings water closer to communities, reducing water collection time. Research indicates that a 15- minute reduction in water collection time increases the proportion of girls attending school by 8- 12 percent.90 The combination of these complementary efforts from other actors, and the school meals provided by WFP are all likely to have contributed to keeping the students (mostly girls) in school. “… we used to have a lot of absenteeism especially by girls during their time [menses], but now that the government has intervened through that program of sanitation, now there is not much absenteeism” Garissa key informant interviews 89 Scott, L., Dopson, S., Montgomery, P., Dolan, C., Ryus, C., 2009. Impact of Providing Sanitary Pads to Poor Girls in Africa. Working Paper. SAID Business School, University of Oxford. 90 Nauges, C. & Strand, J. (2007) “Estimation of Non-Tap Water Demand in Central American Cities”. Resource and Energy Economics 29: 165-182. School Closure Looking after siblings or domestic work Insecurity Working on farm or livestock tending Lack of food Sickness 100.0 80.0 60.0 40.0 20.0 0.0 Baringo Garissa Isiolo Marsabit Nairobi Samburu Tana River Turkana West Percentage Pokot of all days absent 29 120. The other indicator for attendance is primary school completion. Uwezo (2015) found a national primary school completion figures of 96.1 percent.91 This is significantly higher than the baseline, MTR or EoP found in the SMP targeted counties. Table 14 shows that the EoP found a completion rate of 54.4 percent, well below the 80 percent target, and consistent with the findings from the mid-term review. The Uwezo Kenya assessment report of 201692 does not indicate completion rates, focusing instead on other learning outcomes. It is therefore difficult to determine why completion rates are falling in the SMP counties. Interviews with school personnel indicate this may be partly due to incomplete records and/or inaccuracy of the student’s records in some schools, as well as frequent transfer of pupils from one school to another. Other factors include drop-out due to cultural practices such as early marriage, and household migration in search of pasture and water for their livestock. Table 14: Results indicator: Improved student attendance (completion) Target Baseline MTR EoP Percent of students in target schools who start grade one and complete their last grade of primary school 80 76.2 56.4 54.4 2.2.3.1.2 MGD SO 1.4: Increased student enrolment 121. Nationally, primary school enrolment in Kenya has been high with Uwezo describing rates of more than 90% each year between 2009-2012.93 Secondary data indicates that net enrolment is increasing (Annex 22), and this is supported by The World Bank, which found enrolment rates of around 90% between 2013-2015.94At the time of the SMP design (2013), the national net primary enrolment was 83.5% for boys, and 84.5% for girls (Table 15).95 Gender parity in enrolment has been achieved across the country, however the Uwezo 2016 report specifically mentions the ASALs as an exception, where more boys are enrolled than girls.96 Table 15: Net enrolment rates 2014-2016 Baseline MTR EoP National net primary school enrolment rates 83.5 boys, 84.5 girls 84.2 76.2 Average net primary school enrolment rates, SMP counties 61.7 84.5 122. At the mid-term evaluation, the average net enrolment rate in the SMP counties was 61.7 percent97, much lower than the national average. In 2016, the national net enrolment rate fell to 76.2%,98 while in the SMP counties it improved, with the latest enrolment rates average 84.5%. This is confirmed by school assessment data from the EoP evaluation, which found significant improvement99 in enrolment rates since the midline of 3.8 percent for boys, and 6.1 percent for girls, both above the 4 percent target 91 Uwezo (2015): Are Our Children Learning? The State of Education in Kenya in 2015 and Beyond. Nairobi: Twaweza East Africa. 92 Uwezo (2016) Are our children learning (2016)? Uwezo Kenya sixth learning assessment report. December 2016. 93 Uwezo (2015): Are Our Children Learning? The State of Education in Kenya in 2015 and Beyond. Nairobi: Twaweza East Africa. 94 The World Bank (2016) https://data.worldbank.org/indicator/SE.PRM.NENR 95 UNICEF (2015) State of the World’s Children: 2014 data 96 Uwezo (2016) Are our children learning (2016)? Uwezo Kenya sixth learning assessment report. December 2016. 97 At MTR, Garissa and Turkana counties performing particularly poorly with 39.9 percent and 54.4 percent respectively 98 http://data.uis.unesco.org/?queryid=142 99 The significance testing on the enrolment results can be found in Annex 31. 30 (Table 16). A MoEYS/WFP verification exercise in September 2017 found that girl’s enrolment in all the targeted counties is still below 50%, while boys enrolment is 51- 63%.100 This gender disparity is notable and largely explained by cultural practises such as girl’s early marriages, and community preference to take boys to school rather than girls. This finding highlights the importance of community education on the importance of education (discussed ahead), and the need for incentivizing girl’s participation in education. Table 16: Results indicator: Increased student enrolment Target Baseline MTR EoP Percentage increase in boys enrolled in school because of USDA assistance 4 3 9.2 3.8 Percentage increase in girls enrolled in school because of USDA assistance 4 3 7.7 6.1 123. Provision of food commodities to schools for the SMP is based on school enrolment numbers. WFP has therefore put in several verification procedures to ensure these numbers are accurate. WFP’s Field Officers make impromptu visits to the SMP schools to verify such information. This is done by comparing class attendance registers for the specific days against the quantity of food commodities that storekeepers provide for preparation on that given day. Despite this, the evaluation found instances of exaggerated enrolment numbers albeit often for well-intended reasons. For example, in Turkana County it was noted that the Head Teacher or teachers in-charge of the SMP had exaggerated the number of pupils so that they are able to provide school meals to the ECD children as well. 124. The evaluation notes that both WFP and the MoE are well aware of this issue. The MoE is planning to address this by using biometrics and working towards a National Integrated Information System (NIEMIS) to help provide clearer enrolment numbers. The NIEMIS will be launched in 2017. In the interim, WFP does a complete verification of school enrolment in every county under the transition to the HGSMP, and will be doing a verification in all remaining counties at the start of the next round of funding. 2.2.3.1.3 MGD SO 1.3.5: Increased community understanding of the benefits of education 125. Key informants indicate that many households in the arid counties have low recognition of the benefits of education. As described above under the student enrolment findings, households often prioritize pastoralism, and cultural practices such as early marriage and female genital mutilation that stop children (particularly girls) from accessing or completing their education. WFP supports the MoE to sensitize the PTAs and SMCs on the importance of education for both boys and girls. 126. WFP worked with partners to develop messages to be shared with members of PTAs and SMCs during organized trainings. The messages aimed to encourage parents and the community to embrace school enrolment, retention and completion for both boysand girls. Over the course of the SMP, 119 events including County Education Days, radio spots and advertising campaigns were held, well above the target of 44. These campaigns have reached 170 percent of the targeted number of beneficiaries (Table 17). Although it 100 MoEYS/WFP (2017) School Meals Programme – Verification of school enrolment in arid counties. Report, September 2017. 31 is not possible to attribute the improvements in enrolment to these campaigns, key informants rated them highly and encouraged WFP to implement such activities on a regular basis. Table 17: Activity indicator: Raising awareness on the importance of education Indicator Target Achievement Total 2014 2015 2016 Number of events, radio spots, advertising campaigns held 44 20 81 18 119 Number of community members benefiting from events, radio spots, add campaigns held 78,800 29,426 70,000 35,000 134,426 127. The EoP evaluation found that Garissa, Baringo and Samburu counties had achieved the target of 80 percent of parents being able to name three benefits of education (Annex 23) however, on average, only 68 percent of parents could name at least three benefits (Table 18). Ongoing campaigns are therefore necessary to ensure the messages are consolidated. Key informant findings indicate that the MoE is also engagingsupport from local chiefs and church groups to spread a message of the importance of education (and particularly for girls), and this is to be encouraged and continued. Table 18: Results indicator: Increased community understanding on the benefits of education Target Baseline MTR EoP Percent of parents in target communities who can name at least three benefits of primary education 80 66 88101 68 2.2.4 Thematic area: Food utilization and food safety 2.2.4.1.1 MGD Strategic Objective 2: Increased use of health and dietary practices 128. An important component of this project is the emphasis on hygienic food preparation and appropriate food storage. Having appropriate food storage in place not only improves the life of the SMP food but also gives the schools a good grounding in appropriate food storage so they will be well prepared to transition to the HGSMP. For 101 Data from Garissa was excluded due to enumerator error. Summary of results on the participation in education • Enrolment rates in the targeted arid counties were well below the national average, but have improved over the course of this program. The community sensitization campaigns on the importance of education, particularly for girls may have contributed to this. • No significant change was noted in attendance rates, which were already high (86%) atbaseline. Key informant interviews provide multiple reasons for school absences, of which sickness it the main reason. • Both the MoE and WFP have engaged support from local chiefs and church groups, but more work is needed to enable parents to better understand the benefits of education. • Primary school completion rates remain low (54%). 32 schools to be able to appropriately store and prepare the food provided by WFP, the SMP provides training to teachers on food preparation and storage practices. The objectives of which is to encourage schools to store the WFP food off the ground, and prepare it in hygienic conditions. 129. WFP provided two trainings a year (first and third terms) in all the targeted districts and in all the SMP schools. At the district level, the training targeted SCDEs, School Feeding Program Officers and QASOs to increase their capacity to conduct on- the-job training for teachers during routine monitoring. At the school level, WFP provided trainings to teachers, cooks and storekeepers. This training covered aspects of reporting and commodity management with more emphasis on accountability at the school level. 130. Figures 9 and 10 shows that WFP has exceeded its targets on the number of trainings provided (390%) and the number of teachers trained in food preparation and storage practices (272%) (Annex 24). Figure 9: Number of trainings provided on food preparation and storage techniques Figure 10: Number of teachers trained in food preparation and storage practices 131. Despite all trainees passing the training, Table 19 shows that only 76.5 percent of schools stored the food off the ground at EoP. Only in one county, Isiolo, did all surveyed schools store food off the ground. Three other counties - Garissa, Baringo and Samburu were within 10 percent of achieving the target for this indicator. Clearly, there are other factors involved in the decision of how to store food, not just a lack of knowledge of appropriate practices. Annex 24 shows the dataset of school having dedicated storerooms, by county, and also provides some photos of substandard food storage noted by the evaluation team during school visits. Planned Achieved 100 80 60 40 20 0 2014 2015 2016 Total Planned Achieved 8000 6000 4000 2000 0 2014 2015 2016 Total Number of trainings provided Number of teachers trained 33 Table 19: Results indicator: Increased knowledge of safe food preparation and storage practices Target Baseline MTR EoP Percent of food preparers at target schools who achieve a passing score on a test of safe food preparation and storage 100 87 97.9 100102 Percent of schools in target communities that store food off the ground 100 67 96 76.5 2.2.4.1.2 MGD Strategic Objective 2.6: Increased access to requisite food preparation and storage tools and equipment 132. It is important to note that although the SMP provides training to storekeepers and cooks on food preparation and storage, the program does not provide financial resources for food preparation or storage tools and equipment. Instead, WFP supports the MoE to improve their infrastructure though other sources of funding. WFP also has a complementary project supporting schools to acquire modern energy-efficient stoves (jikos). In total 400 energy-saving jikos have been installed in SMP schools between 2014- 2016. 133. The evaluation finds that this component of the SMP has been well implemented, with great improvements in the number of schools having dedicated kitchens and storerooms by the EoP and both results indicators have been reached (Table 20).103 All counties except West Pokot and Garissa, have met the 80 percent target. School without dedicated kitchen facilities prepared food in open spaces. Annex 25 shows some photos of substandard kitchen, and provides the data set of schools having dedicated kitchens, by county. Table 20: Results indicator: Increased access to requisite food preparation and storage tools and equipment Target Baseline MTR EoP Percent of target schools with improved preparation and storage equipment Dedicated kitchen 80 60 81 82 Dedicated storeroom 80 67 80 80 Food preparation equipment 134. The evaluation found that although most schools (82%) had dedicated kitchens, only half (53%) used energy saving stoves (Figure 11). Evaluation findings presented in Annex 25 shows that while all surveyed schools (100%) in Nairobi and Isiolo had energy efficient stoves, none were found in West Pokot or Baringo. Although WFP did provide 102 The EoP survey assessed only 10 attributes out of the initial 16 at baseline. These constitute of 62.5% of the attributes. 50% of the 62.5 gives a threshold score of 31.3%. If a 50% passing mark of the 10 attributes is used, the proportion of food preparers who achieved a passing score comes down to 87.4%. 103 One-Sample Wilcoxon Signed Rank Test indicate that the number of schools having dedicated kitchens and storerooms by the EoP attained the program targets (kitchens p = 1.000; storerooms p =0.317). In both cases, the observed median percentage was not significantly different from the projected median percent change. 34 stoves to West Pokot, financial constraints meant that Baringo did not receive any energy￾efficient stoves. Figure 11: Attributes of the SMP school cooks and kitchens 135. The evaluation also found that although 82 percent of kitchens were clean, only 29 percent of cooks reported having training in safe food preparation and storage, and less than half (46%) had a valid health certificate. In addition, only 37 percent of cooks reported having a uniform or apron, and the evaluation team found only 64 percent of cooks to be clean and well-groomed. Almost all cooks (99%) reported that children washed their hands before meals (99%). 136. Key informants in schools without dedicated kitchens reported the main reasonfor not having such as space was competing priorities for limited financial resources. Schools without dedicated kitchen facilities said that parents and community lacked the financial means to support construction of a kitchen.In those schools, the cooks prepared the meals on a fire. The lack of modern facilities was reported as an inefficiency in meal preparation as a greater quantity of fuel (firewood) was used, and the cooking time was greatly increased, sometimes delaying meals. Storage tools 137. The evaluation notes that while the two main storage indicators have been met, several food storage issues were identified. Figure 12 shows that while most surveyed storerooms were lockable (95.2%) and properly ventilated (79.8%) only around half the surveyed storekeepers (58.3%) have been trained in safe food preparation and storage practices. This may account for the high proportion of surveyed schools where evidenceof presence of rodents (44%) or presence of insects (50%) was noted, despite most schools reporting carrying out pest/insect control measures (69%). Evidence of spillage or leakage was found in 19 percent of surveyed schools. Mould and excess humidity were not major issues (7.1%). 100 80 60 40 20 0 Cook is Cook is clean Cook has a trained in safe and well uniform or food preparation and storage groomed apron Kitchen is Kitchen has a Kitchen has Children wash clean valid health an energy hands before certificate efficient stove meals Percentage of cooks and kitchens 35 Figure 12: Attributes of SMP food stores 138. Key informants indicate that for the relatively few schools that lacked lockable food storage, the storage and security of the food is a challenge. Some schools used classrooms or offices to store food, particularly when there were no classes. This means that multiple people had access and the appropriate food storage conditions cannot be met. “… another one of the key challenges is storage; for example, in our school we have only three rooms, we have four classes. We are using part of our classes as a store. That is a challenge because I believe it is making a class looks like a shop.” West Pokot key informant interviews 139. The evaluation also collected data on the sanitation (toilet) infrastructure in the SMP schools. The findings indicate that all schools have toilet facilities, with 87 percent of schools providing separate toilets for girls and boys (Annex 26). 100 80 60 40 20 0 Storekeeper Food store is Food store is Evidence of Evidence of Evidence of Evidence of Use of School carries trained in safe lockable food preparation and storage practices properly presence of presence of mould and ventilated rodents in the insects excess of spillage or leakage improvised out raised pallets pest/insect store humidity for control commodities measures storage Percentage of schools Summary of results on food utilization and food safety: • WFP provided two trainings a year on food preparation and storage in all the targeted districts and in all the SMP schools. In total, 86 trainings were provided, to more than 7,000 teachers. • The USDA program does not provide the financial resources for food preparation or storage tools and equipment. Instead, WFP supports the MoE to improve their infrastructure though other complementary funding mechanisms. Kitchens • Most SMP schools have dedicated kitchens, which is a considerable achievement from the baseline. Schools without dedicated kitchen facilities reported inefficiencies in program implementation, including longer cooking times, and greater use of firewood. • Half the schools (53%) use energy-efficient stoves. Although all surveyed schools (100%) in Nairobi and Isiolo had energy efficient stoves, none were found in West Pokot or Baringo. • The evaluation found that although 82 percent of kitchens were clean, more work needs to be done to ensure that cooks have adequate training, and all kitchens have a valid health certificate. Storerooms • Most SMP schools have dedicated storerooms, and store their food off the ground. More work is required to ensure that pest/insect control measures are adequately carried out, as evidence of presence of rodents or presence of insects was found in almost half the stores. 36 2.2.5 Thematic area: Increased national capacity 2.2.5.1.1 MGD Strategic Objective 1.4.1: Increased capacity of government institutions 140. WFP supports capacity building activities for the MoE at county, district, and school level to ensure that appropriate procedures are in place for a transition to the HGSMP. This includes supporting the MoE in the production and distribution of two manuals: the HGSMP and the National School Health Guidelines. • The HGSMP manual contains information related to carrying out a successful school feeding program. This includes meal planning, budgeting, warehousing and storage, reporting and monitoring, procurement and government policy. • The National School Health guidelines cover aspects of safe food preparation, handling and storage practices. 141. Over the course of the SMP, WFP has distributed more than 16,000 manuals to over 8,000 MoE staff (Figures 13 & 14). This has exceeded WFPs targets (188% and 129% respectively) (Annex 26). The intention of distributing the manuals is to enable schools to set up appropriate food procurement and distribution procedures, and help ensure that appropriate infrastructure is put in place. Figure 13: Number of HGSMP manuals distributed to MoE by WFP Figure 14: Number of MoE officers benefiting from HGSMP manual distribution 142. Table 21 shows that all (100%) of the targeted schools currently have these in place including a system of record keeping and monthly physical inventory records in place at the county level. Planned Achieved 20,000 15,000 10,000 5,000 0 2014 2015 2016 Total Planned Achieved 10,000 8,000 6,000 4,000 2,000 0 2014 2015 2016 Total 37 Table 21: Results indicator: Increased capacity of government institutions Target Baseline MTR EoP Percent of districts in which food procurement and distribution procedures and infrastructure are in place 100 85.4 100 100 2.2.5.1.2 MGD 1.4.2: Improved policy and regulatory framework 143. Another component of WFP’s capacity building work is supporting the development of relevant national policies, regulations and procedures. During this phase of implementation, WFP has supported a total of five Government policies or procedures (Table 22). WFP supported the development of the National School Health Policy and the School Meals and Nutrition Strategy, which are both at Stage 2 – they have been drafted and presented for public/stakeholder consultation. The HGSMP Implementation Guidelines are now at Stage 4 – passed/approved after being supported during a previous phase of the SMP (in 2011). Other policies passed in 2014 and 2015 with WFP’s support include the Basic Education Act (2014), and the Kenya Health Policy 2012-2030. Table 22: Results indicator: Improved policy and regulatory framework Target Baseline EoP Number of child health and nutrition policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance 3 3 5 144. Overall, the evaluation found that WFP plays a key support role to the MoE and other relevant ministries on policy development. Key informants provided overwhelmingly positive feedback on WFP’s capacity building work, reflecting the importance of the program in the ASALs. Key informants recognized WFP as an authority on food procurement, distribution and storage, and were pleased with activities that shared some of their experience such as the trainings for teachers, and distributionof manuals. 145. WFP’s indicator of national capacity for this program is the School Feeding National Capacity Index. This is a subjective indicator based on the five Systems Approach for Better Education Results (SABER) policy goals104 and calculated by WFP and MoE together. The Index improved from 13 at baseline to 14 at latest follow up (December 2015). Although this does not meet the target of 18, there is some improvement. 2.2.5.1.3 MGD SO 1.4.4: Increased engagement of local organizations and community groups 146. WFP uses a participatory approach to the implementation of SMP, by engaging with district and school level MoE officers, and encouraging community engagement through PTAs and SMCs. Table 23 shows that more than 80 percent of schools reported that their PTAs and SMCs contribute to their school. However, the evaluation also notes that when surveyed, only 60 percent of households reported being aware of the existence of the SMC. 104 World Bank/WFP (2013) Systems Approach for Better Education Results: School Health and School Feeding 38 Table 23: Results indicator: Increased engagement of local organizations and community groups Target Baseline MTR EoP Percentage of PTAs and SMCs contributing to their school as a result of USDA assistance 80 70 PTAs: 93.8 SMCs: 66.7 PTAs: 82 SMCs: 81 147. The school data above was confirmed by the household interviews which indicates that most households provide 5-30 KES per day as a contribution to the SMP (Table 24). Household interviews and key informant interviews confirmed that in addition to the financial support, parents were heavily involved in the non-food, supportive activities required for the implementation of the SMP (Figure 15). On average, parents provide more than 80 percent of the water, firewood, cooking and eating utensils, cleaning products and cook’s salary. Some key informant interviews noted that in many cases, parents ask their children to collect the water and firewood on the way to school. Teachers noted that this often adversely affects attendance, with students coming late to school. In the unplanned settlements of Nairobi, parents have also initiated income generating activities such as greenhouse farming, fishponds which acts as a food and income source. Table 24: Household reporting of monies provided for school meals County Minimum provided Maximum provided Median provided Tana River County 2 40 5 Isiolo County 2 25 5 Nairobi County 10 51 20 Samburu County 20 20 20 Turkana County 5 50 25 Garissa County 30 30 30 Marsabit County 30 30 30 Figure 15: Proportion of support to non-food inputs, provided by parents 148. WFP actively encourages women’s participation in SMP decision making, resulting in equal representation of women within the School Management Committees (SMCs). Although Table 25 shows that WFP has not achieved the targets in these areas, there has been considerable improvement since the baseline. 100 80 60 40 20 0 Water Firewood Cooking utensils Cleaning products Eating utensils Cooks salary Other Percentage of households 39 Table 25: WFP cross cutting indicators - gender Target Baseline Latest follow up Proportion of beneficiaries in leadership positions of project management committees >50 20 30 Proportion of women project management committee members trained in modalities of food, cash or voucher distribution 100 20 67 149. In addition to community engagement, the SMP has resulted in the formation of several public-private partnerships. The WFP Standard Programme Reports over the 2014-2016 period indicate thirteen new public-private partnerships were formed because of USDA assistance. These include the following organizations: International Paper, DSM, FEED, Caterpillar, Earth Holdings, Government of Kenya, Unilever, Drew Barrymore, Princess Haya WPD, IRB, JAWFP, LG Electronics and Goodeed Association. The value of these partnerships amounts to US$15,381,303. No additional public-private partnerships were identified by the EoP evaluation. 2.2.5.1.4 WFP Protection indicator: Proportion of assisted people who experience threats to safety problems travelling to, from and/or at WFP program site 150. As well as the MGD program performance indicators, in all WFP programs, the proportion of assisted people who experience safety issues at program sites is monitored. Table 25 shows the security threats identified by parents at WFP SMP sites. The EoP evaluation found lower levels of rape, sexual harassment and drug abuse than found at the Mid-Term Evaluation. However, the level of robbery, animal attacks and bullying have increased considerably. When disaggregated by county, the data shows that the counties reporting the highest number of security threats were Marsabit (n=239), Tana River (n = 205), and Isiolo (n= 133) (Table 26). Both Marsabit and Tana River had high levels of animal attacks, while households in Isiolo report robbery and bullying (Annex 28). Table 26: Security threats identified at WFP SMP sites Target Baseline MTR EoP Proportion of assisted people who experience threats to safety problems travelling to, from and/or at WFP program site 0 _ Rape 4.9% Rape 0.2% Sexual harassment 4.9% Sexual harassment 0.4% Robbery 3.7% Robbery 9% Animal attacks 8.2% Animal attacks 41.5% Bullying 3.5% Bullying 14.9% Drug abuse 4.4% Drug abuse 0.7% 151. Annex 28 shows additional findings re whether security threats were related to the commute to school, or around the school feeding set-up per se. Overall, the majority of threats (78%) relate to the commute to school. 40 152. In response to a recommendation from the mid-term evaluation, in 2016 WFP introduced a feedback hotline in all USDA supported counties. This was intended to improve accountability and enable people to contact WFP and make a complaint or provide feedback about the program. The EoP evaluation was conducted just after the hotline was rolled out, and it found that only 36.9% of parents were aware that there were any avenues through which complaints can be made. Out of the group aware that they could make complaints, only 14 households (3.4%) mentioned the hotline (Table27). Most of those households were in Isiolo and Turkana Counties (Annex 29). Other reported ways to provide feedback included meetings with School Administrators (46.7%), or with the SMC (39.8%). More work evidently needs to be done on community awareness raising of the presence of the hotline, and the possibility to provide feedback. WFP is currently in discussion with the MoE to support the introduction of a helpline in government￾supported HGSMP. Table 27: Household reporting of possible ways they can make complaints Avenue Frequency Percent Valid Percent Regular meetings with SMC 165 14.7 39.8 Regular meetings with School Administrators 194 17.3 46.7 Suggestion Box 2 0.2 0.5 Hotline 14 1.2 3.4 Other (specify) 40 3.6 9.6 Total 415 36.9 100 Missing System 709 63.1 Total 1124 100 Summary of results on national capacity: • WFP provides considerable support to the MoE to build their capacity to implement the HGSMP after transitioning from SMP. The government key informants provided overwhelmingly positive feedback on WFP in this regard. • The MoE recognizes WFP as an authority on food procurement, distribution and storage. • WFP has contributed to the development of five national policies and strategies and helped the MoE produce and distribute two manuals: the HGSMP and the National School Health Guidelines. • All (100%) of SMP schools have set up appropriate food procurement and distribution procedures, including a system of record keeping and monthly physical inventory records in place at the county level. • WFP’s indicator of national capacity for this program is the School Feeding National Capacity Index. The Index improved from 13 at baseline to 14 at latest follow up (December 2015). • WFP’s participatory approach to SMP implementation has promoted active engagement in the SMP particularly by the PTA and SMC. The approach has also improved gender equalityas there are equal numbers of women and men on the SMCs. • There has been significant improvement in the inclusion of women in the SMC and the inclusion of women in leadership positions. However, this figure still remains under target, as set at the beginning of the project. • The evaluation found some security threats at WFP SMP sites, including high levels of animal attacks (41.5% of parents reporting), and bullying (14.9%). • Only 36.9% of parents were aware that there were any avenues through which complaints can be made to WFP. 41 2.2.6 Thematic area: Efficiency of program delivery 153. For this evaluation, the efficiency of program delivery has been assessed largely by qualitative means, asking key informants whether the SMP activities were implemented in the most efficient way compared to alternatives. The team also assessed whether there were any internal or external factors influencing the efficiency of the program (attainment of the planned outputs, cost factors, logistics and pipeline performance). 2.2.6.1.1 Perceived efficiency of SMP in delivering its outputs 154. Key informants were asked to name issues that affected the efficiency of SMP implementation – particularly the provision of food to the targeted schools for the provision of school meals. Figure 16 shows the three areas most frequently positively identified by key informants were the WFP food delivery system (“transportation by WFP”), the procurement of cereals, and WFP’s training (of MoE personnel and teachers). Figure 16: Key informant views on SMP implementation efficiency 155. Less frequently mentioned, but also considered positively, was the community involvement in supporting the SMP. Key informant interviews with WFP personnel identified the use of an integrated supply chain management system known as Logistics Execution Support System (LESS)105 as positively supporting efficiency. The LESS aided the monitoring and documentation of the supply while ensuring seamless supply ofschool meals. On the negative side, food transportation by the SCDE was the most frequently mentioned issue. Food storage within the schools, insecurity, food loss/theft and the SMP reporting systems were also frequently mentioned. SMP schools did not report any major issues with WFP transportation of food until pipeline breaks occurred in 2015 due to funding constraints from non-USDA donors, including another key donor who was unexpectedly unable to maintain their commitments 156. Table 28 shows that in 2015 only 52% of planned commodities were distributed. These pipeline breaks together with the national teachers strike from 31 August to 5 October 2015 account for the low volume of food distributed in 2015 compared to planned. 105 LESS is a project that was developed by WFP in the year 2014 to achieve integrated supply chain management. The project integrates supply chain, inventory accounting and real-time tracking systems for improved delivery of Logistics Function Frequency Transportation by SCDE Training Transportation by WFP Storage within schools Insecurity Schools SMP reporting Procurement of cereals Food losses Community provisions Involvement of local Untrained teachers adminstration MoEST supervision MoH and MoEST coordination Overlapping activities Sale of food -ve Pipeline breaks Intensity +ve 42 Table 28: Planned vs. Actual commodities distributed (2015) Commodity Planned distribution (mt) Actual distribution (mt) % Actual vs. Planned Beans 260 - - Bulgur Wheat 4,348 6,382 146.8% Corn Soya Blend (CSB) 1,544 595 38.5% Iodized Salt 447 220 49.1% Maize 16,095 4,399 27.3% Rice - 2 - Split Peas 4,934 2,750 55.7% Vegetable Oil 746 397 53.3% TOTAL 28,375 14,744 52% 157. Figure 17 provides a summary of the key activities related to the provision of food for SMP. Counties implementing SMP identified the storage and transportation by the SCDEs to the schools, as the weakest link in the pipeline. Interviews with Head Teachers indicated that relying on the SCDE for transportation of food to schools led to delays in delivery, reducing the number of days that the schools could provide meals. Head Teachers also reported occasionally contributing their private funds to pay transporters to deliver the food products from the SCDE storage, and sometimes, when funds were low, payment was done by offering food products to the transporter instead of paying cash. Figure 17: Flow of SMP food provision “At least we should have enough transporters so that the food can be delivered in time, so that schools can start smoothly without delays”. Turkana key informant interviews 158. Key informant interviews also identified unclear roles and responsibilities, especially after transitioning to the HGSMP. The responsibility of meeting the cost of loading and off-loading the trucks was not clear, with several Head Teachers reported having to pay for both loading and off-loading. This was observed in almost all the counties except Nairobi, especially in the more remote areas. Key informants felt the main reason behind this was a lack of proper organization in contracting the transport by Targeting and School Selection Training of the Schools Procurement of Food Supplies Warehousing atthe regional Offices Transportation by WFP to DEOs Daily provisions for Preparation Storage at Schools Receipt at Schools Transportation by SCDEs to Schools Storage by SCDEs Preparation of the food Serving the food to the pupils School Based Reporting on SMP 6 KEY Blue- WFP led processes Red- SCDE managed processes Black- School Managed Process 43 the sub county education office, which led to pipeline problems and delays in delivery to schools. 159. Schools that had already transitioned to the HGSMP also reported reduced number of days of school feeding but for different reasons. In the HGSMP schools the delays were mainly because of delayed disbursement of funds to schools by the National Treasury. These delays were more significant, as they delayed the whole food procurement process – from the tendering, procurement by traders and delivery of food to the schools. Overall, the evaluation found that the management by WFP on the provision of food was fit-for￾purpose. WFP has managed the program efficiently, contributing to the successful provision of food commodities. Effective planning, control and communication processes were also found to be in place. 160. At school level, lack of water and firewood for cooking were the two priority issues identified by key informants, both in SMP and HGSMP schools despite them being rated positively in Figure 19. The lack of water and cooking fuel (firewood) affects the school’s ability to provide meals in a timely manner. When combined with a lack of appropriate kitchen facilities and food storage as previously identified, meal preparation time isgreatly extended, often resulting in late meal times in schools. The evaluators also noted inefficiencies in the time between the completion of program monitoring forms SM6 and the arrival of the data at WFP, up to 3 months, which greatly reduces the utility of the information. The evaluation therefore finds that a more efficient way of collecting, collating and dispatching the data would be appropriate. A cost comparison (Table 29) shows that the paper system is approximately four times more expensive than an electronic alternative. Table 29: Hypothetical cost comparison of paper versus technology enabled reporting106 Estimated current cost of paper system Estimated cost of SMP6 reporting using electronic system USD USD Reproduction cost per SMP6 form 0.05 Purchase of tablets and solar enabled power packs 120 Data entry cost of SMP 6 form (Per Form) 1 Design of system 2,500 Return transportation costs per unit 1 Utility cost per month per school (subscription) 2 Data verification cost per month 3 Data costs per month per school 3 TOTAL cost $5.05 per form Number of Schools 1400 Number of schools 1400 Number of SMP 6 forms per term (Triplicate) 90 Total one-off cost $170,500 Total recurrent cost per year USD$ 636,666 Total recurrent cost per year $63,000 Total cost over life of project $1,910,000 Total cost over life of project $359,500 106 This is compared to a paper based approach and uses a customisable Open Data Kit program that runs on mobile devices running Google’s android operating system. 44 161. The evaluation team understand that that an electronic system would rely on mobile phone coverage and/or internet connections, and would require training. However, the team also note that WFP Kenya has been successfully implementing mobileVulnerability Analysis and Mapping (mVAM)107, a mobile-based platform for monitoring of the supply chain using both voice calls and SMS. Key informant interviews indicate that implementation of this project has been very useful, even in the insecure areas of Mandera – and the project management team can get real-time data in Nairobi. Overall, the evaluators established that the current paper reporting system was less efficient and more costly than possible electronic options. 2.2.7 Summary of findings – Evaluation Question 2 162. Table 30 provides a summary of the key results of the SMP and indicates whether targets have been met. The full updated Performance Monitoring Plan can be found in Annex 30. Table 30: Summary of key results - Evaluation Question 2 Result Indicator Target Baseline EoP Contributions to Feed the Future Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance <1,020,483 767,108 763,490 Improved Literacy of School-Age Children Percent of students (girls/boys) who, by the end of 2 years of school demonstrate reading comprehension equivalent to their grade level as defined by national standards at USDA supported ≥48.8 48.8 46.1 107 This project was launched in 2013, and uses mobile technology to track food security trends in real time, providing high frequency data that supports humanitarian decision-making. In order to achieve high performance, data collection methods are tailored to the needs of the country in which the project operates. Summary of results on efficiency of program delivery: • The three areas most frequently positively identified by key informants were the WFP food delivery system (“transportation by WFP”), the procurement of cereals, and WFP’s training (of MoE personnel and teachers). Less frequently mentioned, but also considered positively, was the community involvement in supporting the SMP. • WFP personnel identified the use of an integrated supply chain management system known as Logistics Execution Support System (LESS) as positively supporting efficiency. The LESS aided the monitoring and documentation of the supply while ensuring seamless supply of school meals. • The counties implementing SMP identified the storage and transportation by the SCDEs to the schools, as the weakest link in the pipeline. • Pipeline breaks occurred in 2015 due to funding constraints from non-USDA donors, including another key donor who was unexpectedly unable to maintain their commitments. In 2015 only 52% of planned commodities were distributed, and overall, 71% of planned commodities have been provided. • At school level, lack of water and firewood for cooking were the two priority issues identified by key informants, both in SMP and HGSMP schools despite them being rated positively • The evaluators established that the current paper reporting system was less efficient and more costly than possible electronic options. 45 schools Number of total individuals benefiting directly from USDA￾funded interventions <1,020,483 767,108 763,490 Number of total individuals benefiting indirectly from USDA￾funded interventions <1,135,467 536,758 527,984 Increased Capacity of Government Institutions Percent of districts in which food procurement and distribution procedures and infrastructure are in place 100 85.4 100 More Consistent Teacher Attendance Percent of teachers in target schools who attend and teach school at least 90% of scheduled school days per year 90 51 89.7 Improved Attentiveness Percent of students in classrooms identified as inattentive by their teachers ≤20 20 15 Reduced Short Term Hunger Percent of students in target schools who regularly consume a meal before the school day 85 41 53 Percent of students in target schools who regularly consume a meal during the school day 90 80 67 Increased Access to Food (School Feeding) Percent of students in target schools consuming daily meals at school 100 100 91.5 Improved Student Attendance Total students regularly (80%) attending USDA supported classrooms/schools 95 85.6 86.4 Percent of students in target schools who start grade one and complete their last grade of primary schools 80 76.2 54.4 Increased Use of Health and Dietary Practices (See RF 2) Percent of schools in target communities that store food off the ground 100 67 76.5 Increased Engagement of Local Organizations and Community Groups Percentage of PTAs and SMCs contributing to their school as a result of USDA assistance 80 70 PTAs: 82 SMCs: 81 Improved Policy and Regulatory Framework Number of child health and nutrition policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance 3 3 5 Increased Knowledge of Safe Food Prep and Storage Practices Percent of food preparers at target schools who achieve a passing score on a test of safe food preparation and storage 100 87 100 46 Increased Access to Requisite Food Prep and Storage Tools and Equipment Percent of target schools with improved preparation and storage equipment 80 Dedicated Kitchen: 82 Dedicated Storage: 80 Increased Student Enrolment Percentage increase in boys enrolled in school as a result of USDA assistance 4 3 3.8 Percentage increase in girls enrolled in school as a result of USDA assistance 4 3 6.1 Increased Community Understanding of Benefits of Education Percent of parents in target communities who can name at least three benefits of primary education 80 66 68 Summary of key findings Evaluation Question 2 – What were the results of the School Meals Program? • Through the support of multiple donors WFP has successfully provided almost 52,794 MT of high quality food commodities, to 1,766 targeted SMP schools. This includes 19,260MT from USDA. This has enabled the provision of school meals to almost two million school children between 2014-2016. School performance • The targeted arid SMP counties were all below the national average of literacy and numeracy at the start of the program period, and remain so in 2016. It is clear that without the inclusion of specific literacy-related activities within the programme, school feeding cannot improve school performance as the responsibility for achieving education outcomes lies largely with the Ministry of Education (MoE). Nationally, literacy and numeracy rates have shown little progress over the course of the SMP, and the Government of Kenya is engaging in several initiatives to improve this. • WFP’s contribution to school performance is to provide a school meal each day, to enable children to concentrate of their studies. An objective, composite indicator shows that the number of inattentive children has shown significant improvement since baseline. Baringo County was the only county that has not achieved the ≤20 percent target. Teachers reported that hunger was the main cause of inattention in class (38%). Provision of school meals • Only around half the surveyed students consume a meal before attending school, so the school meal makes a significant contribution to the food security of the children. However, almost a third of surveyed parents reported reducing the portion of food served to their children after school. This reduces the benefit of the school meals as a food security intervention. • The evaluation found that school meals have been provided 65-70% of school days over the course of the program, in part due to funding constraints, and partly due delays in food delivery by the SCDE. This again reduces the contribution of the SMP to food security. Participation in education • Primary school enrolment in the targeted counties has improved since the baseline, and the campaigns on the importance of education, particularly for girls by MoE/WFP may have contributed to this. • No significant change was noted in attendance rates, which were already high (86%) atbaseline. • Primary school completion rates remain low (54%). 47 Food utilization and safety • WFP provides training on food preparation and storage in all the targeted districts and in all the SMP schools. WFP then supports the MoE to improve their infrastructure though alternative funding mechanisms. • The program has achieved the targets of the proportion of schools with dedicated school, dedicated storerooms, and storing food off the ground. However, more work is needed to ensure all schools have access to energy efficient stoves, other appropriate food storage practices and adequately trained kitchen and store personnel. National capacity • WFP provides considerable levels of support to the MoE to build their capacity to implement the HGSMP after transitioning from SMP. The government key informants provided overwhelmingly positive feedback on WFP in this regard. The MoE recognizes WFP as an authority on food procurement, distribution and storage. • WFP’s participatory approach to SMP implementation has promoted active engagement in the SMP particularly by the PTA and SMC, and there has been significant improvement in the inclusion of women in the SMC and the inclusion of women in leadership positions. Efficiency of program delivery • The three areas most frequently positively identified by key informants were the WFP food delivery system (“transportation by WFP”), the procurement of cereals, and WFP’s training (of MoE personnel and teachers). Less frequently mentioned, but also considered positively, was the community involvement in supporting the SMP. • The counties implementing SMP identified the storage and transportation by the SCDEs to the schools, as the weakest link in the pipeline. • The evaluation found some security threats at WFP SMP sites, including high levels of animal attacks and bullying. However, only 36.9% of parents were aware that there were any avenues through which complaints can be made to WFP. 2.3 Evaluation Question 3: How and why did the School Meals Program produce the results? 2.3.1 Internal issues affecting the implementation of SMP 163. WFP Logistics systems and operational guidelines and standards: WFP has substantial expertise in procurement, storage and delivery of food commodities. They also have many forms of operational guidance to support their logistics systems, including strict quality standards. The SMP has implemented food delivery using this system, although they have given the responsibility of the transport and delivery of the commodities to contracted transporters. High quality food was delivered to the counties in a timely manner with no complaints, and minimal loss of food during transportation was reported. 164. WFP Monitoring system: A comprehensive monitoring system has been utilized by WFP and the findings and analysis from it has contributed to the efficient implementation of the program, and changes in implementation as required. WFP has effectively monitored the required 12 results indicators and 21 activity indicators as per the agreed USDA/WFP agreement. 165. A limitation of the current monitoring system is that the monitoring of the provision of food for schools is done through paper forms. The forms capture the amount of food received, daily consumption and monthly summaries. While the volume of work was found to be cumbersome for teachers, the summary is effective at indicating the balance of food from the previous month, the receipt for the month, consumption, losses 48 and reasons for the losses like rodent infestation, theft or condemned. Since food deliveries were based strictly on reconciliation, this summary form is key. The Mid-Term Evaluation recommended engaging in capacity building activities at the school level to strengthen record-keeping and filing practices, and this has been done. 166. The evaluation notes the high level of commitment of the staff involved in transportation, receipt and usage. It also finds that an electronic system would be more efficient and cost-effective. 167. The main issues of concern in relation to SMP monitoring was poor supervision in more remote locations, particularly by MoE personnel. This was reported by key informants in multiple areas including Baringo, Samburu, and Garissa. The Mid-Term Evaluation recommended strengthening the joint monitoring system by increasing awareness of it at county level and considering a cost-sharing mechanism between WFP, MOE and MoH. Although this has not yet fully been addressed, the evaluation recognizes that this has been budgeted for in the 2016 grant from USDA to WFP and plans are in place, once county level coordination committees are established. 2.3.2 External issues affecting the implementation of SMP 168. Funding: The SMP is a multi-donor project, and although USDA has provided all the commodity and financial contributions as stipulated in the grant agreement, overall the project has been under-resourced. This has resulted in lower than planned commodity volumes being provided, contributing to schools being unable to provide school meals every school day. 169. Government of Kenya financial allocations and systems: Late disbursement of government funds to schools from the National Treasury, was the main reported hindrance to the HGSMP implementation in the counties that have transitioned during this period. While this does not affect SMP significantly, it makes transition to HGSMP more difficult. The government’s level of funding for HGSMP is also inconsistent year to year, significantly affecting county level officials’ capacity to implement and monitor HGSMP activities. More information on this can be found in Section 2.4.2. 170. MoE responsibilities: Support from the MoE is critical to the success of the SMP as they are responsible for all the infrastructure and personnel connected to the schools. The MoE has provided significant financial support and budget allocation for theSMP including providing the funds for schools to purchase pens, books and chalks for thepupils under the free primary education (FPE) program. Under the FPE, schools are allocated 1,020 KES per child, annually for these costs.108 However, key program indicators such as literacy and completion rates have not been met and require ongoing commitment and support from the MoE. 171. County governance: The evaluation received numerous complaints from teachers related to the transportation of food to the schools from the SCDE storage site. Although officially the SCDE is supposed to deliver food directly to the school, in some locations the SCDE organized food delivery to a central location and then the Head Teachers had to organize additional transport from there. As a result, several teachers reported using their own private funds for the transportation of food items to the school. In addition, the lack of proper organization within the County Governments in 108 Republic of Kenya Ministry of Education, “Education for All” National Review 2014 49 contracting the transporters, leads to delays in food distribution which resulted in some schools in remote areas getting their food late into the school term. 172. Another issue related to county governance is the devolution of ECD centres. In 2013 ECDs were removed from the SMP program as the responsibility for them shifted from the MoE to the county governments. However, for various reasons, primary schools still get ECD pupils in their school and need to provide meals for them, leading to reduced food servings per pupil. The County’s role in supporting ECD feeding program is still unstable. The evaluators note that at central level, the MoE commitment to the SMP/HGSMP is strong. However, county level authorities have different levels of commitment to both the SMP and to ECD, as well as different levels of capacity. 173. Partnerships and Collaborations: Aside from the MoE, WFP has collaborated with various government ministries including the Ministry of Health through the Public Health Officers, who inspected the storage facilities, the kitchens and trained the Head Teachers, cooks and School Meal Officers on food safety and handling. Most key informants correlated this with reduced cases of food contamination. 174. WFP also coordinated with several Non-Governmental Organizations and with UNICEF on activities aimed to improve attendance in schools. For example, Welthungerhilfe (German Agro-Action) donated water tanks in Tana River, built water reservoirs for rain water tapping, provided energy saving stoves, and drilled boreholes in the schools and in the communities around the schools. World Vision and other agencies also provided sanitary products for girls to ensure school attendance. These activities have contributed to the support to households in the SMP targeted areas. 175. Parental engagement: In many schools, parental engagement in the SMP has been high, with parents taking leading roles on the SMCs and participating in the PTAs. Parental engagement varies by county, with some locations able to provide less financial support. This affects the construction of appropriate facilities – particularly kitchens. In some counties that cost is prohibitive and requires support and collaboration from other actors. 176. Physical Environment: Defining characteristic of the SMP counties is the harsh terrain and long distances from Nairobi. These present several challenges to the distribution of food. Food commodities are often transported during the holidays to give the transporters ample time to deliver on time. The terrain has also hindered regular supervision of the SMP by MoE personnel. Seasonal rains exacerbate this problem, making access even more difficult especially in Turkana and West Pokot where flooding occurs. At other times of the year there can be severe water shortages, with peopletravelling long distances to collect water for cooking and drinking. 177. Insecurity: Due to frequent inter-community conflicts, especially between the Pokot and Turkana communities, transporting food and conducting monitoring and supervision of the SMP was hampered. Some cases of theft were also reported. A Head Teacher in Tana River reported loss of the food to theft from the school stores which meant that the little food left had to be stretched out to the end of the term thus reducing the rations per child. 50 2.3.3 Summary of findings – Evaluation Question 3 Key findings Evaluation question 3: Why and how did the program produce the observed results? The evaluation has found that there were several internal and external factors that have affected the SMP results. Internal factors • Logistics systems: High quality food was delivered to counties in a timely manner with no complaints, and minimal loss of food during transportation • Monitoring system: A comprehensive monitoring system has been utilized by WFP and the findings and analysis from it has contributed to the efficient implementation of the program, and changes in implementation as required. An electronic system would be more cost=effective, and allow the Nairobi office to receive information faster. External factors • Funding: Although USDA has provided all the commodity and financial contributions as stipulated in the grant agreement, overall the project has been under-resourced. This has resulted in lower than planned commodity volumes being provided, contributing to schools being unable to provide school meals every school day • Support from MoE: Support from the MoE is critical to the success of the SMP as they are responsible for all the infrastructure and personnel connected to the schools. The MoE has provided significant financial support and budget allocation for the SMP However, program indicators such as literacy and completion rates have not been met and require ongoing commitment and support from the MoE. • County governance: The evaluation identified complaints from teachers regarding county governments not fulfilling its delivery commitment, of taking food directly to the schools. As a result, teachers have sometimes used their private funds to transport food to the school. • Parental engagement: Parental engagement in the SMP has been high, with parents taking leading roles on the SMCs and participating in the PTAs. 2.4 Evaluation Question 4: How sustainable is the School Meals Program? 178. To assess the sustainability of the SMP, the evaluators looked at two main aspects of the program: The national government’s level of commitment to the program, and the contributions it makes. 2.4.1 National Government Commitment 179. The SMP in Kenya’s arid and semi-arid lands of Kenya started in 1980. Kenya therefore represents one of the most long-standing school feeding programs in WFP’s global portfolio. The global financial crisis in 2008 brought about considerable change when WFP’s funding reduced substantially and WFP Kenya was forced to reduce its school feeding program. At the same time, WFP and the Government of Kenya agreed on a transition strategy that commenced with the first handover of 540,000 children in 2009 and subsequent annual handovers of 50,000. By the end of 2015 the number of school children supported by the HGSMP stood at over 900,000. The evaluation finds that this agreement is holding, and the current transition arrangement exceeds the initial plan. 180. Since the handover strategy has been in place, WFP has included capacity building elements to ensure that the governments HGSMP is implemented successfully. During the next phase of the SMP (2016-2020) WFP plans to transition six counties to HGSMP, while continuing to build the capacity for national and county-level actors to manage school feeding programs. The fact that the transition arrangement is strong, and still 51 holding after eight years indicates high level commitment from both the government and WFP. It also shows the strength of the relationship between the government and WFP. At a policy level, the national government’s commitment to the SMP is again demonstrated by the MoE’s Sessional Paper (2015) on ‘Education and Training’,109 and the National Education Sector Plan (NESP 2013-2018)110which stress its commitment to implement the requirements of the Basic Education Act (2013). This is backed by the stated commitment in the Strategic Plan of the Ministry of Education (2013- 2017)111 and the Medium-Term Plan Two (2013-2018).112 181. Both the Government policies and the transition strategy require the national government to allocate resources for the SMP. Despite this, the evaluation team notes that several factors have negatively influenced the implementation and outcomes of the program, some of which are due to a lack of human resources, or financial resources within the MoE. This indicates a lack of prioritization for the program by the national government outside of the MoE, which has resulted in a lack of resources being made available to the MoE to implement the program effectively. The evaluation notes that the period of transition to HGSMP has been extended from one term to one year, in order to better support the change to HGSMP in the arid counties. Key informants welcomed this extended support. 2.4.2 National Government Contribution 182. The evaluation found the value and predictability of the MoE financial contribution to the program, and the timeliness of disbursement were problematic. The national government’s financial contribution to school feeding program has fluctuated over the last three years (Figure 18) reducing the effectiveness of implementation. Figure 18: Government contributions to SMP (2014-2016) 183. During the 2014/15 financial year, the National government allocated 2.3 billion KES113 to the SMP. The following year (2015/16) however, there was a significant drop to 1 billion KES,114 followed by an increase to 2.6 billion in 2016/17. 115 In total, over 2014- 2016, the government has allocated 5.9 billion KES to the SMP (US$56.8 million). This is 300% of the MGD/WFP contribution over the same period, however key informants raised concerns about the level of the government contribution compared to the demand. 109 Government of Kenya (2015) Sessional Paper (2015) on ‘Education and Training’. Ministry of Education. 110 Government of Kenya (2012) National Education Sector Plan (2013-2018). Ministry of Education. 111 Government of Kenya (2012) Strategic Plan of the Ministry of Education (2013- 2017). Ministry of Education. 112 Government of Kenya (2012) Medium-Term Plan Two (2013-2018). 113 http://www.businessdailyafrica.com/Tough-Budget-balancing-act-for-Treasury-chief/-/539546/2345158/-/wly984/-/index.html 114 https://www2.deloitte.com/content/dam/Deloitte/ke/Documents/tax/Tax_Budget%20Highlights%202015_KE.pdf 115 https://www2.deloitte.com/content/dam/Deloitte/ke/Documents/tax/Tax_budget-highlights%202016-KE.pdf 8,000,000,000 6,000,000,000 4,000,000,000 2,000,000,000 0 2014/15 2015/16 2016/17 TOTAL Kenyan Shillings (KSh) 52 “… the biggest challenge that we are facing right now is that the government has not increased the financial allocation which is commensurate to the number of children that they are supporting … Children are supposed to be fed for 195 school days, that is for one year … however, if you look at what we have to support children from the HGSMP, it is insufficient.” Samburu key informant interviews 184. The evaluation recognizes that WFP has been having ongoing discussions with the MoE on how to rectify these financial issues. To date, WFP has supported changes including ensuring Treasury is on the Technical School Meals Committee as recommended by the Mid-Term Review. The MoE has also agreed in principle to include Treasury, Ministry of Labour and Social Services, Ministry of Water, Ministry of Planning and Devolution and the Teachers Service Commission (TSC) in the Technical committee. There is some evidence that the national commitment towards SMP is increasing, as the government has increased the funding by KES.1.6 million in the 2017/2018 financial year. Furthermore, in response to drought in early 2017, the government allocated KES 622 million towards provision of meals when WFP did not have resources. WFP plans to continue undertaking advocacy activities in this regard. 185. It is clear that these financial issues have negative effects on the implementation of the program: schools are unable to plan properly, procurement is delayed, transportation costs cannot be paid in a timely manner, and teachers reported using their own private funds to support the implementation. These issues have also been identified by other school feeding evaluations in Kenya.116 Sustainability of the SMP can only be achieved through the National government ensuring allocations commensurate with requirements, increasing year on year to keep up with market prices and increasing student numbers. 186. The Mid-Term Evaluation recommended that the government consider ring￾fencing the SFP budget line to secure funds allocation for the program, and perhaps bring the SFP under the National Safety Net Program (NSNP). WFP is currently discussing with the government to explore the possibilities of having the SFP funding integrated under the NSNP, to make funding more regular and protected from reallocations. 2.4.3 Summary of findings – Evaluation Question 4 116 Including Dunn, S & Kariuki, W (2015) External evaluation of the WFP Cash Transfer to Schools Pilot Project, March 2013 – March 2015. WFP Kenya; Haag, P (2014) External Evaluation of Kenya´s Home-Grown School Meals Program (2009 – 2013). Final Report. Summary of key findings Evaluation question 4: How sustainable is the SMP? The evaluation finds that the SMP is sustainable as it is in process of transitioning to the government-led HGSMP. • The National government has demonstrated a high level of ownership of the program and a commitment to providing school meals through their HGSMP. There are numerous policies and strategies that require the government to allocate resources to SMP. However, Government financial contributions have not been stable resulting in negative effects on the implementation of the program. Sustainability of the SMP can only be achieved through the Government ensuring allocations commensurate with requirements, increasing year on year to keep up with market prices and increasing student numbers. • The government also has a commitment to take over all the WFP SMP schools over time, and the handover agreement is holding. There is also some evidence that the national commitment towards SMP is increasing. 53 3 Conclusions 187. The following section describes the main conclusions of the evaluation. The conclusions are organized as per the international evaluation criteria: relevance, effectiveness, impact, efficiency and sustainability. Relevance 188. The long-term objective of the SMP is to promote universal primary education of socio-economically disadvantaged and nutritionally vulnerable children, especially girls, within the primary schools in targeted areas. The relevance and importance of this objective was confirmed by key informant interviews with multiple Government officials at all levels: national, county and district. At the time of the SMP design, all ten targeted counties were behind the national average education figures – enrolment, attendanceand literacy and numeracy, indicating that educational support programs were appropriate. 189. The SMP aligns well with Government of Kenya policies and strategies, and with WFP’s own corporate guidance. Although WFP provides food assistance to the SMP by in￾kind transfer, which is incongruent with the government’s cash modality used in the HGSMP, the evaluation recognises that the continued use of in-kind assistance is only a temporary measure while counties transition to cash transfers (through the HGSMP), and therefore appropriate. The SMP also aligns with other development actors, and WFP coordinates with others to ensure that their targeted communities receive other complementary assistance that WFP cannot provide. 190. Overall, this EoP evaluation found the design of the SMP to be appropriate to the education, food security, and gender context, and coherent to the policy framework of the Government of Kenya as well as WFP corporate guidance. Effectiveness 191. The EoP evaluation finds that overall, the SMP has been well implemented and effective at meeting program results and targets. 192. The SMP has been very effective at improving the capacity of the MoE to implement SMP and transition to HGSMP. In addition, WFP has effectively contributed to Kenya’s education policy direction, by supporting the development of multiple government policies and strategies. WFP has also provided numerous trainings for teachers and MoE officials and key informant interviews indicate that these have helped schools manage the program more effectively. WFP has also supported the distribution ofHGSMP and National School Health Guidelines, which has contributed to all schools nowhaving appropriate food procurement and distribution procedures and infrastructure in place. 193. The provision of school meals has been implemented as planned, with both targets and actual delivery decreasing annually, as counties transition to the government’s HGSMP. These meals have been effective in reducing school children’s short-term hunger, as the evaluation found that only half the surveyed children eat a meal at home before coming to school. Parents also reported that if school meals were not provided at school, 30 percent of children would not eat lunch, and 12.5 percent of children would go home for lunch but then not return to school. This indicates the role that the SMP plays not only in food security, but in keeping children in school all day. 194. The presence of SMP has contributed to improving school enrolment, and has been critical for improving student attentiveness in class. Although inattentiveness has numerous causes, teachers identified hunger as the primary reason. In counties where more parents reported that food was sometimes not provided at school, teachers were 54 more likely to report that hunger was the main cause of inattention. In those schools, teachers reported that the lack of school meals accounted for 71% of student inattentiveness. 195. Attendance rates have been high (>80%) since baseline, with no significant change over the course of this period of implementation. However, all school level key informants reported that when food is not available, student attendance reduces, so thereis clearly a link between the SMP and attendance. This makes it even more important that meals are available every school day. The evaluation found that school meals were provided only 65- 70% of school days over the course of the program in part due to funding constraints from other (non-USDA) donors and pipeline delays, as well as occasional insufficient firewood and water provision by the communities. This is likely to have reduced potential education outcomes as some pupils may not have attended on those days. 196. Despite the role of SMP in improving enrolment, attentiveness of students and keeping them in school all day, the intended school performance outcome - improved literacy - has not improved. Literacy rates are largely the responsibility of the MoE and are affected by the lack of trained teachers and lack of educational materials in some schools. The evaluation found that literacy is poor throughout Kenya, and that the Government is taking steps to address this. 197. The training that WFP has done to train school personnel in appropriate food storage and food preparation practices has been effective, with 82% of schools now having dedicated kitchen facilities and 80% having dedicated food storage facilities. These results are also testament to the effective model of parental engagement employed by the program. In counties without dedicated kitchen spaces, schools reported that it is mainly due to competing priorities for limited financial resources. 198. WFP’s participatory approach to implementation of SMP has been effective at engaging parents in SMP implementation. WFP have also managed to improve gender equality in SMP management with most schools now having equal representation of men and women on the SMC. 199. Overall, the evaluation found that some counties performed poorly on multiple indicators including school attendance, children eating meals before school, the percentage of days that school meals are provided, attentiveness of students, and the presence of energy efficient stoves. These counties - Baringo, West Pokot, Turkana and Marsabit - will need tailored support depending on their results, to bring them up to the standard of the other counties. Impact 200. It is clear from evaluation interviews that the SMP plays a significant role in encouraging parents to enrol their children and to send them to school regularly. The contribution WFP has made to coordinating with other development partners to implement complementary programs to improve girl’s attendance at school is also likely to result in improved gender parity in education. More discussion on gender aspects has not been possible due to the nature of the evaluation questionnaires, and a lack of adequate gender assessment in the ASALs. 201. The SMP clearly plays a significant safety net function, improving the food security of the school children. The evaluation found that only half the children eat a meal at home before attending school. However, almost a third of parents reported that they reduced the portion of food on the weekdays (school days) compared to the weekends because of the SMP. This negates some of the food security benefit of the school meal. 55 Evaluation findings also indicate that the SMP directly contributes to some household’s food security because children taken a portion of their SMP food home for their siblings. 202. Key informants believe the SMP supporting the nutritional status of school children. The SMP provides fortified commodities (wheat, oil and salt), increasing the micronutrient content of the school meal. No measurement of nutritional status is included in program monitoring so it not possible to categorically state that these are indeed impacts of the program. Efficiency 203. The evaluation found that the WFP system of delivering food to schools was efficient. In general, WFP provided high quality food commodities with no complaints, and minimal loss of food during transportation. Reported pipeline breaks occurred in 2015 due to funding constraints caused by a reduction in non-USDA donor support. 204. A comprehensive monitoring system has been utilized by WFP and the findings and analysis from it has contributed to the efficient implementation of the program, and changes in implementation as required. Poor supervision by MoE staff in more remote locations was an area identified for improvement. WFP and MoE are in the process of strengthening joint monitoring activities so these issues may be resolved soon. 205. The complementary funds provided by WFP to provide energy efficient stoves to schools, has also contributed greatly to improving the efficiency of meal preparation. 206. Although useful for project monitoring, teachers found the paper based system cumbersome and time consuming, and the evaluation found it to be expensive. The evaluation also found that the components of the program implemented by the SCDE, including food storage and transportation introduced inefficiencies in terms of delays and extra costs for Head Teachers. Sustainability 207. The evaluation has found that overall, the SMP is relevant and coherent with government priorities. At a policy level, the National Government’s commitment to the SMP is demonstrated by its numerous policies and strategies that include school feeding. These policies, as well as the transition strategy for the handover of all WFP SMP schools to the government-led HGSMP require that the national government allocate resources for the SMP. 208. Government financial contributions have not been stable during this period of implementation resulting in negative effects on the implementation of the program. Sustainability of the SMP can only be achieved through the Government ensuring allocations commensurate with requirements, increasing year on year to keep up with market prices and increasing student numbers. There is some evidence this is happening, as the government has increased the funding by KES.1.6 million in the 2017/2018 financial year. 209. Overall, the partnership between WFP and the Government of Kenya is strong and the agreement for transitioning the SMP to the government’s HGSMP is holding. With ongoing support over the next phase of the program, the government key informants felt that they would be able to take over all the school feeding in the country, resulting in a sustainable program. The three counties that have successfully transitioned to the HGSMP during this phase of program are testament that a full handover of the program will eventually be possible. 56 4 Recommendations The evaluation has found that overall, the SMP is relevant and coherent with government priorities. There are also many components of the program that have been effectively implemented. Given that there is a handover strategy to the government in place, and holding, the evaluation generally recommends a continuation of the program in its current form so that a full handover to the government will be possible. Overall program strategy Recommendation 1: WFP should continue to implement SMP in the ASAL counties while supporting the handover of counties to HGSMP as per the current transition plan. The support recommended for continuation includes: • WFP should continue to support the MoE capacity development. This includes supporting schools to understand the requirements for implementing SMP through a cash-based modality. This is coherent with the government’s HGSMP approach. • WFP should continue to collaborate with other development actors such as World Vision to support interventions that contribute to increased school attendance, especially for girls. • WFP should continue to support the MoE to actively seek partnerships for the provision of school infrastructure that WFP currently provides support for including kitchen facilities, energy efficient stoves, food storage, and water. • WFP should continue to invest in public awareness campaigns to encourage school attendance and improve parents understanding on the benefits of education. This should be done together with local authorities, church groups or other well-respected parties. Priority 1: To be actioned within the next six months Recommendation 2: In recognition that WFP Kenya has been successfully implementing mVAM117 in the arid counties, this evaluation recommends that the WFP CO support the MoE to digitize their School Meals reporting system. This would improve the timeliness of data delivery and support improved implementation. Use of electronic reporting that utilizes mobile network for instant uploading of data, in place of the paper based form (SMP 6, 7 and 8) could be piloted in one county and if successful scaled up to cover the entire program. Integration with mVAM monitoring system could also be considered. The evaluation team recommends that WFP pilot digital monitoring in some or all of the SMP locations, either as a stand-alone monitoring strategy or combined with SMS, or paper-based monitoring. Results for the pilot should then be used to identify challenges before upscaling to areas where connectivity, security or other concerns makes it possible. Recommendation 3: As per the Mid-Term Evaluation findings, this evaluation recommends that WFP Kenya and its partners carry out community-level sensitizationon the threats to pupils’ safety. This is due to the high level of threats still reported by parents in relation to the commute to school. SFP stakeholders should also increase 117 This project was launched in 2013, and uses mobile technology to track food security trends in real time, providing high frequency data that supports humanitarian decision-making. In order to achieve high performance, data collection methods are tailored to the needs of the country in which the project operates. 57 awareness on these topics during the program implementation. Strategic partnerships with agencies focusing on Child Protection (i.e. UNICEF, Plan International, Save the Children etc.) would be an added value in helping reinforcing synergies and complementarity with the SFP. Recommendation 4: WFP Kenya, with support from the Regional Bureau as needed, should implement additional awareness campaigns on the presence and purpose of programme feedback mechanisms, especially the new feedback hotline. The evaluation found that less than half the surveyed households were aware of any feedback mechanism, and only 14 households mentioned the possibility of using the hotline. Recommendation 5: WFP Kenya should ensure that all cooks and storekeepers in SMP schools provided with training on safe food preparation and appropriate food storage. More work is needed to ensure that all school kitchens have a valid health certificate, and that pest/insect control measures are adequately carried out, as the evaluation found evidence of presence of rodents or presence of insects in almost half the stores. Recommendation 6: WFP Kenya should assess why Baringo, West Pokot, Turkana and Marsabit counties have consistently achieved poorer results on several indicators than other counties. The specific indicators include school attendance, children eating meals before school, the percentage of days that school meals are provided, attentiveness of students, and the presence of energy efficient stoves. Each county has different areas of weakness, so WFP Kenya should ensure that each county receive tailored, additional support on the areas where they are weak, to bring them up to the standard of the other counties, and ensure that they will eventually be able to successfully transition to the HGSMP. Priority 2: To be actioned within the next one year Recommendation 7: WFP Kenya, with support from the Regional Bureau as needed, should conduct a gender and protection assessment to identify and contextualise issues related to school feeding in the ASALs. The assessment should provide recommendations for implementing targeted measures to improve gender parity in school participation as latest figures show that girls enrolment in all the targeted counties is still below 50%, while boys enrolment is 51-63%.118 Priority 2: To be actioned on an ongoing basis until full handover of the SMP to the HGSMP Recommendation 8: Based on the financial challenges of the MoE, this evaluation recommends that WFP Kenya continue to support the Government of Kenya to find solutions to improve the management of financial resources for the SMP. This should include the following: • Helping to identify ways to advocate for, and allocate a sufficient, and annually indexed budget to the HGSMP. The budget should include sufficient funds for procurement, transportation, storage and preparation of school meals each school day. 118 MoEYS/WFP (2017) School Meals Programme – Verification of school enrolment in arid counties. Report, September 2017. 58 • Helping to identify ways to ensure existing SMP funding is not reallocated, while helping to secure additional funds. • Providing technical support to improve the efficiency of financial allocations to the schools 59 5 Bibliography Braidotti, F (2015) A Mid-Term Evaluation of WFP’s USDA McGovern-Dole International Food for Education and Child Nutrition Program’s support (2013-2015) in Kenya from September 2013to December 2014. Final report. October 2015 Bundy, D (2009) “Rethinking School Feeding: Social Safety Nets, Child Development, and the Education Sector Dunn, S & Kariuki, W (2015) External evaluation of the WFP Cash Transfer to Schools Pilot Project, March 2013 – March 2015. WFP Kenya. Education Development Trust (2016) https://www.educationdevelopmenttrust.com/en￾GB/news-and-blogs/news/2016/barriers-to-girls-education-kenya Feed the Future: https://www.feedthefuture.gov/country/kenya Goe, L (2008) Teacher quality and student achievement: Making the most of recent research. TQ Research and Policy Brief. National Comprehensive Centre for Teacher Quality. Washington DC. Government of Kenya (2009). National School Health Policy. Government of Kenya (2010) The Constitution of Kenya – Chapter 4: The Bill of Rights. http://www.wipo.int/wipolex/en/text.jsp?file_id=207673 Government of Kenya (2011) Kenya National Social Protection Policy. Ministry of Gender, Children and Social Development. Government of Kenya (2011). Food and Nutrition Security Policy. Government of Kenya (2011). National School Health Strategy Implementation Plan (2011-2015). Ministry of Public Health & Sanitation, and the Ministry of Education. Government of Kenya (2012) Medium-Term Plan Two (2013-2018). Government of Kenya (2012) National Education Sector Plan (2013-2018). Ministry of Education. Government of Kenya (2012) Strategic Plan of the Ministry of Education (2013- 2017). Ministry of Education. Government of Kenya (2012) The National Nutrition Action Plan (2012-2017). Ministry of Public Health and Sanitation Government of Kenya (2015) Sessional Paper (2015) on ‘Education and Training’. Ministry of Education. Government of Kenya (2016) School nutrition and meals strategy for Kenya. Ministry of Education, Ministry of Health, and the Ministry of Agriculture, Livestock & Fisheries. Draft – February 2016. Haag, P (2014) External Evaluation of Kenya´s Home-Grown School Meals Program (2009 – 2013). Final Report. Hincks Dellcrest Centre - The Child with Attention Problems - The Inattentive Child. https://www.hincksdellcrest.org/ABC/Teacher-Resource/The-Child-with-Attention￾Problems/The-Inattentive-Child.aspx Hopkins, D. (1997) powerful learning, powerful school. London Chapman; Guthric J.W (1970) A survey of school effectiveness. In do Teachers make a difference? Washington D.C Department of health, Education and Welfare http://data.uis.unesco.org/?queryid=142 http://www.businessdailyafrica.com/Tough-Budget-balancing-act-for-Treasury-chief/- /539546/2345158/-/wly984/-/index.html 60 http://www.ipcinfo.org/ipcinfo-countries/ipcinfo-eastern-middle-africa/Kenya https://www2.deloitte.com/content/dam/Deloitte/ke/Documents/tax/Tax_Budget%20Highligh ts%202015_KE.pdf https://www2.deloitte.com/content/dam/Deloitte/ke/Documents/tax/Tax_budget￾highlights%202016-KE.pdf Kimani, E (2014) The nutrition paradox in Kenya: What are we doing? Global Nutrition Report. Langinger, N. (2011) School Feeding Programs in Kenya: Transitioning to a Home-grown Approach. Stanford Journal of International Relations. Fall 2011. MoEYS/WFP (2017) School Meals Programme – Verification of school enrolment in arid counties. Report, September 2017. Molinas, L. & Regnault de la Mothe, M. (2009) The multiple impacts of school feeding: a new approach for reaching sustainability. In: WFP (2010) Revolution: From food aid to food assistance: Thematic Areas, Chapter 14, p217-230. Nauges, C. & Strand, J. (2007) “Estimation of Non-Tap Water Demand in Central American Cities”. Resource and Energy Economics 29: 165-182. Oxfam (2005) Beyond the Mainstream, Education and Gender Equality Series, Program Insights. Oxfam GB. December 2005. www.ungei.org/resources/files/oxfam_edPaper4.pdf Republic of Kenya Ministry of Education, “Education For All” National Review 2014 Scott, L., Dopson, S., Montgomery, P., Dolan, C., Ryus, C., 2009. Impact of Providing Sanitary Pads to Poor Girls in Africa. Working Paper. SAID Business School, University of Oxford. UNICEF (2014) UNICEF Annual Report 2014 – Kenya. UNICEF (2015) State of the World’s Children: 2014 data United Nations Development Program (2014) “Human Development Report 2015” United Nations Development Program (2015) Human Development Report Office. http://report.hdr.undp.org Uwezo (2014) Literacy and numeracy across East Africa: Are our children learning? Uwezo (2014): Are Our Children Learning? Literacy and Numeracy in Kenya 2014. Nairobi: Twaweza East Africa Uwezo (2015): Are Our Children Learning? The State of Education in Kenya in 2015 and Beyond. Nairobi: Twaweza East Africa. Uwezo (2016) Are our children learning (2016)? Uwezo Kenya sixth learning assessment report. December 2016. Uwezo Kenya (2012): Are Our Children Learning? Annual Learning Assessment Report 2012, Nairobi: Twaweza East Africa WFP (2009) Gender policy: promoting gender equality and the empowerment of women in addressing food and nutrition challenges. WFP/EB.1/2009/5-A/Rev.1 WFP (2013) Revised School Feeding Policy. Promoting innovation to achieve national ownership. November 2013. WFP (2014) Baseline survey for the USDA McGovern-Dole International Food for Education and Child Nutrition Program’s Support (2013-2015) to WFP Kenya Country Program. WFP Kenya (2014) Standard Program Report WFP Kenya (2015) Standard Program Report WFP Kenya (2016) Standard Program Report 61 Woodhouse, Allison and Lamport, Mark A. (2012) "The Relationship of Food and Academic Performance: A Preliminary Examination of the Factors of Nutritional Neuroscience, Malnutrition, and Diet Adequacy," Christian Perspectives in Education, 5(1). Available at: http://digitalcommons.liberty.edu/cpe/vol5/iss1/1 World Bank (2016) http://data.worldbank.org/indicator/SP.RUR.TOTL.ZS & Unknown author/date. Kenya: Situation analysis to transform nutrition. http://pubs.iied.org/pdfs/G02742.pdf World Bank/WFP (2013) Systems Approach for Better Education Results: School Health and School Feeding 62 6 Annexes Annex 1: Evaluation Terms of Reference Introduction These Terms of Reference (TOR) are for the final evaluation of the World Food Program (WFP) McGovern-Dole (MGD) International Food for Education and Child Nutrition Program (FFE 615- 2013/041/00) in Kenya. This evaluation is commissioned by WFP Kenya Country Office and will cover the period from 2014 to 2016. School feeding in Kenya is a multi-donor project. This TOR were prepared by the WFP Kenya Country Office M&E unit based upon an initial document review and consultation with stakeholders and following a standard template. The purpose of the TOR is twofold. Firstly, it provides key information to the evaluation team and helps guide them throughout the evaluation process; and secondly, it provides key information to stakeholders about the proposed evaluation. Reasons for the Evaluation The reasons for the evaluation being commissioned are presented below: Rationale This evaluation follows on the baseline evaluation (conducted between March-July 2014) and the mid-term evaluation (conducted between February and July 2015) and is being commissioned for the following reasons: USDA manages the MGD Food for Education program which is a major funding mechanism for school feeding worldwide. It aims to reduce hunger and improve literacy and primary education and has, more recently, incorporated goals related to boosting teacher attendance and capacity as well as students’ academic performance. The program provides U.S. produced agricultural commodities and financial assistance, and supports capacity development and enhanced monitoring and reporting. Sustainability is an important consideration, and the grantees are expected to work to support government and community ownership. MGD is one of the longest -standing, important donors to WFP School feeding in Kenya. Most recently, WFP Kenya was awarded a total of US$20.2 million in support for the period 2014-2016 The grant agreement incorporates 12 specific performance indicators and 21 results indicators against which performance of the program will to be measured. In the evaluation plan agreed between with USDA, WFP commits to conducting a final evaluation to measure performance of the program for accountability and learning purposes. For this reason, WFP is commissioning an evaluation at the final-point of project implementation. Objectives The main objective of this evaluation is to assess and report on the performance and results achieved (intended or unintended, positive or negative) of USDA MGD support to WFP School Feeding Program in Kenya from 2014 to 2016. The Evaluations will serve the dual and mutually reinforcing objectives of accountability and learning. Accountability – The evaluation will assess and report on the performance and results of the USDA MGD support to WFP School Feeding Program in Kenya from 2014 to 2016. Learning – The evaluation will determine the reasons why certain results occurred or not to draw lessons, derive good practices and pointers for learning. It will provide evidence-based findings to inform operational and strategic decision-making. Findings will be actively disseminated and lessons will be incorporated into relevant lesson sharing systems. Stakeholders and Users A number of stakeholders both inside and outside of WFP have interests in the results of the evaluation and some of these will be asked to play a role in the evaluation process. Table 1 below provides a preliminary stakeholder analysis, which should be deepened by the evaluation team as part of the inception phase. Accountability to affected populations is tied to WFP’s commitments to include beneficiaries as key stakeholders in WFP’s work. As such, WFP is committed to ensuring gender equality and women’s empowerment in the evaluation process, with participation and consultation in the evaluation by women, men, boys and girls from different groups. 63 Table 1: Preliminary Stakeholders’ analysis Stakeholders Interest in the evaluation and likely uses of evaluation report to this stakeholder INTERNAL STAKEHOLDERS Country Office (CO) Kenya Responsible for the country level planning and operations implementation, it has a direct stake in the evaluation and an interest in learning from experience to inform decision-making. It is also called upon to account internally as well as to its beneficiaries and partners for performance and results of its operation. Regional Bureau (RB) Nairobi Responsible for both oversight of COs and technical guidance and support, the RB management has an interest in an independent account of the operational performance as well as in learning from the evaluation findings to apply this learning to other country offices. WFP HQ WFP has an interest in the lessons that emerge from evaluations, particularly as they relate to WFP strategies, policies, thematic areas, or delivery modality with wider relevance to WFP programming. Office of Evaluation (OEV) OEV has a stake in ensuring that decentralized evaluations deliver quality, useful and credible evaluations. OEV management has an interest in providing decision-makers and stakeholders with independent accountability for results and with learning to inform policy, strategic and programmatic decisions. WFP Executive Board (EB) The WFP governing body has an interest in being informed about the effectiveness of WFP operations. This evaluation will not be presented to the EB but its findings may feed into annual syntheses and into corporate learning processes. EXTERNAL STAKEHOLDERS Beneficiaries As the ultimate recipients of food assistance, beneficiaries have a stake in WFP determining whether its assistance is appropriate and effective. As such, the level of participation in the evaluation of women, men, boys and girls from different groups will be determined and their respective perspectives will be sought. Government, National and County Levels Both county and national Government have a direct interest in knowing whether WFP activities in the country are aligned with its priorities, harmonised with the action of other partners and meet the expected results. For SFP, the government has the overall ownership of the school feeding program, and shares the interest in learning lessonsfor design of future programs, including transition to cash model. The key line Ministries are:’ Ministry of Education, Ministry of Agriculture, Livestock and Fisheries, Ministry of Health, Treasury including relevant Ministries at county level. County and Sub￾County Education Officers, School Management Committees are also key as they are involved in program implementation and policy support. UN and Development Partners The Kenya United Nations Development Assistance Framework (UNDAF) should contribute to the realisation of the government developmental objectives. Kenya United Nations Country Team (UNCT) has therefore an interest in ensuring that WFP operation is effective in contributing to the UN concerted efforts. WFP implements theprogram within a wider UN system of support to government priorities. The partner agencies are interested in learning to what extent WFP interventions are contributingto the overall outcomes committed to the UNDAF particularly UNICEF, UNESCO, FAO, UNDAF thematic working groups, the Education Sector Donors Groups, The World Bank. NGOs [Feed the children, Partnership for Child Development (PCD) and SNV.] NGOs are WFP’s partners for the implementation of some activities while at the same time having their own interventions. Some NGOs are members of the national school feeding technical committee where coordination and joint monitoring of the overall national program - of which this project fits within, is done. The results of theevaluation might affect future implementation modalities, strategic orientations and partnerships. Donors [USDA, Canada/DFATD, Australia, Russia, Private donors] WFP operations are voluntarily funded by a number of donors. The school feeding program is a multi-donor donor initiative in which USDA support complements and supplements other donors. As such, other donors will have an interest in knowing whether their funds have been spent efficiently and if WFP’s work has been effective and contributed to their own strategies and programs. 64 The primary users of this evaluation will be: • The Kenya country office and its partners in decision-making, notably related to program implementation and/or design, Country Strategy and partnerships • Given the core functions of the Regional Bureau (RB), the RB is expected to use the evaluation findings to provide strategic guidance, program support, and oversight • WFP HQ may use evaluations for wider organizational learning and accountability • OEV may use the evaluation findings, as appropriate, to feed into evaluation syntheses as well as for annual reporting to the Executive Board. Context and subject of the Evaluation Context Kenya has a population of 44 million people. It has diverse natural resources and highly varied terrain. The country's highlands comprise one of the most successful farming regions in Africa; the port of Mombasa is a major regional hub; and the unique geography supports abundant and diverse wildlife of great economic value. In September 2014, the World Bank reclassified Kenya's economy as lower-middle income. However, poverty, food insecurity, under-nutrition and income inequality remain high; 45.6 percent of Kenyans live below the national poverty line. The most severe conditions exist in the arid north, which is underdeveloped, drought-prone and is often disrupted by local conflicts. Food availability is constrained by poor roads and long distances to markets. Kenya is a food-deficit country, ranking 145 of 188 countries in the 2015 Human Development Index (two positions up from previous year).119 The country's 2015 Global Hunger Index was 24, ranking 67th out of 117 assessed countries. Many parts of the county, especially the arid and semi-arid lands which comprise 80 percent of Kenya's land area, are characterized by undernourishment, wasting, stunting, and child mortality. Global acute malnutrition among children aged 6 - 59 months in arid areas often exceeds 15 percent while micronutrient deficiencies are above 50 percent. Education is fundamental to the Government's strategy for socio-economic development. The 2015 Kenya Economic Survey stated that national net enrolment in primary education was 88 percent with 78.5 percent completion rates (2014 data). However, in several northern counties net enrolment is still below 50percent. Poverty is linked with worsening droughts and flooding that force poor households to resort to negative coping mechanisms such as withdrawing children from school and selling productive assets. Kenya has a ten￾year Ending Drought Emergencies plan which aims to create “a more conducive environment for building drought resilience” by investing in infrastructure, security, human capital and improved financing for drought risk management. Kenya has several social-assistance programs which cover only 27 percent of the poor; 90 percent of the funding comes from development partners. In the 2012 social-protection policy aimed to increase access to services for vulnerable populations, school feeding is a major social safety net. Education is fundamental to the Government’s strategy for socio-economic development. In 2010, national net enrolment in primary education was 93 percent for boys with 88 percent completion, and 92 percent for girls with 78 percent completion.120 In the north-eastern counties net enrolment dropped to 40 percent with 35 percent completion, and adult literacy was 8 percent;121 education in these areas is frequently disrupted by conflict, drought and flooding. Girls’ enrolment improved from 0.96 in 2008 to 1.0 in 2012, but gender disparities persist.122 Retention and educational quality are ongoing challenges. Early childhood development (ECD), education and care are weak and reach only half of pre-school-age children. The National Education Sector Support Program (2013–2018)123 aims to enhance basic education in terms of access and quality. The 2010 National School Health Strategy includes access to safe water and sanitation components. Of children under 5, 84 percent are deficient in vitamin A, 73 percent in iron and 51 percent in zinc; a quarter of children have inadequate iodine intake. Iron deficiency affects 55 percent of pregnant women, 46percent of adolescents in refugee camps and 21 percent of schoolgirls in western Kenya.10 Many households cannot afford a nutritious diet, and an estimated 1.8 million children are chronically undernourished; high stunting levels persist. The 2012 National Food and Nutrition Security Policy aims to: i) improve nutrition; ii) ensure that adequate food is accessible and affordable; and iii) protect vulnerable populations through safety nets linked to long-term development. It prioritizes the prevention of nutrition-related vulnerabilities in the first 1,000 days of life and links nutrition education with targeted 119 United Nations Development Program (2014). “Human Development Report 2015”. 120 MOEST administrative data. 121 121 Government of Kenya (2015). “National Education Sector Plan: Volume One”. Nairobi: MOEST. 122 Government of Kenya. (2012). “Second Medium-Term Plan, 2013–2017” Nairobi. 123 Government of Kenya (2015). “National Education Sector Plan: Volume One”. Nairobi: MOEST. 65 nutrition interventions. Kenya joined Scaling Up Nutrition (SUN) in 2012, and is developing its National Nutrition Action Plan implementation strategy. Subject of the evaluation The Government of Kenya (Ministry of Education (MoE) and WFP have since 1980 carried out a school meals program in food insecure regions of Kenya with the objectives of encouraging parents to enrol and keep their children in school, and to encourage pupils to learn. By 2008, the number of pupils receiving school meals had grown from an initial 240,000 to 1.2 million in 3,850 primary schools in Kenya’s arid andsemi-arid lands. To pursue greater national ownership and sustainability of the program, MoE established the Home￾Grown School Meals Program (HGSMP), which in 2009 took over an initial 540,000 pupils in semi-arid lands until 2014 when WFP started handing over school feeding in arid counties beginning with Isiolo County. Nairobi County also switched from in kind food assistance to cash in September 2015. While the Government and WFP work to gradually expand the coverage of the HGSMP, WFP continues to support children in all public schools in the arid lands and in targeted schools in the informal settlements of Nairobi, where food insecurity continues to be widespread, and education indicators are below the national average. WFP also prioritises capacity development of the Government to manage and extend the HGSMP to the arid lands. WFP provides regular hot mid-day meals in primary and pre-primary schools. Primary school pupils receive a lunch of 198 grams comprising cereals, pulses, fortified vegetable oil and iodized salt to provide 30 percent of the recommended daily energy intake, and pre-primary pupils receive this lunch and an additional morning porridge made from Super Cereal. Meals are provided every school day, for a total of 195 days a year. In addition to providing school lunch to schools, WFP is engaged in capacity development activities to enhance the capacity of the government to sustainably expand and manage the school meals program. The activities include training, equipment support, south to south learning initiatives and policy support among others. Currently WFP is supporting revision of the HGSMP manual to incorporate lessons learned in cash transfer to schools in arid areas; School Health, Nutrition and Meals Strategy; School Health policy; Micronutrient powder in Schools Policy and incorporating nutrition education in primary school curriculum. WFP implements its school feeding program in close collaboration with MOEST. An annual joint work plan is formulated, and regular meetings at central and local levels organized to coordinate activities. At the county level, WFP works with county-level education officials. School Management Committees already established in each school are in charge of day-to-day implementation. The activities are monitored as part of WFP’s regular monitoring and through joint monitoring missions with MOEST. McGovern-Dole is one of the longest-standing donors to the SFP in Kenya. It’s most recent contribution of US$20.2 million supports the SFP during 2014 to 2016. This period spans two WFP Country Programs (CPs). During the design of the new CP, there were many decisions made with the GoK, which altered plans and sequencing of the SFP program. The $20.2 million agreement between USDA and WFP was signed in September 2013 while the food commodities (Bulgur wheat) arrived between February–March 2014. Distribution to schools could therefore not commence until Term 2, 2014. Through this support, WFP provides school meals, raises awareness on the importance of education, trains stakeholders on appropriate food preparation and storage practices and supports capacity building. The objectives of MGD support include boosting pupils’ enrolment, attendance, literacy and attentiveness, reducing short term hunger and guaranteeing access to food for school children. The project also aims to enhance teacher attendance, spread awareness on the benefits of education among the community, engage local organizations and community groups, increase knowledge about safe food preparation and storage and provide equipment for this purpose. Finally, to ensure sustainability, the objectives include building government capacity and improving the policy and regulatory framework in support of child health and nutrition (See Annex 3: results framework). Evaluation Approach Scope The evaluation will be of MGD-supported WFP School feeding activities implemented from 2014 to 2016. The evaluation will cover arid counties and the informal settlements of Nairobi where these activities were implemented during the above-mentioned period. The final evaluation will use the internationally agreed criteria of relevance, effectiveness, efficiency, sustainability and impact. As per the agreed-on evaluation plan, this evaluation will put greater emphasis than the midterm evaluation on the effectiveness, impact and sustainability of the program. It’s noteworthy that the midterm evaluation focused on the implementation of the program with the evaluation findings targeted at adjustments or program management decisions that were to help improve implementation. As such, the mid‐term evaluation was focused on interim or anticipated results, partnerships, implementation arrangements and systems, and any factors affecting the results achieved at the mid‐point. . This evaluation is focused on accountability (against intended results) and learning (for the continuance of the school 66 feeding in Kenya). The final evaluation will assess the impact of the program against the following objectives: • Contribution to feed the future • Improved literacy of school –age children • Increased capacity of Government institutions • More consistent teacher attendance • Improved attentiveness • Reduced short term hunger • Increased access to Food (School Feeding) • Improved student attendance • Increased use of health and dietary practices • Increased engagement of local organizations and community groups • Improved policy and regulatory frame work • Increased knowledge of safe food prep and storage practises • Increased access to requisite food prep and storage tools and equipment • Increased student enrolment. • Increased community understanding of education benefits The evaluation will not cover WFP’s accountability for literacy results but will document the trends in literacy achievement from students in program schools and non‐program schools, using available national data in line with WFP’s commitment to the principle of using nationally available data and systems where possible. National reports produced by UWEZO will therefore be used. UWEZO is the Government of Kenya’s recognized source of numeracy and literacy data. UWEZO identifies and adheres to country specific policies and guidelines that relate to methodology (sampling) and test development for national assessments. In addition to this, the UWESO processes are benchmarked alongside Pratham’s Annual Status of Education Report (ASER) from which the UWESO methodology is derived. UWESO applies a rigorous research design which uses a two-step sampling approach. The stratum is the sub-county and all counties are included. This ensures that there is representativeness of the sample. The detailed report also involves careful efforts to eliminate biases. On the literacy and numeracy tests, a set of questions is administered to children of school age in the sampled households to test their level of understanding. The Standards Manual124 gives the details of organizational standards for the Assessment. The evaluation will take into consideration that school feeding program in Kenya is a multi-donor initiative. Evaluation Criteria and Questions Evaluation Criteria: The evaluation will apply the international evaluation criteria of Relevance, Effectiveness, Efficiency, Impact and Sustainability. Gender Equality and the Empowerment of women (GEEW) should be mainstreamed throughout. Evaluation Questions: Allied to the evaluation criteria, the evaluation will address the following key questions, which will be further developed by the evaluation team during the inception phase. Collectively, the questions aim at highlighting the key lessons and performance of the WFP’s McGovern-Dole International Food for Education and Child Nutrition Program support (2014-2016), which could inform future strategic and operational decisions. Below are the key criteria and broad questions to be evaluated: Criteria Evaluation Questions Relevance Areas for analysis will include the extent to which the objectives, targeting, choice of activities and of transfer modalities: Were appropriate to the needs of the target population; Were aligned with relevant stated national policies, including sector policies and strategies and seek complementarity with the interventions of relevant humanitarian and development partners Were aligned with WFP strategies, policies and normative guidance Were aligned with partner UN agency and donor policies and priorities? Effectiveness Has the SFP achieved its stated outputs, objectives and outcomes? What were the major factors (Both internal and external) influencing the achievement or 124 http://www.uwezo.net/wp-content/uploads/2012/08/RO_2012_UwezoStandardsManual.pdf 67 non-achievement of the outputs, outcomes/objectives of the intervention? Why and how did the operation produce the observed results? The evaluation should generate insights into the main internal and external factors that caused the observed changes and affected how results were achieved. The inquiry is likely to focus, amongst others: Internally (factors within WFP’s control): the processes, systems and tools in place to support the operation design, implementation, monitoring/evaluation and reporting; the governance structure and institutional arrangements (including issues related to staffing, capacity and technical backstopping from RB/HQ); the partnership and coordination arrangements; Externally (factors outside WFP’s control): the external operating environment; the funding climate; external incentives and pressures; etc. Efficiency Were activities cost-efficient? Were the activities implemented in the most efficient way compared to alternatives? What were the external and internal factors influencing efficiency of the program (attainment of the planned outputs, cost factors, logistics and pipeline performance)? Impact What were the short- and medium-term effects of the program on beneficiaries’ lives? Are assisted schools moving in the right direction of improving education outcomes and sustaining school feeding? Did any negative effects occur for beneficiaries? What were the gender-specific impacts, especially regarding enrolment and attendance? What are the main drivers of positive impacts? (Partnerships, capacity, ownership, etc.) What were the intended and unintended impacts of the program Sustainability To what extent is the country taking ownership of the program? (e.g. demonstrated commitment and contribution to the program); What is the national readiness to implement the program? E.g. demonstrated capacity at central and sub-national levels to manage the program? Data Availability The following are the main sources of data. • Baseline and mid-term evaluation reports • WFP strategic Results framework • Kenya Country Program 200680 (2014-2018) project document and log frame • Kenya Country Program 106680 (2009-2014) project document and log frame • School feeding handbook • WFP School feeding policy • 2013 t0 2014 Standard Project Reports (SPRs). • M&E monthly monitoring reports • Strategy to Strengthen & Expand the Home-Grown School Meals (HGSM) Program into the Arid Lands of Kenya (Validated version 2013) • USDA commitment letter for Agreement FFE-615-2013/041-00 Kenya • Government of Kenya Education related policies and strategies • External Evaluation of WFP’s Cash Transfer to Schools Pilot Project (March 2013- March 2015 • HGSMP Evaluation May 2014 • UWEZO annual reports Concerning the quality of data and information, the evaluation team should: Assess data availability and reliability as part of the inception phase expanding on the information provided in section 4.3. This assessment will inform the data collection Systematically check accuracy, consistency and validity of collected data and information and acknowledge any limitations/caveats in drawing conclusions using the data. Methodology The methodology will be designed by the evaluation team during the inception phase. It should: Employ the relevant evaluation criteria above [relevance, efficiency, effectiveness, impact and sustainability] Demonstrate impartiality and lack of biases by relying on a cross-section of information sources (stakeholder groups, including beneficiaries, etc.) The selection of field visit sites will also need to demonstrate impartiality. Using mixed methods (quantitative, qualitative, participatory etc.) to ensure triangulation of information through a variety of means. 68 Apply an evaluation matrix geared towards addressing the key evaluation questions taking into account the data availability challenges, the budget and timing constraints; Ensure through the use of mixed methods that women, girls, men and boys from different stakeholder groups participate and that their different voices are heard and used; Mainstream gender equality and women’s empowerment, as above; The evaluation team is expected to elaborate appropriate sampling methods for collecting primary quantitative and qualitative data. The evaluation team will draw a statistically representative sample from the sample frame consisting of the total number of schools (1668) spread across 10 counties (Mandera, Wajir, Garissa, Marsabit, Isiolo, Samburu, Turkana, Tana River and West Pokot) and the unplanned settlements of Nairobi, covered by this program (See table on program coverage and Annex 1). As with the Mid Term Evaluation, the Final Evaluation will take a program theory approach based on the results framework. In its execution, the evaluation will draw on the existing body of documented data as far as possible. The evaluation will use mixed methods and triangulate information from different methods and sources to enhance the reliability of findings. In particular, the evaluation will combine qualitative and quantitative approaches to collect field‐level data and information from the arid counties and unplanned settlements of Nairobi under school meals program. Separate questionnaires will be applied to the different primary sources of information, focusing on infrastructure, staff, enrolment and attendance, exam scores, completion rates and community involvement in the program. The qualitative component of the evaluation will use participatory methods where relevant to highlight lessons learned and case studies representative of the interventions. In particular, the methodology will involve focus group discussions with head teachers, school management committee members, education officials, pupils and key informants drawn from education stakeholders. This component will employ relevant interview schedules as a key data collection method which will be collated to provide general impressions of the program. Fieldwork will be based on a follow‐up to the baseline and mid-term evaluations conducted. Where possible and relevant, before/after comparison will be done through design of comparable sampling strategy. The following mechanisms for independence and impartiality will be employed: use of an Evaluation Committee and an Evaluation Reference Group and referring to the Technical Note on Independence and Impartiality for guidance Quality Assurance WFP’s Decentralized Evaluation Quality Assurance System (DEQAS) defines the quality standards expected from this evaluation and sets out processes with in-built steps for Quality Assurance, Templates for evaluation products and Checklists for their review. DEQAS is closely aligned to the WFP’s evaluation quality assurance system (EQAS) and is based on the UNEG norms and standards and good practice of the international evaluation community and aims to ensure that the evaluation process and products conform to best practice. DEQAS will be systematically applied to this evaluation. The WFP Evaluation Manager will be responsible for ensuring that the evaluation progresses as per the DEQAS Step by Step Process Guide and for conducting a rigorous quality control of the evaluation products ahead of their finalization. WFP has developed a set of Quality Assurance Checklists for its decentralized evaluations. This includes Checklists for feedback on quality for each of the evaluation products. The relevant Checklist will be applied at each stage, to ensure the quality of the evaluation process and outputs. In addition, to enhance the quality and credibility of this evaluation, an external reviewer directly managed by WFP’s Office of Evaluation in Headquarter will provide: • systematic feedback on the quality of the draft inception and evaluation reports; and • Recommendations on how to improve the quality of the evaluation. This quality assurance process does not interfere with the views and independence of the evaluation team, but ensures the report provides the necessary evidence in a clear and convincing way and draws its conclusions on that basis. The evaluation team will be required to ensure the quality of data (validity, consistency and accuracy) throughout the analytical and reporting phases. The evaluation team should be assured of the accessibility of all relevant documentation within the provisions of the directive on disclosure of information. This is available in WFP’s Directive (#CP2010/001) on Information Disclosure. Phases and Deliverables The evaluation will proceed through the 5 following phases. The evaluation schedule annex provides a detailed breakdown of the proposed timeline for each phase over the full timeframe. A summary of the deliverables and deadlines for each phase are as follows: 69 Figure 1: Summary Process Map Preparation phase (February–March 2016): The evaluation manager will conduct background research and consultation to frame the evaluation; prepare the TOR; select the evaluation team and contract the company for the management and conduct of the evaluation. The TOR will be shared with USDA for comments and or inputs. Inception phase (April): This phase aims to prepare the evaluation team for the evaluation phase by ensuring that it has a good grasp of the expectations for the evaluation and a clear plan for conducting it. The inception phase will include a desk review of secondary data and initial interaction with the main stakeholders. The inception report will be shared with USDA for comments and or inputs. Evaluation phase (May - June): The fieldwork will span over a period of two months and will include visits to project sites and primary and secondary data collection from local stakeholders. A debriefing session will be held upon completion of the field work. Reporting phase (Mid-June - August): The evaluation team will analyse the data collected during the desk review and the field work, conduct additional consultations with stakeholders, as required, and draft the evaluation report. The draft evaluation report will be submitted to the evaluation manager for quality assurance. Stakeholders will be invited to provide comments, which will be recorded in a matrix by the evaluation manager and provided to the evaluation team for their consideration before report finalisation. Follow-up and dissemination phase: The final evaluation report will be shared with the relevant stakeholders. The management responsible will respond to the evaluation recommendations by providing actions that will be taken to address each recommendation and estimated timelines for taking those actions. The evaluation report will also be subject to external post-hoc quality review to report independently on the quality, credibility and utility of the evaluation in line with evaluation norms and standards. The final evaluation report will be published on the WFP public website. Findings will be disseminated and lessons will be incorporated into other relevant lesson sharing systems. Organization of the Evaluation Evaluation Conduct The evaluation team will conduct the evaluation under the direction of its team leader and in close communication with the independent evaluation manager appointed by WFP to manage the evaluation. The team will be hired following agreement with WFP on its composition and in line with the evaluation schedule provided in a separate annex. The team members will not have been involved in the design or implementation of the subject of evaluation or have any other conflicts of interest. Further, they will act impartially and respect the code of conduct of the evaluation profession. Team composition and competencies The Team Leader should be a senior evaluator with at least 10 years of experience in evaluation with demonstrated expertise in managing multidisciplinary and mixed quantitative and qualitative method evaluations, complemented with good understanding of School Meals programs and additional significant experience in other development and management positions. The Team leader will also have expertise in designing methodology and data collection tools and demonstrated experience in leading similar evaluations. She/he will also have leadership and communication skills, including a track record of excellent writing and presentation skills. Her/his primary responsibilities will be: i) defining the evaluation approach and methodology; ii) guiding and managing the team; iii) leading the evaluation mission and representing the evaluation team; iv) drafting and revising, as required, the inception report, the end of field work i.e (exit)debriefing presentation and evaluation report in line with EQAS. The team must include strong demonstrated knowledge of qualitative and quantitative data and statistical analysis. It should include both women and men and at least one team member should be familiar with WFP’s FFE work and with USDA M&E Policy. The team will be multi-disciplinary and include members who together include an appropriate balance of expertise and practical knowledge in the following areas: • Education • Nutrition • Food security • Gender • Sampling and statistical analysis • Capacity development Preparation Inception Inception Report Evaluation Reporting Evaluation Report Dissemination and follow-up 70 All team members should have strong analytical and communication skills, evaluation experience and familiarity with Kenya or the Horn of Africa. The team members will bring together a complementary combination of the technical expertise required and have a track record of written work on similar assignments. Team members will: i) contribute to the methodology in their area of expertise based on a document review; ii) conduct field work; iii) participate in team meetings and meetings with stakeholders; iv) contribute to the drafting and revision of the evaluation products in their technical area(s). All members of the evaluation team will abide by the Code of Conduct for evaluators (Attached to individual contracts), ensuring they maintain impartiality and professionalism Security Considerations Security clearance: where required is to be obtained from WFP Kenya office. As an ‘independent supplier’ of evaluation services to WFP, the evaluation company is responsible for ensuring the security of all persons contracted, including adequate arrangements for evacuation for medical or situational reasons. The consultants contracted by the evaluation company do not fall under the UN Department of Safety & Security (UNDSS) system for UN personnel. Consultants hired independently are covered by the UN Department of Safety & Security (UNDSS) system for UN personnel which cover WFP staff and consultants contracted directly by WFP. Independent consultants must obtain UNDSS security clearance for travelling to be obtained from designated duty station and complete the UN system’s Basic and Advance Security in the Field courses in advance, print out their certificates and take them with them.125 However, to avoid any security incidents, the Evaluation Manager is requested to ensure that: The WFP CO registers the team members with the Security Officer on arrival in country and arranges a security briefing for them to gain an understanding of the security situation on the ground. The team members observe applicable UN security rules and regulations. The WFP CO registers the team members with the Security Officer on arrival in country and arranges a security briefing for them to gain an understanding of the security situation on the ground. The team members observe applicable UN security rules and regulations – e.g. curfews etc. Roles and Responsibilities of Stakeholders The Kenya Country Office: The Kenya country Office management (Deputy Country director will take responsibility to: • Ensure an independent Evaluation Manager for the evaluation: • Compose the internal evaluation committee and the external evaluation reference group • Approve the final TOR, inception and evaluation reports. • Ensure the independence and impartiality of the evaluation at all stages, including establishment of an Evaluation Committee and of a Reference Group • Participate in discussions with the evaluation team on the evaluation design and the evaluation subject, its performance and results with the Evaluation Manager and the evaluation team • Organise and participate in two separate debriefings, one internal and one with external stakeholders • Oversee dissemination and follow-up processes, including the preparation of a Management Response to the evaluation recommendations Evaluation Manager: • Manages the evaluation process through all phases including drafting this TOR • Ensure quality assurance mechanisms are operational • Consolidate and share comments on draft TOR, inception and evaluation reports with the evaluation team • Ensures expected use of quality assurance mechanisms (checklists, quality support etc.) • Ensure that the team has access to all documentation and information necessary to the evaluation; facilitate the team’s contacts with local stakeholders; set up meetings, field visits; provide logistic support during the fieldwork; and arrange for interpretation, if required. • Organise security briefings for the evaluation team and provide any materials as required • Chairs the External Reference Group meetings An Internal Evaluation Committee has been formed as part of ensuring the independence and impartiality of the evaluation. The membership includes M&E officer, evaluation manager, technical unit in charge of school feeding program, Deputy (country director programs), One staff each from finance and 125 Field Courses: Basic https://dss.un.org/bsitf/; Advanced http://dss.un.org/asitf 71 logistics unit. The key roles and responsibilities of this team, includes providing input to evaluation process and commenting on evaluation products. An External Evaluation Reference group has also been formed, with representation from USDA/FAS, Canada, Ministry of Education, Feed the Children, WFP Country office and Regional Bureau and will review the evaluation products as further safeguard against bias and influence (See annex 5; External reference Group TOR) The Regional Bureau. The RB management will be responsible to: • Assign focal point for the evaluation. • Participate in discussions with the evaluation team on the evaluation design and on the operation, its performance and results. In particular, the RB should participate in the evaluation debriefing and discussions with the evaluation manager and team, as required. • Provide comments on the TORs, inception report and the evaluation report. Headquarters. Some HQ divisions might, as relevant, be asked to discuss WFP strategies, policies or systems in their area of responsibility and to comment on the evaluation TOR and report. Other Stakeholders (Government, NGOs, and UN agencies) will be identified for interviews by the evaluation team in addition to the list provided by WFP which will be based on the preliminary stakeholder analysis detailed in table 1. Government and USDA and other partners will provide inputs into the draft evaluation report before its finalized. The Office of Evaluation (OEV). OEV will advise the Evaluation Manager and provide support to the evaluation process where appropriate. It is responsible to provide access to independent quality support mechanisms reviewing draft inception and evaluation reports from an evaluation perspective. It also ensures a help desk function upon request from the Regional Bureaus. Communication and budget Communication To enhance the learning from this evaluation, the evaluation team should place emphasis on transparent and open communication with key stakeholders. These may for example take place by ensuring a clear agreement on channels and frequency of communication with and between key stakeholders. Communication with evaluation team and stakeholders should go through the Evaluation manager. As part of the international standards for evaluation, WFP requires that all evaluations are made publicly available. Following the approval of the final evaluation report, dissemination will be broad and workshops will be conducted both internally and with partners, looking at the recommendations and the way forward. Specifically; WFP Kenya Country Office will organize an internal workshop to discuss evaluation findings and recommendations, where the consultant will present the key findings; WFP in collaboration with the Ministry of Education, will organize a workshop targeting relevant external audiences, where the consultant will present the key findings. WFP will discuss the report with USDA and disseminate the findings and recommendations in various ways, including through discussions with WFP senior management and staff as well as with the key partners including the Ministry of Education, non-governmental partners and UN agencies. 72 Annex 2: Counties where the USDA-supported SMP is implemented County Number of schools 1 Baringo 112 2 Garissa 167 3 Isiolo 98 4 Mandera 185 5 Marsabit 167 6 Nairobi 92 7 Samburu 148 8 Tana River 161 9 Turkana 331 10 Wajir 191 11 West Pokot 114 TOTAL 1,766 73 Annex 3: UWEZO 2014 report - school performance rankings for the targeted SMP targeted counties County District Mean pass rate combined test National ranking (out 0f 155 Districts) Nairobi126 West 77.6 15 Baringo North 63.7 69 Central 63.6 72 Marsabit 57.3 106 Moyale 54.2 116 Garissa 45.2 138 Isiolo 42.4 140 Tana River 34.4 144 Pokot Central 56.1 111 West 54.2 115 North 36.3 142 East 33.9 145 Samburu East 30.6 149 Turkana East Central 46.3 136 South 32.9 146 North 28.2 151 Wajir West 47.9 131 South 23.7 153 Mandera East 71.4 35 Central 20.5 154 West 17.0 155 126 Nairobi is included in the SMP because of the unplanned settlements around it where there is high levels of food insecurity. 74 Annex 4: Recommendations of the SMP Mid-Term Evaluation Immediate and short-term key recommendations (requiring mid-course corrections or initiation of action) are as follows: 1. For many pupils in the arid lands the SFP meal is the most substantial meal they have during the day. In fact, 56.1 percent of parents reduce the portions of the evening meal when lunch is provided in schools and only 59 percent of pupils consume a meal before school. Recommendation: Consider providing the SFP meal one or two hours earlier (11am instead of 12.30pm) to address problems of short term hunger, particularly in counties where the intake of breakfast at household level is low. Particular attention should be given to Marsabit, Turkana and West Pokot Counties since they performed more poorly than the other Counties for the abovementioned indicators. 2. Results indicate that pupils experience significant threats to their safety while walking to and from school. This is particular prevalent in Nairobi. Recommendation: SFP stakeholders should increase awareness on these topics during program implementation. WFP should consider carrying out community-level sensitization on the threats to pupils’ safety. Strategic partnerships with agencies focusing on Child Protection (i.e. Unicef, Plan International, Save the Children etc) would be an added value in helping reinforcing synergies and complementarity with the SFP. These interventions should be prioritized in Nairobi informal settlements. 3. The GoK financing flows under which SFP operates are very inefficient and not timely and this significantly delays program implementation. This affects both the government-led HGSMP and the in-kind program supported by WFP (due to commodity delivery delays by MOEST at county level). Recommendations: WFP and the GoK should consider establishing a national and independent entity to manage the SFP, housed outside of MOEST, with the aim of increasing implementing partners’ involvement and accountability. The Treasury should be represented within this independent entity to ensure that any arising issues related to funding channels are promptly addressed. The institutional arrangement could be lighter at national level while increasing the support to county-level structures. A potential disadvantage of this option is the danger of having another autonomous agency that doesn’t integrate closely enough with the core ministries. In this regard, an open discussion to seek a proper balance is encouraged. In addition, GoK should consider ring-fencing the SFP budget line in order to secure funds allocation for the program. In this regard, a strategy could be to bring the SFP under the National Safety Net Programs (NSNP). This will enhance coordination; help the SFP program to work more coherently, efficiently and effectively with the others Kenya’s safety net programs andin addition it could ensure a “ring-fenced” budget. This can be done integrating the SFP with the National Social Protection Secretariat (and the Council when the bill is enacted) within the NSNP. MOEST should participate in the national steering committee and, at Director level, inthe management and technical working groups. Synergies could be sought by integrating the MOEST data (NIEMIS) into the social protection single registry for example. In order to further secure funds for the future implementation of the SFP program in a sustainable manner, a scale-up of the current advocacy campaign is recommended especially targeting the National Assembly Budget Committee, the Council Governance and the Treasury. Student Councils formed at school level could be also involved for an effective advocacy campaign. 4. There is high degree of variance and inaccuracy in the methods used to distribute food among pupils. Recommendation: Provide a unified scoop measure to all supported schools, both under the SFP and the HGSMP. Implementers should also consider providing weighing scales to all supported schools along with adequate training on how to use them. 75 5. Even though the SFP increases the attendance rate, this alone does not translate into good literacy outcomes of the pupils in the arid counties. Other factors impacting negatively on literacy outcomes include poor quality of teaching as documented by the SDI Report 2012. Recommendation: This evaluation recognizes that influencing the quality of teaching is beyond WFP’s mandate and responsibility. Recognizing WFP efforts in the review of the Kenya national curriculum alongside other development partners, Kimetrica recommends continued synergy with partners who are supporting the GoK to address issues related to the quality of education and teaching, particularly in the context of the two recently launched national programs – the Global Partnership for Education (GPE) and Tusome – that are principally funded by the World Bank and USAID. Medium to long-term recommendations are as follows: 1. There is a low level of coordination between MOEST and MOH at county level. Coordination may be increased through more frequent and regular meetings of MOEST and MoH officials and through their involvement in the joint monitoring activities. Recommendation: Strengthen county level school committees, with a wide representation of program stakeholders, to increase coordination and accountability. In addition, MoH should ensure that food inspections are carried out in the District Education Officer (DEO) warehouses as soon as WFP delivers the commodities, especially if there is insufficient funding to carry out food inspection at school level. The establishment of a national independent entity beyondMOEST and integrating SFP into the NSNP, as outline in point 3, would help to increase the involvement and participation of other Ministries (including MoALF), ring-fence the funds and foster coordination. 2. Multiple reporting lines related to the SFP implementation create issues of accountability. Teachers, including the SFP Teacher, report to the TSC while the program is under the purview of MOEST. Recommendation: Strengthen the Technical School Feeding Committee by involving additional partners, such as the TSC, to address issues of accountability and multiple reporting lines, and enlarging its operational budget. The involvement of the Treasury within the Technical School Feeding Committee would also be beneficial to address lesson learned number 3 mentioned above. 3. There is high staff turnover, both within schools and ministerial offices. One SFP management training per project cycle (once every 2.5 years) may not be effective, despitethe additional supplementary visits and on-the-job training carried out by WFP. Recommendation: Consider increasing the frequency of training (once a year), especially for officials that have not yet been trained in the context of the SFP. 4. Record-keeping at school level is poor and this affects the monitoring process. Recommendation: Project implementers should engage in capacity building activities at the school level to strengthen record-keeping and filing practices. More frequent SFP management trainings at the county level (recommendation number 6) would increase the schools’ and government’s capacity to implement the HGSMP and in-kind contribution program. Project implementers should consider making commodity delivery conditional upon reconciled records (both SFP and School records). 1. KI interviews in Garissa and Turkana revealed that monitoring at the school level by MOEST is a challenge due to lack of funds at county level. In addition, government officials were not aware of the joint monitoring arrangements in place with WFP. Recommendation: SFP implementing partners should raise awareness of the joint monitoring arrangement at county level. WFP should consider involving the MoH, and MoALF in addition to MOEST during their field visits. Furthermore, MoH should ensure that food inspections are carried out in the DEO warehouses as soon as WFP delivers the commodities, especially if there is insufficient funding to carry out food inspection at school level. A cost- 76 sharing mechanism between WFP, MOEST and MoH should be considered to cover the expenses related to joint monitoring. This evaluation recommends developing a Memorandum of Understanding (MoU) between WFP, MOEST, MoH and MoALF (and TSC) to determine the cost￾sharing mechanism: a possible solution could include WFP providing transport means (as is already happening) with MOEST and MoH ensuring their officials’ daily allowance and fuel is covered. MOEST and WFP should also consider involving additional partners in their monitoring activities (such as UNICEF) to increase complementarity of activities with key partners. They should use this opportunity to conduct joint sensitization sessions of pupils and teachers on the issues addressed by their programs. 2. The schools often inflate their enrolment figures, as verified by WFP in Samburu County. Recommendation: Finalize the already initiated process of creating a common MIS to reflect real-time changes in school enrolment and attendance figures. 3. The WFP helpline/complaint mechanism proved to be quite effective in the County where it was piloted. Recommendation: Following the pilot exercise conducted by WFP, the GoK should prioritize the implementation of a helpline/complaint mechanism in all counties covered by the SFP and HGSMP, with a dual objective of increasing community knowledge and awareness of the program and creating an avenue for raising complaints and issues. 77 Annex 5: Evaluation criteria This evaluation considered a globally accepted evaluation criteria. The questions proposed were in line with the considerations of the evaluation questions, and offered thebest opportunity towards evaluating the SMP with reference to the situation at baseline and midline surveys. Overall the evaluation incorporated gender based approaches towards the data collection, analysis and reporting. Relevance •In assessing relevance, the evaluation questions sought to determine the extent to which the SMP was suited to the priorities and policies of the target group, recipient, and donor. In doing this, we considered the following questions: To what extent were the objectives of the program still valid? Were the activities and outputs of the program consistent with the overall goal and the attainment of its objectives?; Were the activities and outputs of the program consistent with the intended impacts and effects? Effectiveness •The evaluation assessed the extent to which SMP attained its objectives. To achieve this the following questions were considered: To what extent were the objectives achieved / are likely to be achieved?; what were the major factors influencing the achievement or non-achievement of the objectives? Efficiency •Our approach towards assessment of efficiency shall consider measuring the outputs both qualitatively and qualitatively in relation to the inputs. This is more or less an economic approach, which questions whether SMP used the least costly resources possible in order to achieve the desired results. This aimed at comparing alternative approaches to achieving the same outputs, to see whether WFP and partners adopted the most efficient process. The following questions were used: Were activities cost-efficient?; were objectives achieved on time?; was the program or project implemented in the most efficient way compared to alternatives? Impact •The evaluation sought to assess the positive and negative changes produced by SMP, directly or indirectly, intended or unintended. The evaluation was concerned with both intended and unintended results including positive and negative impact of external factors, such as changes in terms of the environment, education systems, and financial conditions. The following questions were considered: What has happened because of the program or project; what real difference has the activity made to the beneficiaries; how many people have been affected? Sustainability •In assessing sustainability, the evaluation sought to measure whether the benefits of the SMP are likely to continue after donor funding has been withdrawn. The evaluation sought to determine; to what extent did/will the benefits of a SMP continue after donor funding ceased; the major factors which were/are likely the achievement or non-achievement of sustainability of the program or project. 78 Annex 6: Evaluation Matrix Criteria Evaluation Questions Measure/ Indicator of Progress Main Sources of information Data Collection Methods Data Analysis Methods Evidence availability/ reliability Relevance Areas for analysis will include the extent to which the objectives, targeting, choice of activities and of transfer modalities: Qualitative and quantitative assessment of the targeting, project implementation and the target population profile to determine all the parameters of relevance. WFP documents, Key Informants, Target Population a. Face to face interviews b. Review of literature and data Thematic analysis using Miles and Hubberman approach for the qualitative data. This will be triangulated by analysis of quantitative data to deduce the extent to which the needs were reached. 3-Strong Were appropriate to the needs of the target population; Were aligned with relevant stated national policies, including sector policies and strategies and seek complementarity with the interventions of relevant humanitarian and development partners Were aligned with WFP strategies, policies and normative guidance Were aligned with partner UN agency and donor policies and priorities? Effectiveness a. Has the SMP achieved its stated outputs, objectives and outcomes? 1. % increase in enrolment 2. % increase of pupils attending school 3. Number of methods used in assessing literacy levels 4. % increase in the number of teachers attending school as a result of SMP 5. % of MoE officials trained by WFP in 2014? School based assessment MoE Officials face to face interviews with head teachers, patrons and MoE officials Varied as per PMP 2-Fair b. What were the major factors (Both internal and external) influencing the achievement or non-achievement of the outputs, outcomes/objectives of the intervention? Analytical evidence of difference in the different schools based performance Assessment of the reported Secondary Data and Primary Data a. Secondary Data Provided by WFP b. Secondary data collected from other Thematic analysis of qualitative data and triangulating with performance measures 3-Strong 79 Criteria Evaluation Questions Measure/ Indicator of Progress Main Sources of information Data Collection Methods Data Analysis Methods Evidence availability/ reliability c. Why and how did the operation produce the observed results? The evaluation should generate insights into the main internal and external factors that caused the observed changes and affected how results were achieved. internal and external factors by stakeholders and target populations Assessment of other complementary projects within the project area? Assessment of other documented issues within the community such as security, conflict, climate change etc. sources but relevant to the subject area. C. Face to Face Interviews d. Internally (factors within WFP’s control): the processes, systems and tools in place to support the operation design, implementation, monitoring/evaluation and reporting; the governance structure and institutional arrangements (including issues related to staffing, capacity and technical backstopping from RB/HQ); the partnership and coordination arrangements; The processes, systems and tools in place to support the operation design, implementation, monitoring/evaluation and reporting; the governance structure and institutional arrangements (including issues related to staffing, capacity and technical backstopping from RB/HQ); the partnership and coordination arrangements a. Primary Data collected from WFP staff and project stakeholders b. Various Project Reports WFP a. Face to face interviews b. Review of literature and data Thematic analysis of qualitative data and triangulating with performance measures 3-Strong e. Externally (factors outside WFP’s control): the external operating environment; the funding climate; external incentives and pressures; etc. Factors outside WFP that affected the project performance a. Primary Data collected from WFP staff and project stakeholdersb. Various Project Reports WFP a. Face to face interviewsb. Review of literature and data Thematic analysis of qualitative data and triangulating with performance measures 2-Fair Efficiency a. Were activities cost-efficient? b. Were the activities implemented in the most efficient way compared to alternatives? c. What were the external and internal factors influencing efficiency of the program (attainment of the planned outputs, cost factors, logistics and pipeline performance)? Financial Cost per Key Performance Indicator a. Primary Data collected from WFP staff and project stakeholders b. Various Project Reports WFP a. Review of Literature/Data and Progress reports from WFP b. Face to Face Interviews with WFP c. F2F interviews with key Stakeholders d. Review of financial documentations Assess the cost versus the program output e.g. cost per percentage increase in enrolment, cost per unit performance measure 1-Weak Time Cost per unit measure of key performance indicator Perceived efficiency of SMP in delivering its outputs Qualitative Thematic analysis 3-Strong Ease of carrying out activities Qualitative Thematic analysis 3-Strong Decision making process and involvement of community Qualitative Thematic analysis 3-Strong 80 Criteria Evaluation Questions Measure/ Indicator of Progress Main Sources of information Data Collection Methods Data Analysis Methods Evidence availability/ reliability Cost of overheads visa vise direct project costs Qualitative Thematic analysis 3-Strong Duplication of effort and management of duplication Qualitative Thematic analysis 3-Strong Reported and documented issues around efficiency Qualitative Thematic analysis 3-Strong Deliverables achieved in time and on budget Qualitative Thematic analysis 3-Strong Impact a. What were the short- and medium term effects of the program on beneficiaries’ lives? 1. Proportion of students who by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. 2. Percent of food preparers at target schools who achieve a passing score on a test of safe food preparation and storage 3. Percent of teachers in target schools who attend school and teach at least 90 percent of scheduled school days per year. 4. Percent of students (girls/boys) regularly attending supported schools 5. Percent of districts in which food procurement and distribution procedures and infrastructure are in place 6. Number of child health and nutrition policies, regulation and/or administrative procedures in place School based assessment Face to faceinterviews with head teachers, pupils and WFP county and national officials Desk review Varied as per PMP 3-Strong 81 Criteria Evaluation Questions Measure/ Indicator of Progress Main Sources of information Data Collection Methods Data Analysis Methods Evidence availability/ reliability b. Are assisted schools moving in the right direction of improving education outcomes and sustaining school feeding? 7. Proportion of students who by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. 8. Percent of food preparers at target schools who achieve a passing score on a test of safe food preparation and storage 9. Percent of teachers in target schools who attend school and teach at least 90 percent of scheduled school days per year. 10. Percent of students (girls/boys) regularly attending supported school. 11. Percent increase in girls/boys enrolled in schools 12. Percent of students in targeted schools consuming daily meals (lunch) School based assessment Face to face interviews with head teachers, pupils Varied as per PMP 3-Strong Sustainability measures put in by the school School based assessment Observations Review of data from schools F2F interviews with school heads, patrons, cooks, parents, etc. Qualitative Thematic analysis 3-Strong c. Did any negative effects occur for beneficiaries? Negative effects among beneficiaries School based assessment F2F interviews with school heads, patrons, project implementation stakeholders, parents, and pupils Qualitative Thematic analysisQuantitative based proportion (%) quantification of the negative effects 3-Strong 82 Criteria Evaluation Questions Measure/ Indicator of Progress Main Sources of information Data Collection Methods Data Analysis Methods Evidence availability/ reliability d. What were the gender-specific impacts, especially regarding enrolment and attendance? Gender based measures School based assessment a. Primary data collection among pupils’ b. Review and segmentation of all relevant secondary data by gender. C. Thematic assessment of issues while assessing for gender based differences. Analysis of all relevant findings by gender. Includes a sampling approach of pupils that addresses the need for separate gender quotas 3-Strong e. What are the main drivers of positive impacts? (Partnerships, capacity, ownership, etc.) Positive and negative drivers of impact f. What were the intended and unintended impacts of the program Number of total individuals benefiting indirectly from USDA￾funded interventions Face to face interviews with parents and pupils School based assessments Varied as per PMP 3-Strong Sustainability a. To what extent isthe country taking ownership of the program? (e.g. demonstrated commitment and contribution to the program); Ownership of the program Reports and Literature, F2F interviews with stakeholders Review of Literature/Data F2F data Thematic analysis of qualitative data 3-Strong b. What is the national readiness to implement the program? E.g. demonstrated capacity at central and sub-national levels to manage the program? SWOT analysis of the central and subnational levels Reports and Literature, F2F interviews with stakeholders 83 Annex 8: Quantitative data collection tools SCHOOL QUESTIONNAIRE A (To be filled by Head-teacher in advance) Section 1: Respondent Identification This section is to document information about the location school and the name and contact of the respondent and head teacher. # Question Response Code Q101 County Q102 Sub-County- District Q103 Zone Q105 School Name Q106 Latitude Q107 Longitude Q108 Respondent name Q109 Which Position do you occupy in this school 1. Head Teacher/Principal 2. Deputy Head Teacher 3. Teacher 4. Administrative officer 5. Other Q110 Director/Headmaster name (only if Respondent is not the Head Teacher) Q111 What is the school category 1. Boys School 2. Girls School 3. Mixed gender School Q112 Kindly indicate your Mobile Phone number Q113 Date of Interview / / Day/Month/Year (e.g. 15 / 04 / 2015) Q114 Name of Enumerator 84 Section 2: School Feeding Program (To be filled by Head-teacher in advance) Question Response Code Q201 What are the avenues or communication channels through which parents and pupils get information about SMP or make complaints about the program (WFP: Protection and Accountability to affected population)? (Circle all that apply) 1. None 2. Regular meetings with SMC 3. Regular meetings with School Administrators 4. Suggestion Box 5. Helpline 6. Other (specify) Q202 How many people are involved in the preparation of the food (food preparers, cook and store keeper) Q203 Does the school have a Parent Teacher Association (PTA)? 1. Yes 2. No Q204 Does the school have a School Meal Committee (SMC)? 1. Yes 2. No (skip to Q206) Q205 What is the level of participation and engagement of the SMC in the SMP? 1. High 2. Medium 3. Low Q206a What means of transport would someone use to the nearest food market? (Mark all that apply) 1. By Foot 2. Bicycle 3. Motorcycle 4. Auto RickShaw (Tuk Tuk) 5. Private Car 6. Public Bus/Matatu 7. Donkey/Camel 8. Other 9. Other 10. Other Q206b For the means of transport mentioned what time in minutes does it take to the nearest food market? (Mark all that apply) 1. By Foot: 2. Bicycle: 3. Motorcycle: 4. Auto RickShaw (Tuk Tuk): 5. Private Car: 6. Public Bus/Matatu: _ 7. Donkey/Camel: _ 8. Other: : 9. Other: : 10. Other: : Q207a What means of transport would someone use from this school the nearest educational office? 1. By Foot 2. Bicycle 3. Motorcycle 4. Auto RickShaw (Tuk Tuk) 5. Private Car 6. Public Bus/Matatu 85 7. Donkey/Camel 8. Other 9. Other 10. Other Q207b For the means of transport mentioned what time in minutes does it take from this school the nearest educational office? 1. By Foot: 2. Bicycle: 3. Motorcycle: 4. Auto RickShaw (Tuk Tuk): 5. Private Car 6. Public Bus/Matatu: _ 7. Donkey/Camel: _ 8. Other: : 9. Other: : 10. Other: : Section 3. Non-food contribution (To be filled by Head-teacher in advance) This section assesses the non–food contributions by schools, parents, external donors, or a combination of sources. Q208_1a: Did the SCHOOL make any non-food contributions in 2015? 1. Yes 2. No Non-food items contributed by school? Approximate % of requirement met (Cannot be more than 100%) Q208_2a Q208_3a Water 1. Yes 2. No (go to Next) Firewood 1. Yes 2. No (go to Next) Cooking Utensils 1. Yes 2. No (go to Next) Cleaning Products 1. Yes 2. No (go to Next) Plates and cutlery for pupils 1. Yes 2. No (go to Next) Cooks Salary 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Q208_1b: Did the PARENTS make any non-food contributions in 2015? 1. Yes 2. No Non-food items contributed by school? Approximate % of requirement met (Cannot be more than 100%) Q208_2b Q208_3b Water 1. Yes 2. No (go to Next) 86 Firewood 1. Yes 2. No (go to Next) Cooking Utensils 1. Yes 2. No (go to Next) Cleaning Products 1. Yes 2. No (go to Next) Plates and cutlery for pupils 1. Yes 2. No (go to Next) Cooks Salary 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Q208_1c: Did the EXTERNAL DONORS make any non-food contributions in 2015? 1. Yes 2. No Non-food items contributed by school? Approximate % of requirement met (Cannot be more than 100%) Q208_2c Q208_3c Water 1. Yes 2. No (go to Next) Firewood 1. Yes 2. No (go to Next) Cooking Utensils 1. Yes 2. No (go to Next) Cleaning Products 1. Yes 2. No (go to Next) Plates and cutlery for pupils 1. Yes 2. No (go to Next) Cooks Salary 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Q208_1d: Did the ANY OTHER PERSON make any non-food contributions in 2015? 1. Yes (Specify _) 2. No Non-food items contributed by school? Approximate % of requirement met (Cannot be more than 100%) Q208_2d Q208_3d Water 1. Yes 2. No (go to Next) Firewood 1. Yes 2. No (go to Next) Cooking Utensils 1. Yes 2. No (go to Next) Cleaning Products 1. Yes 2. No (go to Next) Plates and cutlery for pupils 1. Yes 2. No (go to Next) Cooks Salary 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Q208_1e: Did the ANY OTHER PERSON make any non-food contributions in 2015? 1. Yes (Specify _) 2. No Non-food items contributed by school? Approximate % of requirement met (Cannot be more than 100%) 87 Q208_2e Q208_3e Water 1. Yes 2. No (go to Next) Firewood 1. Yes 2. No (go to Next) Cooking Utensils 1. Yes 2. No (go to Next) Cleaning Products 1. Yes 2. No (go to Next) Plates and cutlery for pupils 1. Yes 2. No (go to Next) Cooks Salary 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Q208_1f: Did the ANY OTHER PERSON make any non-food contributions in 2015? 1. Yes (Specify _) 2. No Non-food items contributed by school? Approximate % of requirement met (Cannot be more than 100%) Q208_2f Q208_3f Water 1. Yes 2. No (go to Next) Firewood 1. Yes 2. No (go to Next) Cooking Utensils 1. Yes 2. No (go to Next) Cleaning Products 1. Yes 2. No (go to Next) Plates and cutlery for pupils 1. Yes 2. No (go to Next) Cooks Salary 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) Other (specify) 1. Yes 2. No (go to Next) 88 Section 4: Teacher-Head Teacher (To be filled by Head-teacher in advance) ID Teacher Name (Optional) Sex Age Qualification Position in the school Years of Years of Full time/ Part Teachers’ Teacherstrained 1. Senior Graduate 1. Head teacher/ teaching experience time attendance in in safe food 1. Male 2. GT1 principal experience as Head 2015 preparation and 2. Female 3. GT2 4. ATS 1 2. Deputy head teacher Teacher (for Head￾1. Full-time 2. Part-time (in percent) storage practices 5. ATS 2 3. Senior teacher Teacher 1. Yes 6. ATS3 4. Teacher only) 2. No 7. ATS4 (government) 5. Teacher (paid 8. P1 9. Untrained contract) 6. Teacher (volunteer) 7. Other (specify) 10. Other Q401 Q402 Q403 Q404 Q405 Q406 Q407 Q408 Q409 Q410 Q411 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 89 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 90 Section 6: Student record # Academic year Total number of students enrolled: (Male) Total number of students enrolled: (Female) Total number of students: Transferred OUT (Male) Total number of students: Transferred OUT (Female) Total number of students: Dropout (Male) Total number of students: Dropout (Female) Q601 Pre-Primary Primary Pre-Primary Primary Pre-Primary Primary Pre-Primary Primary Pre-Primary Primary Pre-Primary Primary Q602 Q603 Q604 Q605 Q606 Q607 Q608 Q609 Q610 Q611 Q612 Q613 1 2015 2 2014 3 2013 4 2012 2015 class 8 cohort Q614 Total number of students enrolled in 2008: (Male) Q615 Total number of students enrolled in 2008: (Female) Q616 Students cohort who completed Primary within the same school: (Male) Q617 Students cohort who completed Primary within the same school (Female) Q618 Total number of students: Transferred OUT but completed Primary elsewhere between 2008-2015 (Male) Q619 Total number of students: Transferred OUT but completed Primary elsewhere between 2008-2015 (Female) Q620 Total number of students: Dropout between 2008-2015 (Male) Q621 Total number of students: Dropout between 2008-2015 (Female) 91 Section 7: School & Student Performance Q700 Average KCPE Score (2015) Term 1 (2015) Term 2 (2015) Term 3 (2015) Average Mark for 2015 (out of 500) Q701 Q702 Q703 Q704 Average mark for Class 1 pupils (out of 500) Average mark for Class 2 pupils (out of 500) Average mark for Class 3 pupils (out of 500) Average mark for Class 4 pupils (out of 500) Average mark for Class 5 pupils (out of 500) Average mark for Class 6 pupils (out of 500) Average mark for Class 7 pupils (out of 500) Average mark for Class 8 pupils (out of 500) No. Pupils Name (Optional) Sex 1. Male 2. Female Class School Attendance (days) Average Pupil Mark (out of 500) Term 1 Term 2 Term 3 Term 1 Term 2 Term 3 Q705 Q706 Q707 Q708 Q709 Q710 Q711 Q712 Q713 1 2 3 4 5 6 7 8 92 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 93 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 94 95 Section 8: Record attendance & Food Utilization (SMP Form 6) YEAR 2016 Term Day Date Pre-Primary Attendance Primary Attendance Total Attendance School Feeding Provided 1. Yes 2. No (go to Q815) Maize / bulgar wheat (Kg) Pulses (Kg) Veg. Oil (Kg) Salt (Kg) CSB (Kg) Top 3 reasons for missed meals. (Use code below) What was the source of lunch eaten on those days when meal was not provided? (Use code below) Boys Girls Boys Girls Q801 Q802 Q803 Q804 Q805 Q806 Q807 Q808 Q809 Q810 Q811 Q812 Q813 Q814 Q815 Q816 Term1 1 Term1 2 Term1 3 Term1 4 Term1 5 Term1 6 Term1 7 Term1 8 Term1 9 Term1 10 Term1 11 Term1 12 Term1 13 Term1 14 Term1 15 Term1 16 Term1 17 Term1 18 96 Term1 19 Term1 20 Term1 21 Term1 22 Term1 23 Term1 24 Term1 25 Term1 26 Term1 27 Term1 28 Term1 29 Term1 30 Term1 31 Term1 32 Term1 33 Term1 34 Term1 35 Term1 36 Term1 37 Term1 38 Term1 39 Term1 40 Term1 41 Term1 42 Term1 43 Term1 44 Term1 45 Term1 46 Term1 47 97 Term1 48 Term1 49 Term1 50 Term1 51 Term1 52 Term1 53 Term1 54 Term1 55 Term1 56 Term1 57 Term1 58 Term1 59 Term1 60 Codes for Q815 Codes for Q816 1. Over scooping 2. Less food was delivered to the school than required 3. Lack of firewood therefore did not cook on certain days 4. No cook to prepare the food 5. Food was not delivered on time 6. Lack of cooking utensils 7. Other (Specify) 1. Pupils brought their own lunch from home 2. Pupils were sent home to eat during lunch time 3. Lunch purchased by school using school fees 4. Pupils did not eat lunch at all 5. Other (specify) 98 Section 8: Record attendance & Food Utilization (SMP Form 6) (Continued) YEAR 2016 Term Day Date Pre-Primary Attendance Primary Attendance Total Attendance School Feeding Provided 1. Yes 2. No (go to Q815) Maize / bulgar wheat (Kg) Pulses (Kg) Veg. Oil (Kg) Salt (Kg) CSB (Kg) Top 3 reasons for missed meals. (Use code below) What was the source of lunch eaten on those days when meal was not provided? (Use code below) Boys Girls Boys Girls Q801 Q802 Q803 Q804 Q805 Q806 Q807 Q808 Q809 Q810 Q811 Q812 Q813 Q814 Q815 Q816 Term2 1 Term2 2 Term2 3 Term2 4 Term2 5 Term2 6 Term2 7 Term2 8 Term2 9 Term2 10 Term2 11 Term2 12 Term2 13 Term2 14 Term2 15 Term2 16 Term2 17 Term2 18 Term2 19 99 Term2 20 Term2 21 Term2 22 Term2 23 Term2 24 Term2 25 Term2 26 Term2 27 Term2 28 Term2 29 Term2 30 Term2 31 Term2 32 Term2 33 Term2 34 Term2 35 Term2 36 Term2 37 Term2 38 Term2 39 Term2 40 Term2 41 Term2 42 Term2 43 Term2 44 Term2 45 Term2 46 Term2 47 Term2 48 100 Term2 49 Term2 50 Term2 51 Term2 52 Term2 53 Term2 54 Term2 55 Term2 56 Term2 57 Term2 58 Term2 59 Term2 60 Term2 61 Term2 62 Term2 63 Term2 64 Term2 65 Term2 66 Term2 67 Term2 68 Term2 69 Term2 70 Codes for Q815 Codes for Q816 1. Over scooping 2. Less food was delivered to the school than required 3. Lack of firewood therefore did not cook on certain days 4. No cook to prepare the food 5. Food was not delivered on time 1. Pupils brought their own lunch from home 2. Pupils were sent home to eat during lunch time 3. Lunch purchased by school using school fees 4. Pupils did not eat lunch at all 5. Other (specify) 101 6. Lack of cooking utensils 7. Other (Specify) 102 Section 8: Record attendance & Food Utilization (SMP Form 6) (Continued) YEAR 2015 Term Day Date Pre-Primary Attendance Primary Attendance Total Attendance School Feeding Provided 1. Yes 2. No (go to Q815) Maize / bulgar wheat (Kg) Pulses (Kg) Veg. Oil (Kg) Salt (Kg) CSB (Kg) Top 3 reasons for missed meals. (Use code below) What was the source of lunch eaten on those days when meal was not provided? (Use code below) Boys Girls Boys Girls Q801 Q802 Q803 Q804 Q805 Q806 Q807 Q808 Q809 Q810 Q811 Q812 Q813 Q814 Q815 Q816 Term1 1 Term1 2 Term1 3 Term1 4 Term1 5 Term1 6 Term1 7 Term1 8 Term1 9 Term1 10 Term1 11 Term1 12 Term1 13 Term1 14 Term1 15 Term1 16 Term1 17 Term1 18 Term1 19 103 Term1 20 Term1 21 Term1 22 Term1 23 Term1 24 Term1 25 Term1 26 Term1 27 Term1 28 Term1 29 Term1 30 Term1 31 Term1 32 Term1 33 Term1 34 Term1 35 Term1 36 Term1 37 Term1 38 Term1 39 Term1 40 Term1 41 Term1 42 Term1 43 Term1 44 Term1 45 Term1 46 Term1 47 Term1 48 104 Term1 49 Term1 50 Term1 51 Term1 52 Term1 53 Term1 54 Term1 55 Term1 56 Term1 57 Term1 58 Term1 59 Term1 60 Codes for Q815 Codes for Q816 1. Over scooping 2. Less food was delivered to the school than required 3. Lack of firewood therefore did not cook on certain days 4. No cook to prepare the food 5. Food was not delivered on time 6. Lack of cooking utensils 7. Other (Specify) 1. Pupils brought their own lunch from home 2. Pupils were sent home to eat during lunch time 3. Lunch purchased by school using school fees 4. Pupils did not eat lunch at all 5. Other (specify) 105 Section 8: Record attendance & Food Utilization (SMP Form 6) (Continued) YEAR 2015 Term Day Date Pre-Primary Attendance Primary Attendance Total Attendance School Feeding Provided 1. Yes 2. No (go to Q815) Maize / bulgar wheat (Kg) Pulses (Kg) Veg. Oil (Kg) Salt (Kg) CSB (Kg) Top 3 reasons for missed meals. (Use code below) What was the source of lunch eaten on those days when meal was not provided? (Use code below) Boys Girls Boys Girls Q801 Q802 Q803 Q804 Q805 Q806 Q807 Q808 Q809 Q810 Q811 Q812 Q813 Q814 Q815 Q816 Term2 1 Term2 2 Term2 3 Term2 4 Term2 5 Term2 6 Term2 7 Term2 8 Term2 9 Term2 10 Term2 11 Term2 12 Term2 13 Term2 14 Term2 15 Term2 16 Term2 17 Term2 18 Term2 19 106 Term2 20 Term2 21 Term2 22 Term2 23 Term2 24 Term2 25 Term2 26 Term2 27 Term2 28 Term2 29 Term2 30 Term2 31 Term2 32 Term2 33 Term2 34 Term2 35 Term2 36 Term2 37 Term2 38 Term2 39 Term2 40 Term2 41 Term2 42 Term2 43 Term2 44 Term2 45 Term2 46 Term2 47 Term2 48 107 Term2 49 Term2 50 Term2 51 Term2 52 Term2 53 Term2 54 Term2 55 Term2 56 Term2 57 Term2 58 Term2 59 Term2 60 Codes for Q815 Codes for Q816 1. Over scooping 2. Less food was delivered to the school than required 3. Lack of firewood therefore did not cook on certain days 4. No cook to prepare the food 5. Food was not delivered on time 6. Lack of cooking utensils 7. Other (Specify) 1. Pupils brought their own lunch from home 2. Pupils were sent home to eat during lunch time 3. Lunch purchased by school using school fees 4. Pupils did not eat lunch at all 5. Other (specify) 108 Section 8: Record attendance & Food Utilization (SMP Form 6) (Continued) YEAR 2015 Term Day Date Pre-Primary Attendance Primary Attendance Total Attendance School Feeding Provided 1. Yes 2. No (go to Q815) Maize / bulgar wheat (Kg) Pulses (Kg) Veg. Oil (Kg) Salt (Kg) CSB (Kg) Top 3 reasons for missed meals. (Use code below) What was the source of lunch eaten on those days when meal was not provided? (Use code below) Boys Girls Boys Girls Q801 Q802 Q803 Q804 Q805 Q806 Q807 Q808 Q809 Q810 Q811 Q812 Q813 Q814 Q815 Q816 Term3 1 Term3 2 Term3 3 Term3 4 Term3 5 Term3 6 Term3 7 Term3 8 Term3 9 Term3 10 Term3 11 Term3 12 Term3 13 Term3 14 Term3 15 Term3 16 Term3 17 Term3 18 Term3 19 Term3 20 109 Term3 21 Term3 22 Term3 23 Term3 24 Term3 25 Term3 26 Term3 27 Term3 28 Term3 29 Term3 30 Term3 31 Term3 32 Term3 33 Term3 34 Term3 35 Term3 36 Term3 37 Term3 38 Term3 39 Term3 40 Term3 41 Term3 42 Term3 43 Term3 44 Term3 45 Term3 46 Term3 47 Term3 48 Term3 49 110 Term3 50 Term3 51 Term3 52 Term3 53 Term3 54 Term3 55 Term3 56 Term3 57 Term3 58 Term3 59 Term3 60 Term3 61 Term3 62 Term3 63 Term3 64 Term3 65 Term3 66 Term3 67 Term3 68 Term3 69 Term3 70 Codes for Q815 Codes for Q816 1. Over scooping 2. Less food was delivered to the school than required 3. Lack of firewood therefore did not cook on certain days 4. No cook to prepare the food 5. Food was not delivered on time 6. Lack of cooking utensils 1. Pupils brought their own lunch from home 2. Pupils were sent home to eat during lunch time 3. Lunch purchased by school using school fees 4. Pupils did not eat lunch at all 5. Other (specify) 111 7. Other (Specify) 112 Section 9: Delivery Record Q900a: What quantity of food was received during the academic year 2016? Was there any carryoverstock for this item at the beginning of Term 2 of 2016 (January)? 1. Yes 2. No (skip to Q920) Amount in kg Did school receive this item in Term 2 of 2016? 1. Yes 2. No(skip to Q922) Amount in kg Any loss incurred during Term 2 of 2016? 1. Yes 2. No(skip to Q924) Amount in kg Actual Delivery Date Expected delivery date (based on Distribution Plan) Q901a Q902a Q903a Q904a Q905a Q906a Q907a Q908a Q909a 1 Maize 2 Bulgar Wheat/Rice 3 Pulses 4 Vegetable Oil 5 CSB/CSM/UNIMIX 6 Salt 7 Other 8 Other #2 9 Other #3 _ 113 Section 9: Delivery Record (CONTINUED) Q900b: What quantity of food was received during the academic year 2016? Was there any carryoverstock for this item at the beginning of Term 1 of 2016 (January)? 1. Yes 2. No (skip to Q920) Amount in kg Did school receive this item in Term 1 of 2016? 1. Yes 2. No(skip to Q922) Amount in kg Any loss incurred during Term 1 of 2016? 1. Yes 2. No(skip to Q924) Amount in kg Actual Delivery Date Expected delivery date (based on Distribution Plan) Q901b Q902b Q903b Q904b Q905b Q906b Q907b Q908b Q909b 1 Maize 2 Bulgar Wheat/Rice 3 Pulses 4 Vegetable Oil 5 CSB/CSM/UNIMIX 6 Salt 7 Other 8 Other #2 9 Other #3 _ 114 Section 9: Delivery Record (CONTINUED) Q900c: What quantity of food was received during the academic year 2015? Was there any carryoverstock for this item at the beginning of Term 3 of 2015 (September)? 1. Yes 2. No (skip to Q904) Amount in kg Did school receive this item in Term 3 of 2015? 1. Yes 2. No(skip to Q906) Amount in kg Any loss incurred during Term 3 of 2015? 1. Yes 2. No(skip to Q908) Amount in kg Actual Delivery Date Expected delivery date (based on Distribution Plan) Q901c Q902c Q903c Q904c Q905c Q906c Q907c Q908c Q909c 1 Maize 2 Bulgar Wheat/Rice 3 Pulses 4 Vegetable Oil 5 CSB/CSM/UNIMIX 6 Salt 7 Other 8 Other #2 9 Other #3 _ 115 Section 9: Delivery Record (CONTINUED) Q900d: What quantity of food was received during the academic year 2015? Was there any carryoverstock for this item at the beginning of Term 2 of 2015 (May)? 1. Yes 2. No (skip to Q912) Amount in kg Did school receive this item in Term 2 of 2015? 1. Yes 2. No(skip to Q914) Amount in kg Any loss incurred during Term 2 of 2015? 1. Yes 2. No(skip to Q916) Amount in kg Actual Delivery Date Expected delivery date (based on Distribution Plan) Q901d Q902d Q903d Q904d Q905d Q906d Q907d Q908d Q909d 1 Maize 2 Bulgar Wheat/Rice 3 Pulses 4 Vegetable Oil 5 CSB/CSM/UNIMIX 6 Salt 7 Other 8 Other #2 9 Other #3 _ 116 Section 9: Delivery Record (CONTINUED) Q900e: What quantity of food was received during the academic year 2015? Was there any carryoverstock for this item at the beginning of Term 1 of 2015 (January)? 1. Yes 2. No (skip to Q920) Amount in kg Did school receive this item in Term 1 of 2015? 1. Yes 2. No(skip to Q922) Amount in kg Any loss incurred during Term 1 of 2015? 1. Yes 2. No(skip to Q924) Amount in kg Actual Delivery Date Expected delivery date (based on Distribution Plan) Q901e Q902e Q903e Q904e Q905e Q906e Q907e Q908e Q909e 1 Maize 2 Bulgar Wheat/Rice 3 Pulses 4 Vegetable Oil 5 CSB/CSM/UNIMIX 6 Salt 7 Other 8 Other #2 9 Other #3 _ 117 6.1.1.1.1 School Questionnaire B: Observation of School Facilities (To be conducted by interviewer on the day of school visit) Section 1: School Identification: Section 2: Observation of School Facilities (To be administered by interviewer during the actual school visit) This section looks at various facilities available within the school. Question Response Code Q301 Number of classrooms Q302 Where does the school store its supplementary books 1. A dedicated room for a library 2. Staff room 3. No Supplementary books at all (go to Q304) 4. Other room (Specify ) Q303 How many supplementary books does the school have? Q304 Does your school have a dedicated kitchen? 1. Yes (go to Q306) 2. No Q305 If not, where is the food normally prepared? 1. In a classroom 2. Open space 3. Other # Question Response Code Q101 County Q102 Sub-County- District Q103 Zone Q105 School Name Q106 Latitude Q107 Longitude Q111 What is the school category 1. Boys School 2. Girls School 3. Mixed gender School Q112 Kindly indicate your Mobile Phone number Q113 Date of Interview / / Day/Month/Year (e.g. 15 / 04 / 2015) Q114 Name of Enumerator 118 Q306 Does your school use energy saving stoves? 1. Yes 2. No Q307 What is the main source of DRINKING water for the school? 1. Piped water into dwelling, plot, or yard 2. Public tap/standpipe 3. Tube well/borehole 4. Protected dug well 5. Protected spring 6. Rainwater collection 7. Unprotected spring 8. Cart with small tank/drum 9. Tanker truck 10 Surface water (river, dam, lake, pond, stream, canal, or irrigation channel) 11 Bottled water 12. Children carry water from home 13. Other Q308 Does the school have toilets for pupils? OBSERVATION 1. Yes 2. No (go to section 4) Q309 Do the toilets have hand washing facilities within or nearby? OBSERVATION 1. Yes 2. Some of them 3. No Q310 How many toilets blocks have hand washing facilities within or nearby? Q311 Do girls have separate toilets from boys? OBSERVATION 1. Yes 2. No What type of toilet does the school have Does your school have this type of toilet? 1. Yes 2. No (go to next) Number of boys’ toilets Number of girls’ toilets Number of mixed toilets Q312 Q313 Q314 Q315 Q316 1 Flush or pour/flush facilities connected to a: (Piped sewer, septic, pit latrine) 2 Flush or pour/flush toilets without a sewer connection 3 Pit latrines with a slab 4 Pit latrines without slab/open pit 5 Ventilated improved pit latrines 6 Composting toilets 7 Bucket latrines 8 Hanging toilets/latrines 9 No facilities, open defecation 119 School Questionnaire C: Observation of Attendance in School To be conducted by an interviewer during the school visit Section 1: School Identification: Section 2: Observation School Survey (To be administered by interviewer during the actual school visit) Pupils attending class during survey Day of Interview Compute from Roster 2 days before interview Compute from Roster 2 days before interview Total Number of Registered Pupils in Class Q501a Q501b Q501c Q501d Standard 1 Standard 2 Standard 3 Standard 4 Standard 5 Standard 6 Standard 7 Teachers present in school during survey Day of Interview Compute from Roster 1 day before interview Compute from Roster 2 days before interview Total Number of Teachers employed in the school Q502a Q502b Q502c Q502d # Question Response Code Q101 County Q102 Sub-County- District Q103 Zone Q105 School Name Q106 Latitude Q107 Longitude Q111 What is the school category 1. Boys School 2. Girls School 3. Mixed gender School Q112 Kindly indicate your Mobile Phone number Q113 Date of Interview / / Day/Month/Year (e.g. 15 / 04 / 2015) Q114 Name of Enumerator 120 6.1.1.1.2 School Questionnaire D: Observation of Lunch Serving in the School To be conducted by an interviewer during the school visit during lunch hour Section 1: School Identification: # Question Response Code Q101 County Q102 Sub-County- District Q103 Zone Q105 School Name Q106 Latitude Q107 Longitude Q111 What is the school category 1. Boys School 2. Girls School 3. Mixed gender School Q112 Kindly indicate your Mobile Phone number Q113 Date of Interview / / Day/Month/Year (e.g. 15 / 04 / 2015) Q114 Name of Enumerator Section 2: Observation of the Lunch Servings The enumerator weighs 20 random pupils’ rations (in grams) Which Standard (class) is the pupil Observations Day of Interview Day 2 Q503 Q504 1 2 3 4 5 6 7 8 9 10 11 12 13 121 14 15 16 17 18 19 20 How many teachers eat lunch with food coming from the same pot used to feed the pupils? (FROM OBSERVATION) Day 1 Q510 122 6.1.1.1.3 School Questionnaire E: Observation of Commodities Provided for Cooking To be conducted by an interviewer during the school visit during the provision of Commodities Section 1: School Identification: # Question Response Code Q101 County Q102 Sub-County- District Q103 Zone Q105 School Name Q106 Latitude Q107 Longitude Q111 What is the school category 1. Boys School 2. Girls School 3. Mixed gender School Q112 Kindly indicate your Mobile Phone number Q113 Date of Interview / / Day/Month/Year (e.g. 15 / 04 / 2015) Q114 Name of Enumerator The enumerator weighs commodities cooked on each school survey day (distinguish between commodities provided by WFP and by School/Parent contribution) (in grams) No Items WFP School Parents Q513 Q514 Q515 1 Maize 2 Bulgar Wheat/Rice 4 Pulses 5 Vegetable Oil 6 CSB/CSM/UNIMIX 7 Salt 8 Fruits 9 Vegetables 10 Meat 11 Other (specify) 12 Other (specify) 123 No Non-food item contribution by School/Parents Observed during visit 1. Yes 2. No Q520 1 Water 2 Firewood 3 Cooking Utensils 4 Cleaning Products 5 Plates and cutlery for pupils 6 Cooks Salary 7 Other (specify) 8 Other (specify) 9 Other (specify) 6.1.1.1.4 Teacher Questionnaire Respondent Identification # Question Response Q101 County Sub-County- District Zone School name Q102 Date of Interview / / Q103 Name of Enumerator # Teacher Name (optional) Class In your observation, on average in 2015, what percentage of students in your classes was inattentive? According to you, what is the main reason for students’ inattentiveness? 1. Hunger 2. Tiredness from domestic or income-generating work 3. Sickness 4. Perceived unimportance of primary education 5. Other (specify) According to you, if WFP SMP would stop today, what would be the consequence on pupil attendance? 1. No consequence, attendance remains the same 2. Attendance will drop slightly (10%) 3. Attendance will drop drastically (over 30%) Q104 Q105 Q106 Q107 Q108 Q109 1 2 3 4 5 6 7 8 124 9 10 Comment: Note to World Food Program: In as much as the above was used to measure attentiveness at midline and end line, we would like to propose an additional approach that considers each and every pupil who had been sampled as opposed to general assessment of the entire class by the class teacher. At analysis, engagement will be measured as percentage of sampled pupils who were rated positively by their teachers out of all pupils sampled in the survey. Additionally, an average index ranging from 1 to 5 will be used to represent engagement score. This will be an additional measure beyond the above. For each and every Class Teacher: As in relation to the specific pupil who was sampled Question On a scale of 1 to 5 (1 Never, 2 Rarely, 3 Don’t Know, 4 Sometimes, 5 Always, 98 -Not Applicable) Pupil 1 (Female) Pupil 2 (Male) The student asks questions or contributed to discussions in class? The student seeks advice from academic staff? The student makes class presentation? The student uses library resources in school? The student comes to class having completed readings or assignments? The student keeps up to date with your studies? The student works with other students on projects during class? The student works with other students outside class to prepare assignments? The student is attentive in class? 125 6.1.1.1.5 Pupil Questionnaire Respondent Identification # Question Response Code Q101 County Q102 Sub-County- District Q103 Zone Q101 School name Q102 Student name Q103 Sex 1=Male 2=Female Q104 Age Q105 Current Grade/Class Q106 Father' name Q107 Mother's name Q108 Date of Interview / / Day/Month/Year (e.g. 15 / 04 / 2016) Q109 Name of Enumerator (alias/code name) Section 2 Question Response Code Q201 Did you eat a meal during lunch time yesterday/last school day? 1=Yes 2=No (go to Q205) Q202 Where did this meal come from? 1. Lunch provided by school 2. Bring own food and eat at school 3. Pay for lunch e.g. kiosk or school canteen 4. Went home for lunch and then came back 5. Other Q203 Were you still hungry after the meal? 1=Yes 2=No Q204 Did you leave anything on the plate? 1=Yes 2=No Q205 In the past 5 school days, how many days did you eat breakfast before going to school? 0 to 5 Q206 In the past 5 school days, how many days did you eat a meal after going to school (and before going to bed)? 0 to 5 Q207 Normally, if you become aware that the school food is finished, do you come to school the next day? 1=Yes 2=No 126 Q208 How long does it take to get to school? 1=less than 15 minutes, 2=between 15 and 30 minutes, 3=between 30 and 60 minutes, 4=more than 1 hour Q209 How do you travel to school? 1=on foot, 2=by bicycle, 3=by car, 4=by bus, 5=by motorbike, 6=other 6.1.1.1.6 Household Questionnaire Respondent Identification # Question Response Code Q101 County Sub-County- District Zone School Name Q102 Household address Q103 Household size Q104 Number of pupils in pre-primary and primary school within the HH Q105 Number of school years attended by the head of the household? Q106 What is the main occupation of the head of the household? 1=Too old to work 2= Student 3= Farmer 4= Pastoralist 5= Salaried Employee 6= Casual Labourer 7= Self-employed business 8= Not currently working 9= Other Q107 What is the head of the household’s current marital status? 1 = Married or living together 2 = Divorced or separated 3 = Widowed 4 = Single Q108 Date of Interview Q109 Enumerator Name Section 1: Demographics 127 # Name of Respondent Age of Respondent Sex 1=Male 2=Female Phone Number Relationship to the head of the household 1= head 2=spouse 3=elder son/daughter 4=brother/sister 5=other relative Q110 Q111 Q112 Q113 Q114 Q115 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 128 Section 2: Questions are related exclusively to the pupil through which this HH member was selected Respondent ID (from previous section) Which class/grade does child attend? In the past 5 school days, how many days did the child eat breakfast before going to school? (0 to 5) In the past 5 school days, how many days did the child eat ameal after going to school (and before going to bed)? (0 to 5) On school days, when school meals are provided, do you reduce the portion of food provided to the pupil compared to the week end? 1. Yes 2. No (go to Q207) On average how much do you reduce the portion compared to the week end? 1. 25% 2. 50% 3. 75% Was there a time in 2015 when the school did not provide a meal? 1. Yes 2. No (go to Q209) 3. I don’t know If yes, how did the child eat lunch at school? 1. Child brought own food and eat at school 2. Gave cash to child to buy lunch 4. Child came home for lunch and then went back to school 5. Child remained home and eat home 6. No lunch 7. Other (Specify) Q201 Q202 Q203 Q204 Q205 Q206 Q207 Q208 1 2 3 4 5 6 7 8 9 10 129 Section 3: Questions are related exclusively to the pupil through which this HH member was selected (CONTINUE) Respondent ID (from previous section) Did the school asked payment specific for school meals during the academic year 2015? 1. Yes (go to Q210) 2. No (go to Q211) How much? (In Kshs) During the academic year 2015, approximately how many days was the child absent from school? Days absent for school closure during the school terms Days absent for child work on farm or livestock tending Days absent for child looking after siblings or domestic work Days absent because of lack of food Days absent because of insecurity, fear of going to school Days absent for sickness Days absent for ceremonies and family events Days absent due to skipping school Q201 Q209 Q210 Q211 Q212 Q213 Q214 Q215 Q216 Q217 Q218 Q219 1 2 3 4 5 6 7 8 9 10 130 Section 4: School related questions Question Response Code Q301 Please list at least 3 benefits of Primary education? 1. Improves literacy rate 2. Social Skill Development 3. Increases ability to learn new skills (adoption of technology) 4. Girls remain more in school and early marriages are delayed 5. Improves cohesion in the community 6. Helps break the cycle of poverty 7. Increases the chances of the pupils' future economic self-reliance 8. Through girls' education, improves the general wellbeing of households (nutrition, health etc) Q302 Do any avenues exist through which you can get information or make complaints on the SMP? 1=Yes 2=No (go to Q304) Q303 If yes, which avenues exist? 1 .Regular meetings with SMC 2. Regular meetings with School Administrators 3. Suggestion Box 4. Hotline 5. Other (specify) Q304 Are you aware of the existence of the School Meal Committee (SMC)? 1=Yes 2=No (go to Q306) Q305 If Yes, what is your perception of the SMCs involvement in the SMP. 1=High 2=Medium 3=Low Q306: In the past 30 days, please indicate the number of cases of physical and/or emotional threats to the safety of your child that he/she was exposed to when walking to and from school or at school. ID Type of threat Number of cases child was exposed to this threat Q306_1 Q306_2 Q306_3 1 Rape 2 Sexual Harassment 3 Robbed 4 Animal Attacks 5 Bullying 6 Abuse of drugs 131 6.1.1.1.7 Cook Questionnaire Respondent Identification # Question Response Code Q101 County Sub-County- District Zone School name Q102 Respondent name Q103 Sex of the Respondent: 1. Male 2. Female Q104 Respondent's Position: 1. Cook 2. Food Preparer Q105 Date of Interview / / Day/Month/Year (e.g. 15 / 04 / 2015) Q106 Name of Enumerator 132 Section 2: School feeding program information # Question Response Code Q201 Are you trained in safe food preparation and storage practices (confirm with any records, if available)? 1. Yes 2. No (go to Q203) Q202 If Yes, how many times have you received training in 2015? Q203 Do you have a valid health certificate? 1. Yes (go to Q205) 2. No Q204 if no, what is the main reason? 1. Cannot afford the fee 2. Didn't have time to go to the health centre 3. Don't know how to get one 4. Don't think I need one 5. Other (specify) Q205 Do children wash their hands before the meal? 1. Yes 2. No Q206 When is the last time you received your salary? / MONTH/YEAR Q207 Do you receive your salary regularly every month? 1. Yes 2. No Q208 Do you use energy saving stoves? 1. Yes 2. No Q209 Are the cooks clean and well groomed? DIRECT OBSERVATION 1. Yes 2. No Section 3: Safe Food Preparation Practices (for Cook) # Question Response Code Q301 Do you have a uniform or apron for use in the kitchen? 1. Yes 2. No Q302 When do you clean your kitchen? 1. Every morning before food preparation, often during the day and after use 2. After food preparation 3. At the end of the week Q303 Which is the best source of water for cleaning and cooking food? 1. Piped water, rain water and boreholes which are well protected 2. Water from the river/streams 3. Water from a pond Q304 When do you usually wash your hands for food preparation? 1. Before handling food and often during food preparation 2. After using the latrine 3. After finishing food preparation 4. Never Q305 How do you ensure that food is clean before cooking? 1. Rinse it in water and cook 2. Remove foreign matters then cook 3. Use clean containers to collect it from the store, remove foreign matters and then wash it with clean water thoroughly before cooking 133 Q306 When do you wash your cooking utensils (cooking pots, lids, scoops, knives, plates etc.) with clean water and soap 1. After use 2. Prior to using them 3. Prior to, after using them and drying them in a rack before storage Q307 Are there measures in place to prevent food from contamination from pests and rodents? Name them: 1. 2. Q308 What is the most important thing to check in food before cooking? 1. Expiry date, packaging, color of the food, presence of pests 2. Source of food 3. Colour of the package Q309 How do you store cooked food prior to serving the pupils? 1. Store cooked food in covered cooking pots in a clean, safe place before serving the pupils 2. Store cooked food in open containers 3. Store cooked food outside the kitchen without covers 134 6.1.1.1.8 Storekeeper Questionnaire Respondent identification # Question Response Code Q101 County Sub-County- District Zone School name Q102 Respondent name Q103 Sex of the Respondent: 1. Male 2. Female Q104 Date of Interview / / Day/Month/Year (e.g. 15 / 04 / 2015) Q105 Name of Enumerator Section 2: School feeding program information # Question Response Code Q201 Does your school have a dedicated food store room? 1. Yes 2. No Q202 Is the food store room lockable? ASK QUESTION + OBSERVATION 1. Yes 2. No Q203 Is the store room properly ventilated? DIRECT OBSERVATION 1. Yes 2. No Q204 Is there any evidence of presence of rodents in the store? ASK QUESTION + OBSERVATION 1. Yes 2. No Q205 Is there any evidence of presence of insects (weevil and others)? ASK QUESTION + OBSERVATION 1. Yes 2. No Q206 Is there any evidence of mould and excess of humidity? ASK QUESTION + OBSERVATION 1. Yes 2. No Q207 Is there any evidence of spillage or leakage? ASK QUESTION + OBSERVATION 1. Yes 2. No Q208 Is the food stored off the ground? ASK QUESTION + OBSERVATION 1. Yes 2. No (go to Q210) Q209 If yes, does the school use improvised raised pallets for commodities'storage? ASK QUESTION + OBSERVATION 1. Yes 2. No Q210 Does the school carry out pest/insects control measures? 1. Yes 2. No Q211 Are you trained in safe food preparation and storage practices? 1. Yes 2. No (End of interview) Q212 If Yes, how many times have you received training in 2015? 135 Annex 9: Qualitative data collection tools 6.1.1.1.9 Key Informant Documentation and Introduction Form My name is and I work for TNS RMS East Africa, a research agency conducting economic and social research in different disciplines. We are currently conducting an evaluation of the School Meals Program implemented by World Food Programbetween 2014 and 2016. The main of the evaluation is to assess the achievements, if any, of the program and document the lessons learnt by the program. You have been selected as one of the key informants who could provide relevant information for the evaluation. Your participation in this evaluation is voluntary. Your responses will be treated with confidence since the information you provide will be combined with other responses to give a general picture of the assessment. During this interview, I will record the interview to allow for later transcription and analysis of data. If you have any questions please feel free to call TNS Public Affairs team, specifically Stephen Aloo or Bernard Nyauchi who are the team leaders on +254 (0)709 842 000 during office hours. Do you have any questions for me? Do you agree to participate in this evaluation? Moderator Document Consent: Yes/No Name of Key Informant Organisation Position in the Organisation Years with the organisation Current Location of Key Informant Date of Interview Method of Interview 1. Phone Based Interview 2. Face to Face Interview Name of Interview 6.1.1.1.10 Key Informant Guide WFP Staff WFP Officers Nairobi Logistics Team: 1. Location: where are you currently based? 2. What is your role in in the project? (Probe for how long they have been involved inthe specific component on the project) 3. Whom do you work with in the project?Food Distribution 4. How is the quantity of food to be distributed to the schools calculated at DEOs' level? o Probe: Is it based on enrolment information received from the schools or onfood availability only? 136 5. Regarding the Distribution Plans, is data on actual food received by the schoolsrecorded following the delivery? 6. Confirm the pipeline supply chain, particularly the secondary transport￾are thelogistics teams in Nairobi able to control the timely delivery of the secondary transport? 7. Does WFP ever experience delays in the dispatch of food due to customs andclearance issues? o If yes, what are the main reasons for the pipeline breaks when they occur? o Probe: Tender process for identification of trucks for primary transport-experiences and lessons learned. Food Storage 8. Relationship with DEO and local government officials – lessons learnt and recommendations. 9. What are some of the losses that WFP have experienced in its own warehouses?(Probe for when) If so, due to what reasons? 10. What are the procedures adopted for pest control and management in the DEOswarehouse? o How common is it for these warehouses to incur food losses due to pest/insectattacks? o What are the main reasons for food losses in DEOs warehouses?Program Handover 11. In terms of the handover of the program to the government (HGSMP), are there any plans for WFP to handover some of the logistics aspects/arrangements? o If yes, elaborate which aspects o If no, which aspects would you recommend Recommendations 12. In your opinion, are there any areas where WFP could provide additional support? i.e. procurement and logistics aspects, technical assistance 13. What are some of the factors that have affected implementation of the SMP? o Negative o Positive 14. Do you have any recommendation regarding specific aspects that should be addressed by the TNS Evaluation team? 6.1.1.1.11 Key Informant Guide WFP Staff This guide will be administered to the School Meal program team based in Nairobi and at the counties School Meal Program Team: 1. Location: where are you currently based? 2. What is your role in in the project? (Probe for how long they have been involved inthe specific component on the project) 3. Brain storm main points for benefits of primary education (related to thequestionnaires). 4. According to your observations, is there an actual difference in the ration size of pre-primary and primary children or do they all receive the same amount of food? 137 o Probe: if and how the schools record these different ration sizes distributed tochildren? 5. Regarding the meal rations, what are the schools' coping mechanisms if they receiveless food? Do they re-calculate the rations based on food available? 6. According to your observations, what has been the general experience with o Teachers' attendance? o Pupils’ attendance? 7. What are the community’s contribution to the SMP o Probe: Food and non-food items 8. What are WFP's generic impressions on the hygiene conditions of the differentschools across counties? Are the basic standards generally met? 9. Do schools usually have both a cook and a store keeper or are these functions coveredby the same person? 10. According to your experience, do schools have energy efficient stoves? 11. Do you have any recommendations regarding specific aspects that should beaddressed by the TNS Evaluation Team? 6.1.1.1.12 Key Informant Guide WFP Staff Senior Management and Key Program Staff: Director Level 1. Location: where are you currently based? 2. What is your role in in the project? (Probe for how long they have been involved inthe specific component on the project) Targeting 1. Generally, what are your views about the targeting for the WFP SMP? (Probe fortarget population, success, challenges) 2. Which category of respondents or counties benefited the most from the program(Probe for reasons) 3. What changes in the targeting did your team implement along the progamme? (Probefor reasons for the specific mentions). Success: 4. What would you say are some of the successes of the WFP SMP program? 5. What are the major reasons for the successes of the WFP SMP program?Challenges 6. What are some of the key challenges experienced in the implementation of theprogram? (Probe for specific responses?) 7. What were the coping measures for each challenge? 8. What recommendations would you make for improving the challenges in future?SMP Handover 9. How is the handover process going? o Probe: whether it is going according to WFP Country Progam plans o If no, what are the changes? 10. What are the plans to capacity build the GoK for the HGSMP in relations tosecondary transport in particular Commodity Management and Bidding process to select transporters as well as Quality Assurance? What are some of the successes around thesame? What are some of the challenges around this? Sustainability 11. What is your understanding of the sustainability of the SMP? (Probe for what theymean by sustainability) Recommendations 138 12. What recommendations would you give to improve any future implementation ofSMP? 6.1.1.1.13 Key Informant Guide for Donors USDA Representative in Nairobi and Washington (Donor) 1. Location: where are you based? 2. What is role of USDA in the project? 3. Whom do you work with in the project? o Probe for synergy with WFP and other stakeholders 4. How has your experience WFP management of current agreement been like? 5. What are the major challenges with the implementation of the SMP 6. What is your view on the pipeline breaks and delay of commodity? 7. What is your view on the handover strategy and the involvement of USDA in theSMP beyond 2016? o Probe for difference in cash versus the in-Kind Department of Foreign Affairs, Trade and Development Canada (DFATD) – (Donor) 1. What is role of DFATD in the SMP? 2. Whom do you work with in the project? o Probe for synergies with WFP and other stakeholders 3. What is your view on the handover strategy? 4. What is the DFATD’s future funding plan for SMP? 5. What are the key lessons learnt? 6. What recommendations would you give to improve any future implementation of SMP? 6.1.1.1.14 Key Informant Guide for MoE County Directors of Education (CDE) and District Education Officers (DEOs) (MOE): 1. Location: where are you currently based? 2. What is your role in in the project? (Probe for how long they have been involved inthe specific component on the project) Targeting 3. Generally what are your views about the targeting for the WFP SMP? (Probe fortarget population, success, challenges) 4. Which category of respondents or counties benefited the most from the program(Probe for reasons) 5. What changes in the targeting did your team implement along the progamme? (Probefor reasons for the specific mentions). 6. What is the position and relevance of the SMP within the broader education policiesand programs? 7. What are the education policies updated, issued or are under discussion between 2014and 2015? o Probe: number and examples of the policies 8. What is the involvement of MoE in the design of WFP M&E tools and the selectionof USDA indicators? Collaboration 9. How does MoE collaborate/interact with MoH in regards to the SMP? o Probe: how they are linked and distinct role for each o Probe for policy linkages between MoE and MoH 139 Effect 10. What is the impact of the SMP on enrolment? o Probe: inflated figures and how it has been addressed 11. What is the impact of SMP on attendance of pupils? 12. What is your view on the methods used to assess literacy level? o Probe for the methods used to assess literacy 13. What is the impact of SMP on attendance of teachers and the quality of teaching? 14. Were any of the MoE officials trained by WFP in 2014? o If yes, probe on number of people trained, type of trainings, frequency andperceived impact Success: 15. What would you say are some of the successes of the WFP SMP program? 16. What are the major reasons for the successes of the WFP SMP program?Challenges 17. What are some of the key challenges experienced in the implementation of theprogram? (Probe for specific responses?) o Probe for: Practice/implementation challenges o Probe for policy challenges 18. What were the coping measures for each challenge? 19. What recommendations would you make for improving the challenges in future?Sustainability 20.What is your understanding of the sustainability of the SMP? (Probe for what theymean by sustainability) 21. Probe what they consider the devolved governments could do to improve the SMP 22.What measures have been put in place by the national government towards the futureof SMP? (Probe for the next 5 years; ask for any documentation) 23.What measures have been put in place by the county government towards the futureof SMP? (Probe for the next 5 years; ask for any documentation) Recommendations 24.What recommendations would you give to improve any future implementation ofSMP? 6.1.1.1.15 Key Informant Guide for Other Respondents Section A of this guide is applicable to Parents, Head teachers, and partners. However, Section B varies based on additional questions that need to be addressed by specific respondents. Section A Icebreaker 1. Location: where are you currently based?Relevance of the Program 2. Generally what would you say are the key needs of children in arid and semi-aridcommunities? (Probe for specificity) 3. To the best of your knowledge, what is currently being done to address the needs thatyou have mentioned and who is implementing it? (Probe for each and every need mentioned) 4. What is your understanding of the WFP SMP?Effectiveness of the SMP 5. What would you say were some of the achievements of the School Meals Program(SMP)? 140 6. What contributed to achievements you just mentioned? (Probe for each achievement)Efficiency 7. In your opinion what would you say about the time taken in the implementation of theSMP program? (Probe for entire project, delivery of food, trainings for teachers etc.; base this on the respondents understanding of SMP). 8. How easy was it for WFP to implement the various activities? (Prompt based onunderstanding of WFP SMP) 9. If respondent mentions delays, what could have caused the delays in theimplementation? 10. What is your comment about WFP involving the community in decision making forthe SMP program? (Probe specific for respondent, community, teachers, parents, stakeholders) 11. What has been the coordination and cooperation of various partners who are doingsimilar projects to WFP? ( Base this on response to 3 above and understanding of WFP) Duplication of effort and management of duplication Sustainability 12. In your opinion who should be responsible for the School Meals Program? (Probe forhow) 13. Generally speaking, what measures have been put in place by the national governmenttowards the future of SMP? (Probe for the next 5 years; ask for any documentation) 14. Generally speaking, what measures have been put in place by the county governmenttowards the future of SMP? (Probe for the next 5 years; ask for any documentation) 15. Generally speaking, what measures have been put in place by schools towards thefuture of SMP? 16. Generally speaking, what measures have been put in place by the community towardsthe future of SMP? 17. What is your understanding of the sustainability of the SMP? (Probe for what theymean by sustainability) Challenges 18. What are some of the key challenges experienced in the implementation of theprogram? (Probe for specific responses?) 19. What were the coping measures for each challenge? 20.What recommendations would you make for improving the challenges in future?Recommendations 21. What recommendations would you give to improve any future implementation of SMP? Section B Additional Questions Specific to MoH (National Level) 1. What are the child health and nutrition policies updated, issued or under discussion in2015 and 2016? Probe: number and examples of the policies 2. Which MoH officials were trained by WFP in 2015? (probe on number of peopletrained, type of trainings, frequency and perceived impact) 3. What is the MoH collaboration/interaction with MoE in regards to the SMP? 141 Additional questions to feed the Children 1. Have you ever experienced pipeline breaks i.e. delays in food deliveries? o Probe: what is the average delay time? o Probe: What were the reasons for the delay? 2. What are the timelines for delivery by FTC once you receive food from WFP? 3. What is the condition of the storage facilities in Nairobi? o Probe for pest infestation and fumigation procedures 4. Does FTC play a role in the monitoring procedure? o If yes, what is the role of FTC? 5. What is the view on the SMP implementation and its performance? o Probe for the following o Ration size, o Enrolment versus attendance, o Actual feeding days o Hygienic standards and storage procedures 6. What are the key lessons learnt? Additional questions to Public Health Officers (PHOs) (MoH): (County Level) 1. Relationship with WFP and SMP implementation – discussion on frequency/type of interaction, confirmation of roles and responsibilities, lessons learnt and recommendations. 2. Training received through the support of WFP in 2014 – frequency, type, number of people trained. 3. Discussion of status of facilities (store room, kitchen, etc) and storing procedures in SMP schools. 4. Discussion of monitoring procedures for quality of the food delivered and main challenges, if any. 5. Lesson learnt, areas of improvements, suggestions and recommendations to improve SMP. 6. In your opinion, are there any areas where collaboration with WFP could be strengthened? 7. Any recommendations or suggestions you may have for the evaluators. Additional questions to Education officials (Quality Assurance Standard Officer, School feeding Officer, Inspectors, TSC): 1. Relationship with WFP and SMP implementation – discussion on frequency/type of interaction, confirmation of roles and responsibilities, lessons learnt and recommendations; 2. Perceived impact of SMP on pupils’ attendance and education; 3. Discussion on main education constraints in the county and how these are addressed by SMP; 4. In your opinion, what is the status of facilities in SMP schools (store room, kitchen, etc). 5. In your opinion, are there any areas where WFP could provide additional support? 6. What recommendations would you give to improve any future implementation of SMP? 142 Additional questions to Evidence action –Deworm the World Initiative (Complementary partner) 1. Confirmation of Evidence Actions’ main role and activities implemented within the education and child support sector, how their activities complement the SMP and contribute to its relevance. 2. Frequency and geographic coverage of Evidence Action activities linked to the SMP; main issues and recommendations. 3. What is the future fundraising trends and the effect on continuation of Evidence Action Programs 4. What recommendations would you give to improve any future implementation of SMP? Additional questions to UNICEF (UN Counterpart in Education) 1. What is the main role of UNICEF’s in the SMP? o Probe for complementarity and synergies with WFP, MoE and GoK o Probe for geographic coverage in relations to SMP 2. What activities is UNICEF implementing within the education and child support sector in Kenya. 3. What recommendations would you give to improve any future implementation of SMP? Additional questions to Partnership for Child Development (Complementary partner) 4. What is the main role of PCD in the SMP? o Probe for complementarity and synergies with WFP o Probe for geographic coverage in relations to SMP o Probe for challenges encountered 5. What activities is PCD implementing within the education and child support sector inKenya. 6. What recommendations would you give to improve any future implementation of SMP? Additional questions to SNV (complementary partner) 1. What is role of SNV in the project? 2. Whom do you work with in the project? o Probe for complementarity and synergies with WFP and other stakeholders 3. What is your view on the impact of the SMP? 4. What is your view on the handover strategy? 5. What is the SNV’s future funding plan for SMP? 6. What are the key lessons learnt? 143 Annex 10: List of schools selected for field visits County District School Name 1. NAIROBI MAKADARA KALOLENI PRIMARY STAREHE MATHARE NORTH PRIMARY SCHOOL WESTLANDS KANGEMI YOUTH CENTRE LANGATA MAKINA SELF HELP PRIMARY STAREHE NDURURUNO PRIMARY SCHOOL DAGORETTI KAGIRA PRIMARY STAREHE VALLEY BRIDGE PRIMARY LANGATA STARA RESCUE CENTRE LANGATA LAINI SABA (CHRIST THE KING) WESTLANDS WARUKU PRIMARY LANGATA MAKINA BAPTIST PRIMARY LANGATA LANGATA PRIMARY 2. GARISSA GARISSA NAJAH PRIMARY GARISSA AL NUR PRIMARY GARISSA SANKURI PRIMARY GARISSA HYUGA PRIMARY GARISSA ATHELEY PRIMARY GARISSA BALICH PRIMARY GARISSA ALFAROUQ PRIMARY GARISSA JARIROT PRIMARY GARISSA RAYA PRIMARY GARISSA LANGO PRIMARY GARISSA KAZUKO PRIMARY GARISSA JARIBU PRIMARY 3. BARINGO EAST POKOT SUKUT EAST POKOT TAMKAL EAST POKOT CHEMERIL EAST POKOT CHEWARA EAST POKOT NYAKWAL PRIMARY EAST POKOT NYAUNYAU EAST POKOT AKWICHATIS EAST POKOT LEMUYEK EAST POKOT CHEPTURU EAST POKOT KREZEE EAST POKOT LOSIKIRIAMOI EAST POKOT CHEPKARERAT PRIMARY 4. MARSABIT MARSABIT CENTRAL QACHACHA PRIMARY MARSABIT CENTRAL SEGEL PRIMARY MOYALE /SOLOLO MUKH GURA PRIMARY LAISAMIS LAISAMIS PRIMARY LAISAMIS MARTI PRIMARY 144 MARSABIT CENTRAL ILPUS PRIMARY MARSABIT CENTRAL LAKARTINYA PRIMARY LAISAMIS ARGE PRIMARY MOYALE QATE PRIMARY MOYALE /SOLOLO D/ELELE PRIMARY LAISAMIS GUURAM PRIMARY MOYALE /SOLOLO HAWECHA PRIMARY 5. ISIOLO MERTI KORBESA PRIMARY SCHOOL MERTI DOLOLODAKIYE PRIMARY SCHOOL MERTI GODA PRIMARY SCHOOL ISIOLO ST. KIZITO PRIMARY SCHOOL GARBATULLA KULAMAWE PRIMARY SCHOOL ISIOLO DAABA PRIMARY SCHOOL GARBATULLA WAKO WARIO PRIMARY SCHOOL ISIOLO AKILI PRIMARY SCHOOL ISIOLO PEPO LA TUMAINI PRIMARY SCHOOL GARBATULLA DABASITI PRIMARY SCHOOL ISIOLO TUALE PRIMARY SCHOOL GARBATULLA YAQ BARSADI PRIMARY SCHOOL 6. SAMBURU SAMBURU CENTRAL NKORIKA PRIMARY SAMBURU CENTRAL LPARTUK PRIMARY SAMBURU CENTRAL AMAIYA PRIMARY SAMBURU NORTH SUMURWAI PRIMARY SAMBURU NORTH SIMITI PRIMARY SAMBURU CENTRAL LPUSI PRIMARY SAMBURU CENTRAL SEKETET PRIMARY SAMBURU CENTRAL SUNONI PRIMARY SAMBURU NORTH SERERIT PRIMARY SAMBURU NORTH BARAGOI PRIMARY SAMBURU CENTRAL LOKUTO PRIMARY SAMBURU CENTRAL LEIRR PRIMARY 7. TURKANA TURKANA SOUTH LOUPWALA PRIMARY TURKANA SOUTH KAEKUNYUK PRIMARY TURKANA SOUTH NAMORTUNGA PRIMARY TURKANA CENTRAL NAPUU PRIMARY TURKANA CENTRAL NADOTO PRIMARY TURKANA NORTH AIC LOKICHOGGIO GIRLS PRIMARY TURKANA CENTRAL KOONO PRIMARY TURKANA CENTRAL NASEKON PRIMARY TURKANA NORTH TODONYANG PRIMARY TURKANA CENTRAL KALOKOL MIXED PRIMARY TURKANA NORTH ABUR PRIMARY TURKANA CENTRAL KALIMAPUS PRIMARY 8. TANA RIVER TANA DELTA BILISA PRIMARY 145 TANA RIVER ANOLE TANA RIVER TITILA TANA RIVER HOSINGO TANA RIVER MATAGALA TANA DELTA DIDAADE PRIMARY TANA RIVER DARIME TANA DELTA KINYADU PRIMARY TANA DELTA GOLBANTI PRIMARY TANA DELTA KILELENGWANI PRIMARY TANA DELTA MNAZINI PRIMARY TANA DELTA MAREMBO PRIMARY 9. WEST POKOT NORTH POKOT KALUKUNA PRIMARY NORTH POKOT WASAT NORTH POKOT NGOTUT NORTH POKOT TIGHOT NORTH POKOT KASITET NORTH POKOT KONYAO DORCAS NORTH POKOT MORWEBONG NORTH POKOT KOSIA NORTH POKOT NAPITIRO NORTH POKOT ROCK-VALLEY NORTH POKOT LOPUSHENIKOU NORTH POKOT KIWAWA 146 Annex 11: Detailed methodology General Approach The evaluation used a cross-sectional study design that adopted both quantitative and qualitative approaches to collection, analysis and reporting from both primary and secondary data sources. The methodology used offered comparability to baseline and midline assessment on most of the parameters and made some improvements aimed at improving reliability and segmented analysis context for the evaluation. The evaluation was implemented in four (4) distinct and overlapping methodological steps: Figure: Methodological Steps for the Evaluation M1: Desk Based Review of Surveys, Reports, and Data Desk based review of previous reports and data (baseline and midline), other program monitoring reports and data that were availed by WFP, data and reports from other relevant bodies such as MoE to establish some background information about SMP and education. In addition, analysis of data/reports from the UWEZO was undertaken to benchmark literacy and numeracy127. The secondary review of reports and data was broadened to cover reports/data that were relevant to the evaluation subject but not within the categories mentioned. The desk review was guided by the evaluation questions. The desk review was conducted alongside the M2 and M3 components. M2: School Based Assessment School-based assessment was conducted using an interviewer administered face-to￾face survey with pupils, parents, teachers, cooks and storekeepers. The interviewers used a semi- structured interview approach that allowed for collection of both quantitative and some qualitative data. Data was collected using electronic data collection approach. Interviewers ensured that specific focus on gender was incorporated at all times during the questioning to deduce any gender related issues. M3 and M4: Qualitative Data Collection Approach The M3 and M4 segments used Key Informant Interviews (KIIs) which were conducted by highly experienced moderators. The approach allowed for content￾focused collection of information from the Key Informants based on the informant’s role in the implementation ofthe 2014-2016 SMP. The moderators used KII guides that were linked to evaluation questions and structured based on the category of informants. With consent from the respondents, the interviews were audio-recorded and then later transcribed for analysis using N-Vivo. The survey achieved a sample of 158 Key Informant Interviews and 9 Complete Case Studies Sampling Approach Study Population The study population for the M2 component of the evaluation was pupils, parents, head teachers, patrons and cooks from the implementation schools of the WFP SMP. Local administrators, education officers, school management committee (SMC) M3: County Based M4: National Based M1: Desk Based Review of Surveys, Reports and Data M2: School Based Assessment of Pupils, Parents, and Teachers Implementers an Assessment of d Implementers an Assessment of d Stakeholders Stakeholders 147 members also referred as Board of Management (BOM) and other stakeholders within the catchment area of the M2 sampled schools was considered for M3. Geographically this evaluation considered nine of the 11-implementation counties (excluding Wajir and Mandera) due to security reasons. Inclusion and Exclusion Criteria During the evaluation, only the project targeted counties and schools were included in this evaluation. For the specific respondents, the inclusion criteria considered; respondents who provided verbal informed consent for participation in this evaluation. While for the specific respondents, the exclusion criteria entailed; • Respondent’s from counties where the government authority denied consent to have the county included in the study • Respondents who had been exposed to the baseline and midline survey (this was only applicable for the pupils and parents) • Respondents with diminished autonomy Sample Size for Pupils, Parents and Schools under M2 The total pupils used to assess on pupils related indicators was the overall determinant of how many parents, schools, and pupils were included in the sample. We proposed a minimum sample size of 900 pupils (450 girls and 450 boys) and 900 parents/guardians. The sample size of 900 was based on the following assumptions; a response rate of 95 %, a design effect of 1.5, a power of 70 %, detection of 7 % difference in desired behaviour indicator from midline, and an average 50 % value of indicator at midline. We however managed to achieve a sample size of 552 boys and 557 girls. The split of the sample into boys and girls was to allow for gender￾based comparison of the pupils related indicators being considered in this evaluation. Ninety schools were sampled with each school providing 10 pupils. The end line allowed us a power to offer any additional insights based on segmentation of the sample during analysis. Given the bias towards this approach, we calculated school based weights that adjusted for the school’s distribution per county based on the list of schools provided by WFP. These weights were applied while calculating totals for various indicators. Sample Size for Head Teachers, Program Patrons and Cooks The evaluation included all the head teachers and school feeding teachers/heads from the 90schools selected above. We interviewed 90 heads and 85 store keepers and 87 cooks. Sample Size Determination for Qualitative Component (M3, M4) Considering that, the qualitative component is designed to answer additional componentsfor the evaluation, and give explanatory elements of the study. For the M3 methodology, the evaluation considered 175 key informant interviews. Specifically, each county had local administrators, education officers, school management committee members and Program Partners included in this component. 127UWEZO is the Government of Kenya’s recognized source of numeracy and literacy data. 148 Sampling the Counties and Schools The primary sampling unit was the schools, since this determined the subsequent respondents to be sampled under proximity of the school. The evaluation used a multistage sampling approach to be able to achieve a sample of the desired schools: All the 10 intervention counties were purposively selected except Wajir and Mandera due to security reasons. Within each county, the schools were clustered based on the sub counties. (WFP provided a list of all schools that benefited from the intervention). The number of schools included per sub-county was determined through a direct division of the 10 schools desired by the number of sub-counties. To be able to avoid thinning out of the sample, we further clustered sub-counties to ensure that at any time, we have at least three schools within a sub county/ cluster i.e. a county had a maximum of three sub-counties or clusters. A random sampling approach was used to select the schools. This was mutually exclusive for each sub-county/cluster. From the selected schools, a catchment radius of 5km and 10 km was assumed for urban and rural respectively (except Nairobi, which had a radius of 2 km). Sampling the Pupils and Parents Pupils at the school were stratified into eight classes according to the education system in Kenya. The pupils interviewed were selected using simple random sampling using class registers. A random sampling table was used to identify two pupils (1 girl and 1 boy) to sample in each class in the schools. It should be noted that as opposed to the midline, each school had a different randomnumber for selection of a pupil in each class i.e. the sampling within each school wasmutually exclusive. Midline assessment used same index number for the same classes acrossall schools. To address the gender sensitivity in data collection, the evaluation ensured to thebest extent possible the boys and girls are interviewed by data collectors of the same gender. The evaluation team contacted the school administration (at least 5 days in advance) torequest the randomly selected pupils to invite their parents on specific dates for theinterview while giving very limited information about the study to avoid bias. In case the pupils or parents were not willing to participate in the survey, they were not be replaced since the selection of 2 pupils per class per school allowed for 16 pupils while we only needed a minimum number of 10 pupils. This is how the evaluation ended up with 1119 pupils and 1119 parents. Sampling Respondents for M3 and M4 Qualitative Interviews Only respondents drawn from the nine selected counties were included in the survey. Where a county has more than one potential respondent per category of responded e.g. local administration, the evaluation team gave priority to the respondents drawn from the proximity of the randomly selected school. However, where the respondents were from within the County an attempt were made to interview the primary respondents using face-to-face approach unless the respondent was not available or had been snowballed for some minor confirmation then the interviewer booked a telephonic interview with the respondents. Similarly, the national based respondents were interviewed using face-to-face approach and telephonic interviews used only in extreme scenarios. A detailed list of the specific respondents is presented as Annex of this document. Data Management Data Collection Tools The data collection tools were aligned to the methodology above, with the evaluation 149 having broadly had two categories of data collection tools. The quantitative questionnaires were semi-structured in nature and were electronically scripted using TNS’ N-Field Mobile Data Collection Software. While the qualitative ones used key informant guides. Data Processing Quantitative data was collected using tablets in the field. These allowed the data to be transmitted immediately to a centralized cloud database that is owned by TNS. The data was checked for consistency and quality controlled. Use of electronic data eliminated some paper based data cleaning elements like wrong skip application and thus minimised the data cleaning while on field and allowed for day to day data cleaning from a central place; this increased comparative assessment of data from the different teams and individuals while cleaning. The qualitative data was audio recorded, transcribed, and later analyzed. Data Cleaning The Data Management team conducted rigorous data checking and cleaning on the data using an edit specification and cleaning rules. Checking involved checking filter, subgroups and different breakdowns of various cells checked on live data. Entries with mistakes/routing problems were identified and feedback given to Field Department. 15 % of the primary collected data was quality controlled. The main objective of QC checks as follows; • To ensure the best quality and professional ethics of data collection throughout all project phases. • Ensure project execution is in conformity with the WFP’s specifications, methodology and other briefing instructions are met. • Ensure collaboration and uniformity of all fieldwork operation in all manners is achieved. Data Analysis Approach Quantitative data was analyzed using IBM SPSS. Indicators measured at a single point in time were analyzed using univariate analysis to give frequencies, means and/or proportions as appropriate. In determining, the significance of changes in indicators between baseline, midline, and end line appropriate test for significance was used depending on the statistical nature of the indicator. A further statistical analysis was conducted to assess for possible causal relationships. Qualitative data was analyzed using the N-Vivo application using Miles and Hubberman (1994) approach towards qualitative analysis. The resulting product was a detailed thematic ordered matrix the themes being the main evaluation criteria with the sub-themes being the specific evaluation questions and variants of issues coming from them. The qualitative findings are presented as general summaries and where necessary reported as verbatim quotes. Given that the baseline and midline did not have as much gender considerations, comparison to the two was done on aggregated gender level. Limitations and Risks The study was conducted in the selected counties, schools and households around the program schools. The survey instrument was developed in English and was translated into the local language during the interviews, which could have potentially been misinterpreted. To reduce this challenge a pre-test was conducted in nearby non￾selected schools to look at the context relevance, while pilots study done within the selected counties to assess for methodological relevance and flaws. This was not wide spread as the teams of data collectors understood local languages and culture. The prior sampling approach applied previous evaluations for the pupils and parents was subject to bias however, during the end line evaluation the interviewers verified school register to ascertain that the selected respondent was actually the one 150 contacted by the school administration. In addition, the evaluation team ensured that they limit the information provided to the school on recruitment. During the evaluation, there was need to collect some recall information since implementation (2014) and given the recall bias, this was problematic. The evaluation team therefore, limited the recall to a shorter period dependent on the information being requested. Research ethics TNS RMS is a member of various research organizations and completely complies with the ethical guidelines on the conduct of research that this entails, and other laws of the countryin which we are operating. The TNS research team has received training in Human Subjects Protection (HSP), and this is renewed annually. As an active member of ESOMAR, the world organization for the promotion of best practice with respect to market and social research and advocating self-regulation, TNS fully adheres to the International Code of Market and Social Research. The details of the ESOMAR guidelines for interviewing children and minors are presented as Appendix D. TNS will also adhere to the Plan’s Child protection Policy and Code of Conduct Guiding principles in the Belmont Report that TNS is fully cognizant of and applies when conducting social research includes respects for persons, beneficence and justice (fairness in the selection of potential respondents). At the individual level – TNS seeks verbal and/ or written consent from all respondents to any study or assessment. The voluntary nature of participation in the research is made clear in the recruitment process. TNS does not subject respondents to any form of coercion, any undue inducement, or intimidation. The respondents will be enlightened on the purpose and procedure of the survey before the interview commences. The end-benefits of participating in the survey will also be clearly explained to the respondents. Dissemination of Findings TNS proposes a delta (participatory) approach to dissemination of the findings of this evaluation; this is a participatory approach where the participants interact with some of the raw findings and make deductions around key evaluation questions. As opposed to most disseminations, this give the chance to the participant to better appreciate the findings and conclusions drawn from the findings. The delta approach allows the evaluators to structurally put the feedback from the workshop into the main report of the evaluation. A delta approach provides better ways of ensuring that where multiple stakeholders wereinvolved, the team is able to arrive at more or less similar understanding and consensus of the findings This approach utilizes the use of a participatory workshop, which follows the following distinct steps: • Participants are grouped into various groups and then given analyzed findings from the report • The groups then come up with key issues based on the findings and recommendations from the data using groupings for similar findings. • The groups present the key issues and recommendations • TNS then presents its findings, analysis, and recommendations • The larger group then discusses and adopts the findings and recommendations It is envisioned that WFP will be responsible for identifying and inviting the participants for the one day dissemination workshop while TNS will only offer technical facilitation and pay for the actual workshop venue of the dissemination workshop. Annex 12: Secondary data on education outcomes in the SMP counties • UWEZO enrolment rankings for the targeted SMP counties (2013) 151 County Enrolment (%) Poverty (%) Rank Nationally (out of 47 Counties) Nairobi 93.5 4.5 1 Baringo 90.0 78.9 34 Tana River 83.9 81.7 37 Isiolo 83.3 85.8 40 West Pokot 87.3 87.1 41 Wajir 85.2 79.0 42 Garissa 78.8 59.8 43 Marsabit 75.6 84.1 44 Turkana 86.6 93.3 45 Samburu 80.8 90.1 46 Mandera 84.1` 82.6 47 Source: UWEZO 2015 report 159 50 45.2 46.3 45 40 36.3 35 34.4 33.9 30.6 30 28.2 25 23.7 20.5 20 17 15 10 5 0 Nati • Mean pass rate, combined literacy and numeracy test, SMP targeted counties (2015) onal averNagaeirobi North Central Baringo Marsabit Moyale GarissaTana Rive Central West North East r Pokot East Samburu East Central South North Turkana West South Wajir East Central West Mandera Source: UWEZO, 2015 160 Annex 13: Household survey findings re commuity needs Source: EoPE key informant interviews 2016 Frequency Hunger Food Insecurity Poor Health Care Water Shortage Unemployment Gender Exclusion Malnutrion Insecurity Illiteracy Distance to Schools -ve Intensity +ve 161 Annex 14: Other development actors working in the SMP locations Development actor Activity Partner location Action Against Hunger/United States (AAH/USA) Humanitarian Coordination and Information Management, Humanitarian Studies, Analysis, or Applications National Catholic Diocese Food and nutrition, water and sanitation, public health and human livelihoods National FAO Agriculture and food security National Feed Kenya (feed the children) Food and nutrition. Trainings on health Nairobi, Rift valley and Western Netherlands Development Organization (SNV) HGSMP Kenya ASALs Partnership for Child Development (PCD) HGSMP, health and nutrition, human livelihoods National UNESCO Access to quality education National UNICEF Nutrition National UNICEF Nutrition National Welthungerhilfe (German Agro-Action) Agriculture Kenya ASALs World Vision Food and nutrition, water and sanitation, public health National 162 100% 80% 60% 40% 29% 20% 10% 14% 15% 18% 16% 16% 6% 10% 13% 2% 9% 0% Annex 15: Additional UWEZO education data (2015) • UWEZO literacy and numeracy figures, SMP counties (2015) Percentage of illiterate children Source: UWEZO 2015 Percentage of innumerate children Source UWEZO 2015 100% 80% 60% 40% 28% 20% 20% 20% 10% 16% 15% 15% 17% 17% 21% 11% 3% 0% 163 Annex 16: Additional analysis re attentiveness in class • Causes of inattentiveness in Schools Lack of food at school accounts for 70.7% of the variation in student inattentiveness (r2 =0.707, p < 0.01) Model Summary Model R R Square Adjusted R Square Std. Error of the Estimate 1 .841a .707 .666 3.836 a. Predictors: (Constant), Ever reported that food was not provided at school Coefficients Unstandardized Coefficients Standardized Coefficients Beta t Sig. Collinearity Statistics Model B Std. Error Tolerance VIF 1 (Constant) 1.199 3.971 .302 .772 Ever reported that food was not provided at school 43.220 10.505 .841 4.114 .004 1.000 1.000 a. Dependent Variable: Inattentiveness 164 Annex 17: Activity indicator – Number of boys and girls receiving school meals (2014-2016) Planne d Actua l % Achievement Male Femal e Total Male Femal e Total Mal e Femal e Tota l Pre￾primary and primary children receiving school meals. 201 4 440,000 375,00 0 815,000 417,86 5 378,251 796,116 95.0 100.9 97.7 Pre￾primary and primary children receiving school meals. 201 5 326,150 266,85 0 593,000 328,21 6 257,884 586,100 100. 6 96.6 98.8 Primary school children receiving school meals 201 6 243,000 207,00 0 450,000 244,32 1 186,088 430,409 100. 5 89.9 95.6 TOTAL 1,009,1 50 848,85 0 1,858,0 00 990,4 02 822,22 3 1,812,6 25 98.7 96 97.6 165 Annex 18: Additional analysis re parental reporting of meals before and after school • Percent of students in target schools who regularly consume a meal before and after school (as reported by parents) County Before school After school Meal consumed every day of thelast 5 days Meal consumed at least 3 days out oflast 5 Meal consumed every day of thelast 5 days Meal consumed at least 3 days out of last 5 Samburu 81.1 95.5 84.1 96.2 Isiolo 72.7 83.9 65.7 86.0 Nairobi 63.0 75.9 86.1 99.1 Garissa 59.2 83.5 80.6 90.3 Tana River 49.4 75.0 75.0 91.9 West Pokot 44.2 67.4 54.7 85.3 Baringo 25.7 61.4 34.3 73.6 Turkana 24.4 31.1 83.2 87.4 Marsabit 29.8 69.4 35.5 71.0 Average 50.1 71.8 66.2 86.6 166 Annex 19: Activity indicator – Total quantity of commodities provided to SMP from all sources Commodit ies distribute d 2014 2015 2016 TOTAL Target Achieve d Targe t Achieve d Targe t Achieve d Targe t Achieved Wheat 5,725.174 5,394.45 303.35 11,422.97 Oil 531.786 332.082 109.179 973.05 Pulses 4,699.67 2 2,201.6 1046.52 7 7,947.80 Maize 11,539.94 3,121.3 3,366.11 8 18,027.36 Rice 308.966 0 0 308.97 Corn soya blend 1,261.02 5 503.35 0 1,764.38 Micro￾nutrient powder 0.014 0 5.528 5.54 Iodized salt 290 181 73.95 544.95 Total quantity of commodities (tons) provided 6,364 24,356. 58 8,320 11,733.7 8 3,083 4,904. 65 17,76 7 40,995.0 1 % Achieved 382 % 141% 159% 230 % 167 Annex 20: Parental reporting of whether there were times that meals were not provided at school, by county County Yes No I don’t know Total Baringo County 61.4% 33.6% 5.0% 100% Tana River County 43.8% 48.8% 7.5% 100% Isiolo County 43.4% 49.7% 7.0% 100% Marsabit County 37.1% 41.9% 21.0% 100% Garissa County 35.9% 48.5% 15.5% 100% Samburu County 29.5% 69.7% 0.8% 100% West Pokot County 27.4% 71.6% 1.1% 100% Turkana County 26.9% 63.9% 9.2% 100% Nairobi County 16.7% 74.1% 9.3% 100% Total 37.0% 54.6% 8.4% 100% • Reported reduction in food portions on week days compared to weekends County Food proportion reduction 25% 50% 75% Baringo County 50.0 47.1 2.9 Garissa County 67.7 25.8 6.5 Isiolo County 78.0 19.5 2.4 Marsabit County 18.9 56.8 24.3 Nairobi County 46.8 46.8 6.4 Samburu County 68.4 23.7 7.9 Tana River County 78.9 21.1 0.0 Turkana County 68.4 31.6 0.0 West Pokot County 75.0 25.0 0.0 Total 57.5% 36.3% 6.2% 168 Annex 21: Additional analysis re school attendance (students) • Causes of Absenteeism in Schools Responses from household interview County Days Absent School Closure Working on farm or livestock tending Looking after siblings or domestic work Lack of food Insecurity Sickness Ceremonies and family events Skipping school Baringo 2664 864 1349 1340 1489 2158 884 1366 1053 Tana River 689 82 52 14 33 61 605 67 34 Isiolo 493 958 0 2 11 68 411 15 51 West Pokot 385 2 10 13 23 6 346 0 0 Samburu 257 7 25 23 15 9 225 9 2 Marsabit 192 166 9 10 32 12 218 4 2 Turkana 192 32 1 0 12 31 261 0 82 Garissa 172 118 16 8 48 36 246 10 11 Nairobi 146 1803 2 2 5 4 101 32 2 Family events and sickness account for 99.7% of absenteeism (r2 = 0.997, p < 0.001) Model Summary Model R R Square Adjusted R Square Std. Error of the Estimate 1 .980a .961 .955 169.455 2 .998b .997 .996 53.434 a. Predictors: (Constant), Family Events b. Predictors: (Constant), Family events, Sickness Coefficients Model Unstandardized Coefficients Standardized Coefficients t Sig. Collinearity Statistics B Std. Error Beta Toleranc e VIF 1 (Constant) 284.544 60.701 4.688 .002 Family Events 1.749 .133 .980 13.14 2 .000 1.000 1.000 2 (Constant) -37.302 44.439 -.839 .433 Family Events 1.266 .073 .710 17.26 7 .000 .327 3.054 Sickness 1.099 .137 .330 8.025 .000 .327 3.054 169 • Student Profiles Students that recorded high absenteeism were also high on the following 9 attributes: Attribute Coefficient p value 1) Absent due to ceremonies and family events .980** 0.000 2) Absent Working on farm or livestock tending .979** 0.000 3) Absent due to Insecurity .979** 0.000 4) Absent looking after siblings or domestic work .975** 0.000 5) Absent for lack of food .975** 0.000 6) Absent for skipping school .975** 0.000 7) Absent due to sickness .912** 0.001 8) Ever reported that food was not provided at school .825** 0.006 9) Inattentiveness .804** 0.009 170 Annex 22: Additional analysis re school enrolment • Percentage increase in boys and girls enrolled in school as a result of USDA assistance Statisti c Std. Error Boys Change Endline Mean 7.5259 6.01353 95% Confidence Interval for Mean Lower Bound -4.5115 Upper Bound 19.5633 5% Trimmed Mean 3.8077 Median 1.0000 Variance 2133.593 Std. Deviation 46.1908 3 Minimum -80.29 Maximum 283.33 Range 363.63 Interquartile Range 24.80 Skewness 3.714 .311 Kurtosis 21.889 .613 Statistic Std. Erro r Girls Change Endline Mean 8.9437 4.97713 95% Confidence Interval for Mean Lower Bound -1.0228 Upper Bound 18.9102 5% Trimmed Mean 6.0798 Median 3.1578 Variance 1436.765 Std. Deviation 37.90468 Minimum -65.08 Maximum 145.45 Range 210.53 Interquartile Range 21.56 Skewness 1.751 .314 Kurtosis 5.259 .618 171 Hypothesis Test Summary Null Hypothesis Test Sig. Decision 1 The median of Boys Change EoP equals 4.00. One-Sample Wilcoxon Signed Rank Test 0.850 Retain the null hypothesis. 2 The median of Girls Change EoP equals 4.00. One-Sample Wilcoxon Signed Rank Test 0.323 Retain the null hypothesis. Asymptotic significances are displayed. The significance level is .05. Primary school net enrolment rates – Kenya Primary School 2007 2008 2009 2010 2011 2012 2013 2014 2015 Total 112.63 112.3 113.27 ... 116.55 116.13 ... 111.4 108.97 Female 111.85 111.17 112.04 ... 116.89 116.47 ... 111.58 108.68 Male 113.4 113.42 114.48 ... 116.22 115.79 ... 111.22 109.26 Total 86.33 82.08 82.78 ... ... 84.87 ... ... ... Female 86.39 82.65 83.31 ... ... 86.6 ... ... ... Male 86.27 81.5 82.26 ... ... 83.17 ... ... ... Source: UNESCO (http://uis.unesco.org/country/KE) Notes 1). One-Sample Wilcoxon Signed Rank Test indicate that the enrolment rates for boys was within the set threshold of 4-percentage point increase between baseline and end of project(p = 0.850 - the observed median percentage was NOT significantly different from the projected median percent change). 2). One-Sample Wilcoxon Signed Rank Test indicate that the enrolment rates for girls was within the set threshold of 4-percentage point increase between baseline and end of project(p = 0.323 - the observed median percentage was NOT significantly different from the projected median percent change) 172 Annex 23: Percentage of households able to name benefits of education, by county County 3 Benefits Less than 3 benefits Garissa County 99.0 1.0 Baringo County 92.1 7.9 Samburu County 81.8 18.2 Marsabit County 64.5 35.5 Turkana County 62.2 37.8 Isiolo County 59.4 40.6 Tana River County 55.6 44.4 West Pokot County 51.6 48.4 Nairobi County 42.6 57.4 Total 67.8% 32.2% 173 Annex 24: Additional analysis re WFP provided training on food preparation and storage practices • Activity indicators - Training: Food preparation and storage practices 2014 2015 2016 TOTAL Indicator Target Achieved Target Achieved Target Achieved Target Achieved Number of trainings provided in food preparation and storage practices 11 54 6 17 5 15 22 86 Number of teachers trained in food preparation and storage practices 600 3,455 1,000 1,753 1,000 1,870 2,600 7,078 • Percentage of schools in target counties that store food off the ground County Yes Isiolo 100.0 Garissa 90.9 Baringo 90.0 Samburu 90.0 Turkana 80.0 Marsabit 80.0 Tana River 60.0 Nairobi 60.0 West Pokot 50.0 Total 76.5% 174 • Percentage of schools in target counties with dedicated food store rooms County Yes Nairobi 100.0 Garissa 90.9 Tana River 90.0 Turkana 90.0 Samburu 90.0 West Pokot 75.0 Isiolo 75.0 Baringo 50.0 Marsabit 50.0 Total 78.8% • Photos of inappropriate food storage in visited schools Illustration of inadequate storage within a classroom (a) and a proper storage (b) 175 Annex 25: Additional analysis re food preparation and hygiene • Percentage of schools with a dedicated kitchen, by county County Yes Isiolo 100.0 Nairobi 100.0 Tana River 88.9 Turkana 88.9 Samburu 88.9 Marsabit 85.7 Baringo 83.3 West Pokot 60.0 Garissa 50.0 Total 82.3% • Percentage of schools using energy saving stoves, by county County Yes Isiolo 100.0 Nairobi 100.0 Samburu 70.0 Marsabit 60.0 Garissa 50.0 Turkana 50.0 Tana River 45.5 Baringo 0.0 West Pokot 0.0 Total 52.9% • Photos of substandard kitchen facilities in visited schools 176 Annex 26: Additional data re sanitation (toilet) facilities in surveyed schools • Does the school have toilets for pupils? County Yes Baringo 100% Tana River 100% Garissa 100% Turkana 100% West Pokot 100% Samburu 100% Isiolo 100% Marsabit 100% Nairobi 100% Total 100% • Do girls have separate toilets from boys? County Yes Marsabit 100.0 Nairobi 100.0 Isiolo 90.0 Tana River 88.9 Turkana 88.9 Samburu 88.9 Baringo 83.3 West Pokot 80.0 Garissa 70.0 Total 87.3% • Toilet Type (Multiple Responses allowed) Toilet Type N Percent of Cases Pit latrines with a slab 57 72.2% Ventilated improved pit latrines 39 49.4% Piped sewer, septic, pit latrine 16 20.3% Pit latrines without slab/open pit 11 13.9% Flush or pour/flush toilets without a sewer connection 2 2.5% No facilities, open defecation 2 2.5% 177 Annex 27: Activity indicator – Capacity building: local, regional and national level 2014 2015 2016 TOTAL Target Achieved Targe t Achieved Target Achieved Target Achieved Number of Home Grown School Feeding Program manuals distributed 3,500 1,770 0 824 3,000 5,770 6,500 8,364 Number of MOE Officers benefiting from distribution 3,500 6,000 3,000 4,232 2,000 5,770 8,500 16,002 178 Annex 28: Additional data re reported security threats by county • School- and commuter-related threats by county County School Related Commuter Related Total f % f % f % Marsabit County 0 0% 239 100% 239 32% Tana River County 39 19% 166 81% 205 27% Isiolo County 62 47% 71 53% 133 18% Baringo County 24 35% 44 65% 68 9% Samburu County 19 33% 39 67% 58 8% Garissa County 15 58% 11 42% 26 3% Turkana County 6 55% 5 45% 11 1% Nairobi County 2 22% 7 78% 9 1% Total 167 22% 582 78% 749 100% • Reported security threats, by county County Number of households mentioning threats Bullying Animal Total attacks Robbery Rape Sexual harassment Drug abuse Marsabit County 0 228 0 1 2 8 239 Tana River County 39 157 8 1 0 0 205 Isiolo County 62 1 70 0 0 0 133 Baringo County 24 36 8 0 0 0 68 Samburu County 19 39 0 0 0 0 58 Garissa County 15 2 8 0 1 0 26 Turkana County 6 4 1 0 0 0 11 Nairobi County 2 0 6 0 1 0 9 Total 167 467 101 2 4 8 749 179 Annex 29: Additional data re locations of households reporting knowledge of the feedback hotline County f % Isiolo County 4 29% Turkana County 4 29% Marsabit County 2 14% Tana River County 2 14% Nairobi County 1 7% Samburu County 1 7% Baringo County 0 0% Garissa County 0 0% West Pokot County 0 0% Total 14 100% 180 Annex 30: Updated performance monitoring plan Theme Outcome Performance Indicator Data Source Methodology Details/Analysis Plan Baseline Value Mid-line Value EoP Evaluation Increased performan ce MGD SO 1 – Improved literacyof school age children 1. Proportion of students who by theend of two grades ofprimary schooling, demonstrate that they can read and understand the meaning of grade level text. UWEZO Annual Learning Report 2014 Endline Survey: School Questionnaire, Section 7:School & Student Performance MOE Secondary data Service Delivery Indicators (SDI) Survey2012 (World Bank) Baseline used UWEZO report 2012. For the mid-line evaluation, the UWEZO report 2014 was used. We propose use of UWEZO 2015 report. Considering that literacy test was discouraged by MOE during the midline, we propose to use similar approach where proxy indicators areused: - Average KCPE Score (2014) versus(2015) - Average mark for Class 2 pupils(out of 500) - Average mark for Class 8 pupils(out of 500) Average Performance rate of pupilsby gender at County level obtainedfrom MOE. Average - 48.8 Between 33.8% (Samburu) and 62.6% (Baringo) Uwezo: 64% average (Samburu East 30.6, Baringo 63.7%) Average KCPE Score(2014): 188.21128 Average mark for Class2 pupils (out of 500): 274.73129 Average mark for Class8 pupils (out of 500): 220.56130 Average KCPE at national level for 2013: 250.05 (253.23 for boys and 246.88 for girls)(MOE 2013) Average literacy (Kiswahili) – 46.1 Average numeracy andliteracy for class 3 children: 32% Who can read letters - between 71% (Turkana) and 98% (Nairobi) Who can read numbers - between 72% (Turkana) and 97%(Nairobi) 2. Number total individuals benefiting directly from USDA￾funded interventions School monitoring data(SMP 8) WFP Reports Secondary data review of WFPrecords 2013- Girls: 341,673 Boys: 425,435 Total: 767,108 753,139 763,490 3. Number of total individuals benefiting indirectlyfrom USDA-funded interventions WFP reports Endline Survey: HH/Parent Questionnaire,Q104. Use method applied during the baseline and midline survey. Interviews with parents will determine the average number of children per HH going to school. HHaverage size is will be computed say X. Number of HHs= direct beneficiaries (Indicator No. 2) /the number of children per HH going to 536,758 Average number of children per HH goingto school: 2.84. Indirect beneficiaries =838,000 527,984 181 128 The average KCPE 2014 score per county was: Garissa: 163.25, Marsabit: 200, Nairobi: 91.75, Tana River: 159.25, Turkana: 257.28, West Pokot: 264.33. 129 Garissa had the worst performance among the sampled counties with the average mark for Class 2 pupils being 203.25. 130 From lowest to highest, the sampled counties’ performance for average Class 8 pupils’ mark was: Tana River: 163.75, Nairobi: 184.28, Marsabit: 200.5, Garissa: 225, Turkana: 257.37 and West Pokot: 262.83. 182 school (obtained from HHquestionnaire Q104). Number of indirect beneficiaries =number of HHs * (X- number of children per HH going to school). MGD 1.1.1 – More consistent teacher attendance 4. Percent of teachers in target schools who attend school and teach at least 90 percent of scheduledschool days per year. Endline Survey: School Questionnaire, Section 4Teacher￾Head Teacher (Q410) and Section 5: Observation School Survey (Q504-Q506) SDI Survey 2012 – WorldBank Data will be collected directly from the Head teacher and will be triangulated with data from the SDI 2012 survey. It will also be comparedwith teacher attendance during the field survey. 51% Attendance of at least 90% of scheduled days:74.8% (323 teachers)131General attendance of teachers: 88.79%132 Average teacher attendance during the 3day surveys: 81.88%133 Average pupil teacher ratio from MOE data forKenya: 34.5134 89.7% 5. Number of MOE officers trained in promoting consistent teacher attendance WFP reports The baseline data reported zero meaning that no teachers’ trainings onpromoting consistent teacher attendance were conducted. For the mid-line value, secondarydata from WFP will be analysed. 0 200 Year 2014: 200 2015: 295 2016: 89 Total: 584 6. Number of trainings in promoting teacherattendance conducted for MOEofficers WFP reports 0 17135 Year 2014: 12 2015: 15 2016: 15 Total: 42 MGD 1.2 – Improved Attentiven ess 7. Percent of studentsin classrooms identified as inattentive by theirteachers Endline Survey: TeachersQuestionnaire The data will be collected using the teachers’ perception. Teachers from all classed will be interviewed. The average percentage will be calculatedfor this indicator. 20% 20.05%136 Main reason for inattentiveness: hunger (43.7%) 15% Main reason for inattentiveness: (hunger38%). Participati onin Education MGD 1.3 – Improved student 8. Percent of students (girls/boys) Endline Survey: School Questionnaire, Section 7. School data on attendance will be captured for the ten sampled pupils in 85.6% 88% (attendance vsenrolment as for 86.4% 131Nairobi and Turkana counties achieved exceptionally high values for this indicator, while Garissa’s performance was quite low. Breakdown by county: Garissa: 42.2%, Marsabit: 87.03%, Nairobi: 183 97.7%, Tana River: 68.3%, Turkana: 98.6%, West Pokot: 76.6%. 132 The breakdown by county is as follows: Garissa: 56.95%, Marsabit: 88.27%, Nairobi: 96.88%, Tana River: 81.7%, Turkana: 97.08%, West Pokot: 92.57%. 133 Average teacher attendance during the survey, by county: Garissa: 97.91%, Marsabit: 69.66%, Nairobi: 97.19%, Tana River: 68.12%, Turkana: 81.66%, West Pokot: 88.15%. 134 Average pupil teacher ratios in the sampled counties for 2014 (MOE): Garissa 46.5, Marsabit 37.4, Nairobi 36.2, Tana River 36.6, Turkana 71.7, West Pokot 37.2. 135In 2014, the SMP Management Training was carried out in East Pokot, Garissa, Marsabit, Tana River, West Pokot and Samburu Counties. 136 Percent of inattentive students by county: Garissa: 11.95%, Marsabit: 25.26%, Nairobi: 15.58%, Tana River: 19.87%, Turkana: 20.73%, West Pokot: 26.37%. 184 attendance regularly attending supported schools School & Student Performance each school. baseline) 72.5% regularly attending (Girls: 73.02%137 Boys: 72.28%138) 9. Percent of students in target schools who start grade one and complete the last grade of primary school Endline Survey: School Questionnaire, Section 6:Student record (Q614- Q621) SDI Survey 2012 – WorldBank At school level the calculation will bebased on school records and interviews with head teachers following backwards the co-hort of pupils who graduated in the year 2014. In the count, students who initially enrolled in the sampled school but completed the Primary Education with another school shouldbe considered. Average % will be applied. This will be triangulated with datafrom the SDI Report. 2013: Girls 72.8% Boys 78.7% Total 76.2% 56.44% 54.4% MGD 1.3.4 – Increased student enrolment 10. Percent increase in girls enrolled in schools Endline Survey: School Questionnaire, Section 6:Student record (Q601- Q613) MOE data at county level At school level, the trend will be calculated from the enrolment figurefor the last 4 years (2012- 2015) fromheadmaster records. This data will be triangulated withMOE data at county level. 3% 7.7%139 3.8% 11. Percent increase in boys enrolled in schools 3% 9.2%140 6.1% 12. Number of events, radio spots, and campaigns held WFP reports Secondary data from WFP. 0 20 radio spots 119 13. Number of community members benefitingfrom events, radio spots, and campaigns held WFP reports 0 65,204 134,426 MGD 1.3.5 – Increase d communi ty 14. Percent of parents in target communitieswho can name at Endline Survey: HH/parent Questionnaire,Section 3: School related Parents will be asked to name the benefits of education they are awareof. Parents should mention at least 3 66% 87.8%141 68% 137 The breakdown by county was: Garissa: 80.95%, Marsabit: 100%, Nairobi: 21.42%, Tana River: 83.78%, Turkana: 100%, West Pokot: 41.17%. 138 The breakdown by county was: Garissa: 91.66%, Marsabit: 85.71%, Nairobi: 29.41%, Tana River: 85.36%, Turkana: 81.48%, West Pokot: 41.37%. 185 139 This figure is only for pupils in primary school. The corresponding value for the increase in enrolment from 2012 to 2013 is 17.3% and for 2014 to 2015, 0.9%. The percent increase in enrolment for pre-primary girls from 2013 to 2014 was 2.7%. 140 This figure is only for pupils in primary school. The corresponding value for the increase in enrolment from 2012 to 2013 is 13.4% and for 2014 to 2015, 14.9%. The percent increase in enrolment for pre-primary boys from 2013 to 2014 was 7.4%. 141 Data from Garissa were not included in the analysis as enumerators misinterpreted the way they were supposed to pose the question. 186 understandin g ofbenefits of education least three benefitsof primary education. questions (Q301) benefits. This is consistent with baseline but we shall use a pre￾defined standard list (while baseline was kept open). The average % fromrespondents will be used to calculate this indicator. Provision ofschool meals MGD 1.2.1 Reduced shortterm hunger 15. Percent of studentsin target schools who regularly consume a meal before the school day Endline Survey: Pupil Questionnaire (Section 2 – Q205); HH/Parent questionnaire (Section 2:Questions are related exclusively to the pupil through which this HH member was selected – Q203) Data to be collected through pupil’s interviews in each sampled school as well as from interviews with parents. Students consuming a meal before theschool day are considered those who regularly had that meal during the last5 days (answer is 5). Always: 41% Sometimes: 39% 59.1%142143 Girls: 57.3% Boys: 61.5% 53% 16. Percent of studentsin target schools who regularly consume a meal during the school day Endline Survey: School Questionnaire, Section 8.Record attendance & Food Utilization (SMP Form 6) WFP reports: SMPmonitoring system Data collected from all terms on a daily basis from form SMP Form 6Data will be triangulated with WFPMonitoring data. 70% 68.1%144 67% MGD 1.2.1.1 – Increased access tofood (school feeding) 17. Number of daily school meals (breakfast, snack, lunch) provided as a result of USDA assistance* WFP reports Secondary data from WFP N/A 76,000,000 15,529,293 18. Total quantity of commodities provided for schoolmeals provided to students as a result of USDA assistance WFP reports: SMP monitoring system, WFPSPRs Review of WFP secondary data andreports. 19,017mt 14,810mt 19,260 mt 19. Number of students receiving school meals as a result of USDA assistance WFP reports: SMP monitoring system, WFPSPRs Secondary data review of WFPrecords 767,108 753,139 763,490 187 20. Percent of students WFP Records At school level we shall take 100% 100% 91.5% 142 The percent of students who regularly consume a meal before the school day by county: Garissa: 78.8%, Marsabit: 78.8%, Nairobi: 54.1%, Tana River: 73.8%, Turkana: 24.3%, West Pokot: 36.2%. 143 Only 49.3% of parents said that their child had had breakfast in the last 5 days. 144 The percentage of school feeding days in 2014 as reported in the SMP 6 forms in each county were: Garissa: 66.1%, Marsabit: 69.8%, Nairobi: 75.7%, Tana River: 50.9%, Turkana: 71.6%, West Pokot: 74.6%. 188 in targeted schools consuming daily meals (lunch) Endline Survey using the school’s records related toschool information from interviews with teachers and head teachers. In addition, observations will be made during the school survey to verify what proportion of students present in school eats a meal. 21. Number of trainings provided in food preparation and storage practices WFP Training reports The data will be obtained from theWFP training reports. This data will be triangulated and counterchecked though interviews atschool level with cooks and head teacher. 54 17145 15 22. Number of teachers trained in food preparation and storage practices Endline Survey throughinterview with head teacher, cooks and store keeper, WFP Training reports 3,455 1,753 1,870 Feed the Future indicators 23. Number of socialassistance beneficiaries participating in productive safetynets as a result of USDA support WFP reports: SMP monitoring system, WFPSPRs Analysis from secondary data 767,108 753,139 763,490 Increas ed nationa l capacity MGD 1.4.1 – Increased capacityof government institutions 24. Percent of districts in which food procurement and distribution procedures and infrastructure are in place. MOE data Ministry of Educationthrough the county education office Key informant interviewsthrough visit at Turkana and Garissa Counties Analysis of secondary data fromMOE. 85.4% 100% 100% 25. Number of MOE officers benefitingfrom home -grown school feeding manuals distributed MOE data Analysis of secondary data fromMOE. 6,000 4,232146 8,364 MGD 1.4.2 – Improved Policyand Regulatory Framework 26. Number of child health and nutritionpolicies, regulationand/or administrative procedures in place MOE data Analysis of secondary data fromMOE. 3 (Food and Nutrition Security Policy; National Social Protection 2 policies at Stage 2: Drafted and presentedfor public/stakeholder consultation147 . 8 policies at Stage 5: 2 policies at Stage 2: Drafted and presentedfor public/stakeholder consultation149 . 8 policies at Stage 5: 189 145 In 2014, the SMP Management Training was carried out in East Pokot, Garissa, Marsabit, Tana River, West Pokot and Samburu which covered also food preparation and storage practices. 146 This indicator was calculated by multiplying the number of home grown school feeding manuals distributed (2,116) times an average of two beneficiaries per manual. 147Child Health Policy (being finalized in 2015) and National School Health, Nutrition And Meals Program Strategy (drafted in 2011). 190 Policy; National School Health Policy) Passed for which implementation hasbegun148. Passed for which implementation hasbegun150. 27. Number of Home￾grown feeding manuals distributed MOE data Analysis of secondary data fromMOE. 1,770 824 16,002 1.4.4 – Increased Engagement of Local Organizations and Community Groups 28. Number of PTAs and SMCs contributing to theirschool (use percentage) Survey through interviewwith Head teacher and HH/Parents Data obtained through interview withheadmaster and HH/Parents. Particular emphasis will be given in understanding the frequency and type of the contribution (food and non-food items). 70% 93.8% of PTAs, 66.7% of SMCs151 82% PTAs 81% SMCs 29. Number of public￾private partnerships formed as a result ofUSDA assistance* WFP reports Secondary data from WFP N/A 13152 13 30. Value of new public and private sector investments leveraged as a resultof USDA assistance* WFP reports Secondary data from WFP N/A USD 15,381,303 USD 15,381,303 Food utilizati onand food safety MGD SO 2 – Increased use of health and dietary practices 31. Percent of schools intarget communities that store food off the ground Endline Survey at schoollevel through physical observations, interviews with person in charge of the store. Interview store keeper and observation of storage. Criteria thatwill be used: off ground/on ground,aeration, vermin and pest free. 67% 96%153 76.5% MGD 2.2 – Increased 32. Percent of food preparers at target Endline Survey throughinterview with cooks Kimetrica will administer the same test to cooks used during the baseline 87% 97.9%154 100% 149Child Health Policy (being finalized in 2015) and National School Health, Nutrition And Meals Program Strategy (drafted in 2011). 148 School Health Nutrition Policy – (being reviewed in 2015), Diarrhoea Policy (updated in 2014), Basic Education Act (2014), National Food and Nutrition Security Policy (2011) (being reviewed in 2015), Kenya National Social Protection Policy (2011), National School Health Policy (2009) and Kenya Health Policy 2012-2030 (2012). 150 School Health Nutrition Policy – (being reviewed in 2015), Diarrhoea Policy (updated in 2014), Basic Education Act (2014), National Food and Nutrition Security Policy (2011) (being reviewed in 2015), Kenya National Social Protection Policy (2011), National School Health Policy (2009) and Kenya Health Policy 2012-2030 (2012). 151 While water, firewood, cooking utensils, cleaning products and plates were provided for children in 2014 in at least 87% of cases, during the survey, no fruits and meat were contributed and 191 vegetables in only 2 schools in Turkana. 152 New partnerships with: International Paper, DSM, FEED, Caterpillar, Earth Holdings, Government of Kenya, Unilever, Drew Barrymore, Princess Haya WPD, IRB, JAWFP, LG Electronics and Goodeed Association. 153 The break down by county is as follows: Garissa: 100%, Marsabit: 96%, Nairobi: 87.5%, Tana River: 100%, Turkana: 100%, West Pokot: 100%. 154 The mid-term evaluation recommends using in the future a passing threshold should be 80%. Using this threshold, only 29.2% of cooks passed the test. The break down by county is as follows: Garissa: 12.5%, Marsabit: 14.3%, Nairobi: 50%, Tana River: 37.5%, Turkana: 33.3%, West Pokot: 25%. 192 knowledge of safefood preparation and storage practices schools who achievea passing score on a test of safe food preparation and storage WFP records survey with some improvements. Threshold for passing: 50% correctanswers. MGD 2.6 Increased access to requisite food preparation and storage tools and equipment 33. Percent of target schools with improved food preparation and storage equipment Endline Survey through interviews with headmaster (School Questionnaire) and cook and physical observations Data obtained from interviews withHead teachers and cooks through questionnaires. Indicators will be calculated from our sample by dividing the schools with improvedstorage equipment by the total number of schools visited. Food preparation: 60% Food stores: 67% -Energy saving stoves:37.5% of schools155156 -Food preparation (schools with kitchens)81%. -Food store 80%. -Raised wooden pallets: 95.8% of schools. 80% of schools havededicated storerooms 82% of schools havededicated kitchens WFP Protection indicator Proportion of assisted people whoexperiences threats to safety problems travelling to, from and/or at WFP program site Endline Survey through Household Questionnaire Parents were asked whether their child had been exposed to any threatsto his/her safety in the past month. N/A rape: 4.9%, sexual harassment: 4.9%, robberies: 3.7%, animal attacks: 8.2%, bullying:3.5%, abuse of drugs: 4.4%157 Rape: 0.2%, Sexual harassment: 0.4%, Robbery 9%, Animal attack: 41.5%, Bullying 14.9%, Drug abuse 0.7% 155 The break down by county is as follows: Garissa: 12.5%, Marsabit: 37.5%, Nairobi: 87.5%, Tana River: 12.5%, Turkana: 12.5%, West Pokot: 62.5%. 193 156 27.1% of cooks confirmed that they use energy saving stoves. 157 Rape : Nairobi 12.7%, Garissa 1.2%. Sexual harassment: Nairobi 14.5%. Robbed: Nairobi 5.4%. Animal attacks: Turkana: 22.7%, Nairobi 9.1%, Tana River 2.5%. Bullying: Tana River 2.5%. Abuse of drugs: Nairobi 11%. Annex 31: Additional data - Significance test results Significance Testing - Enrolment Hypothesis Test Summary Null Hypothesis Test Sig. Decision 1 The median of Boys Change EoP equals 4.00. One-Sample Wilcoxon Signed Rank Test 0.850 Retain the null hypothesis. 2 The median of Girls Change EoP equals 4.00. One-Sample Wilcoxon Signed Rank Test 0.323 Retain the null hypothesis. Asymptotic significances are displayed. The significance level is .05. Significance Testing – USDA Direct Beneficiaries Hypothesis Test Summary Null Hypothesis Test Sig. Decision 1 The median of Number of total individuals benefiting directly from USDA-funded interventions equals 1,020,480 One-Sample Wilcoxon Signed Rank Test 0.102 Retain the null hypothesis. Asymptotic significances are displayed. The significance level is .05. The median number of beneficiaries benefiting directly from USDA-funded interventions is 763,490 which is below the set target of <1,020,480. There is a slow (p =0.102) regression of thetotal individuals benefiting directly from USDA-funded interventions. The variance change is within the set target given that p > 0.05. Significance Testing – USDA Indirect Beneficiaries Hypothesis Test Summary Null Hypothesis Test Sig. Decision 1 The median of Number of total individuals benefiting indirectly from USDA-funded interventions equals 1,135,470. One-Sample Wilcoxon Signed Rank Test 0.180 Retain the null hypothesis. Asymptotic significances are displayed. The significance level is .05. Notes 1). One-Sample Wilcoxon Signed Rank Test indicate that the enrolment rates for boys was withinthe set threshold of 4-percentage point increase between baseline and end of project (p = 0.850 -the observed median percentage was NOT significantly different from the projected median percent change). 2). One-Sample Wilcoxon Signed Rank Test indicate that the enrolment rates for girls was withinthe set threshold of 4-percentage point increase between baseline and end of project (p = 0.323 -the observed median percentage was NOT significantly different from the projected median percent change) The median number of beneficiaries benefiting indirectly from USDA-funded interventions is 777,373 which is below the set target of <1,020,480. There is a slow (p =0.180) regression of thetotal individuals benefiting indirectly from USDA-funded interventions. The variance change is within the set target given that p > 0.05. Significance Testing - Teachers in target schools who attend school and teach atleast 90 percent of scheduled school days per year ONE-SAMPLE TEST Teachers in target schools who attend school and teach at least 90 percent of scheduled school days per year Test value = 90 t df sig. (2- tailed) Mean difference 95% Confidence interval of the difference lower upper -.366 34 .717 -.27200 -1.7823 1.2383 The overall mean attendance for the teachers is 89.7% (Test value + Mean Difference) of scheduled school days in a year. This is 0.3 percentage point below the target of 90%. However,the difference is statistically insignificant (p>0.05). In this regard, the target was attained. Thismeans that the 36% of the teachers that attend less than 90% of the scheduled sessions do not adversely affect the overall mean teacher attendance. WFP Kenya Country Office