Prepared Decentralized Evaluation En d lin e Eva lu a t ion of Un it e d St a t e s De p a rt m e n t of Agricu lt u re (USDA) McGove rn -Dole Gra n t Food for Ed u ca t ion Progra m m e for WFP Ca m b od ia (Agre e m e n t Nu m b e r: FFE-442-2016/015-00) FY 2017-2019 January 2020 Com m issioned by WFP Cam bodia Evaluation Manager: Bunthang Chhe Prepared by Sophia Dunn, Team Leader Jean-Pierre Silvéréano, Evaluator Sovith Sin, Evaluator Prem Bhandari, Evaluator DECENTRALIZED EVALUATION FOR EVIDENCE-BASED DECISION MAKING Disclaimer The opinions expressed in this report are those of the Evaluation Team , and do not necessarily reflect those of the World Food Program m e. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this docum ent does not im ply endorsem ent by WFP of the opinions expressed. The designation em ployed and the presentation of m aterial in m aps do no im ply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delim itation of frontiers. This publication was produced at the request of the United States Departm ent of Agriculture. It was prepared by an independent third-party evaluation firm . The author’s views expressed in this publication do not necessarily reflect the views of the United States Departm ent of Agriculture or the United States Governm ent. Accessibility Note: An accessible version of this docum ent can be m ade available by contacting fas.m onitoring.evaluation@usda.gov Acknowledgements The evaluation team would like to thank all stakeholders who participated in this evaluation and gave their tim e and opinions. We gratefully acknowledge the input from representatives of several m inistries within the Royal Governm ent of Cam bodia including the Ministry of Education, Youth and Sport at national, provincial and district levels, the Ministry of Economy and Finance, the Ministry of Agriculture and the Council of Agricultural and Rural Developm ent. We also acknowledge the input from USDA and other donors, WFP Cam bodia, the Plan International, World Vision, World Education and KAPE team s, the WFP Regional Bureau and all other stakeholders for their contribution towards the evaluation. Special thanks to the World Food Program m e in Cam bodia for their tim e and effort in supporting the evaluation, including organizing all the key inform ant interviews. Our thanks are especially due to Chhe Bunthang who served as Evaluation Focal Point for this evaluation, and to Riguen Thorn for organizing all the secondary docum entation and data. The evaluation team also extends special thanks to Kannitha Kong, Em m a Conlan, Francesca Erdelm ann, Um Nisith, and all the WFP Field Office Team for Battam bang, Siem Reap and Kam pong Thom for your ongoing support over the past two years. Special thanks also go to Indochina Research Lim ited staff especially Chandara Gnim , Morina Heak, Pisei Im , and all the field supervisors and enum erators for their dedication to the survey. The evaluation surveys would not have been possible without your support. Special thanks are also due to Kim haeng Ong and Sovanrith Khull for their translation services during the qualitative field m ission, and to Bunyeth Ho for his support during the quantitative survey. Finally, the team gratefully acknowledges the com m unity participants in the evaluation process – households, teachers, School Directors, school com m ittee m em bers, village chiefs, com m une council m em bers, cooks, parents and school children. We hope that this evaluation will contribute to im proving the school environm ent and education outcom es for all children in Cam bodia. Table of Contents 1 Introduction ..................................................................................................... 1 1.1 Overview of the Evaluation Subject ........................................................................................ 2 1.1. Context ................................................................................................................................... 5 1.2 Previous evaluation recom m endations .................................................................................. 8 1.3 Stakeholder analysis............................................................................................................... 8 1.2. Evaluation Methodology and Lim itations ............................................................................... 8 2 Evaluation Findings........................................................................................ 15 2.1 Evaluation Question 1: How appropriate is the program m e? ...............................................15 2.2 Evaluation Question 2: What are the results of the program m e? .........................................20 2.3 Evaluation Question 3: Why and how has the operation produced the observed results? ...44 2.4 Evaluation Question 4: How sustainable is the program m e m oving forwa rd?......................47 3 Conclusions and Recommendations ...............................................................49 1.3. Overall Assessm ent/Conclusions..........................................................................................49 3.3. Recom m endations ...............................................................................................................51 4 Bibliography ...................................................................................................54 5 Annexes ..........................................................................................................58 6 List of Acronyms.............................................................................................59 List of Tables Table 1: Number of USDA McGovern-Dole sites to be visited (qualitative field mission)....................................10 Table 2: Geographic breakdown of sampled schools and households for the quantitative survey.....................11 Table 3: School mapping information in the school year (SY2018/19) ................................................................ 12 Table 4: Percentage of parents who can name at least three benefits of primary education (n=958)................21 Table 5: Enrolment rate for children 6-11 years old in surveyed schools (SY 2018/19).......................................21 Table 6: Promotion, repetition and drop-out rates (SY 2018/19)..................................................................... 22 Table 7: Regular attendance rate SY 2016/17 – SY 2018/19 ................................................................................ 22 Table 8: Summary of available school feeding-related infrastructure ................................................................. 23 Table 9: Provision of THR by WFP (2017-2019).................................................................................................... 24 Table 10: Planned vs. actual USDA food commodities distributed (MT).............................................................. 25 Table 11: Provision of daily school meals 2017-2019........................................................................................... 25 Table 12: Percent of students in target schools who indicate that they are “hungry” or “very hungry” during the school day.................................................................................................................................................... 26 Table 13: Reported inattentiveness of students................................................................................................. 26 Table 14: Literacy training activities for school administrators and teachers...................................................... 28 Table 15: Number of schools receiving learning materials and stationery packages (SY 2018/19)...................29 Table 16: Demonstrated reading and comprehension - Grade 2 ......................................................................... 30 Table 17: Number of people trained in child health and nutrition (2017-2019).................................................. 31 Table 18: Percentage of schools using clean cooking and eating equipment ...................................................... 33 Table 19: Summary of availability of school water and sanitation infrastructure................................................ 33 Table 20: Absence in class (SY 2018/19) .............................................................................................................. 34 Table 21: Number of days missed due to illness.................................................................................................. 35 Table 22: Parental membership of school committees (SY 2018/19) .................................................................. 36 Table 23: Number of policies developed as a result of USDA assistance (2017-2019)......................................... 39 Table 24: Progress on the achievement of the SABER quality standards............................................................. 43 List of Figures Figure 1: Percentage of teachers reporting child hunger and inattentiveness in USDA supported schools27 Figure 2: Main source of information on good health practices (% of households) ....................................32 Figure 3: Month of year when school gardens are utilized (Percent)..........................................................32 Executive Summary 1. This report is the endline evaluation of the United States Departm ent of Agriculture (USDA) McGovern-Dole Grant Food for Education program m e im plem ented by World Food Program m e (WFP) and partners in Cam bodia (FY 2017-2019) (FFE-442-2016/015-00). This activity evaluation is com m issioned by the WFP Cam bodia Country Office (CO) and is based on the Term s of Reference provided by WFP Cam bodia (Annex 1). The endline evaluation covers the period from Septem ber 2016 to August 2019. The McGovern-Dole program m e is a continuation of the previous phase (2013-2016), which was itself a continuation of support from 2010. The programme is im plem ented in three provinces of Cam bodia: Battam bang (BTB), Siem Reap (SRP) and Kam pong Thom (KTM) all of which received support in the previous phase of the programme. 2. The purpose of the endline evaluation is to critically and objectively review the program m e im plem entation since the 2017 baseline to assess whether targeted beneficiaries received services as expected, while assessing whether the project m et its stated goals and objectives. The m ain objectives of the evaluation are: • Accou n t a b ilit y: To assess and report on the perform ance and results of all the McGovern-Dole grant funded activities as per the Perform ance Monitoring Plan (PMP). • Le a rn in g: To determ ine the reasons why certain results occurred or not, to draw lessons, derive good practices and pointers for learning. It will also provide evidence - based findings to inform operational and strategic decision-m aking. 3. The m ain expected users for this evaluation report include USDA, the CO, its m ain im plem enting partner the Ministry of Education, Youth and Sport (MoEYS) and the other im plem enting partners World Vision (WV), Plan International (Plan), World Education Incorporated (WEI) and Kam puchean Action for Prim ary Education (KAPE). 4. As per the original design of the program m e , during 2017-2019 WFP has continued to hand over the McGovern-Dole program m e to the MoEYS as per the Roadm ap of 2015, with a view to ensuring national ownership of the program m e by 2021. This transition included m oving away from the traditional school m eals com prised m ainly of USDA donated com m odities, to a Hom e -Grown School Feeding (HGSF) m odel, including using a HGSF-hybrid m odel, using both locally purchased and externally sourced food. WFP is also reducing WFP’s THR activities as planned as the governm ent takes them over. 5. Cam bodia has im proved prim ary school enrolm ent and attendance in recent years and reducing gender disparities in education. The m ain challenge now is prim ary school com pletion. Although both repetition and dropout rates have steadily declined in the last five years,1 they rem ain a key concern to MoEYS. Food security and undernutrition also rem ain im portant public health concerns, with a recent study2 finding that the overall dietary intake of school children did not m eet the local recom m ended dietary allowances. Meth odology 1 Final Draft Education Strategic Plan 2019-2023, MoEYS, May 2019. 2 Horiuchi, Y et al (2019) Urban-rural differences in nutritional status and dietary intakes of school-aged children in Cam bodia. Nutrients. 2019 Jan; 11(1):14. 6. The evaluation was designed to assess the 2017-2019 McGovern-Dole programme against each of the OECD-DAC evaluation criteria, with a focus on effectiveness, im pact and sustainability since this is an endline evaluation. The planned transition to governm ent ownership m eans that this evaluation has a strong focus on the sustainability aspect to see whether the plans for governm ent ownership are feasible . Sim ilarly, the evaluation is interested in assessing the im pact of the programme to provide evidence to the governm ent that school feeding is a worthwhile investm ent. The evaluation was designed to answer four m ain questions: How appropriate is the program m e? What are the results of the program m e? How and why has the program m e achieved its results? And how sustainable is the programme? 7. In order to respond to these questions, the evaluation used a theory-based, gender￾responsive and participatory approach, the sam e as used during the 2017 baseline. The evaluation design is quasi-experim ental case -control (com parison) as per the baseline. The evaluation m ethodology used m ixed m ethods, including secondary docum ent review, qualitative interviews and focus group discussions with a range of stakeholders, and a quantitative endline survey to enable com parison of results against the baseline . The evaluation was conducted during July-August 2019. Ke y Fin d in gs Eva lu a t ion q u e st ion 1: How a ppropria t e is t h e progra m m e? 8. The evaluation of the previous phase (2013-2016) established that the range of activities were appropriate , and that rem ains the case . The m ain issues that were highlighted during the baseline were that the m odality of food procurem ent followed by WFP (food im ported/donated from the US) was not sustainable once the governm ent took over the im plem entation of the program m e, and that food-based take -hom e rations did not align with the governm ent’s preference to provide cash-based support. To this end, over the 2017-2019 period, WFP ha s m ade substantial effort to work with the MoEYS to find a m ore appropriate SF m odel for governm ent ownership. 9. WFP has continued to be instrum ental in supporting policy developm ent in Cam bodia during this phase, on m any relevant areas including food and nutrition security, social protection, school health and school feeding. The McGovern-Dole program m e is therefore designed to align with governm ent policies and strategies, and with WFP’s own corporate guidance. The programme also aligns with several SDGs, m ost notably SDGs 2, 4 and 17. The literacy objective of McGovern-Dole also aligns well with the governm ent’s new Education Strategic Plan and with the new WFP Corporate Results Fram ework which aim s to ensure that education quality (literacy) is im proved. Eva lu a t ion q u e st ion 2: Wh a t a re t h e resu lt s of t h e progra m m e? 10. The m ain strategic objective (SO1) of the program m e is to increase the num ber of children who at the end of Grade 2 are able to read grade level text. During 2017-2019, WEI and KAPE im plem ented several literacy-related activities and their end of program m e assessment found that early grade reading im proved from 23.8 percent in SY 2017/18 to 30.5 percent in SY 2018/19. This is a significant im provem ent over a short period. As well a s being effective, the literacy activities have been im plem ented very efficiently, with a sm all num ber of WEI staff collaborating with the MoEYS at all levels to ensure that teacher training was com pleted with the appropriate m aterials, and then followed up in class. 11. The second program m e SO is to im prove the use of health and dietary practices. To do this, the program m e provides infrastructure support to increase practices such as handwashing with soap, using clean latrines, and drinking clean water. The McGovern-Dole programme also includes training on good nutrition to help children and parents m ake sound dietary decisions. All these activities are intended to contribute to reducing health-related absence from school. The ET is confident that the provided training, along with a clean school environm ent are sound starting points for good health for the school children. Further, the continued support to infrastructure developm ent during this phase was a good use of USDA resources, to ensure that all the USDA-supported schools provide children with an appropriate and healthy learning environm ent. 12. This phase has also seen im provem ent in school enrolm ent rates, and m ost stakeholders indicated that the presence of school m eals plays a role in regular student attendance. MoEYS data indicates that dropout rate s were lower in USDA-supported schools (4.9 percent vs. 6.0 percent) and com pletion rate s were higher (81.3 percent vs. 80.1 percent) although these differences m ay not be statistically significant. Girls in both categories of schools perform ed better than boys, with m ore girls being prom oted, and less girls repeating grades or dropping out of school. 13. Aside from the above, the evaluation found that the McGovern-Dole program m e has been effective at im proving children’s access to food, im proving parental understanding on the benefits of prim ary education, im proving school feeding related infrastructure in the USDA-supported schools and prom oting parental/com m unity engagem ent in the program m e. The endline surveys were also able to establish other differences between USDA￾supported schools and com parison schools, presum ably as a result of the program m e. These include reduced hunger in class and reduced inattentiveness (which are positively correlated. The evaluation also indicate s that the program m e saves parents tim e and m oney, enabling them to spend m ore tim e on livelihood activities. The ET also recognize that the McGovern￾Dole program m e prom otes equal participation in the program m e for boys and girls, by providing scholarships for both, ensuring gender separated latrines are present in schools, and providing all students with training on various topics. However, the issue of inadequate rem uneration for the school cooks, alm ost all of whom are wom en, is an ongoing concern. Although both WFP and the MoEYS are aware of this issue, the gender transform ative potential that the program m e could bring to the cooks, is yet to be realized. Eva lu a t ion q u e st ion 3: Wh y a n d h ow h a s t h e op e ra t ion produ ced t h e observed resu lt s? 14. The evaluation identified several internal and external factors that have been instrum ental to the achievem ent of the program m e results, as shown below. • In t e r n a l fa ct ors: Long term engagem ent between WFP and partners, strong program m e m anagem ent and technical capacity, strong logistic/supply chain support and technical support from RBB. High capacity of im plem enting partners, strong m onitoring system s, and use of program m e funds to support research, visits and workshops. • Ext e rn a l fa ct ors: Collaboration with MoEYS, Governm ent com m itm ent to school feeding and literacy program m es, support from other donors, capacity and turnover of MoEYS personnel, and unclear regulations on the use of Com m une Developm ent Funds. Eva lu a t ion q u e st ion 4: How su st a in a ble is t h e progra m m e m ovin g forwa rd? 15. During the 2017-2019 period, the MoEYS has undertaken several pieces of work with the support of WFP, to clarify its position on school feeding. As a result, the governm ent is now clearly articulating its preference for cash-based program m ing (scholarships) and for a HGSF m odel as it benefits the local econom y. The governm ent is also now dem onstrating a stronger com m itm ent to build and own a national school m eals program m e. The evaluation identified a significant change in the lexicon of MoEYS representatives since baseline, with m any representatives now recognising school feeding as an im portant social assistance instrum ent, especially for households in areas experiencing food insecurity and poor educational outcom es. This growing governm ent com m itm ent to school feeding, culm inated in the governm ent developing a concept note for MoEYS im plem entation of school feeding activities in 205 schools in six provinces 3 , including schools in two provinces where the USDA McGovern-Dole program m e is currently im plem ented: SRP (31 schools) and BTB (32 schools). In total, the project will cost at least USD 2 m illion per year. This proposal was recently accepted by the Prim e Minister’s Office and MoEYS is now planning to start im plem entation at the start of the next school year (SY2019/20). Overall, the evaluation team estim ates that capacity of the governm ent to im plem ent school feeding program m es has increased from SABER Level 1 (latent) at baseline to SABER level 2 (em erging). 3 Banteay Meanchey, Kampong Cham, Siem Reap, Battambang, Preah Vihear and Stoeung Traeng Overall conclusions 16. Re le va n ce : During this phase, the governm ent clearly stated their preference for a HGSF approach and WFP has piloted several HGSF m odels to help the governm ent decide which aspects they would like to continue in a nationally owned program m e. WFP is now appropriately transitioning all the USDA-supported schools to a HGSF m odel to align with the governm ent’s preference and with the agreed national transition plan. 17. Efficie n cy: USDA provided USD 15 m illion in funding for this phase of program m ing, 25 percent less than the previous phase. This is indicative of the transition of the program m e to national ownership and reduction in program m e im plem entation requirem ents, and a greate r focus on capacity strengthening activities, which generally require less funds. WFP and partners have im plem ented the m ajority of the planned activities, with som e capacity building activities ongoing that will be com pleted during a no cost extension period to March 2020. Aside from those activities, all others have been com pleted in a tim ely m anner. 18. Effe ct ive n e ss: The program m e has effectively im proved school enrolm ent rates, and m ost stakeholders indicated that the presence of school m eals plays a role in regular student attendance. MoEYS EMIS data indicates that dropout rates were lower in USDA-supported schools (4.9 percent vs. 6.0 percent) and com pletion rates were higher (81.3 percent vs. 80.1 percent). Girls in both types of schools perform ed better than boys, with m ore girls being prom oted, and less girls repeating grades or dropping out of school. The program m e has contributed to increased school enrolm ent, increased com m unity engagem ent in school activities and increased access to food, reducing hunger and inattentiveness in class. 19. Im p a ct : For each programme activity, outputs have contributed to expected outcom es and it is clear that SO1 - im proved literacy - has been achieved, as the percentage of children able to read and com prehend grade level text has increased. Sim ilarly, the effectiveness of the SO2 activities have contributed to producing a healthy and conducive learning environm ent in schools. The differences noted from the MoEYS EMIS data between USDA￾supported and com parison schools in com pletion and dropout, and the im proved literacy rates found by WEI indicate the potential of the McGovern-Dole program m e to have long￾term im pacts on the level of education achieved by children in this program m e. The evaluation also found a signficant difference in hunger (reduced) and attentiveness (im proved) in USDA-supported schools com pared with the counterfactual. The evaluation also found that total inattentivene ss in class was significantly positively correlated with total percent reported hungry in class.4 If the MoEYS take on all aspects of the programme , including the provision of school m eals, and literacy activities and expand it to other areas, there is therefore great potential for im proving the school standards and education outcom es for school children in Cam bodia. 20. Su st a in a b ilit y: The evaluation com m ends WFP and MoEYS on the significant progress m ade during 2017-2019 on transitioning school feeding activities to national ownership. The upcom ing im plem entation of the new McGovern-Dole Grant (FY2020-2024) and the new USDA Local and Regional Procurem ent grant over the sam e period, provides a m ore feasible 4 Correlation (r)=0.378 – a medium correlation, p<0.001 (two tailed) tim e period (to 2025) for the MoEYS to take full ownership of the school m eals program m e and gain sufficient additional capacity in the HGSF m odel to im plem ent the program m e. Re com m e n d a t ion s • St ra t e gic r e com m e n d a t ion s Re com m e n d a t ion 1: The Cam bodia program m e has shown that school feeding can provide a e fficient and effective platform to support im provem ents in literacy, the ET therefore recommend that WFP Headquarters advocate for the expansion of literacy activities into non￾McGovern-Dole school feeding program m es to align with the tim eline of the new WFP CRF. Re com m e n d a t ion 2: The new WFP Strategic Plan outlines a transition in role for WFP from project im plem entation to enabler. As such, WFP Headquarters should develop program m ing tools to guide Country Offices on how best to carry out this transition before the end of the tim eline of the current Strategic Plan. • Opera t ion a l recom m en da t ion s Re com m e n d a t ion 3: WFP Cam bodia , with support from WFP RBB and the WFP Office of School Feeding in headquarters as required, should continue to collaborate with MoEYS over the next phase to provide capacity building support for the im plem entation of a nationally owned HGSF program m e. Re com m e n d a t ion 4: WFP Cam bodia, together with the MoEYS and MEF should undertake a com prehensive costing exercise wthin the next 12 m onths to obtain sufficient information to determine appropriate levels of national funding for school feeding. Re com m e n d a t ion 5: WFP Cam bodia together with the MoEYS, im plem enting partners and other stakeholders as appropriate should ensure that the transition plan is well docum ented and dissem inated at sub-national level within the next 12 m onths. Re com m e n d a t ion 6: WFP Cam bodia and the MoEYS should continue to work together to ensure that the MoEYS’s program m e m onitoring capacity is adequate in the programme locations targeted for governm ent im plem entation by the end of the next phase of McGovern-Dole program m ing. 1 Introduction 1. This report is the endline evaluation of the United States Departm ent of Agriculture (USDA) McGovern-Dole Grant Food for Education program m e im plem ented by World Food Program m e (WFP) and partners in Cam bodia (FY 2017-2019) (FFE-442-2016/015- 00). This activity evaluation is com m issioned by the WFP Cam bodia Country Office (CO) and is based on the Term s of Reference (ToR) provided by WFP Cam bodia (Annex 1). The endline evaluation covers the period from Septem ber 2016 to August 2019. 2. This McGovern-Dole program m e is a continuation of the previous phase (2013- 2016), which was itself a continuation of support from 2010. The program m e provides one school m eal per day to pre -prim ary and prim ary school students in three provinces of Cam bodia: Battam bang (BTB), Siem Reap (SRP) and Kam pong Thom (KTM). The program m e also provides food scholarships (take hom e rations - THR) to girls and boys from vulnerable households as an incentive to keep them in school. Along with the provision of the school m eal and THR, the program m e includes several other activities to im prove literacy of school children, im prove the school environm ent, and educate parents and children in health and nutrition. The activities are described in m ore detail in the Section 1.2. The scope of the evaluation is all activities im plem ented through the McGovern-Dole program m e including the provision of a school m eals program m e (SMP) and THR and the com plem entary activities outlined above in the program m e overview. In addition, all the processes related to the design, im plem entation, resourcing, m onitoring, evaluation, and reporting of these activities are included, as relevant, to answer the evaluation questions. 3. The m ain objectives of the evaluation are: • Accou n t a b ilit y: To assess and report on the perform ance and results of all the McGovern-Dole grant funded activities as per the Perform ance Monitoring Plan (PMP). • Le a rn in g: To determ ine the reasons why certain results occurred or not, to draw lessons, derive good practices and pointers for learning. It will also provide evidence -based findings to inform operational and strategic decision-m aking. 4. The evaluation team (ET) has followed the perform ance of this program m e since July 2017 when the endline for the previous phase and the baseline of the current phase were conducted at sam e tim e.5 The sam e ET then conducted the m id-term review (MTR) in 2018.6 The purpose of the endline evaluation is now to critically and objectively review the program m e im plem entation since the baseline to assess whether ta rgeted beneficiaries received services as expected, while assessing whether the project m et its stated goals and objectives. 5. The evaluation is designed to answer four m ain questions: • How appropriate is the program m e? 5 Dunn, S. et al (2017) Baseline Report for WFP Cambodia and the USDA/McGovern Dole Food for Education Programme 2017-2019. 6 Dunn, S., Silvéréano, J & Sin, S. (2018) Mid-term review of USDA McGovern-Dole Grant FFE-442-2016/015-00 for WFP School Feeding in Cambodia covering school years 2016/17 and 2017/18 • What are the results of the program m e? • How and why has the program m e achieved its results? • How sustainable is the program m e? 6. There are m any expected users for this final endline evaluation report including USDA, the CO, its m ain im plem enting partner the Ministry of Education, Youth and Sport (MoEYS) and the other im plem enting partners World Vision (WV), Plan International (Plan), World Education, Inc. (WEI) and Kam puchean Action for Prim ary Education (KAPE). Other governm ent m inistries and United Nations (UN) agencies that provide technical guidance or support to the program m e in other ways will also be interested in the evaluation findings as will USDA, other school feeding (SF) donors, other donors im plem enting lite racy activities in Cam bodia such as USAID and other WFP offices im plem enting sim ilar program m es. Further description of evaluation users can be found in Annex 2. 7. WFP Cam bodia has recently been awarded a new UDSA McGovern-Dole grant for a further four-year period (2020-2023) as well as a USDA Local and Regional Procurem ent (LRP) program m e grant for the sam e im plem entation period. 1.1 Overview of the Evaluation Subject Tim in g 8. The 2017-2019 USDA McGovern-Dole funded programme is a continuation of the 2013-2016 programme . The grant being evaluated was approved in Septem ber 2016 and covers three school years over the period Septem ber 2016 to August 2019 i.e. SY 2016/17, SY 2017/18 and SY 2018/19. Ge ogra p h ic scop e 9. The McGovern-Dole program m e is im plem ented in three provinces of Cam bodia: Battam bang (BTB), Siem Reap (SRP) and Kam pong Thom (KTM) all of which received support in the previous phase of the program m e. Object ives a n d ou t com es 10. The purpose of McGovern-Dole funding7 is to use food com m odities and technical assistance to im prove literacy and prim ary education of school-aged children in developing countries. Instead of a logical fram ework, the program m e uses the two Results Fram eworks (RF) developed by USDA’s Foreign Agricultural Service (FAS). Each of these depicts the theory about how the two Strategic Objectives (SOs) of the programme (im pacts) can be achieved, and it is this Theory of Change (ToC) that the evaluation is following. The program m e has two SOs: 1. To im prove literacy (m easured by early grade reading) and 2. To im prove the health and dietary practices. To achieve the SOs, several outcom es including im proved quality of literacy instruction, im proved student attendance and im proved attentiveness need to be achieved, requiring m ultiple activities including providing school m eals, training and school infrastructure a re needed. Both RFs also show foundational results, including increased capacity of governm ent institutions, im proved policy fram eworks, increased governm ent support and increased engagm enet 7 https://www.fas.usda.gov/programs/mcgovern-dole-food-education-program of local orgnaizations and com m unity groups. The pictorial representation of the RFs can be found in Annex 3. 11. FAS expects that the SOs of the two RFs can begin to be achieved in whole or in part within a four-to-six-year period. Since the period under evaluation is a continuation of the previous phase of the program m e , determ ining whether the SOs have been achieved is a key com ponent of this evaluation. In addition to the achievem ent of the two SOs, the current grant has a strong focus on capacity strengthening to support governm ent m anagem ent of the program m e and assessing the governm ent’s readiness to m anage the program m e is another key elem ent of the evaluation. Ou t pu t s a n d a ct ivit ies 12. As per the original design, during 2017-2019 WFP has continued to hand over school m eals to the MoEYS as per the Roadm ap of 2015. This included m oving away from the traditional school m eals com prised m ainly of USDA donated com m odities, to a Hom e - Grown School Feeding (HGSF m odel, including using a HGSF-hybrid m odel, using both locally purchased and externally sourced food. WFP is also reducing WFP’s THR activities as planned as the governm ent takes them over. As a result, during this phase the USDA￾supported schools are receiving support through five different m odalities: • Som e schools receive a m eal of food m ainly provided by USDA (SMP) • Som e schools receive both SMP and THR (SMP+THR) • Som e schools use the hybrid HGSF m odel (HGSF-hybrid) • Som e schools use both the hybrid HGSF m odel plus THR (HGSF-hybrid + THR) • Som e schools received THR only (THR) although this is now discontinued.8 13. The school m eals include USDA provided com m odities: rice and oil, com plem ented by pulses, anim al protein and salt from other donors. The new m odality, HGSF-hybrid, provides USDA rice and oil and supports schools to purchase locally produced item s as well. Since SY 2018/19, the m ajority of schools are im plem enting the HGSF-hybrid m odel. THR is provided to vulnerable children in grades 4 to 6 to support their progression to higher grades and support m ore equitable access to the governm ent’s scholarship program m e for the poor in grades 7 to 9. Although originally only targeting girls, in som e schools the THR is provided to vulnerable boys and girls. Children receiving the THR are provided with 10kg of rice and 0.91kg of oil twice a year as an incentive to regularly attend school. 14. The program m e also works with school personnel to establish school gardens, provide training on com m odity m anagem ent and food preparation and storage practices, and establish activities to prom ote literacy. The program m e also constructs or rehabilitates school infrastructure to ensure the target schools provide a healthy learning environm ent. In addition, the program m e invests significantly in capacity building at the local, regional and national level to ensure sustainability and governm ent ownership. All these project activities are described in m ore detail in Annex 4. 8 THR only schools were discontinued in SY2017/18. 15. All the above activities are im plem ented in SRP and KTM. The program m e in Battam bang includes all activities except for the literacy com ponent and support to school infrastructure developm ent. This is because , since 2017, those activities were directly implem ented by the Provincial Office of Education, Youth and Sport (PoE), with support from the SRP WFP Area Office.9 WFP Cam bodia has recently been granted a no cost extension (NCE) until end March 2020 to com plete a few ongoing activities. These are noted in the evaluation findings section ahead. Pla n n ed ben eficia ries 16. For the period FY2017-2019 the program m e planned to reach 859 schools in BTB, SRP and KTM provinces while also supporting the com plem entary activities focused on im proving literacy and health and dietary practices described above. As at SY 2018/19 WFP is supporting 585 schools to im plem ent SMP and HGSF as the MoEYS has taken over im plem entation of the THR. A m ap of the program m e locations can be found in Annex 5. The program m e planned to re ach 150,600 pre -school and prim ary school children with school m eals (either through the SMP or the HGSF) and THR in 433 schools, providing 8,400 children from the poorest households with THRs, including 4,750 girls (56.5 percent). Re sou rce requ irem en t s a n d fu n din g sit u a t ion 17. The USDA McGovern-Dole grant to WFP Cam bodia was signed between the two parties in Septem ber 2016 through which USDA allocated up to USD 15,212,698 for donations of com m odities and funding of ocean transportation and othe r authorized expenses during US fiscal years FY 2017-2019. This is 25 percent less than the grant for the last phase (2013-2016). Partners 18. The grant is im plem ented by WFP in partnership with several governm ent m inistries and NGOs. The key partner is the MoEYS which is responsible for the im plem entation of the program m e in the schools. Within the MoEYS, WFP works closely with several departm ents including the Prim ary Education Departm ent/ Scholarship Office (which supports the im plem entation and m onitoring of the program m e), the School Health Departm ent, and the Policy Departm ent. From the end of 2016 the Teacher Training Departm ent and the Curriculum Departm ent of the MoEYS have also been involved in the literacy com ponent which focuses on early grade reading (EGR). Other m inistries involved include the Ministry of Health (MoH), Ministry of Economy and Finance (MEF), Ministry of Agriculture Forestry and Fisheries (MAFF), Ministry of Interior (MoI) and the Council of Agricultural and Rural Developm ent (CARD). 19. The McGovern-Dole program m e is im plem ented together with four NGO partners: Plan and WV for the m ajority of school-based activities and for the literacy com ponent, WEI and their partner KAPE. WFP is responsible for directly im plem enting the activities tha t affect the foundational results, such as building the capcity of the governm ent and supporting im proved poliy and regulatory fram eworks. Several United 9 PLAN supporte d BTB during SY2016-17 after which PoE/DoE with support from WFP has directly im plem ented the programme. Nations agencies also provide technical support to the school feeding program m e including UNICEF, UNESCO, FAO, the World Bank and WHO. Each provides support in their own technical areas: Water, sanitation and hygiene (WASH), education, agriculture (schools gardens), social protection and health. WFP also has Mem orandum s of Understanding (MoU) with GIZ and UNICEF to im prove the m onitoring of school-based WASH program s by custom izing m obile data collection tools. Ge n d e r d im e n sion s of t h e in t erven t ion 20. No specific gender analysis was conducted as baseline for this phase of program m ing, which lim its gender analysis of the program m e. However, WFP has shown increased com m itm em t to gender equality and em powerm ent in Cam bodia . In recent years the CO has undertaken several pieces of gender-related work in order to better understand the gender context. During this phase, additional gender-related work was done, including a review of gender in Cam bodia’s food security and nutrition policies,10 and a study into gender in household decision-m aking.11 The findings of these pieces of work should be incorporated into the design of the program m e and the section on evaluation findings describes whether gender dim ensions have im proved as a result. 1.1. Context 21. The m ost recent data from UNDP shows Cam bodia’s Hum an Developm ent Index12 (HDI) value of 0.582, ranking 146 out of 189 countries. Cam bodia attained lower￾m iddle -incom e country (LMIC) status in m id-2016, with a Gross Dom estic Product (GDP) per capita of USD 1,384.42 in 2017.13 Econom ic growth rem ains robust at 6.8 percent in 2017, and 6.9 percent in both 2018 and 2019. This is projected to accelerate slightly in 2020, buoyed by a rebound in textile and apparel exports as well as tourism and agriculture.14 Over the past two decades, Cam bodia has seen a significantly reduced poverty rate, dropping from 50 percent in 1992 to 47.8 percent in 2007, and then to 13.5 percent by 2014.15 However, although the official poverty rate has fallen dram atically, the poor have not risen very far above the poverty line. In 2017 the Multi-dim ensional Poverty Index (MPI)16 rem ained high 17 at 0.146 with 33 percent of the population affected.18 22. Ge n d e r: Despite progress in for gender equality in Cam bodia in the last two decades, wom en continue to be largely underpresented in politics and public decision m aking, tertiary education and form al em ploym ent. Traditional gender norm s and social attitudes continue favouring m en in both public and private spheres.19 Cam bodia is 10 WFP (2016) Gendered nature of intra-household decision-making in Cambodia. WFP Cambodia. Phnom Penh. 11 Brody, A. (2017) Innovations from the Field: Gender mainstreaming from the ground up. Cambodia Mission Report 12 http://hdr.undp.org/en/countries/profiles/KHM 13 World Bank. World Bank Open Data: http://data.worldbank.org/ 14https://www.worldbank.org/en/news/press-release/2018/05/10/high-growth-in-cambodia-expected-to-continue-world-bank 15 Ministry of Planning. Poverty Estimate in 2014 in Cambodia 16 Multi-dimensional poverty takes into consideration three dimensions: health, education and standard of living. 17 Asian Development Bank (2014) Cambodia Country Poverty Analysis. 18 The index identifies deprivations across the same three dimensions as the HDI and shows the number of people who are multidimensionally poor (suffering deprivations in 33% or more of the weighted indicators) and the number of weighted deprivations with which poor households typically contend with. 19 https://www.kh.undp.org/content/cambodia/en/home/projects/partnership-for-gender-equity-phase-iii.html ranked 116 out of the 160 countries on the m ost recent Gender Inequality Index20 (GII = 0.473). The GII is essentially the loss in hum an developm ent due to inequality between fem ale and m ale achievem ents in the three GII dim ensions: reproductive health, em powerm ent, and econom ic activity. Cam bodia’s low ranking indicates that gender inequality still exists. The 2017 Gender Developm ent Index (GDI) results are better at 0.914 which puts it into Group 4 (second lowest), an im provem ent from being in the lowest category in previous years.21,22,23,24 Although the num ber of wom en working in the private sector is higher than m en in m any provinces 25 largely due to em ploym ent in the garment sector, wom en are typically em ployed at lower levels and paid less. It is estim ated that on average wom en are paid thirty percent less than m en for com m ensurate work.26 23. Ed u ca t ion : Cam bodia has m ade good strides in im proving prim ary education program s and reducing gender disparity in education in rural areas. The net school enrolm ent rate (NER) during SY2015/16 was high - 98.4 percent (99.3 percent for girls), a significant im provement from 81 percent in 2001. Although there is still a need to expand enrolm ent in prim ary schools and pre -schools in som e locations, sustained efforts to globally expand access to school are less relevant than they once were. The m ain challenge now for prim ary school education is com pletion. Although both repetition and dropout rates have steadily declined in the last five years,27 they rem ain a key concern to MoEYS. School dropout is m ost problem atic at the end of the prim ary school cycle as students are m ore likely to leave school rather than repeat a year. School dropout is also m ore likely to happen in rural areas.28 Boys and girls face different challenges rem aining in school, with girls in rural areas dropping out m ainly due to severe poverty then end up caring for younger siblings, working alongside their parents in the rice fields, or m oving to urban centres to find work.29 Boys are also som etim es pressured to leave school and find em ploym ent. National literacy rates for children finishing Grade 2 are not available. Additional figures on som e key prim ary school indicators can be found in Annex 6. 24. Food se cu r it y a n d n u t rit ion : Despite the high econom ic growth and substantial poverty reductions in the last two decades, food security and undernutrition rem ain im portant public health concerns in Cam bodia. The national objectives set for the Cam bodia -specific Millennium Developm ent Goal (MDG) 1 were not m et 30 and the figures for m alnutrition rem ain higher than m ost countries in the region.31 The new SDG 20 http://hdr.undp.org/en/composite/GII 21 http://hdr.undp.org/sites/all/themes/hdr_theme/country-notes/KHM.pdf 22 Ratio of female to male HDI values. Gender Development Index scores range from 0 to 1 with a score of 1 indicating equality between men and women. 23 Gender Development Index groups: Countries are divided into five groups by absolute deviation from gender parity in HDI values. Group 5 comprises countries with low equality in HDI achievements between women and men (absolute deviation from gender parity of more than 10 percent). 24 Human Development Report,2015, UNDP 25 Commune Database 2013, Ministry of Planning 26 CSO report on Cambodian gender issues. 2009 27 Final Draft Education Strategic Plan 2019-2023, MoEYS, May 2019. 28 Heng, K. et al (2016) Research report. School Dropout in Cambodia: A case study of Phnom Penh and Kampong Speu. Korea International Cooperation Agency, Cambodia Country Office. Royal University of Phnom Penh, Faculty of Education 29 UNESCO/UNICEF (2012) Asia/Pacific: End of Decade Notes on Education for All – EFA Goal #5 Gender Equity. Bangkok: UNESCO and UNICEF. 30 Cambodia had an objective of reducing the prevalence of undernourished people to <10%. 31 https://opendevelopmentcambodia.net/topics/sdg-2-zero-hunger/ indicators covering undernourishm ent and dietary diversity, suggest that 14 percent of households continue to consum e less than the m inim um dietary energy requirem ent, while 11.6 percent have inadequate dietary diversity.32 A recent study com paring the nutritional status of school children in Cam bodia with those in the South East Asian Nutrition Survey33 (SEANUTS)34 found the urban/rural difference in Cam bodia to be sim ilar to the SEANUTS survey results. Stunting rates were high (33.2%) and m ore prevalent in rural areas, and in boys aged 13-15 years, and the overall dietary intake of surveyed children did not m eet the local recom m ended dietary allowances. 25. Sch ool fe e d in g: SF is a m ajor com ponent of WFP Cam bodia’s ongoing Country Strategic Plan (CSP) and is currently im plem ented in 8 out of 25 provinces.35 Recognizing the im portance of sustainability of the SF program m e, WFP’s overarching vision is to oversee the transition from a WFP-led SF program m e in Cam bodia to a governm ent-led program m e, as outlined in the School Feeding Roadm ap, agreed and signed between WFP and the MoEYS in May 2015.36 The transition tim eline has recently been revised from 2021 to 2025 to align with MoEYS’s Education Strategic Plan (ESP) and the National Social Protection Policy Fram ework (NSPPF). WFP’s strategy is to utilize USDA com m odities and capacity building funds to increase the readiness of MoEYS to adopt a nationally owned SF program m e. The Governm ent has already confirm ed the continued absorption of WFP’s food and cash take -hom e ration beneficiaries into the national scholarship program m e. To support this, since SY 2017/18, WFP has m ade som e strategic adjustm ents to the school m eals program m e and scaled up the use of the HGSF m odality. This m odel is m ore coherent with the governm ent’s preference for a program m e using local com m odities. The school-feeding capacity developm ent activities described in this report are im plem ented across the 11 WFP-supported provinces including the three with USDA program m ing.37 This is in view of the focus for the 2017-2019 phase of im plem entation on the developm ent of appropriate school feeding for national ownership by 2021. 26. Ot h er in t ern a t ion a l a ssist a n ce : All of the im plem enting partners for this program m e are engaged in other com m unity developm ent or literacy activities, both within the three McGovern-Dole targeted provinces and in other locations. Several other UN agencies work in Cam bodia including UNICEF, FAO, UNESCO, UNDP, UNWom en and WHO, each program m ing to their areas of expertise and their m andates, and in line with the United Nations Developm ent Assistance Fram ework (2019-2023)38, and the new UNDAF is currently under developm ent. The UNDAF focuses on three outcom es: inclusive growth and sustainable developm ent; social developm ent, social protection and hum an capital; and governance and hum an rights. WFP has been im plem enting program m es in Cam bodia since 1979, including school feeding activities since 1999. 32 Cambodia Socioeconomic Survey, 2014, National Institute of Statistics, Ministry of Planning; Available at: https://www.unicef.org/cambodia/6.Maternal.pdf 33 SEANUTS was conducted in Indonesia, Malaysia, Thailand and Vietnam in 2011. 34 Horiuchi, Y et al (2019) Urban-rural differences in nutritional status and dietary intakes of school-aged children in Cambodia. Nutrients. 2019 Jan; 11(1):14. 35 Kampong Chhnang, Pursat, Battambang, Banteay Meanchey, Oddar Meanchey, Siem Reap, Kampong Thom, Preah Vihear, and Stung Treng 36 MoEYS/WFP School Feeding Roadmap. Signed 28 April 2015. 37 WFP is operational in 11 provinces (including the 3 USDA provinces), with an overlap of activities in Siem Reap. 38 http://kh.one.un.org/content/unct/cam bodia/e n/hom e/what-we -do/undaf.htm l Currently, WFP Cam bodia im plem ents activities under the CSP (2019-2023)39 which highlights WFP’s shift from activity im plem entation to strengthening national capacities and building scalable program m e m odels. 1.2 Previous evaluation recommendations 27. The baseline and MTR found the McGovern-Dole program m e to be well im plem ented. The m ain concern was the short tim e fram e planned for the transition to national ownership. In addition, at baseline, the ET found that several USDA supported schools were lacking the full package of infrastructure (kitchens, energy efficient stoves, handwashing facilities etc). Both these areas were included as recom m endations at baseline and then m ore detailed recom m endations were m ade about the transition to national ownership during the MTR (Annex 7). In addition to objectives of the evaluation, the ET assessed whether the recomm endations from both the baseline and MTR have been im plem ented. 1.3 Stakeholder analysis 28. The evaluation included key inform ant interviews (KIIs) and focus group discussions (FGDs) with a range of internal and external stakeholders of the program m e. The ToR provided a com prehensive list of stakeholders for this endline evaluation, essentially the sam e list as developed at baseline. The stakeholders com e from m ultiple groups including WFP and MoEYS representatives (both governm ent and school personnel), other governm ent representatives, im plem enting partners, UN agencies, USDA and other donors. The detailed list of stakeholders and their role in the evaluation can be found in Annex 8. Since accountability to affected populations is a key WFP com m itm ent, the eva luation also included interviews with beneficiary groups, including school children, parents, cooks, teachers, and School Directors. Evaluation approach and m ethodology 1.2. Evaluation Methodology and Limitations 1.3.1 Overall evaluation approach 29. This evaluation uses the same theory-based, gender-responsive and participatory approach used during the 2017 baseline , i.e. an approach that ensures that each of the com ponents of the prorgram m e’s ToC - the USDA RFs - are evaluated, it will also m ainstream gender into the evaluation and assess whether gender dim ensions change as a result of an intervention, and ensures that wom en’s and girl’s voices are prom inent throughout the evaluation, and disaggregated quantitative data by sex where possible . The evaluation also ensures that a wide range of stakeholders are involved in the evaluation. 30. As per the evaluation ToR, the endline evaluation has used the internationally agreed OECD-DAC evaluation criteria 40 to evaluate the McGovern-Dole programm e. A detailed breakdown of the key evaluation questions can be found in Annex 9 and the 39https://docs.wfp.org/api/docum ents/WFP-0000101923/download/?_ga=2.9809142.212291492.1569797484- 633668376.1569186899 40 Relevance, effectiveness, efficiency, impact and sustainability detailed Evaluation Matrix used by the ET is in Annex 10. The planned transition to governm ent ownership m eans that this evaluation has a strong focus on the sustainability criteria to see whether the plans for governm ent ownership are feasible. Sim ilarly, the evaluation is interested in assessing the im pact of the McGovern-Dole programme to provide evidence to the governm ent that school feeding is a worthwhile investment. The four m ain evaluation questions are: How appropriate is the program m e? What are the results of the program m e? How and why has the program m e achieved its results? And how sustainable is the program m e? 31. From 2019, the United Nations System s-Wide Action Plan (UNSWAP) 2.041 is the gender accountability fram ework for all UN agencies. Since the McGovern-Dole programme hopes to achieve positive outcom es for school children, and girls in particular, several output variables are already m onitored by WFP in gender disaggregated form . This data, together with gender disaggregated outcom e data from the quantitative survey has enabled the ET to determ ine if the program m e resulted in different outcom es for girls and boys. The United Nations Evaluation Group (UNEG) guidance on gender 42 has also been used to shape the evaluation approach. 1.3.2 Evaluation Methodology 32. The evaluation was conducted during July-August 2019. The overall evaluation schedule and the detailed quantitative and qualitative field m ission schedules can be found in Annexes 11 -13. 33. The evaluation approach consists of two principal com ponents: qua ntita tive da t a collect ion and a qu a lit a t ive field m ission to analyse the achievem ents of the program m e. Both qualitative and quantitative data collection were conducted in all three provinces that received USDA support during 2017-2019. The evaluation design is quasi￾experim ental case -control (com parison) as per the 2017 baseline. The sam ple selection of schools was driven by criteria agreed WFP during the inception phase which included: geographic scope of the program m e (all three regions where the program m e is im plem ented), com parability of data (schools where only THR was distributed were excluded), control group (non-USDA schools) and se -disaggregated data available and gender- relevant indicators had been tracked. The evaluation m ethodology used m ixed m ethods, including secondary docum ent review, qualitative interviews and focus group discussions with a range of stakeholders, and a quantitative endline survey to enable com parison of results against the 2017 baseline. The evaluation also used existing program m e m onitoring data from WFP, WV, PLAN and WEI to triangulate qualitative evaluation findings, and to understand the overall program m e outputs. The qualitative and quantitative data collection was com plem ented with secon da ry docu m en t review including relevant project docum ents, assessm ents, WFP and governm ent policies and norm ative guidance, as well as the baseline survey and m id-term review reports. Detailed inform ation on the secondary data sources used can be found in Annex 14. The m ixed data collection m ethods selected and applied generated a considerable volum e of 41 UNWom en (2018) UN-SWAP 2.0 Fram ework and Technical Guidance: Accountability fram ework for m ainstream ing gender equality and the em powerm ent of wom en in United Nations entities. Novem ber 2018 42 http://www.uneval.org/document/detail/2133 prim ary and secondary data which enabled thorough triangulation of data from different m ethods and sources. The rationale for selection of each m ethod was as follows: Se con d a ry d ocu m e n t re vie w of relevant project docum ents, including assessm ent on which the program m e was designed, Governm ent policies and norm ative guidance, as well as the baseline survey and m id-term review reports. This m ethod was applied system atically to verify and validate prim ary data collection across the five evaluation crtiteria, and was particularly valuable regarding appropriatness/ relevance, coherence, com plem entarity and coordination. The qu a n t it a t ive e n dlin e su rve y has purposely followed the sam e m ethodology and will use the sam e tools as the 2017 baseline so that the results are directly com parable. The qualitative data elicited stakeholder perceptions that addressed all the criteria and the three m ain guiding questions, but was particularly valuable in regarding appropriatness/ relevance, com plem entarity, coherence and effectiveness. The qu a lit a t ive fie ld m ission exam ined the coherence and sustainability of the diverse school feeding activities and projects im plem ented by WFP Cam bodia under the MGD agreem ent for 2017-2019. The qualitative data collection also generated prim ary data that was used to triangulate the quantitative data m entioned above. The field m ission prioritized the collection of the qualitative inform ation from KIIs, FGDs, and observations during project and school visits. Qu a lit a t ive field m ission 34. The evaluation included key inform ant interviews and FGDs with stakeholders both in Cam bodia and outside Cam bodia.43 In Cam bodia, interviews were done in Phnom Penh and in all three of the target provinces, including in a sam ple of six USDA supported schools and districts. In total, the ET spoke with 277 key inform ants during the endline evaluation. During the baseline and the MTR, the ET visited at least two schools per province, and the sam e m ethodology was repeated for the endline (Table 1). Table 1: Number of USDA McGovern-Dole sites to be visited (qualitative field mission) Provin ce Ba tta m bang Sie m Re ap Ka m pong Thom In te rve n tion s HGSF-h ybrid SMP SMP + THR HGSF￾h ybrid SMP + THR USDA supported sch ools visite d Kouk Pou m School Sam rong School Pongror School Prasat Sa m loa n g School Kou k Nguon School Ch im e a k School District le ve l School Fe e ding Com m itte e s Koas Krala District Soutr Nikom District Prasat Ba kon g District Ka m pong Svay District Santuk District 35. In total, over the im plem entation period, the ET visited 20 different schools, spending at least two hours in each with several school stakeholders: School Directors and Deputy Directors, teachers, parents, school com m ittee m em bers, schoolchildren and cooks. The qualitative interviews also included interviews with district school feeding com m ittees (DSFC). For the endline, the ET visited six schools covering each of the m odalities as well as their accom panying DSFC (Table 1). Qu a n t it a t ive da t a collect ion 43 USDA Washington and Ho Chi Minh City, and representatives from the WFP Regional Bureau in Bangkok. 36. The quantitative survey was conducted by Indochina Research Ltd (IRL), the sam e local research com pany that conducted the 2017 baseline. The survey was conducted in the sam e sam ple of schools used at baseline, using the sam e questionnaires, and largely the sam e enum eration team . The original sam ple of USDA-supported, and com parison schools was constructed for the baseline survey in 2017 through Propensity Score Matching (PSM). This was purposely done to enable a better understanding of differences at endline (now) through difference -in-m eans test. The sam ple construction enables quantification of change in key outcom es at endline, to better attribute results to the presence or absence of the school feeding interventions. 37. USDA su p p ort e d sch ools: Selection of the USDA supported schools was done through random selection. Probability of cluster selection was not proportional to size so that sm all schools have equal chances of being included in the sam ple as bigger schools. Weighting was then applied on the household sam ple to m ake up for this. The intervention sam ple consists of schools in each of the three USDA supported provinces. The number of schools sam pled per province is proportional to the num ber of schools supported by the program m e in the provinces. Table 2 shows the locations of the sam pled schools and households for the quantitative survey, with the m ajority being located in SRP. In total, 160 schools 44 were surveyed as well as 958 households. The households are of 479 boys’ and 479 girl’s households. Table 2: Geographic breakdown of sampled schools and households for the quantitative survey Provin ce Tota l Batta m bang Sie m Re a p Ka m pong Th om Non -USDA supported areas USDA supporte d schools 5 49 32 0 86 Com parison Schools 42 0 0 33 75 House h olds (6 pe r sch ool) 30 292 192 444 958 38. The PSM process m atched som e com parison schools with m ultiple case schools which is why there are less com parison schools. The findings from the com parison schools were weighted to enable findings of 86 m atched pairs of schools. 39. Over the course of this phase of program m ing, the m odality of school m eals in som e schools has changed, as WFP began increasing the use of the HGSF-hybrid m odel in SY 2017/18. This m eans that not all schools in the sam ple are receiving the sam e intervention as at baseline. The ana lysis therefore com pare s all the SMP schools (both USDA com m odities and HGSF-hybrid) against SMP+THR schools as per baseline. Any significant differences or potential change in results due to changing m odalities are noted in the findings if evident. The full list of sam ple schools and interventions provided from baseline to present can be found in Annex 16. 40. Com p a rison sch ools: As per USDAs request at baseline, the ET selected com parison schools largely from non-USDA supported provinces and in areas where there were no school feeding activities funded by any donor. The ET was therefore unable to select schools from the im m ediately neighbouring districts due to the presence of 44 One school was not able to be surveyed due to flood. school feeding program m ing in those districts funded by other donors. The com parison districts were selected together with WFP personnel at baseline, who helped the ET identify locations where there were no SMP program s, funded by USDA or others. As a result, m ost com parison schools are outside the USDA-supported provinces except for BTB where m any non-supported schools were available within the province. The full list of surveyed schools and their locations can be found in Annexes 17 and 18. Annex 19 contains detailed inform ation on the m ethodology behind the initial selection of com parison schools. The 160 surveyed schools included over 40,000 students (48.6% girls), with an average of 236 students per school (Table 3). USDA supported schools were significantly sm aller than the com parison schools. Additional inform ation on the student m akeup of the sam pled schools can be found in Annex 20. Table 3: School mapping information in the school year (SY2018/19) # Childre n (6-11-ye a rs old ) in catchment area # Stu dents (6-11-ye a rs old ) # Stu de n ts (ove r 11 ye ars old ) Students per school Tota l Girls Tota l Girls Tota l Girls All USDA schools 17,704 8,694 16,804 8,192 3,413 1,518 235 SMP on ly^ 2,597 1,277 2,505 1,258 646 277 185 SMP + THR ^^ 15,107 7,417 14,299 6,934 2,767 1,241 247 Com p a rison schools 22,645 10,903 21,060 10,439 4,134 1,802 296 TOTAL students 40,349 19,597 37,864 18,631 7,547 3,320 Ave rage = 236 ^ including HGSF Hybrid, ^^ including HGSF-Hybrid+ THR 41. Su rve y re sp on d e n t s: The quantitative data collection was m ade up of a survey of sam pled schools including a questionnaire for the School Director, shorter questionnaire for teachers and an observation com ponent about the school infrastructure. For the teachers’ survey, up to three teachers were selected from each school when possible, ideally the teachers of Grades 2, 4 and 6. This is designed to capture outcom es of each of the three types of interventions, i.e. Grade 2 (SMP), Grade 4 (SMP + THR), and Grade 6 (THR). However, since som e schools have only one or two teachers in total, selection of teachers by grade was not always possible. A gender balance of interviewees was m aintained when possible. 42. In addition to the school survey, a questionnaire was done with six households connected to each school. The household interviews were conducted after the school interview and were prim arily done with a household head and the m other of school child sam ple. Within each selected school, lists of SMP, SMP + THR and THR beneficiaries with children in Grades 1-6 were provided to determ ine the six households eligible for inclusion in the survey. Using beneficiary lists, system atic random sam pling was conducted by program m e type to select children/ households for inclusion. At this stage, gender balance was ensured by selecting an equal num ber (three) of girls and boys for inclusion. 1.3.3 Data collection tools 43. The sch ool a ssessm ent includes basic inform ation on the com position and perform ance of the school (num ber of children enrolled, attending, availability of advanced teaching m aterials, literacy testing success rates, etc.) as well as specific inform ation on the school’s infrastructure (latrines, water sources, kitchens, school gardens, etc.). The prim ary respondent was the school’s principal or chief adm inistrator. The school survey was adapted from a WFP school survey instrum ent with slight m odifications. The questionnaire can be found in Annex 21. 44. The teacher assessm ent includes inform ation on student attentiveness in class, and short-term hunger. This short survey was carried out in all case and com parison schools and answered by prim ary school teachers to collect data on estim ated short-term hunger reduction and im proved student attentiveness. The questionnaire can be found in Annex 22. 45. The h ou seh old su rvey includes inform ation on the com position and education level of the household m em bers, household incom e and expenditure, food consum ption and coping strategies. It also includes inform ation on child health, parents’ involvem ent in the McGovern-Dole program m e, parental reasons for sending their children to school, and inform ation on som e of the effects of school feeding. The questionnaire can be found in Annex 23. 46. The qualitative data collection tools: the key inform ant interview guides and the focus group discussion guides can be found in Annexes 24 and 25. 1.3.4 Data collection, analysis and reporting 47. Quantitative data collection was perform ed from July 22 to August 10, 2019, and qualitative data collection from July 29 to August 17, 2019. The school and teacher assessm ents were collected through face -to-face interviews using a m anual paper-based form at and later entered into a CSPro database program me . The household survey was digitized in SurveytoGo, and the data was collected through a face -to-face interview. Both ways allowed for quality control. Prior to data collection, instrum ents were pre -tested with a num ber of schools and households around Phnom Penh. 48. A num ber of quality control procedures were applied. First, data collection 45 and data entry program s 46 were designed that allowed m anual data collection and identification of data entry errors. Second, during data collection, an IRL survey team leader and a supervisor accom panied the interviewers and a supervisor checked responses from call back from about 15 percent of the respondents. In addition, the IRL Project Manager, Fieldwork Manager and Fieldwork Supervisors also random ly observed directly about 15 percent of the fieldwork. On top of this, an external consultant engaged by the ET visited 16 schools in three provinces and teachers and households of school children associated with those schools to verify data collection and check the quality of data collection. 45 Using a tablet for household survey 46 For school and teacher surveys 49. Quantitative analysis of data was done by the KonTerra team using SPSS, SAS and Excel software wherever appropriate. After receiving the data from IRL, the data was cleaned and recoded for analysis. Descriptive statistics, as done at baseline were perform ed for case -control com parisons at endline as well as for baseline -endline com parisons. Wherever appropriate, difference in m eans tests were exam ined using independent t-test to com pare the m eans. Sim ilarly, a difference -in-difference (DID)47 approach using regression technique was used to exam ine the effect of treatm ent on several outcom es using baseline -endline com bined data. Da t a a va ila bilit y a n d relia bilit y 50. As part of the 2017 baseline evaluation, the ET assessed the data availability and verified various m onitoring data provided by WFP and partners during school visits to establish data reliability. When no data gaps were noted after the baseline qualitative field m ission, the ET used program m e m onitoring data from WFP and partners for the evaluation of literacy, and to assess program m e outputs against targets. The updated m onitoring data has been used again for this endline evaluation. Further verification of output m onitoring data wa s done by the ET during the qualitative field m ission and triangulated with the quantitative survey results. Data on the breakdown of the costs of im plem enting the McGovern-Dole program m e was difficult to access due to WFP Cam bodia’s pooled system of funds. This has reduced the ETs capacity to fully evaluate the efficiency of the program m e. 1.3.5 Ethics 51. Interviews were carried out in accordance with UNEG guidelines, notably to ensure that all key inform ants understood that their participation was voluntary, and that confidentiality would be respected. The field notes from the qualitative m ission are confidential and will not be turned over to public or private agencies. In addition, steps were taken to ensure that m en, wom en, boys and girls felt that interviews were conducted in appropriate locations so that they were able to freely express their views and concerns without fear of reprisal. Schools were notified in advance that they would be part of the survey, to give them tim e to pass the m essage to parents. Perm ission to interview children at school was sought from their teachers, and from parents during household interviews. Safeguarding principles were also adhered to, with children interviewed in groups, with no specific child required to answer any question if they didn’t volunteer an answer. Children were also interviewed outdoors in full view of other evaluation team m em bers and/or teachers, or interviewed inside with m ore than one team member present. 1.3.6 Limitations and risks 52. The potential risks and lim itations of the evaluation was identified during the previous endline/baseline of the current phase and m itigated where possible during the endline evaluation. These risks included schools closing early for the end of school year before data collection could be com pelted, disruptions to the travel schedule due to the rainy season, poor quality data collection on som e survey questions, and schools not 47 Est(Y) = β0 + β1*POST + β2*TR + β3*POST*TR + έ being aware of the evaluation visits. Mitigating actions included having regular com m unication with the CO throughout the evaluation period, using professional translators, seeking early perm ission from the MoEYS to undertake the surveys, using an experienced local research com pany, and using KonTerra personnel to random ly conduct survey spot checks on the quantitative data collection. Each of these m itigation m ethods were utilized again at endline so the evaluation has proceeded sm oothly. As described earlier, the evaluation was lim ited by the lack of available breakdown of the program m e spending, which reduced the ability to fully evaluate the efficient of the program m e. Lastly, the quantitative field work was affected by flooding of roads preventing IRL from surveying one com parison school in Preah Vihear. This reduced the com parison sam ple from 75 schools to 74. 2 Evaluation Findings 2.1 Evaluation Question 1: How appropriate is the programme? 53. The baseline for this phase was conducted at the sam e tim e as the endline of the previous phase 48. At that tim e, this ET established that the targeting of the programme was appropriate. In 2013, when the schools under evaluation were first selected, WFP used a three -step targeting approach: geographic, school selection then household selection/THR beneficiary selection. All the schools supported during the 2017-2019 phase were supported in the previous phase as well so for the endline, the ET have not repeated the assessm ent of the targeting process. 2.1.1 Are the activities and transfers appropriate to the needs of the target population and communities? 54. The evaluation of the previous phase also established that the range of activities were appropriate. The m ain issues that were highlighted during the baseline were that the m odality of food procurem ent followed by WFP (food im ported/donated from the US) wa s not sustainable once the governm ent took over the im plem entation of the program m e, and that food-based THR did not align with the governm ent’s preference to provide cash-based support. 55. Over the 2017-2019 period, WFP ha s m ade substantial effort to work with the MoEYS to find a m ore appropriate SF m odel for governm ent ownership. In 2017, USDA funding supported an MoEYS Feasibility Study on HGSF to help the governm ent determ ine appropriateness and feasibility. Afterwards, the governm ent declared its preference for the HGSF approach and since then WFP have im plem ented m ultiple m odalities based on successful piloting of various m odels since 2015. Interviews with Governm ent representatives from MoEYS indicated that WFP’s support to HGSF research, and their readiness to provide additional inform ation to help the governm ent m ake a decision on the m ost appropriate m odel, and how to transition, has been m uch appreciated and has guided the overall governm ent direction. Transitioning the USDA McGovern-Dole program m e from the traditional m odel (SMP) of using im ported food is 48 Dunn, S. et al (2017) Endline Evaluation of United States Department of Agriculture (USDA) McGovern-Dole Grant (MDG) Food for Education Programme for WFP Cambodia (2013-2016). Final report. The KonTerra Group. therefore an appropriate approach, as the governm ent is willing to take on a sim ilar m odel. Since SY 2018/19, the m ajority of USDA-supported schools are im plem enting the HGSF-hybrid m odel, supplem enting the USDA rice and oil by procuring anim al protein and vegetables from local suppliers through the governm ent procurem ent process using com petitive bidding. 56. During this phase, WFP has also appropriately handed over alm ost all its THR activities to MoEYS, who will continue to provide scholarships through a cash-based m odality. The evaluation found that all other aspects of the McGovern-Dole programme are appropriate for the context. In particular, the capacity strengthening of the governm ent to im plem ent school feeding activities, the literacy activities and the support to school infrastructure all cam e out positively during key inform ant interviews. The capacity strengthening activities are discussed in detail in Section 3.2.6 57. The 2017 Early Grade Reading Assessm ent (EGRA) conducted as the baseline 49 and the national learning assessm ents conducted by the MoEYS’s Education Quality Assurance Departm ent (EQAD)50 have shown that a large proportion of children in Cam bodia are not acquiring the basic proficiency of Khm er reading, writing and m ath skills, and that these difficulties start in early grades. As part of a package of interventions to address this, the Ed u ca t ion St ra t e gic Pla n (ESP) indicates that the MoEYS “will prioritize extra support interventions for students struggling with reading and writing in at least all primary grades” 51. While aim ing to im prove the literacy of school-aged children, the education objective of the McGovern-Dole program m e is fully appropriate and will rem ain aligned with national education policies in the com ing years.52 Putting increased em phasis on the achievem ent of literacy objectives during this phase was therefore appropriate. 58. The endline evaluation of the previous phase 53 found that not all USDA supported schools had a m inim um package of infrastructure needed to im plem ent a school m eals programme.54 It was therefore recom m ended that the current phase of program m ing continue with infrastructure support to ensure that all USDA-supported schools have sufficient infrastructure to im plem ent a school feeding program m e. The continuation of this activity was therefore appropriate, and the findings can be found ahead in Section 3. 2.1.2 Alignment with government policies and complementarity with other development actors 59. The baseline established that the McGovern-Dole program m e aligned well with governm ent policies and social protection direction during the program m e design phase. The endline has therefore assessed whether the governm ent’s direction still holds by discussing the issue with governm ent and other stakeholders and by reviewing new policies and strategies developed during the current phase . 49 Dunn, S. et al (2017) Baseline Report for WFP Cambodia and the USDA/McGovern Dole Food for Education Programme, WFP Cambodia, December 2017. The KonTerra Group 50 Grade Six Survey 2013: 2nd Draft, EQAD, MoEYS, March 2015. 51 Final Draft Education Strategic Plan 2019-2023, MoEYS, May 2019. 52 See Core Breakthrough Indicators (CBis) in ESP 2019-2023. 53 Dunn, S. et al (2017) Endline Evaluation of United States Department of Agriculture (USDA) McGovern-Dole Grant (MDG) Food for Education Programme for WFP Cambodia (2013-2016). Final report. The KonTerra Group. 54 The minimum package recommended during the 2013-2016 evaluation includes a kitchen, energy-efficient stove, sufficient utensils for meal preparation, easy access to sufficient clean water, a food store, handwashing facilities and soap. 60. At the highest strategic level, the governm ent’s long-term developm ent aim s are docum ented in Vision 2030, a roadm ap towards upper-m iddle -incom e status. Vision 2030 is operationalized through two overarching developm ent strategies: The Re ct a n gu la r St ra t e gy55 and the Na t ion a l St ra t e gic De ve lop m e n t Pla n (NSDP),56 which integrate the Sustainable Developm ent Goals (SDGs) into the developm ent priorities. Overall, the McGovern-Dole programme is aligned with all these docum ents and rem ains coherent with SDG 2 which seeks to “end hunger, achieve food security and improve nutrition and promote sustainable agriculture.” In addition, since school feeding program m es are generally m ulti-sectoral in nature, engaging with m ultiple m inistries on school feeding is therefore required and aligns with SDG 17: strengthening partnerships. The aim of im proving literacy of school children is also fully aligned with SDG 4: Ensure inclusive and equitable quality education and prom ote lifelong opportunities for all. Under the Rectangular Strategy the Ministry of Wom en’s Affairs (MoWA), in partnership with relevant m inistries/institutions and other partners, has developed the strategic plan Neary Rattanak IV (2014-2018) to continue the governm ent’s efforts to m eet SDG 5 and prom ote gender equality and the em powerm ent of wom en in Cam bodia. The strategic plan focuses on increasing participation of wom en in the econom y and their contribution to national goals. 61. The endline evaluation of the previous phase indicated that the McGovern-Dole programme was in line with the objectives of the Ed u ca t ion St ra t e gic Pla n (ESP) (2014- 2018) as it included a focus on strengthening basic education and quality, especially building reading and m athem atics skills during the early grades of prim ary education. These objectives have been em phasised again in the new ESP (2019-2023)57. The new ESP is also linked to the SDGs and includes a roadm ap defining five policy priorities including that all girls and boys have access to quality early education and free and equitable basic education. 58 Still on education, the McGovern-Dole’s literacy activities align well with the fram ework of the USAID education strategy for the period 2016-2021: All Ch ild re n Re a d in g Ca m b od ia (ACRC). The McGovern-Dole literacy partners, WEI and KAPE a re both part of the USAID project as well, so the programme activities on EGR are harm onized with other actors working in literacy (EGR) in Cam bodia such as UNESCO and Room t o Re a d . 62. WFP has a strong relationship with the RGC and is a key stakeholder in supporting governm ent’s policy direction, particularly in the areas of social protection, food and nutrition security and of course, school feeding. There are therefore several policies and strategies that have been developed with direct involvem ent from WFP. The Na t ion a l St ra t e gy for Food Se cu rit y a n d Nu t rit ion (NSFSN) (2019-2023)59 is one such policy, focusing on several areas relevant to school feeding including linking food security and nutrition to social protection to prom ote equity and access, nutrition-sensitive and safe food system s and im proved nutrition and food security through sm all-scale farm s. All 55 Royal Government of Cambodia (2017) Rectangular Strategy Phase IV, 2018-2023 56 Royal Government of Cambodia (2018) National Strategic Development Plan, 2019-2023 57 Royal Government of Cambodia (2018) Education Strategic Plan, 2019-2023 58 Primary and lower-secondary education 59 Royal Government of Cambodia (2018) National Strategy for Food Security and Nutrition, 2019-2023 WFP’s SF activities including the McGovern-Dole program m e will therefore contribute to the im plem entation of this strategy. 63. The Na t ion a l Socia l Prot e ct ion Policy Fra m e w ork (NSPPF) (2016-2025)60 aim s to harm onize and strengthen existing social protection schem es. The transition to a nationally owned school feeding program m e has been docum ented in the NSPPF and m entions SF as social assistance instrum ent. The NSPPF also includes the projected scale - up of scholarships and school m eals by 2021 to help achieve the governm ent’s developm ent objectives. Interivews with governm ent representatives from m ultiple m inistries indicates that WFP was actively working with the governm ent to ensure that school feeding was integrated into the NSPF, and to build the understanding of the governm ent that school feeding is a social safety net. This in turn, has led to increased governm ent interest in having a nationally owned school feeding program m e. In addition to the NSPPF, MoEYS and CARD, together with the Social Protection Secretariat drafted a dedicated HGSF Im p le m e n t a t ion Fra m e w ork. 61 The developm ent of the HGSF Im plem entation Fram ework creates an opportunity to develop m ore com prehensive linkages for school health and nutrition, and to fully integrate HGSF into the NSPPF. Representative s from the MoEYS and from CARD reported that WFP has been instrum ental in designing the new HGSF Guidelines for the MoEYS, with the Cam bodia HGSF Guidelines based on the WFP/FAO HGSF Re sou rce Fra m e w ork 62 and adapted for the Cam bodian context including aligning im plem entation processes with existing governm ent processes such as procurem ent. 64. The new Na t ion a l Policy on Sch ool He a lt h 63 was approved in April 2019 and provides guidance on the practical im plem entation for the prevention of com m unicable diseases. The policy directly focuses on school feeding, aim ing to strengthen safe food nutrition program in schools by encouraging students to take safe and healthy food choice s. The experience of the USDA McGovern-Dole program me has contributed significantly to the developm ent and im plem entation of this policy. The MoEYS provided a positive review of WFP’s involvem ent and support to the developm ent of this policy. Lastly, during this phase, WFP has been in discussion with the MoEYS on the need to have a national school feeding policy. This has now been agreed and the MoEYS is currently in the process of developing the policy with the support of WFP and other partners. 65. The McGovern-Dole program m e also aligns well with the work of other developm ent actors in Cam bodia . The new Un it e d Na t ion s De ve lop m e n t Assist a n ce Fra m e w or k (UNDAF) (2019-2023), 64 focuses on social protection, and nutrition to a ccelerate the successful im plem entation of the UNDAF and the achievem ent of the SDGs in Cam bodia.65 The programme aligns with outcom e 2 - Social Developm ent, Social Protection and Hum an Capital, which aim s to provide m ore people with quality social services. 60 Royal Government of Cambodia (2016) National Social Protection Policy Framework, 2016-2025 61 Unpublished. Work ongoing. 62 WFP, FAO et al (2018) Home-Grown School Feeding Resource Framework: Technical Document. Rome. 63 Royal Government of Cambodia (2018) National Policy on School Health, Ministry of Education, Youth and Sport 64 UNDP (2018) United Nations Development Assistance Framework 2019-2023, Cambodia. DRAFT 14 August 2018 65 The UNDAF has five priority areas: People – Expanding social opportunities; Prosperity – Expanding economic opportunities; Planet – Sustainable living; Peace – Participation and accountability; and Managing urbanization. 2.1.3 Are the activities and transfers aligned with WFP strategies, policies and normative guidance? 66. At baseline, the ET felt that there was a disconnect between the USDA McGovern￾Dole objective of im proving the quality of education (literacy) and the WFP school feeding objective of im proving food and nutrition security. The increased support to literacy over this phase through the partnership with WEI and KAPE has helped to better align these objectives. Further, the new WFP Corp ora t e Re su lt s Fr a m e w ork (CRF)66 has brought new outcom e indicators. In addition to the standard enrolm ent, attendance and graduation rates, the revised CRF recom m ends the use of a new outcom e indicator, which is an exact replication of the existing USDA SO1 outcom e indicator.67 The new CRF m akes clear that: “School feeding activities can be designed to improve literacy as part of a holistic approach to ensure that school-age children are physically, nutritionally and cognitively fit to succeed in school”. At endline, the ET found that although there is still a m isalignm ent between the objectives of the McGovern-Dole program m e in Cam bodia and the other WFP Cam bodia school feeding activities (non-USDA funded), the literacy objectives of McGovern-Dole and the new WFP CRF are m ore closely aligned. Literacy activities are also now well aligned with other actors work in Cam bodia and with the direction of MoEYS. 67. Aside from the above issue , the McGovern-Dole program m e aligns well with m ultiple WFP policies and strategies including the WFP St ra t e gic Plan (2017-2021)68 which focuses on achieving the SDGs, elim inating hunger and m alnutrition, working in partnership and im proving literacy, by direct im plem entation and/or by strengthening country response capacities. The programme is also a key part of WFP Cam bodia’s own CSP. The WFP Re vise d Sch ool Fe e d in g Policy (2013) 69 increased the alignm ent between school feeding, nutrition and social protection, focusing on five objectives including strengthening national capacity for school feeding. The work that WFP is doing in Cambodia with the MoEYS and other m inistries is very m uch in line with that areas of policy focus. Sim ilarly, the WFP Dra ft Nu t r it ion Policy (2017-2021)70 reaffirm s WFP’s com m itm ent to prioritizing nutrition, by prom oting healthy diets that m e et nutrient requirem ents. 68. The WFP Sa fe t y Ne t Policy (2012)71 outlines priorities and im plications for WFP engagem ent in providing technical support and practical expertise for safety nets, ensuring that food and nutrition security objectives are em bedded in safety nets and supporting governm ents in building system s of safety nets. This is very m uch in line with WFPs work in Cam bodia as the team supports the governm ent to recognise school feeding as a social assistance instrum ent. Lastly, the WFP Ge n d e r Policy (2015-2020)72 aim s to ensure that WFP’s program m es prom ote gender equality and wom en’s em powerm ent. The McGovern-Dole programme targets both girls and boys, with girls prioritized where gender differences in education still exist. 66 WFP (2018) Revised Corporate Results Framework (2017-2021), WFP/EB.2/2018/5-B/Rev.1. 67 Percentage of students who, by the end of two grades of primary schooling, demonstrate ability to read and understand grade level text. 68 WFP (2016) WFP Strategic Plan 2017-2021. WFP/EB.2/2016/4-A/1/Rev.2* 69 WFP (2013) Revised School Feeding Policy: Promoting innovation to achieve national ownership. WFP/EB.2/2013/4-C 70 WFP (2017) Nutrition Policy. WFP/EB.1/2017/4-C 71 WFP (2012) Update on WFP’s Safety Nets Policy: The role of food assistance in social protection. WFP/EB.A/2012/5-A 72 WFP (2014) WFP Gender Policy 2015-2020. 69. Overall, the evaluation found that the program m e is appropriate and that it aligns well with governm ent preferences (HGSF m odality), with WFP’s own policies and strategies, and with the work of other actors. No negative feedback was received from stakeholders on this aspect of the program m e. 2.2 Evaluation Question 2: What are the results of the programme? 70. The following section describes the results of the program m e . The evaluation surveys have resulted in a significant volum e of data, m uch of which is in the report annexes. The m ain body of the report describes som e of the key findings, pa rticularly areas which had changed significantly since baseline, or where there was a clear difference between USDA supported and com parison schools. The full table of program m e m onitoring indicators can be found in Annex 26. The following key is followed throughout the report: Target achieved (or within 5%) Target within 10% of being achieved Target not achieved 2.2.1 Achievement of Strategic Objective 1: Improved literacy (Early Grade Reading) 71. As described earlier, the USDA RF1 indicates that to achieve im proved literacy, children m ust attend school, be attentive in class, and teachers m ust provide quality literacy instruction. The following section follows USDA RF1 (Annex 3), describing the re sults WFP and partners have achieved during the im plem entation of the relevant activities and determ ines whether funding has been efficiently utilized to im plem ent effective activities to im prove children’s literacy, m easured through early grade reading levels. Result stream 1: Improved attendance 72. The USDA McGovern-Dole Results Fram ework encourages enrolm ent through cam paigns on the im portance of education, and the daily provision of a school m eal. Both awareness cam paigns and school m eals have been im plem ented in the USDA supported schools now for several years (som e since 2011). At baseline, 65 percent of parents from both USDA-supported and com parison schools reported that the m ain reason they send their children to school was just because it was close to their hom e. Less than half the surveyed parents 73 said that they sent their children to school to get an education. At endline, these figures have im proved considerably, with all parents (99.8-100%) from both USDA-supported and com parison schools reporting that the m ain reason they send their children to school was because of the good quality of education at that schools, along with good quality of school infrastructure, and their wish to im prove the career prospects of their children. This aligns well with the high rate of prim ary school enrolm ent. Table 4 shows that m ore than 90 percent of parents in USDA-supported schools can nam e at least three benefits of prim ary education. This is significantly greater than parents in com parison schools. 73 49.3% of surveyed parents from USDA supported schools and 39.4% of parents from comparison schools Table 4: Percentage of parents who can name at least three benefits of primary education (n=958) Ta rge t (%) All USDAsch ools (%)74 SMP on ly ^ 65 SMP + Com p a rison En dlin e (SY 2018/19) 85 90.5*** 94.0 89.6 83.6 Ba se lin e (SY 2016/17) 85 83 89 79 88 Source : Endline evaluation ^ including HGSF Hybrid, ^^ including HGSF-Hybrid+ THR *** p<=.001; ** p<=.01; * p<.05; 73. At baseline, the net enrolm ent rate for children 6-11 years in the surveyed schools was 87.6 percent. This has now increased to 97.5 percent (Table 5). No significant difference was found in total or girl’s enrolm ent rates in USDA and com parison schools.75 However, the MoEYS Education Managem ent Inform ation System (EMIS) data shows that the USDA supported districts had higher net enrolm ent rates than non-assisted districts (99.6 percent versus 95.2 percent).76 EMIS also indicates that 52 percent boys and 48 percent girls in Cam bodia attend prim ary school. This is the sam e proportion in urban and rural area.77 Table 5: Enrolment rate for children 6-11 years old in surveyed schools (SY 2018/19) Ne t e nrolm e nt rate (%) Tota l Girls All USDA sch ools 96.7 96.5 SMP on ly^ 96.5NS 98.4NS SMP + THR ^^ 96.7 96.0 Com parison schools 98.4 99.4 All su rve ye d sch ools 97.5 98 Base lin e (SY 2016/17) – All USDA su pporte d sch ools 87.6 91.4 Source : Endline evaluation *** t-test (two-tailed) p<=.001; ** p<=.01; * p<.05; 74. Once children are enrolled, it is im portant to keep them in school until at least the end of prim ary education. According to Cam bodia’s new ESP, drop out is of key concern, especially for over-aged children abandoning school. This is com pounded by difficulties in reading and writing com prehension, which increasingly affects student’s ability in all other subjects at the higher grades. A lack of higher grades in som e schools also contributes to school dropout rates as children are unable to continue their studies without m oving to other areas. Table 6 shows that the endline results are sim ilar to the findings at baseline in that there is no statistically significant difference in prom otion rates, repetition rates and dropout rates between USDA supported and com parison schools.78 74 # independent sample t-test (two-tailed) for difference in means of USDA supported schools and comparison schools (significant) as well as between SMP and SMP + THR schools (not significant), p<=.001; ** p<=.01; * p<.05 75 T-test (two-tailed) for independent samples resulted in no significant difference between USDA-supported schools and comparison schools and also between SMP and SMP + THR schools. 76 MoEYS EMIS data SY 2017/18 and SY 2018/19 77 Terms of Reference, Activity Evaluation of USDA McGovern-Dole Grants FFE for WFP School Feeding in Cambodia from 2016 to 2019, WFP Cambodia Country Office. 78 Overall, t-test (two-tailed) for independent samples resulted in no significant difference between USDA and non-USDA schools. Significant differences were observed in total and girl’s dropout rates between SMP only and SMP+THR schools only. Table 6: Promotion, repetition and drop-out rates (SY 2018/19) Prom otion rate (%) Re pe tition rate (%) Drop-out rate (%) Tota l Girls Tota l Girls Tota l Girls All USDAsch ools 84.4 88.1 11.8 8.9 3.7 3.0 SMP on ly ^ 80.8 85.1 12.4 9.8 6.8** 5.0* SMP + THR^^ 85.3 88.8 11.7 8.7 3.0 3.0 Com parison schools 84.9 88.2 11.5 8.6 3.6 3.2 Base lin e (SY 2016/17) – All USDA supporte d schools 85.2 88.8 10.7 7.9 6.1 4.8 Source : Endline evaluation ^ including HGSF Hybrid, ^^ including HGSF-Hybrid+ THR *** p<=.001; ** p<=.01; * p<.05 75. This is also supported by the results of difference -in-difference regression results (not shown), which are statistically not significant, suggesting no im pact of the McGovern￾Dole program m e during this phase on these indicators. However, the MoEYS EMIS data which includes all schools in the country indicates that dropout rate was lower in USDA￾supported schools (4.9 percent vs. 6.0 percent) and com pletion rate was higher in USDA￾supported schools (81.3 percent vs. 80.1 percent). Girls in both types of schools perform ed better than boys, with m ore girls being prom oted, and less girls repeating grades or dropping out of school. 76. EMIS data also indicates that the retention rate in USDA-supported schools was alm ost ten percent higher than the target for 2018.79 These results are particularly com m endable since the schools participating in the m eals program m e are generally located in m ore rem ote, vulnerable districts and initially showed poorer education perform ance than non-supported schools. WFP m onitoring data indicates the number of children regularly attending school80 has been m ore than targeted throughout this phase of program m ing (Table 7). Table 7: Regular attendance rate SY 2016/17 – SY 2018/19 Ta rge t Baseline MTR En dlin e Nu m be r of stu de n ts re gu larly (80%) atte n ding USDA supporte d sch ools 127,600 195,400 167,600 192,723 Achie ve m e n t towards ta rge t e ach school ye ar 153% 131% 151% Source : WFP Monitoring data 2.2.2 Improved school infrastructure 77. The McGovern-Dole program m e includes support to schools to construct or rehabilitate key school feeding-related infrastructure: kitchens, food storeroom s, and energy-efficient stoves. The endline evaluation of the previous phase 81 recommended that WFP and partners continue this work to ensure that all the USDA supported schools had adequate infrastructure when the governm ent took over the program m e. The results 79 WFP (2018) WFP Cambodia Annual Country Report 2018. 80 Attending class more than 80% of school days 81 Dunn, S. et al (2017) Endline Evaluation of United States Department of Agriculture (USDA) McGovern-Dole Grant (MDG) Food for Education Programme for WFP Cambodia (2013-2016). Final report. The KonTerra Group. found now at endline are therefore a culm ination of the work done over m ultiple programme phases. 78. The percentage of USDA-supported schools with the required infrastructure has increased since baseline. This is especially noted in the increased percentage of schools using energy-efficient stoves (Table 8), and a significant im provem ent in school storing food off the ground.82 The CO also put in considerable effort during this period to rehabilitate school kitchens. Interviews with school personnel, parents and MoEYS all provided positive feedback on the infrastructure developm ent work done through this project. Water and sanitation infrastructure (water, latrines and handwashing facilities) are discussed under SO2. Table 8: Summary of available school feeding-related infrastructure USDA sch ools (%) Com p arison sch ools (%) Baseline (SY 2016/17) En dlin e (SY 2018/19) En dlin e Kitch e n 95 98 1.2 (n =1) Food store room 74.4 70.5 0 Food store d off the ground 65.6*** 88 0 Ene rgy e fficie nt stove s 60.5 72.1 0 Source : Endline evaluation Note: Only one com parison school has a kitchen. The school reported that the school lacks kitchen utensils and leaking roof. *** p<=.001; ** p<=.01; * p<.05; 79. Two issues were raised by evaluation stakeholders about the infrastructure support to schools. Firstly, there is a lack of guidance on school infrastructure construction, which results in lim ited uniform ity in school infrastructure construction, particularly on kitchens. WFP is aware of this issue and have included kitchen construction guidance in the new HGSF Guidelines. Secondly, som e stakeholders are concerned that the MoEYS does not provide schools with sufficient school budget for infrastructure m aintenance and regular provision of school feeding related m aterials such as kitchen equipm ent. The issue of infrastructure m aintenance was assessed during the school surveys with enum erators observing the condition of kitchens, food stores, and latrines. The survey found that 57 percent of USDA supported schools lacked kitchen utensils, and som e kitchens and stoves require m inor repairs. In general, however, the endline evaluation found that existing school infrastructure in USDA-supported schools to be well constructed and well m aintained. Additional inform ation on the school infrastructure including the condition at the tim e of the endline survey, can be found in Annex 27. 2.2.3 Increased economic and cultural incentives 80. To help im prove access, attendance, retention and com pletion of prim ary education of poor students in schools, and other vulnerable, m arginalized and disadvantaged children, the MoEYS launched a prim ary scholarship program m e in 82 Overall, t-test for independent samples between baseline and endline resulted in no significant difference between USDA-supported schools and comparison schools except the storing off the ground food (where pallets have been used for food storage). SY2012/13, with external support from the Global Partnership for Education (GPE) and the WFP. 81. Food scholarships (THR) of 10kgs of rice and 0.91kg of vegetable oil per student per m onth were provided three tim es per year as part of the McGovern-Dole programme to help students facing econom ic or cultural obstacles to attend school and prevent dropout. Since 2015, the THR has been gradually integrated into the MoEYS’s national cash scholarship program m e 83 During 2017-2019, WFP’s THR activities have been progressively handed over to the MoEYS Scholarship Office and the MoEYS is now fully operating the scholarships in the USDA supported provinces. The prim ary scholarship program m e is exclusively funded by the national budget and fully institutionalized. 84 During 2017-2019 the scholarship office provided scholarships to vulnerable boys and girls in Grades 4-6 to enable sm ooth transition to the existing governm ent scholarships for lower secondary and then secondary grades. In 2019, the MoEYS approved the expansion of the scholarship program m e to Grades 1-3 so from SY 2019/20 the Scholarship Office will provide cash-based scholarships to vulnerable boys and girls from Grade 1-12. Overall, WFP provided m ore than 28,000 children with THR during this period, 128 percent of the planned target (Table 9). Table 9: Provision of THR by WFP (2017-2019) Ta rge t Ba se lin e Midlin e En dlin e Achie ve m e n t Nu m be r of in dividia u ls re ce ivin g THR a s a re su lt of USDA assistan ce 22,500 0 15,004 28,855 128% Source : WFP Monitoring data 82. A recent evaluation of the scholarships provided by the MoEYS85 found that there is clear evidence that the scholarships contributed to recipients staying in school longer. The evaluation also found that there is clear evidence that scholarships were spent on educational m aterial or expenses, or food. Result stream 2: Improved attentiveness 83. Once children have enrolled in school, the school environm ent m ust be such that they can concentrate on their studies. The provision of a daily school m eal therefore plays a key role in ensuring that all children eat before class, enabling them to concentrate. 2.2.4 Increased access to food 84. In total, the program m e planned to provide approxim atively 5,900 m etric tons (MT) of fortified rice and 330 MT of vegetable oil over the three -year period. During 2017- 2019 WFP received a total of 6,455.3 MT of USDA rice and oil for the McGovern-Dole program m e (Table 10) which was slightly m ore than planned due to cheaper prices of com m odities at the tim e of purcha se . As m ost of the USDA com m odities have been utilized and distributed to schools during the im plem entation period as planned, around 83 In 2019, the government approved cash scholarship from Grade 1 to 3. Grade 1 to 9 will get 60 USD per year and Grade 10 to 12 will get 90 USD per year. 84 Royal Government of Cambodia (2019) Equity-focused Primary Scholarship Framework 2019-2023, MoEYS, May 2019. 85 MacAuslan, I. et al (2019) Country-led evaluation of the National Education Scholarship Programmes of the Ministry of Education, Youth and Sport in Cambodia, 2015-2018. Final Report – Volume 1. UNICEF, Phnom Penh 200MT of food, m ainly oil, rem ains at school level to be utilized next school year. USDA has agreed to an NCE until 31 March 2020 to use the rem aining com m odities. Table 10: Planned vs. actual USDA food commodities distributed (MT) Planne d Re ce ive d Actu a l Achie ve m e n t Rice 5,900 6,047.6 5,865.31 99.4% Oil 330 397.77 372.17 113% TOTAL 6,230 MT 6445.4MT 6,237.5MT 100% Source : WFP Supply Chain data 85. These attendance variances also m ean that WFP has not m et their target for the num ber of school m eals provided (Table 11). WFP output m onitoring data indicates that since SY 2016/17 more than 150,000 children have received daily school m eals each year. This is a total of over 32 m illion school m eals during 2017-2019. Table 11: Provision of daily school meals 2017-2019 Ta rge t Ba se lin e MTR En dlin e Achie ve m e n t Nu m be r of daily school m e als (bre a kfast, sn ack, lunch) provide d as a re sult of USDA assistan ce 38,397,00 0 0 14,887,00 0 32,895,84 2 85.7% Num ber of school-age ch ildre n re ce ivin g da ily sch ool m e als (bre akfast, snack, lunch) as a re sult of USDA assistan ce ) 189,600 0 150,570 175,397 92.5% Source : WFP Monitoring data 86. The provision of the school m eal at school level is well im plem ented and managed. Interviews with school children indicate they are happy with the taste of the m eal, and they enjoy com ing to school to eat breakfast with their friends. Many children indicated they also had food at hom e before com ing to schools, so the m eal at schools appears to be additional food for m any children while for som e it is there only breakfast. “I eat at home but when I come to school and smell food, I feel hungry again.” “The food at school is better than a t home because it is mixed with vegetable and meat like lunch.”86 87. Table 12 shows the teacher’s estim ations of the num ber of school children who are hungry in class. In USDA-supported schools, 6.4 percent of children were reported to be hungry in class on a daily basis, com pared to 13.7 percent in com parison schools. Sim ilarly, in total, 14.4 percent of children in USDA-supported schools were reported to be hungry (daily or som etim es), com pared to 26.6 percent in com parison schools. Both of these differences are statistically significant, indicating the positive role that the school m eal has in reducing hunger. The survey also found a statistically significant reduction in girls reporting hunger in class – from 20.3 percent at baseline, to 14.1 percent at endline. 86 Quotes from interviews with school children during endline evaluation Table 12: Percent of students in target schools who indicate that they are “hungry” or “very hungry” during the school day Ta rge t All USDA schools SMP on ly^ SMP + THR ^^ Com p a rison schools Percentage of total students reported to be hungry in cla ss Da ily ≤10% 6.4*** 2.5** 7.4 13.7 Som e tim e s 8.0*** 8.0NS 8.2 13.0 Tota l ≤10% 14.4*** 10.7NS 15.3 26.6 Girls 14.1*** 9.6NS 15.2 26.3 Base line (SY 2016/17) Tot al ≤10% 20.9 20.8 21.1 34.4 Girls 20.3 19 22.8 31.4 Source : Endline evaluation ^ including HGSF Hybrid, ^^ including HGSF-Hybrid+ THR *** t-test (two tailed) p<=.001; ** p<=.01; * p<.05; NS Statistically not significant 88. The survey with teachers also found a reduction in inattentiveness of students in USDA supported schools, from 11.3 percent at baseline, to 9 percent at endline (Table 13). Table 13: Reported inattentiveness of students Ta rge t All USDA SMP on ly^ SMP + THR ^^ Com p a rison schools Percentage of ch ildre n who are in a tte n tive (sle e py, in a ctive ) during classe s Eve ryda y ≤10% 2.8** 2.8 6.0 4.7 Som e tim e s 6.2*** 5.6 5.0 8.7 Tota l 9.0*** 8.7 9.1 13.3 Girls 5.4** 7.2 5.0 8.0 Baseline (SY 2016/17) Tot a l ≤10% 11.3 11.6 10.8 14.8 Girls 7.4 7.2 7.8 9.4 Source : Endline evaluation ^ including HGSF Hybrid, ^^ including HGSF-Hybrid+ THR *** t-test (two tailed) p<=.001; ** p<=.01; * p<.05; 89. Teachers in com parison schools estim ated 13.3 percent inattentiveness at endline. Overall, the percentage of inattentive students were significantly lower in USDA￾supported schools com pared with com parison schools.87 In addition, the survey found that total inattentiveness in class is significantly positively correlated with total percentage of students reported hungry in class. 88 90. Teachers were asked to report the m onths in which they felt that hunger was an issue. In both USDA-supported and com parison schools, the period May-July was nom inated as having with highest rates of hunger, with an estim ate of 20 percent of children or m ore reported to be hungry (Figure 1). This is the rice planting season, before the harvest. Teachers in com parison schools also reported high rates of hunger during January and March, when USDA supported schools did not. Further, teachers in both USDA-supported (57.3%) and com parison schools (73.3%) reported that children in the 87 In comparison schools, teachers estimated 13.3% inattentiveness compared to 9% inattentive in USDA supported schools. No significant difference was found between SMP only^ and SMP+THR^^ schools. 88 Correlation (r)=0.378 – a medium correlation, p<0.001 (two tailed) m orning shift 89 are m ore likely to be hungry when they com e to school as they often go to school without eating at hom e. This is understandable for the USDA-supported schools since parents know that their children will receive a m eal at school. Children in the afternoon shift have usually eaten either/ both breakfast and lunch at hom e before com ing to school and therefore do not benefit from WFP school feeding. Children rotate which shft they attend on a m onthly basis – one m onth attending m orning shift, the next month in afternoon shift. Data from the household survey indicates that 83 percent of parents from com parison schools reported that their children usually eat breakfast. Figure 1: Percentage of teachers reporting child hunger and inattentiveness in USDA supported schools Source : Endline evaluation Result stream 3: Improved quality of literacy instruction 91. Results stream s 1 and 2 provide the supportive environm ent to m ake schools a safe and conducive environm ent for learning. The third result stream under this SO directly supports im proved literacy of school-age children but im proving the quality of literacy instruction in schools. To achieve im provem ent in literacy rates, significant effort is needed from the MoEYS through supporting teachers to learn and im plem ent quality literacy instruction. Since literacy is not a core area of expertise for WFP, in SY2016/17 they established a partnership with WEI and KAPE to ensure achievem ent of RF1. Literacy activities were im plem ented in USDA-supported schools in SRP and KTM only because schools in BTB was already alm ost handed over the governm ent. The MTR noted this partnership resulted in rapid progress in the literacy arena in both SRP and KTM. 92. National Assessm ent System s subject tests of Grades 3, 6, 8 in Khm er language and Math were instituted by MoEYS in 2006 and later followed in 2010. The 2010 EGRA conducted for 24,000 students in grade 1-6 found that 33 percent could not read at all, while 47 percent could not com prehend what they had read 90. In response to these low national literacy scores am ong prim ary education children, the MoEYS developed new textbooks for Grades 1-3. As part of the im plem entation of the McGovern-Dole programme , WEI has supported MoEYS in its im plem entation of the new curricula and contributed to national efforts to im prove EGR outcom es. Along with this new text book 89 Double -shift teaching is a strategy used to address the shortage of teachers. Teachers teach one group in the m orning and another in the afternoon. 90 World Bank (2012) Implementation Completion and Results Reports: Cambodia Education Sector Support Project. 0 20 40 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Inattentiveness Hunger and curricula for Grades 1-3, WEI also designed an integrated approach to EGR instructions that includes the developm ent of reading benchm arks; literacy coaches; and a Rapid Response System consisting of parental engagem ent, reading gam es, m aterials, and an m Learning app in Khm er that is connected to the national curriculum . According to WEI, students have developed better reading skills and the MoEYS has approved and adopted the m odel, charting a clear path for replication at a far greater scale. 93. Activities developed by WEI and KAPE in cooperation with staff from relevant departm ents within MoEYS (notably Curriculum , Teacher Training, Education Quality Assurance departm ents) include a school-based Teacher Professional Developm ent strategy to help teachers apply EGR techniques into their classroom practice and to engage the school m anagem ent and leadership capacities of school directors in the process. Using USDA funds in support of the new MoEYS national EGR package WEI and KAPE trainers, Provincial Teacher Training Colleges (PTTC) staff, District Training and Monitoring Team s (DTMT) trained under the McGovern-Dole program m e , and Provincial Departm ent of Education (PDoEs) staff were actively engaged in both training of trainers and trainings with Grade 1 teachers and schools directors across Kam pong Thom and Siem Reap provinces. Through applying new skills and knowledge, trained teachers and school directors were expected to enable significant im provem ents in children's reading skills in the early grades. Table 14 shows that WEI and KAPE have achieved all their output training targets for this phase of the program m e. Table 14: Literacy training activities for school administrators and teachers Ta rge t Ka m pong Th om Sie m Re a p Tota l Achie ve m e n t Nu m be r of school a dm inistrators a nd officia ls tra in e d or ce rtifie d a s a re su lt of USDA assistan ce 591 271 443 714 121% Nu m be r of school a dm inistrators in targe te d schools who de m onstra te use of ne w te chnique s or tools 413 198 286 484 118% Nu m be r of te ache rs/e ducators/te a ching assistan ts train e d or ce rtifie d as a re su lt of USDA assistan ce 2,200 1,115 2,097 3,212 146% Nu m be r of te ache rs/e ducators/te a ching assistan ts in targe t sch ools wh o de m onstra te use of ne w an d quality, te aching te chnique s or tools 1,540 681 1,018 1,699 110% Source : WEI Quarterly Report - Year 3 Quarter 3: April 1, 2019 – June 31, 2019. 94. In addition, WEI developed a set of m aterials to address student’s low skills in com prehension and reading fluency. These teaching and learning m aterials have been provided to all target schools during the second year of im plem entation of the program m e (Table 15). Teachers received additional teaching m aterials including colour paper, flip charts, m arkers, and tape the following ye ar. These item s enable teachers to m ake additional learning m aterials and gam es for their students. Table 15 shows that a significantly greater proportion of USDA-supported schools received learning m aterials and stationery m aterials is significantly highe r than in com parison schools during SY 2018/1991. Table 15: Number of schools receiving learning materials and stationery packages (SY 2018/19) # of sch ools re ce ive d le a rn in g package # of sch ools re ce ive d sta tion e ry m a te ria ls % of sch ools re ce ive d le a rn in g package % of school re ce ive d station e ry Ye s No Ye s No m a te ria ls All USDAsch ools 85 1 85 1 98.8* 98.8** SMP on ly^ 16 1 17 0 94.1 100.0 SMP+THR ^^ 69 0 68 1 100.0 98.6 Com p a rison sch o o ls 77 8 76 9 90.6 89.4 Source : Endline evaluation ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR *** t-test (two tailed) p<=.001; ** p<=.01; * p<.05 95. As part of activities im plem ented to prom ote literacy the program m e used popular social m edia and network platform s, such as Facebook and Telegram , to dissem inate resources, reinforce key m essages, send rem inders and channel targeted content to participants and stakeholders. During the 2018 WEI created a Facebook group called Khm er Literacy Resources with the aim of dissem inating Khm er literacy resources, to m otivate teachers and program m e participants, to build an online com m unity of practice, to provide a platform of exchange and support between participants and trainers. WEI have provided teachers who cannot access online content with Micro SD cards with all the program m e’s digital resources. The card also includes SmartBooks, an application created by KAPE, which allows students access to electronic books. 96. WEI’s Aan Khmer m obile app is an educational gam e for Grade 1 and Grade 2 students based on the five key reading skills: Phonem ic Awareness, Alphabet Principle, Reading Fluency, Vocabulary and Reading Com prehension. The contents have been adapted from MoEYS textbook. This app was recently upgraded to function better on sm artphones and is already available for free download on Google Play. 97. WEI plans to leverage the Let’s Read! Digital Library app developed by The Asia Foundation. The app contains over 200 stories and books in Khm er which can be accessed at no cost. Children can access the additional learning content through a parental m obile device or m obile device operated by the school; this content can help children im prove their reading com prehension skills. 98. In addition to equipping schools and trainers with m aterials, WEI cooperated with Plan International to provide key m essages on how parents and com m unity m em bers can take a m ore active role in their children’s literacy developm ent. These m essages were dissem inated during Food Events sessions developed by Plan International. 99. The ET noted som e constraints ham pering the scope of WEI’s intervention. Firstly, cooperation with staff from the national EQAD was lim ited as they were invited m ore as observers than as active participants into the EGRA conducted by WEI. Also, during field 91 t-test (two-tailed) for independent samples show that the percentage of USDA supported schools that reported receiving learning package and stationary materials was significantly greater than those in comparison schools. No statistically significant difference was found between SMP only and SMP+THR schools. visits, som e school directors stressed that the im plem entation of the McGovern-Dole program m e in their school created new and additional responsibilities and tasks for them . Many felt overworked for the im plem entation of the program m e , and this was confirm ed by WEI who found there was lim ited com m itm ent of School Directors to do regular teacher literacy coaching follow-up. 100. During the last quarter of SY2018/19, 517 teacher visits were carried out in USDA supported schools by team s com posed of WEI trainers, DTMTs/PDoEs and PTTCs staff. Only 21 School Directors joined the team s to observe teachers teaching in the classroom and filled their digital teacher observation 92. WEI underlined that this trend has been prevailing throughout the three years of program m e im plem entation. This drawback appears m ore as a sym ptom of a system issue than specific to USDA McGovern-Dole im plem entation: a previous World Bank-com m issioned study had already revealed a teacher training system tha t is not very dynam ic. The study highlighted som e key issues, notably “very little observation, support and supervision of trainer activities in classes by school directors and personnel from MoEYS department” 93 101. Lastly, high turnover of trained personnel requires regular follow up/coaching and training of new staff. Significant flux in the Grades 1-3 teaching staff at target schools has resulted in m ore teachers having been trained than initially planned, as shown in Table 14. 2.2.5 Overall achievement of Strategic Objective 1: Improved literacy of school-age children 102. The m ain objective of the program m e (SO1) is to increase the num ber of children who at the end of Grade 2 are able to read grade level text. During 2017-2019, WEI and KAPE conducted three EGRA in USDA supported schools. The baseline EGRA surveyed children already studying in Grade 3 (instead of Grade 2). For this reason, it has been agreed not to use it for com parison with the present endline. Table 16 shows that at endline 30 percent of children at the end of Grade 2 are able to read and understand grade level text. Table 16: Demonstrated reading and comprehension - Grade 294 Target Ba se lin e MTR En dlin e Percent of students who, by the end of two grades of prim ary schools, dem onstrate that they can read and understand the m eaning of grade level text. 55% N/A95 23.8% 30% Source : WEI Quarterly Report - Year 3 Quarter 3: April 1, 2019 – June 31, 2019. 103. The previous phase of the McGovern-Dole program m e (FY2013-2016) focused on literacy of Grade 6 students while the present phase focused on EGR (Grades 1-3) to align with the SY2016/17 introduction of new National Reading Standards (benchm arks) and associated assessm ent workbooks for Grades 1 and 2, and Guidelines for Grade 3. The 92 World Education (2019) World Education Quarterly Report to WFP Cambodia, Year 3 Quarter 3: April 1, 2019 – June 31, 2019. 93 World Bank (2012) Implementation Completion and Results Reports: Cambodia Education Sector Support Project, 2012. 94 Comprehension is the purpose of reading. Once a child learns the sound-letter relationship (alphabetic principle) and becomes able to decode and read with automaticity, he or she becomes increasingly able to understand the meaning of a text. 95 A 16.1 percent score at baseline has been communicated to the ET. For methodological reason, this figure cannot be used for comparison. Nevertheless, this percentage provides an indication about the level of performance at the starting point of the programme. EGR rates reported in the previous phase are therefore not com parable to the present figures. 104. The result presented in Table 16 show a strong im provem ent over a short period of tim e. However, the im provem ent does not com e close to m eeting the program m e target of 55 percent. The overam bitious target was already identified by the ET during the MTR because of the lim ited human resource capacity of WEI and the short tim e fram e. The level of im provem ent is encouraging and shows the high capacity of WEI and KAPE. The ET note however that although there is a correlation between outcom es and results, establishing a causality is out of reach for the present evaluation. This is m ainly because the WEI baseline lacks a com parison group from non-USDA-supported schools. 105. Overall, the im plem entation of literacy related activities and the activities that contribute to im proved literacy have been im plem ented very efficiently, with a sm all num ber of WEI staff collaborating with the MoEYS at all levels to ensure that teacher training was com pleted with the appropriate m aterials, and then followed up in class. The results indicate that this is both an e fficient and effective way to support im provem ents in literacy. 2.2.6 Achievement of Strategic Objective 2: Improved health and dietary practices 106. This section follows USDA RF2 (Annex 3), describing the results WFP and partners have achieved during the im plem entation of the relevant activities. The McGovern-Dole programme includes several activities to im prove the health and dietary practices of school children, parents and school personnel including training, school garden construction/ rehabilitation, provision of clean water, latrines and handwashing facilitie s and ensuring that children in schools receive deworm ing treatm ent from the MoH. All of these activities contribute to children being healthy. This phase, as the program m e transitions to the HGSF m odel, increasing the dietary diversity of the school m eals, the program m e has also incorporated additional nutrition and food safety instruction. At MTR the ET recom m ended that the school m eals/ HGSF guidelines be revised and ready for use if/when the MoEYS HGSF proposal is funded. Since then, the CO revised the guidelines and they were endorsed by the MoEYS in March 2019. Im p rove d kn ow le d ge of h e a lt h , h ygie n e , n u t rit ion , sa fe food p re p a ra t ion a n d st ora ge pra ct ices 107. Table 17 shows that the McGovern-Dole programme has supported more than 4,000 individuals, including teachers, School Directors, parents and cooks by providing training in child health and nutrition. This is supported by Figure 2 showing that a quarter of all households surveyed reported that NGOs (m ainly Plan and WV) were their m ain source of inform ation on good health practices. Table 17: Number of people trained in child health and nutrition (2017-2019) Ta rge t Ba se lin e MTR En dlin e Achie ve m e n t Nu m be r of in dividu a ls tra in e d in child he alth and nu trition as a re sult of USDA a ssista n ce 3,600 0 2,370 4,392 122% Source : WFP Monitoring data Figure 2: Main source of information on good health practices (% of households) Source : Endline evaluation 108. In addition to the nutrition training provided for school personnel, school children and parents, WFP has supported School Nutrition Days, which provide an opportunity for the com m unity to com e together at the school, share a m eal, play gam es, receive nutrition-related social behaviour change com m unication (SBCC) m essages and learn m ore about nutrition in a less form al way. 109. Sch ool ga r d e n s: In addition to form al training, the McGovern-Dole supports schools to utilize school gardens for nutrition education. Som e schools also use the produce from the garden in their school m eals. Overall, 86 percent of the USDA￾supported schools had school gardens, of which 95 percent were in use at the tim e of the survey. Only 19 percent of com parison schools had gardens. Most of the USDA￾supported schools reported receiving their seeds from their own school budget (88%), from WFP (30%) or from Plan/WV (7%). 110. More than half the surveyed School Directors in USDA-supported schools reported using the school gardens throughout the year (Figure 3) except during school holidays 96. School gardens are m aintained by the school children under the supervision of their teachers. Som e schools reported that they did not have sufficient water to utilize the gardens to their full potential. More inform ation on the issue of water in schools can be found ahead in the section on “increased access to clean water and sanitation services”. Annex 28 provides m ore details on the survey findings about school gardens. Figure 3: Month of year when school gardens are utilized (Percent) Source : Endline evaluation In cre a se d a cce ss t o re q u isit e food p re p a ra t ion a n d st ora ge t ools a n d equ ipm en t 96 The long school holidays are during September/October 0 20 40 NGOs Health center TV School Myself Don’t know Community Meetings Radio Other parents Other All USDA Supported Schools Comparison Schools 0 20 40 60 80 100 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Comparison schools All USDA supported schools 111. The training provided by WFP and im plem enting partners on safe food preparation should help cooks and school personnel provide a hygienic cooking and eating environm ent for the children. However, the evaluation survey found only half the USDA-supported schools were using clean cooking and eating equipm ent in line with WFP guidance (Table 18). This is an im provem ent on baseline but indicates that m ore work needs to be done to prom ote food safety. Table 18: Percentage of schools using clean cooking and eating equipment Ba se lin e En dlin e Percentage of USDA-supported sch ool tha t use cle an cooking and e ating e quipm e nt 28 49 Source : Endline evaluation 112. Given that the new HGSF m odel introduces fresh produce into the school m eals including anim al proteins (m eat and eggs), the CO has sought technical expertise from the RBB to develop guidance for schools to avert potential threats to food safety. The Cam bodia CO, together with the WFP Regional Bureau in Bangkok (RBB) and Sodexo, 97 jointly supported an in-country baseline assessm ent of food safety practices in HGSF system s in 2017, and then supported the developm ent of the MoEYS Guidelines on Food Safety in Schools.98 These country specific guidelines, include gender specific issues such as Listeria and pregnancy, and includes a set of posters, booklets and pam phlets on basic steps to im prove the quality and safety of school m eals. This food safety work is especially im portant given the lack of refrigeration in schools for storing fresh foods, and the fact that the school children are responsible for helping with m any activities in the school including both the cleaning of the latrines and the school grounds, and the serving of the school m eals. 113. The ET note that som e of the food safety related activities planned for this phase have not yet been com pleted due to delays in printing the m aterials. The no-cost extension until March 2020 will therefore include the continuation of som e key nutrition and food sa fety activities including the roll out and dissem ination of a food safety training package for school cooks and supporting the design of a standard kitchen m odel to be used under the national HGSF program . In cre a se d a cce ss t o cle a n w a t e r a n d sa n it a t ion se rvices 114. In USDA supported schools the McGovern-Dole program m e provides funds for the construction and m aintenance of water sources, latrines and handwashing facilities. The survey found that m ost schools (92%) now have separate latrines for boys and girls that are well m aintained, with handwashing facilities and soap nearby (Table 19). This com pared with 77 percent at baseline. Table 19: Summary of availability of school water and sanitation infrastructure USDA supporte d schools (%) Com parison schools (%) Baseline En dlin e En dlin e 97 Sodexo is a French food services and facilities management company, one of the world's largest multinational corporations. 98 Royal Government of Cambodia (2019) Guidelines on food safety in schools. MoEYS, May 2019 Ye a r-rou n d acce ss to cle an wate r 64 66 68 Functioning latrine s (pe rce nt) 90 90 87 Se pa ra te la trin e s for boys a n d girls 77 92 57*** La trin e s a va ila ble for disa ble d students 43 27 11** Ha n d wa sh in g fa cilitie s 86 99 92* Soap at han dwashing station 89.5 100 92** Source : Endline evaluation *** p<=.001; ** p<=.01; * p<.05 (com parison between endline USDA-supported schools and com parison schools) 115. The latrines are cleaned regularly, usually by students on a rota system . USDA supported schools are also m ore likely to have toilets for disabled students,99 than in com parison schools. Som e USDA supported schools had a few latrines that were non￾functional, requiring m aintenance – the reasons were the door was broken, the pit was full, or the com m ode was broken. 116. The m ain concerns of evaluation stakeholders on the water and sanitation related infrastructure was lack of school budget for m aintenance, as with other school infrastructure, and that clean water is not available all year round. At endline only 66 percent of USDA supported schools reported year-round access to clean water from their school grounds (Table 19). This is m ainly because existing water sources dry up during the dry season and this year, Cam bodia had a prolonged dry season due to El Niño. When that happens, children bring water from hom e to enable the cooks to prepare the school m eal. Additional findings regarding water and sanitation services in schools, including the condition of related infrastructure at the tim e of the endline survey can be found in Annex 29. In cre a se d a cce ss t o p re ve n t a t ive h e a lt h in t e rve n t ion s 117. Schools serve as a platform for the delivery of other services including the regular deworm ing treatm ent provided by the MoH with support from the World Health Organization (WHO) and Johnson and Johnson. The endline survey found that 83 percent of children in USDA supported schools received deworm ing treatm ent in SY 2018/19, com pared with 67 percent in com parison schools. Additional findings from the school survey can be found in Annex 30. 2.2.7 Overall achievement of Strategic Objective 2: Improved use of health and dietary practices 118. All the activities im plem ented under SO2 contribute to the im proved health of school children. The RF shows that im proved health should translate to reduced health related absence, helping ensure that children attend the m ajority of their classes. Table 20 shows that overall, there was no significant difference 100 in absence between USDA￾supported and com parison schools, with students m issing 8-10 days per year on average. Table 20: Absence in class (SY 2018/19) All USDA supported schools SMP on ly^ SMP + THR^^ Com p a rison schools 99 Toilets with greater width, with ramp, and with an easy to open door handle 100 t-test (two-tailed) for independent samples show that the percentage of students absent (both total and for females) in class are not significantly different for USDA-supported and comparison schools as well as between USDA supported schools. Percent of absent students Tota l 9.8 10.2 9.7 7.8 Girls 7.8 11.0 7.0 6.2 Source : Endline evaluation ^ including HGSF Hybrid, ^^ including HGSF-Hybrid+ THR *** t-test (two tailed) p<=.001; ** p<=.01; * p<.05; NS Statistically not significant 119. In Cam bodia, it is difficult to attribute these activities to health-related absence as m ost schools do not collect the reason for student absence so there is no data from the baseline survey. Teachers highlighted that children are absent for m any reasons, only som e of which are due to illness. Table 21 shows that on average, children in USDA￾supported schools m issed only 2 days of the year due to illness, which m eets the programme target. Table 21: Number of days missed due to illness Target Baseline Endline Ave rage num be r of sch ool days m isse d due to illne ss ≤12 No data 101 2.1 Source : WFP m onitoring data 120. Regardless of the lack of health-related absence from com parison schools, the evaluation findings show positive results in other aspects of the program m e for RF2. Interviews with parents, teachers and children indicate that the training im plem ented under SO2 has contributed to knowledge of good health and dietary practices, including healthy diets, food safety, and WASH. This training, along with a clean school environm ent through the provision of instructure are sound starting points for good health for the school children. The continued support to infrastructure developm ent during this phase was a good use of USDA resources, to ensure that all the USDA-supported schools provide children with an appropriate, healthy learning environm ent. 2.2.8 Effectiveness of the programme to achieve other outcomes 121. In addition to the above results that are all intended under the USDA RFs, the evaluation found several other results from the program m e, som e intentional, som e not, and m ost not actively m onitored. Com m u n it y e n ga ge m e n t 122. WFP has been encouraging com m unity engagem ent in the school feeding activities for m any years. This includes parents and other com m unity members participating in Parent-Teacher Associations (PTAs), Local School Feeding Com m ittees (LSFCs), and School Support Com m ittees (SSCs). The SSCs are com posed of school directors, com m une council m em bers, village chiefs, teachers, and parents that act as a liaison between the school and com m unity. Alm ost all (99%) of USDA-supported schools have PTAs or sim ilar governance structures in place. Both the SSCs and the LSFCs meet m onthly and are responsible for com m unity contributions to the school, either for m eals such as requesting vegetable s, firewood, salt, and financial contributions to pay the cook (LSFCs) or for fundraising for school infrastructure and m aintenance of the school 101 The ET were unable to determine the number of days missed due to illness, as schools do not keep that information. After the baseline, the CO included this variable as part of their regular monitoring in some schools to ensure a figure would be available for endline. grounds (SSCs). Table 22 shows that this engagem ent with parents is significantly stronger in USDA-supported schools than in com parison schools, with m ore parents sitting on com m ittees and visiting the school m ore often (not including dropping or picking their children from school). Table 22: Parental membership of school committees (SY 2018/19) PTA! LSFC! SSC! Moth e rs’ Com m itte e ! Nu m be r of visits to school# All USDAsupported sch ools 11.3* 6.2*** 2.7* 9.5* 2.35*** SMP on ly^ 10.0 5.0 3.0 11.0 2 SMP+THR ^^ 11.6 6.5 2.7 9.2 2.1 Com p a rison sch o o ls 7.0 1.8 0.7 5.4 1.1 Source : Endline evaluation ^ including HGSF Hybrid, ^^ including HGSF-Hybrid+ THR ! Chi-square test of association between literacy and USDA-supported schools and com parison schools *** p<=.001; ** p<=.01; * p<.05; NS Statistically not significant # t-test (two tailed) *** p<=.001; ** p<=.01; * p<.05; NS Statistically not significant 123. Evaluation interviews found that although parents and com m unity m em bers sit on school com m ittees, their participation in decision m aking and project m anagem ent is lim ited, with the School Director and the Storekeeper taking the m ajority of decisions. In addition, although there is usually a good gender balance in the com m ittee, a gender assessment 102 found that gender expectations and norm s are often reinforced, with m en taking leadership roles and wom en taking on other tasks such as financial m anagem ent (treasury). The assessm ent found that WFP and partners did not have a strategy for achieving a gender balance within the com m ittee. Despite this, FGDs during the evaluation field m ission found that com m ittee members generally felt inform ed about program m e im plem entation and they were willing and able to help out in any way they could. The ET noted an increased understanding from local authorities (village chiefs and com m une councils) since baseline on the ir roles and responsibilities and the support they can provide. The recent change in m odality to HGSF m ean the local authorities have a greater role to play now, to support the supplier bidding process. 124. Since baseline, the ET has com m ended the com m unity participation in the project, with parents providing firewood, water and vegetables for the school m eals as required. In som e schools, parents are also asked to provide financial contributions to purchase condim ents for the school m eal, and to support the payment of the cook. For som e households this contribution is not possible due to their financial circum stance, however all parents interviewed said they were happy to provide whatever they could. 125. The other contribution the com m unity m akes to the school feeding activities is the cook. The school cooks are m em bers of the local com m unity, usually a parent or grandparent who volunteers their tim e to com e to school and cook the m eal each day. The low level of incentive received by the cooks has been perhaps the biggest com plaint about the program m e since baseline and continues to be so. In previous phases, WFP paid the cook, but this was discontinued in SY 2015/16 at which stage the com m unities were asked to take over the paym ent. Since then com m unities have struggled to pay the 102 Brody, A. (2017) Innovations from the Field: Gender mainstreaming from the ground up. Cambodia Mission Report cook a regular am ount, com m ensurate with the work done. Since baseline, WFP and partners have been working hard to correct this issue, engaging the local com m une councils in the discussion and advocating for Com m une Developm ent Funds (CDF) (from MoI) to be used to pay the cook USD 25 per m onth. Interviews with school directors, cooks and com m une councils indicate that this paym ent is rarely received with som e cooks receiving rice only, som e receiving a m ix of rice and cash, and som e receiving cash, m ost around USD 10-15 per m onth, and m any not on a regular basis. The CDF has been increased recently by the MoI, but the focus of investm ent is still heavily on rural infrastructure and less allocation to support social activities such as school infrastructure and cooks. 126. WFP successfully advocated for the MoEYS to include paym ent for the cook of USD 25 per m onth in their proposal to im plem ent school m eals in som e schools from SY 2019/20. Gende r equ a lit y a n d em powerm en t of wom en (GEEW) 127. School feeding provides an opportunity for wom en’s em powerm ent given that wom en play several roles in the program m e including as teachers, School Directors, school com m unity m em bers, com m une council m em bers, and as school cooks. All stakeholders recognized in particular, the im portant role that the school cook plays in the im plem entation of the program m e. Inadequate paym ent of the cook’s incentive was therefore raised by several stakeholders as a m issed opportunity for gender transform ation. 128. The gender assessm ent done in 2017103 confirm ed that school cooks do a lot of work for insufficient paym ent. The change to HGSF m eans that cooks now need to do m ore work often involving waking up at 3am to go to school, m eet the supplier delivering fresh m eat, eggs or vegetables, then prepare the school m eal. This is som etim e s done before she undertakes farm ing or other types of work on their return hom e, as well as unpaid carer responsibilities. At the tim e of the gender assessm ent there were no official m ale cooks, although the ET noted in several schools that the cook’s husband often helps her out (with no additional paym ent). 129. The gender assessm ent also highlighted the need for WFP and partners to actively involve wom en suppliers in the delivery of the HGSF m odel. The new m odel provides an opportunity to increase the incom e of local wom en, potentially reducing the need for m igration in search of work. In 2018, the pilot Interagency Social Protection Assessm ent (ISPA)104 done by FAO found that the HGSF presents no clear evidence of active prom otion of wom en’s econom ic em powerm ent. Addressing the issue of standardized and appropriate paym ent for the cooks would be a good starting point. 130. WFP introduced cooking com petitions for school cooks as a national-wide m eans of prom oting their value while also dissem inating m essages about food safety. While this is also a positive step, without sufficient incentive however, it will m ore difficult to recruit and keep cooks for the program m e. The ET therefore com m ends WFP and MoEYS for 103 Brody, A. (2017) Innovations from the Field: Gender mainstreaming from the ground up. Cambodia Mission Report 104 Royal Government of Cambodia (2018) Report on a pilot application for the ISPA-FSN tool for the Home-Grown School Feeding Programme in Cambodia. Inter-Agency Social Protection Assessment for Food Security and Nutrition. ensuring that the cook’s salary was included in the HGSF proposal for governm ent im plem entation for SY 2019/20. However, m ore work needs to be done to ensure that cooks in all USDA supported school received rem uneration com m ensurate with their workload. 131. On a m ore positive note, stakeholders felt that the McGovern-Dole programme provides equal opportunities for boys and girls to be part of the program m e, and com m ended WFP and MoEYS for providing scholarships to both boys and girls. Hou seh old food a n d n u t rit ion secu rit y 132. The household survey assessed if any changes had occurred at household level during 2017-2019. This included looking at food and nutrition security indicators as well as a range of other topics about the household m em bers. Since the survey sam ple was done by m atching USDA-supported and com parison schools on the basis of household poverty, it is not surprising that the survey results indicate little difference in the socio￾econom ic characteristics of the surveyed households. Both groups of households were prim arily farm ers, earning an average incom e of USD 11-19 per day. 133. The survey did find significant differences in household expenditure patterns, with USDA-supported households spending a lesse r percentage of their incom e on food, less on education, and m ore on non-food item s. USDA-supported households also m ade a larger proportion of their expenditure using credit (Annex 31). The endline evaluation also found no difference in the eating patterns of households, with both USDA-supported and com parison households averaging between 2-3 m eals per day. Children in both types of households consum ed about three m eals a day. However, the average number of m eals consum ed by children in USDA-supported households was significantly greater than those in com parison school households. The households were also sim ilar in term s of overall diet patterns with alm ost all households consum ing rice, protein, and vegetables on a daily basis. The Reduced Coping Strategies Index (rCSI) results also found no significant difference between USDA-supported and com parison households at endline (Annex 32). 134. As part of the household survey parents were also asked to indicate the m ain benefits of school feeding for their children, and for the household. This was to evaluate whether the program m e had any other benefits, aside from the ones that are regularly m onitored by WFP and partners. At baseline, m ost parents reported that the benefits were that their children received food which m akes them healthier and able to learn better. At endline, the m ain reasons rem ained the sam e, but with a larger percentage of parents also reporting the benefits of being m ore attentive in cla ss and having m ore opportunities in their future. 135. Parents also reported that school feeding saves food and m oney to the household, and saves tim e for the parents, including saving their tim e to cook in the m orning. Those parents who reported that they saved tim e, generally used the save d tim e in household chores (between 40-45% households). Around 20 percent of households used the saved tim e for farm ing/livestock work or other incom e ge nerating activitie s. 136. Both m ale and fem ale household m em bers saved tim e as a result of school feeding and/or children’s school attendance. However, it was wom en’s tim e that was prim arily saved as wom en are generally the ones responsible for preparing m orning m eal. Additional inform ation on the benefits of school feeding as reported by parents can be found in Annex 31 and m ore general findings from the household survey can be found in Annex 32. 2.2.9 MoEYS capacity to manage school feeding moving forward 137. WFP does not have any corporate guidelines to help COs build the capacity of governm ent’s on school feeding. Despite this, WFP has been working hard over the last several years to build the capacity of the MoEYS in school feeding im plem entation, and to incorporate school feeding into relevant governm ent policies and strategies as mentioned earlier. This includes supporting the developm ent of the National School Health Policy which was endorsed by the Council Ministry in April 2019, and the ESP with school m eals and HSGF incorporated. Both these docum ents are now at Stage 5 (approved and being im plem ented) (Table 23). WEI has also developed three EGR standards (Grades 1-3). Table 23: Number of policies developed as a result of USDA assistance (2017-2019) Ta rge t Ba se lin e En dlin e Nu m be r of childre n he alth and nutrition policie s, re gu la tion s or a d m in istra tive proce dure s in de ve lopm e n t as a re sult of USDA assistan ce 1 1 (Stage 5) 1 (Stage 5) Nu m be r of e du ca tion a l policie s, re gu la tion s or adm inistrative proce dure s in de ve lopm e n t as a re su lt of USDA assista n ce 4 0 1 (Stage 5) & 3 EGR standards de ve lope d Source : WFP m onitoring data 138. Despite WFP’s strong policy developm ent support, there had been som e key stum bling blocks to national ownership identified during the baseline. These included lack of confirm ation of the governm ent’s preferred m odel of school feeding, lack of governm ent financial com m itm ent to the program m e, and a lack of clear decision-m aking about which provinces the governm ent wanted to prioritize for a nationally owned m odel. During the 2017-2019 period, the MoEYS has undertaken several pieces of work with the support of WFP, to clarify its position on school feeding. The research findings and the capacity gaps identified within them have helped WFP identify the steps required to build governm ent capacity to eventually m anage the program m e. 139. The feasibility study105 on the im plem entation of school feeding conducted by the MoEYS in 2017 was m entioned earlier under appropriateness. The study found that the HGSF m odel was a successful approach for gaining longer term benefits across several sectors including education, live lihoods health and nutrition. Following this study, the governm ent confirm ed that the HGSF was their preferred school feeding m odel. Since then, WFP has been working with the governm ent to expand HGSF, while supporting the 105 Ham, K., Kunthea, K & Chhe, B (2017) The Feasibility of the Implementation of School Feeding Programs in Primary School by the Ministry of Education, Youth and Sport, Department of Policy, MoEYS. developm ent of operational and institutional capacity. The im plem entation of the HGSF￾hybrid m odel106 that is currently being im plem ented in m any USDA supported schools, has proven a successful interim step towards full HGSF.107 The interim m odel has enabled WFP and partners to focus on capacity building: training School Directors, LSFCs, com m une councils, and com m unity m em bers on the required procurem ent processes for HGSF and build relationships with local suppliers and farm ers. 140. In June 2018, MoEYS organized an inter-m inisterial stakeholder consultation workshop on the school m eals program m e which aim ed to identify key benchm arks for the continued transition of the program m e to governm ent ownership. Participants from all 25 provinces and counterparts from various m inistries, NGO partners, and other relevant developm ent partners participated in the workshop which was supported by the WFP Centre of Excellence against Hunger in Brazil and WFP regional school feeding experts. The consultation included a study visit undertaken to schools in the Siem Reap area with a focus on HGSF. This consultation helped solidify the governm ent’s interest in HGSF and school feeding as a social protection instrum ent. As a follow up to that m eeting, MoEYS was requested by the Prim e Minister’s Office to subm it a 3-year concept note for HGSF to MEF. The workshop also saw a com m itm ent from CARD to integrate HSGF as part of the NSFSN and the NSDP. 141. In February 2019, MoEYS and MEF organized a trip to Preah Vihear and Siem Reap Provinces as a joint learning m ission to fam iliarize MEF officials on the school m eals/hom e grown school feeding im plem entation process and to facilitate strategic and operational discussions between the two ministries on the linkages to the ESP and the NSPPF. WFP also participated in the m ission which form ed a step in the appraisal process of HGSF being financed by national budget through MoEYS and MEF. A concept note was then prepared by MoEYS for the ir im plem entation of school feeding activities in 205 schools in six provinces,108 from SY 2019/20 at a cost USD 2 m illion annually for the next three years. This costing was done rapidly by MoEYS and MEF based on rapid calculations from WFP on the cost of their m eal. The proposal has been accepted and MoEYS is now planned to start im plem entation at the start of the SY2019/20. This is in addition to the 2000MT of rice that the governm ent already provides to WFP each year to support SF activities. 142. Furtherm ore, the MoEYS has already dem onstrated their capacity to m anage the scholarship program m e having successfully taken over the im plem entation of WFP’s THR program m e and incorporated it into the existing governm ent cash scholarship programme , which already has a dedicated budget line , policy and m anagem ent structure within the Departm ent of Prim ary Education. 143. While awaiting approval of the above budget for the 205-school proposal, in March 2019, MEF invited WFP to discuss the transition strategy to a nationally owned program m e focusing on WFP’s com m itm ent to provide technical assistance and capacity strengthening support. The Annual WFP School Feeding Workshop conducted in March 106 The HGSF-hybrid model uses USDA donated commodities (rice and oil) and locally procured animal protein, vegetables and salt. 107 The full HGSF model is currently implemented by WFP as part of the school meals activities in non-USDA supported schools with support from other donors. 108 Banteay Meanchey, Kampong Cham, Siem Reap, Battambang, Preah Vihear and Stoeung Traeng 2019 was devoted to review overall program m e achievem ents, challenges, solutions and lessons lea rned. The WFP CSP and the MoEYS transition plan of school m eals/hom e grown school feeding were shared with further discussion on priority targeted areas and possible options for local resource m obilization, and other task required for im m ediate action to move forward the national school m eals program m e. 144. Several key governm ent representatives also attended the 20th Annual Global Child Nutrition Forum (GCNF) in Tunisia, with support from WFP. The forum is organized by the Global Child Nutrition Foundation (and the WFP Centre of Excellence against Hunger, a learning exchange and technical assistance conference to support countries in the developm ent and im plem entation of sustainable school feeding program m es. The forum provided yet another opportunity for the governm ent to com m it to school feeding as an avenue for im proving the nutritional status of school children. 145. As a result of all the above activities, the governm ent is now clearly articulating its preference for cash-ba sed program m ing (scholarships) and for the HGSF m odel as it benefits the local econom y. The governm ent is also now demonstrating a strong com m itm ent to build and own a national school m eals program m e. Key inform ant interviews identified a significant change in the lexicon of MoEYS representatives, now increasingly recognising school feeding as an im portant social assistance instrum ent, especially for households in areas experiencing food insecurity and poor educational outcom es. Further, the next GCNF will be hosted by Cam bodia in Decem ber 2019. This will be the first tim e the Forum will be held in the Southeast Asia Region. Doe s t h e gove rn m e n t h a ve a d e q u a t e in st it u t ion a l a n d HR ca p a cit y t o a ssu m e respon sibilit y for fu t u re sch ool m ea ls a n d sch ola rsh ip a ct ivit ies? 146. The MoEYS has a departm ent dedicated to the m anagem ent of school m eals and scholarships - the Scholarships Office within the Departm ent of Prim ary Education. In addition, the MoEYS has a national Program m e Coordination Com m ittee (PCC) and School Fe eding Task Force (SFTF) providing overall policy and strategy guidance on SF m anagem ent including coordination across line m inistries and developm ent partners, and advocacy for funding support on SF from governm ent. The SFTF carries out day-to￾day activities such as providing technical support, guidance and m onitoring to sub￾national School Feeding Com m ittees to strengthen their capacity for SFP im plem entation and m anagem ent on the ground, then regularly updating relevant info to the PCC. 147. At sub-national levels, Provincial School Feeding Program m e Com m ittee s (PSFC) and DSFCs m anage overall planning, im plem entation, m onitoring and reporting of the program m e at school level (including onsite coaching/m entoring), and providing technical support to the SFCs to ensure im plem entation runs sm oothly. The com m ittees also have a role in ensuring that SF is integrated in provincial, district and school strategic/developm ent plans and sm ooth coordination with NGO partners to arrange capacity developm ent activities to schools, local authorities and com m unity. 148. The SFCs directly im plem ent the SF activities at school level, including daily food m anagem ent and preparation, provision of breakfast to children, reporting on the program m e, closely working with local authorities and com m unities on resource m obilization to SFP (in kind or cash), prom oting sanitation and hygiene practices at schools, organizing education cam paigns, coordinating THR beneficiary selection and THR food distribution. 149. Local authorities (village and com mune chief) lead on local resource m obilization activities for SM/HGSF, including cook incentives, kitchen utensils, feeding shelters, etc.; integrate SM/HGSF into Com m une Investm ent Plan/Com m une Developm ent Plan (CIP/CDP); take lead on Quarterly Coordination m eetings at the com m une level or coordinate ad hoc m eetings where necessary; m onitor the program m e at schools to ensure that the activities run sm oothly and transparently. 150. The ET found that the cooperation and support from DSFCs and all im plem enting partners has im proved since baseline. All five districts visited by the ET have shown progress of their understanding of the roles of and responsibilities of the DSFC since baseline. This includes an understanding of the need for them to support the paym ent of the cook’s incentive. The inclusion of the District Governor and the District Education Officer on the DSFCs bring together the MoEYS and the MoI and this will help ensure that com m une councils receive appropriate guidance on utilization of the com m une budgets for school feeding activities. 151. MoEYS sub-national capacity is affected by the high turnover of staff. Stakeholders cited num erous exam ples of trained personnel leaving their position, or m oving to other locations, resulting in lost knowledge of the program m e. To address this problem , the MoEYS is planned to establish a pool of trainers at provincial level to ensure new MoEYS personnel can be trained on SF im plem entation without support from WFP. The MoEYS also has a group of tra iners in the Provincial Teacher Training centres who can support training/coaching on EGR teaching techniques. 152. Although school feeding using USDA com m odities has been im plem ented by WFP for several years now, the introduction of HGSF m odels are relatively new. To date, the im plem entation of the HGSF m odel has not yet been evaluated and im plem entation guidance and training on the sam e have only been started during this phase of program m ing. Therefore, although the MoEYS has built up considerable institutional capacity for im plem enting school m eals in general, the im plem entation of the HGSF m odel requires ongoing support. The HGSF m odel requires building/strengthening the capacity of schools and the local, regional and national procurem ent and supply chains capacities. This is particularly im portant for Com m une Council m em bers and for School Directors who will be ultim ately responsible for the im plem entation and procurem ent of school m eals at local level. There is a need for WFP’s expertise and continued technical assistance in this area, particularly on the required procurem ent processes which involve establishing relationships with local suppliers and farm ers. 153. During this phase, WFP has invested significant resources on research to help provide inform ation to improve the im plem entation of the HGSF m odality and support governm ent ownership. These include form ative research on adolescent program m ing,109 identification of m icronutrient gaps,110 a Fill the Nutrient Gap Analysis 111 109 Anthrologica & WFP (2018) Formative research to inform adolescent programming in Cambodia: Engagement for health, nutrition and sustaimable development. Summary Report – February 2018 110 WFP (2018) Micronutrient Challenges and Solutions: A stock take. March 2018 111 WFP (2018) Fill the Nutrient Gap – Cambodia Summary Report. October 2018 and a landscape analysis to assist the governm ent to supply fortified rice to the HGSF.112 WFP is also currently supporting a local rice fortification pilot (purchase of kernels for blending governm ent rice contribution) which should be com pleted during the no-cost extension period. The results of this research, together with the recent awarding of the USDA Local and Regional Food Aid Procurem ent (LRP) Program m e to Cam bodia should help address som e of the MoEYS’s capacity gaps on HGSF during the next phase of program m ing. 154. Overall, the ET is satisfied that the MoEYS’s current capacity for im plem enting school m eals has im proved since baseline. At baseline, the ET rated the capacity of the MoEYS on school feeding as System s Approach for Better Education Results (SABER) Level 1. At endline, the ET determ ined that the capacity had increased to the next level (em erging) but confirm ation of this would require a full SABER assessm ent (Table 24). Table 24: Progress on the achievement of the SABER quality standards SABER Sta n da rds Ba se lin e En dlin e Strong policy fra m e works La te n t Em e rgin g Strong institutional stru cture and coordin a tion La te n t Em e rgin g Stable fun ding and budge ting La te n t La te n t Sound progra m m e de sign and im ple m e n ta tion , in clu din g e va lu a tion Em e rgin g SMP and Sch olarsh ips: Em e rgin g HGSF im ple m e n ta tion : La te n t Strong com m unity participation an d owne rship Em e rgin g Em e rgin g Ove ra ll SABER Le ve l 1 SABER Le ve l 2 Source : Evaluation Team ’s own asse ssm ent Is t h e re a fu n ct ion in g gove r n m e n t r e p or t in g a n d M&E system ? 155. To im prove program m e im pact and value -for-m oney, WFP started work on a m ore efficient outcom e m onitoring system in 2015: Platform s for Real-tim e Inform ation SysteMs (PRISM). The system includes m obile data collection devices for routine m onitoring and assessm ent. During the current phase, WFP has put considerable work into the PRISM system to not only finalize its developm ent but also to adjust the system for the MoEYS for m onitoring and reporting scholarship im plem entation. The resulting system is now called the National Cash Scholarship Inform ation System . WFP also supported GIZ, UNICEF and MoEYS to im prove m onitoring of school-based WASH program s by custom izing m obile data collection tools and web-based m apping applications, leveraging PRISM. 156. The transfer of knowledge to MoEYS has included providing training to staff and supplying equipm ent (screens) in the three USDA provinces. This included a training of conducted with 78 education officials at both national and sub-national levels. Through this approach, 1,801 school directors and adm inistrators in the three USDA supported 112 WFP (2018) Cambodian Rice Landscape Analysis: Opportunities in the rice value chain to support rice fortification. December 2018 provinces were trained and will directly use the system to digitalize inform ation for their individual schools. 113 157. The transition from a paper-based m onitoring system to a digital platform has greatly im proved the efficiency of scholarship m onitoring for both WFP and for MoEYS, with data now being available in a m ore tim ely m anner. However, there is still more work to be done to ensure the system is appropriate for SF im plem entation. WFP’s im plem enting partners highlighted the need for continued capacity building for MoEYS staff regarding data m anagem ent. This was confirm ed by the 2019 ISPA that found that after the HGSF is adopted by the governm ent, PRISM m ay not be utilized due to lack of technical capacity.114 With additional capacity support at sub-national levels over the next phase of program m ing, the new m onitoring system has huge potential to im prove the efficiency of school feeding program m e m onitoring as well as the scholarship m onitoring. 158. The MTR noted that despite the advances in m onitoring technology, the current coverage of the USDA program m e m eans that it is difficult to m onitor all the schools without full-tim e personnel. It has been difficult to determ ine whether MoEYS has a m onitoring and results fram ework with clear output and outcom e indicators that are being tracked. The system of inform ation flow from the schools to MoEYS faces som e challenges such as depending on NGO’s support for reporting; some schools are still using paper-based m onitoring and reporting because they lack com puters and because of com m unication difficulties between schools, provincial level, departm ental levels and national MoEYS. There is also a lack of adequate resources for MoEYS personnel to carry out visits to schools. As a result, som e areas are not m onitored due to difficult access and lack of resources from the District Offices of Education (DOE) and PDoEs. 159. In addition to the overall program m e m onitoring system , WFP has e stablished a com plaints/feedback m echanism : an autom ated telephone answering service where people leave voice m essage and their phone num ber and are then called back by WFP personnel to discuss the issue. Since 2016 the system has only received ten calls regarding im plem entation issues, m ost of which were related to the cook being absent and/or the school not providing m eals, and m ore recently, two com plaints about the am ount of paperwork involved in the HGSF activities. 2.3 Evaluation Question 3: Why and how has the operation produced the observed results? 2.3.1 Internal factors 160. Lon g t e rm e n ga ge m e n t b e t w e e n WFP a n d p a rt n e rs: WFP has had partnerships or collaborations with several program m e stakeholders over an extended period, well beyond the current phase of McGovern-Dole im plem entation. This includes in particular, the MoEYS, with whom WFP has worked for m ore than a decade. The strong relationship between them has been critical for the positive results during 2017-2019, and especially for building the interest and com m itm ent from the MoEYS to take on the im plem entation 113 WFP semi-annual report October 2018-March 2019. 114 Royal Government of Cambodia (2018) Report on a pilot application for the ISPA-FSN tool for the Home-Grown School Feeding Programme in Cambodia. Inter-Agency Social Protection Assessment for Food Security and Nutrition. of the program m e m oving forward. Overall, stakeholders recognized WFP Cam bodia as a well-m anaged, technically strong, innovative and flexible office. 161. St ron g WFP p rogra m m e m a n a ge m e n t a n d t e ch n ica l ca p a cit y: The WFP Cam bodia personnel are highly experienced program m e im plem enters. They have strong technical expertise in critical program m e sectors such as education, food and nutrition security, and social protection. This expertise has been essential for designing and im plem enting an appropriate program m e, and also for supporting the work of others in these areas. There are also strong capa city building skills within the team , m ainly focused with the m anagem ent team personnel. The CO is currently going through a Capacity Needs Assessm ent to determ ine where their own capacity gaps lie. 162. St ron g logist ic/ su p p ly ch a in su p p ort : All evaluation stakeholders acknowledged WFP as a leader in com m odity m anagem ent and transportation, and positively reviewed their procurem ent, storage and delivery of food com m odities, including their operational guidance to support their logistics system s, and strict quality standards. Overall, the WFP logistics system has enabled the provision of high-quality food to be delivered to the USDA-supported schools in a tim ely m anner with no com plaints. 163. High ca pa cit y of im plem en t in g pa rt n ers: The McGovern-Dole programme has successfully achieved m ost of the planned outputs largely as a result of the strong field im plem entation by the MoEYS particularly at school level, and by Plan International, WV, WEI, KAPE, with m onitoring by WFP field assistants. Plan and WV are both long term partners of WFP and are highly experienced in program m e m anagem ent. As a result, the McGovern-Dole program m e has been well im plem ented and m onitored. Plan and WV have also conducted m ultiple com m unity m obilization exercises and have docum ented key lessons learned. These lessons will be useful when transitioning the programme to national ownership when local authorities and com m unities take on m ore responsibilities. In addition, the high level of com petency of WEI and KAPE has been recognized by a ll stakeholders including MoEYS, USAID and agencies m anaging the GPE’s grant. 164. St ron g m on it orin g syst e m s: WFP’s developm ent of the PRISM system has helped WFP and partners com prehensively m onitor the program m e. Integrating PRISM into the MoEYS Scholarship Office has also enabled im proved m onitoring at sub-national level. In addition, WFP have supported the digitization of tools for storekeepers to be able to m onitor and report food stocks on their phones. This provides WFP CO with real-tim e inform ation on school level food requirem ents. 165. Te ch n ica l su p p or t from WFP RBB and Headquarters: The RBB and the Office of School Feeding (OSF) in WFP Headquarters have provided technical support to the CO on food safety, SBCC and nutrition during this phase has enabled the program m e to expand their work in these areas and better align with the governm ent’s preferred m odel of HGSF and with the priority areas identified in the School Health Policy. Support to the programme has also been provided by the WFP Centre of Excellence against Hunger in Brazil for school feeding stakeholders in Cam bodia to participate in the Asian Meeting on School Feeding. 166. WFP fin a n cia l su p p ort t o re se a rch , visit s a n d w orksh op s: WFP has supported several pieces of research this phase to inform program m ing decisions for the transition. This includes the nutrition and food safety work, the ISPA, and m ultiple gender studies with IDS. The workshop aimed to identify key benchm arks for the continued transition of the program m e to governm ent ownership and included a study visit to schools in the Siem Reap area with focus on HGSF. This support has been invaluable for the MoEYS in establishing their position on school feeding and for their transition planning. 2.3.2 External factors 167. Colla b ora t ion w it h MoEYS: Support from the MoEYS is critical to the success of the McGovern-Dole programme as they are responsible for all the infrastructure and personnel connected to the schools. Cooperation with relevant technical departm ents within MoEYS, notably the Prim ary Education, Curriculum , Teacher Training, EQAD and School Health Departm ents, as we ll as with decentralized MoEYS personnel (within PTTCs, DTMTs, PDoEs, DOEs) has been key for the successful im plem entation of McGovern-Dole programme activities. 168. Gove r n m e n t ’s com m it m e n t t o sch ool fe e d in g: Aside from the MoEYS, several other governm ent m inistries/entities were involved in school feeding related decision￾m aking during the evaluation period. This includes MEF, MoP, MoI, MAFF, CARD and national school feeding com m ittees at provincial, district, com m une and school levels. The Technical Working Group on Food Security and Nutrition coordinated by CARD is an effective structure for open partnership dialogues between WFP and other stakeholders involved in school feeding. WFP uses this forum to work collaboratively towards inform ation and experience sharing, inform ing policy and decision m akers, and the transition to national ownership. This inter-m inisterial engagem ent has been im portant for supporting cross-sectoral school-feeding-related policy developm ent and deciding program m e direction. The governm ent only confirm ed their preferred m odel of school feeding (HGSF) in 2018. This and the lack of governm ent financial com m itm ent to the programme until later that sam e year, has m ade it difficult for WFP to adhere to the tim eline for handover m entioned in the 2015 RoadMap. 169. Com m it m e n t of a ge n cie s su p p ort in g MoEYS on lit eracy: As part of support to the Cam bodian education sector for the period 2016-2021, the USAID funded ACRC project has assisted MoEYS to harm onize EGR and teaching approaches in Cam bodia. WEI’s partnership with both USAID and WFP contributed to a strategic synchronization of USAID’s and USDA’s literacy work. More recently, in line with the MoEYS EGL program m e and associated five -year plan (2019-2023), USAID has developed an All Children Learning project. While im plem enting the EGR package in USDA-supported schools, WEI contributes to the EGL roll-out.115 170. Su p p or t from ot h e r d on ors: For this phase of the McGovern-Dole program m e , USDA has provided only rice and oil for the school m eals. This m eans that com m odities donated by other donors, such as the tinned fish provided by the Governm ent of Japan, and the yellow split peas purchased using private donor funds have been invaluable. 115 In order to harmonize interventions covering the overall Early Grade Learning (EGL) package, it was agreed that USAID would be in charge of the Early Grade Reading (EGR) aspect while UNESCO and partners would oversee the Early Grade Mathematics (EGM) component. These com m odities are supplem ented by com m unities providing salt and vegetables when available. Together, the com m odities m ake up a nutritionally balanced food basket. 171. Ca p a cit y a n d t u r n ove r of MoEYS p e rson n e l: The turnover of MoEYS staff, particularly in schools and at sub-national level has reduced the effectiveness of capacity building efforts by WFP and im planting partners. 172. Un clea r regu la t ion s on t h e a llowa ble u se of Com m u n e De ve lop m e n t Fu n d s: The ongoing issue of the insufficient paym ent of the school cooks by the com m unity and com m une councils is partially due to unclear regulations on the allowable use of the CDF. There is debate about the am ount and standardization of this paym ent, and the responsibility for it. While the CDF is generally acknowledged to be for infrastructure work and support for com m unity activities within each com m une, it is unclear if this includes schools, and program m es within schools such as school feeding. 2.4 Evaluation Question 4: How sustainable is the programme moving forward? 2.4.1 Is there an exit strategy outlining the timing, allocation of responsibilities on handover to the government and/or other agencies? 173. Since this is the endline of the program m e phase, the qualitative m ission had a focus on sustainability of the program m e. As such, this section is largely based on qualitative interview evidence, rather than on quantitative data as per Evaluation Question 2. 174. WFP’s overarching vision is to oversee the transition from a WFP-led school feeding program m e in Cam bodia to a governm ent-led program m e. The transition to national ownership was agreed in May 2015 with the signing of a School Feeding Roadm ap between MoEYS and WFP. The Roadm ap specifies a move away from external support, to the governm ent taking full responsibility for a com prehensive, nationally owned approach to school feeding by 2021. At the core of this transition is a strategy that includes approaches and tools that strengthen the Governm ent's institutional capacity to design, finance and manage the program m e over tim e. The Roadm ap is organized according to a set of short- (2015-2016), m edium- (2017-2020) and long-term (2021 onward) activities and aligns with the five quality standards of the SABER approach. 175. At the end of the previous phase of the McGovern-Dole program m e, the ET noted that m ost of the short-term activities had been com pleted, with the exception of the establishm ent of a school feeding budget. Key inform ant interviews indicated that the lack of an operational school feeding budget line and a School Feeding Policy were critical barriers to national ownership. At the tim e , there was ongoing debate on the need for a stand-alone policy since the 2017 NSPS and the School Health Policy, which was under developm ent, already included school m eals and scholarships. Interviews with the MoEYS Policy Departm ent indicates that they are now engaged in the form ulation of the School Feeding Policy, the first draft of which should be available at the end of 2019. This is a significant step forward towards governm ent ownership and the sustainability of the program m e. Further, MEF has recently approved USD 2 m illion annually for the next three years as part of the Education Budget is also a significant step forward. However, the relatively late declaration by the MoEYS of a preference for the HGSF m odel and the subsequent requirem ent for additional capacity support on this m odel, m eans that the current end date of the RoadMap is unlikely to be achieved. 176. The ET found a variation in capacities at school level to im plem ent the HGSF program m e. Som e USDA-supported schools are still using the old SMP m odel, som e have been using the HGSF-hybrid for two school years, and other only started the HGSF-hybrid m odel during the last school year. Interviews with school personnel and with m em bers of the DSFCs indicate different capacities with HGSF im plem entation, with som e likely to require significant levels of support from WFP during the next phase. 177. The ongoing transition to national ownership using a HGSF m odel is now being planned over the next five years to 2025. During this period the MoEYS is expected to initially im plem ent HGSF in 205 schools in 6 provinces 116 in SY2019/20, including in 31 USDA-supported schools in Siem Reap and 32 USDA-supported schools in Battam bang. This will gradually be scaled up to approxim ately 650 schools by SY 2024/25. The schedule, and num ber of schools to be handed over, will be reviewed regularly by WFP and the governm ent over the next five years. 178. WFP has recently been awarded a new UDSA McGovern-Dole grant for a further four-year period (2020-2023). The ET notes that the design of the new program m e is appropriate, including the ongoing support for the new five -year MoEYS EGL program m e in line with both USDA’s objectives and with the new WFP CRF. The new phase also includes an expansion of nutrition activities to help m eet SDG 2, and several activities to support national ownership such as facilitating the establishm ent of school feeding budget line and supporting governm ent budgetary m anagem ent. 179. Im plem entation of this additional grant, togethe r with the new USDA LRP grant over alm ost the sam e period, provides the opportunity to finalize the transition to national ownership over a m ore feasible tim e period so that MoEYS can take full ownership of the school m eals program m e and gain sufficient additional capacity in the HGSF m odel to im plem ent the program m e. To w h a t e xt e n t a re p r ogra m m e a ct ivit ies a n d loca l st ru ct u res likely t o be su st a in ed a ft er t h e com plet ion of don or-fu n din g? 180. This evaluation has established that several of the current McGovern-Dole programme activities are likely to be sustained into the next phase of program m ing, and possibly even once donor funding ends. The transition to the MoEYS and therefore the sustainability of the school feeding-related activities and the transition to HGSF m odel is docum ented above in Section 3.4.1, with confirm ation during interviews that MoEYS is ready to start im plem enting the HGSF m odel in schools in SY 2019/20. The MoEYS has already taken over the im plem entation of the scholarship activity (THR) and expanding the scholarships to reach children from grade 1-12. 181. In addition to those activities, the lite racy (EGR) activities im plem ented by WEI and KAPE are currently being scaled-up into the new EGL (Math and Khm er) programme 116 Banteay Meanchey, Kampong Cham, Siem Reap, Battambang, Preah Vihear and Stoeung Traeng adopted by MoEYS in 2018 with a 5-year plan to reach national coverage. WEI is we ll placed to com plem ent existing USAID efforts as it is already im plem enting the first stage of the national roll out with MoEYS unde r a num ber of subcontracts on USAID’s ACRC (2017-2020) and expanded All Children Learning projects. WEI is also assisting GPE in Siem Reap with EGR training im plem entation. Lastly, after successful im plem entation of literacy activities for the McGovern-Dole programme and their collaboration with other literacy actors, WEI is well placed to be assigned a m ajor role in building the capacity of central MoEYS to plan, im plem ent and m anage the EGR training roll out in the rem aining provinces across Cam bodia. 182. The sustainability of the com plem entary McGovern-Dole activities such as the prom otion of nutrition, hygiene and safe food preparation are all included in the new HGSF Guidelines that will be rolled out during the NCE period to March 2020. Further, the infrastructure developm ent work done by Plan and WV is likely to be continued with the support of the com m unities under the guidance of the SSC’s. 3 Conclusions and Recommendations 1.3. Overall Assessment/Conclusions 183. Re le va n ce : School feeding programmes, including the McGovern-Dole , align with several governm ent policies and strategies, and with WFP’s own corporate guidance. The programme also clearly aligns with several SDGs, m ost notably SDGs 2, 4 and 17. The literacy objective of McGovern-Dole also aligns well with the new ESP and with the new WFP CRF which aim s to ensure that education quality is im proved (literacy). WFP has been instrum ental in supporting policy developm ent in Cam bodia on m any relevant areas including food and nutrition security, social protection, school health and school feeding. These policies have helped form the governm ent’s position that school feeding is an im portant social assistance instrum ent, which also provides a strong platform for other activities such as literacy, infrastructure support and teacher training. The McGovern￾Dole program m e is also coherent with other developm ent actors who provide support to schools and with agencies who support literacy, food security and nutrition program m es in Cam bodia. 184. During this phase, the governm ent clearly stated their preference for a HGSF approach and WFP ha s piloted several HGSF m odels to help the governm ent decide which aspects they would like to continue in a nationally owned program m e. WFP is now appropriately transitioning all the USDA-supported schools to a HGSF m odel to align with the governm ent’s preference and with the agreed national transition plan. No negative feedback was received from stakeholders on this aspect of the program m e . 185. Efficie n cy: USDA provided USD 15 m illion in funding for this phase of program m ing, 25 percent less than the previous phase. This is indicative of the transition of the program m e to national ownership and reduction in program m e im plem entation requirem ents, and a greater focus on capacity strengthening activities, which generally require less funds. WFP and partners have im plem ented the m ajority of the planned activities, with some capacity building activities ongoing that will be com pleted during the NCE period to March 2020. Aside from those activities, all others have been com pleted in a tim ely m anner. The grants funds have therefore been sufficient to im plem ent an effective program m e, as seen by the results. 186. The use of the PRISM system has m arkedly im proved the e fficiency of program m e m onitoring for WFP and for partners, m ost notably for the MoEYS. The transition from a paper-based scholarship m onitoring system to a digital platform is a m ajor step forward. However, there is still more work to be done to ensure the system is appropriate for SF im plem entation. WFP’s im plem enting partners highlighted the need for continued capacity building for MoEYS staff regarding data m anagem ent. This was confirm ed by the 2019 ISPA that found that after the HGSF is adopted by the governm ent, PRISM m ay not be utilized due to lack of technical capacity.117 187. Effe ct ive n e ss: This phase of program m ing has seen im provem ent in school enrolm ent rates, and m ost stakeholders indicated that the presence of school m eals plays a role in regular student attendance. MoEYS EMIS data indicates that dropout rate s were lower in USDA-supported schools (4.9 percent vs. 6.0 percent) and com pletion rate s were higher (81.3 percent vs. 80.1 percent). Girls in both types of schools perform ed better than boys, with m ore girls being prom oted, and less girls repeating grades or dropping out of school. These findings indicate that girls generally perform better at school that the boys. 188. Aside from the above, the evaluation survey results show that the McGovern-Dole programme has been effective at im proving children’s access to food, im proving parental understanding on the benefits of prim ary education, im proving school feeding related infrastructure in the USDA-supported schools and prom oting parental/com m unity engagem ent in the program m e. The McGovern-Dole program m e also acts as an effective platform for preventative health inte rventions that include using of clean drinking water, prom oting regular handwashing with soap, and deworm ing. Additional work will be required in the next phase to ensure that USDA-supported schools provide a sufficiently food safe environm ent as new schools’ transition to the HGSF m odel. 189. The endline evaluation surveys were also able to establish other differences between USDA-supported schools and com parison schools, presum ably as a result of the program m e. These include reduced hunger in class and reduced inattentiveness (which are positively correlated), and im proved com m unity engagem ent. The evaluation results also indicate that the program m e saves parents tim e and m oney, enabling them to spend m ore tim e on livelihood activities. 190. The ET recognize that the McGovern-Dole program m e prom otes equal participation in the program m e for boys and girls, by providing scholarships for both, ensuring gender separated latrines are present in schools, and providing all students with training on various topics. However, the issue of inadequate rem uneration for the school cooks, alm ost all of whom are wom en, is an ongoing concern. Although both WFP and the MoEYS are aware of this issue, the gender transform ative potential that the program m e could bring to the cooks, is yet to be realized. 117 Royal Government of Cambodia (2018) Report on a pilot application for the ISPA-FSN tool for the Home-Grown School Feeding Programme in Cambodia. Inter-Agency Social Protection Assessment for Food Security and Nutrition. 191. Im p a ct : As per the USDA RFs, there are two intended im pacts of the program m e: im proved literacy and reduced health-related school absence. For each activity, outputs have contributed to expected outcomes and it is clear that SO1 - im proved literacy - has been achieved, as the percentage of children able to read and com prehend grade level text has increased. Sim ilarly, the effectiveness of the SO2 activities have contributed to producing a healthy and conducive learning environm ent in schools. However, the evaluation was unable to determ ine if they contributed to reduced health-related absences. While this is likely, m ost schools do not have data on the reason for absence to be able to m ake a definitive statem ent. 192. The differences noted from the MoEYS EMIS data between USDA-supported and non-USDA-supported schools in com pletion and dropout, and the im proved literacy rates found by WEI indicate the potential of the McGovern-Dole program m e to have long-term im pacts on the level of education achieved by children in this program m e. The evaluation also found a signficant difference in hunger (reduced) and attentiveness (im proved) in USDA-supported schools com pared with the counterfactual. The evaluation also found that total inattentiveness in class was significantly positively correlated with total percent reported hungry in class.118 193. Su st a in a b ilit y: The evaluation com m ends WFP and MoEYS on the significant progress m ade during 2017-2019 on transitioning school feeding activities to national ownership. This includes the transition of USDA-supported schools to the HGSF-hybrid m odel and handing over of the food based THR activities to the governm ent’s scholarship programme . 194. The declaration by the MoEYS of the ir preference for the HGSF model in 2018 requires that in the next phase of program m ing, WFP shift further away from direct im plem entation and focus m ore on capacity strengthening, i.e.transferring knowledge to MoEYS and other stakeholders on HGSF im plem entation, including providing technical support to policies and operational guidelines, strengthening coordination m echanism s between school feeding stakeholders and ensuring governm ent’s school feeding managem ent structures are in place. All these elem ents are rightfully already included in the new USDA McGovern-Dole program m e . 195. The upcom ing im plem entation of the new McGovern-Dole Grant and the new USDA LRP grant over the sam e period, provides a m ore feasible tim e period (to 2025) for the MoEYS to take full ownership of the school m eals program m e and gain sufficient additional capacity in the HGSF m odel to im plem ent the program m e. 3.3. Recommendations 3.1.1 Strategic recommendations Re com m e n d a t ion 1: The Cam bodia program m e has shown that school feeding can provide a e fficient and effective platform to support im provem ents in literacy and health. The ET therefore recom m ends that WFP Headquarters advocate for the expansion of 118 Correlation (r)=0.378 – a medium correlation, p<0.001 (two tailed) literacy activities into non-McGovern-Dole school feeding program m es to align with the tim eline of the new WFP CRF. Re com m e n d a t ion 2: The new WFP Strategic Plan outlines a change in role for WFP from project im plem enter to enabler. As such, WFP Headquarters should develop program m ing tools to guide Country Offices on how best to carry out this transition before the end of the tim eline of the current Strategic Plan. 3.1.2 Operational recommendations Re com m e n d a t ion 3: WFP Cam bodia, with support from WFP RBB and OSF as required, should continue to collaborate with MoEYS over the next phase to provide capacity building support to MoEYS personnel at central and subnational levels for the im plem entation of a nationally owned HGSF program m e. • This should entail the im m ediate roll out of the HGSF Guidelines in all the program m e locations (already planned), and a continued focus on nutrition and food safety to ensure that adequate procurem ent, storage and food handling procedures are in place before the end of the next program m ing phase. • WFP should also conduct a capacity assessm ent of the MoEYS and other governm ent bodies as appropriate within the next 12 m onths, to ensure that all the stakeholders with key roles and responsibilities in school feeding have sufficient capacity to im plem ent the HGSF program m e. Re com m e n d a t ion 4: WFP Cam bodia in collaboration with MoEYS and MEF should undertake a com prehensive costing exercise wthin the next 12 m onths to obtain sufficient information to determine appropriate levels of national funding for school feeding. Key com ponents of the costing exercise should include internal governm ent costs: staffing, transaction costs, m aterials travel etc. • This should include an adequate budget for the procurem ent of local com m odities for HGSF over the year considering seasonality/availability and changing food prices to ensure that adequate food can be provided throughout the school year. • The costing should include the m aintenance of school-feeding related school infrastructure and all other aspects of the program m e. • The costing should include the regular paym ent of cooks at a level appropriate for the workload to help realize the gender transform ative potential fo the program m e by the end of the next phase of McGovern-Dole program m ing. • WFP Cam bodia should continue to facilitate discussions with MoEYS and MoI at national and sub-national levels to find an agreem ent on the responsibilities for paym ent of the cook’s incentive under national ownership by the end of the next phase of McGovern-Dole program ming. Re com m e n d a t ion 5: WFP Cam bodia together with the MoEYS, im plem enting partners and other stakeholders should ensure that the transition plan is well docum ented and dissem inated at sub-national level within the next 12 m onths. • The plan should be sufficiently detailed to show the tim eline, benchm arks, roles and responsibilities of all stakeholders, as well as the locations and schools that will be im plem ented by the MoEYS during the next phase of program m ing. Re com m e n d a t ion 6: WFP Cam bodia and the MoEYS should continue to work together to ensure that the MoEYS’s program m e m onitoring capacity is adequate in the program m e locations targeted for governm ent im plem entation, by the end of the next phase of McGovern-Dole program m ing. • This will entail WFP training MoEYS personnel at all levels (central and sub￾national) on the use of the new school feeding m onitoring system once it’s finalized, ensuring the targeted locations have adequate physical digital m onitoring equipm ent (tablets and access to internet), and ensuring that inform ation flows efficiently from schools to relevant MoEYS personnel. 4 Bibliography Anthrologica & WFP (2018) Form ative rese arch to inform adolescent program m ing in Cam bodia: Engagem ent for health, nutrition and sustainable developm ent. Sum m ary Report – February 2018 Brody, A. (2017) Innovations from the Field: Gender m ainstream ing from the ground up. Cam bodia Mission Report Asian Developm ent Bank (2014) Cam bodia Country Poverty Analysis. Cam bodia Socioeconom ic Survey, 2014, National Institute of Statistics, Ministry of Planning; Available at: https://www.unicef.org/cam bodia/6.Maternal.pdf Com m une Database 2013, Ministry of Planning CSO report on Cam bodian gender issues. 2009 Dunn, S. et al (2017) Baseline Report for WFP Cam bodia and the USDA/McGovern Dole Food for Education Program m e, WFP Cam bodia, Decem ber 2017. The KonTerra Group Dunn, S. et al (2017) Endline Evaluation of United States Departm ent of Agriculture (USDA) McGovern-Dole Grant (MDG) Food for Education Program m e for WFP Cam bodia (2013- 2016). Final report. The KonTerra Group. Dunn, S., Silvéréano, J & Sin, S. (2018) Mid-term review of USDA McGovern-Dole Grant FFE-442-2016/015-00 for WFP School Feeding in Cam bodia covering school years 2016/17 and 2017/18 Ham , K., Kunthea, K & Chhe, B (2017) The Feasibility of the Im plem entation of School Feeding Program s in Prim ary School by the Ministry of Education, Youth and Sport, Departm ent of Policy, MoEYS. Heng, K. et al (2016) Research report. School Dropout in Cam bodia: A case study of Phnom Penh and Kam pong Speu. Korea International Cooperation Agency, Cam bodia Country Office. Royal University of Phnom Penh, Faculty of Education Horiuchi, Y et al (2019) Urban-rural differences in nutritional status and dietary intakes of school-aged children in Cam bodia. Nutrients. 2019 Jan; 11(1):14. http://hdr.undp.org/en/com posite/GII http://hdr.undp.org/en/countries/profiles/KHM http://hdr.undp.org/sites/all/them es/hdr_them e/country-notes/KHM.pdf http://www.uneval.org/docum ent/detail/2133 https://opendevelopm entcam bodia.net/topics/sdg-2-zero-hunger/ https://www.fas.usda.gov/program s/m cgovern-dole -food-education-program https://www.unwom en.org/- /m edia/headquarters/attachm ents/sections/how%20we%20work/unsystem coordinatio n/un-swap/un-swap-2-fram ework-and-technical-guidance -en.pdf?la=en&vs=1406 https://www.worldbank.org/en/news/press-release/2018/05/10/high-growth-in￾cam bodia -expected-to-continue -world-bank MacAuslan, I. et al (2019) Country-led evaluation of the National Education Scholarship Program m es of the Ministry of Education, Youth and Sport in Cam bodia, 2015-2018. Final Report – Volum e 1. UNICEF, Phnom Penh MoEYS EMIS data SY 2017/18 and SY 2018/19 MoEYS/WFP School Feeding Roadm ap. Signed 28 April 2015. Royal Governm ent of Cam bodia (2014) Poverty Estim ate in 2014 in Cam bodia, Ministry of Planning. Royal Governm ent of Cam bodia (2016) National Social Protection Policy Fram ework, 2016-2025 Royal Governm ent of Cam bodia (2017) Rectangular Strategy Phase IV, 2018-2023 Royal Governm ent of Cam bodia (2018) Education Strategic Plan, 2019-2023 Royal Governm ent of Cam bodia (2018) National Policy on School Health, Ministry of Education, Youth and Sport Royal Governm ent of Cam bodia (2018) National Strategic Developm ent Plan, 2019-2023 Royal Governm ent of Cam bodia (2018) National Strategy for Food Security and Nutrition, 2019-2023 Royal Governm ent of Cam bodia (2018) Report on a pilot application for the ISPA-FSN tool for the Hom e -Grown School Feeding Program m e in Cam bodia. Inter-Agency Social Protection Assessm ent for Food Security and Nutrition. Royal Governm ent of Cam bodia (2019) Equity-focused Prim ary Scholarship Fram ework 2019-2023, MoEYS, May 2019. Royal Governm ent of Cam bodia (2019) Guidelines on food safety in schools. MoEYS, May 2019 Royal Governm ent of Cam bodia (2018) National Strategy for Food Security and Nutrition, 2019-2023 Term s of Reference , Activity Evaluation of USDA McGovern-Dole Grants FFE for WFP School Feeding in Cam bodia from 2016 to 2019, WFP Cam bodia Country Office. UNDP (2015) Hum an Developm ent Report, 2015. UNDP (2018) United Nations Developm ent Assistance Fram ework 2019-2023, Cam bodia. DRAFT 14 August 2018 UNESCO/UNICEF (2012) Asia/Pacific: End of Decade Notes on Education for All – EFA Goal # 5 Gender Equity. Bangkok: UNESCO and UNICEF. UNWom en (2018) UN-SWAP 2.0 Fram ework and Technical Guidance: Accountability fram ework for m ainstream ing gender equality and the em powerm ent of wom en in United Nations entities. Novem ber 2018 WFP (2012) Update on WFP’s Safety Nets Policy: The role of food assistance in social protection. WFP/EB.A/2012/5-A WFP (2013) Revised School Feeding Policy: Prom oting innovation to achieve national ownership. WFP/EB.2/2013/4-C WFP (2014) WFP Gender Policy 2015-2020. WFP (2016) Gendered nature of intra -household decision-m aking in Cam bodia. WFP Cam bodia. Phnom Penh. WFP (2016) WFP Strategic Plan 2017-2021. WFP/EB.2/2016/4-A/1/Rev.2* WFP (2017) Nutrition Policy. WFP/EB.1/2017/4-C WFP (2017) Revised Corporate Results Fram ework (2017-2021), WFP/EB.2/2018/5- B/Rev.1. WFP (2018) Cam bodian Rice Landscape Analysis: Opportunities in the rice value chain to support rice fortification. Decem ber 2018 WFP (2018) Fill the Nutrient Gap – Cam bodia Sum m ary Report. October 2018 WFP (2018) Micronutrient Challenges and Solutions: A stock take. March 2018 WFP (2018) WFP Cam bodia Annual Country Report 2018. WFP, FAO et al (2018) Hom e -Grown School Feeding Resource Fram ework: Technical Docum ent. Rom e. World Bank (2012) Im plem entation Com pletion and Results Reports: Cam bodia Education Sector Support Project, 2012. World Bank. World Bank Open Data: http://data.worldbank.org/ World Education (2019) World Education Quarterly Report to WFP Cam bodia, Year 3 Quarter 3: April 1, 2019 – June 31, 2019. 5 Annexes All the annexes are provided in Volu m e 2 of this report (Annexes). List of Annexes Annex 1: Evaluation Terms of Reference........................................................................................................................................1 Annex 2: Key evaluation report users...........................................................................................................................................14 Annex 3: USDA McGovern-Dole Results Frameworks ..................................................................................................................15 Annex 4: Description of project activities.....................................................................................................................................17 Annex 5: Maps of WFP activities in Cambodia from 2017-2019...................................................................................................19 Annex 6: Key primary education indicators..................................................................................................................................21 Annex 7: Recommendations from the baseline (2017) and MTR (2018)......................................................................................22 Annex 8: Internal and external evaluation stakeholders..............................................................................................................23 Annex 9: Key evaluation questions...............................................................................................................................................26 Annex 10: Evaluation Matrix - Cambodia McGovern-Dole FFE Programme.................................................................................27 Annex 11: Full evaluation timeline ...............................................................................................................................................37 Annex 12: Summary of the quantitative field mission schedule...................................................................................................38 Annex 13: Qualitative field mission schedule ...............................................................................................................................39 Annex 14: Endline survey documents received from WFP CO (as at 30 September)...................................................................44 Annex 15: List of agencies.............................................................................................................................................................45 Annex 16: WFP school feeding interventions in sampled case schools since baseline ................................................................51 Annex 17: Map of target (treatment) provinces and comparison (control) districts ...................................................................53 Annex 18: Full list of case and comparison schools for endline evaluation..................................................................................54 Annex 19: 2017-2019 baseline methodology for selection of comparison schools .....................................................................59 Annex 20: Geographic breakdown of sampled schools and households for the quantitative survey..........................................62 Annex 21: School assessment.......................................................................................................................................................63 Annex 22: Teachers questionnaire ...............................................................................................................................................73 Annex 23: Household questionnaire.............................................................................................................................................75 Annex 24: Key informant guides...................................................................................................................................................93 Annex 25: Focus Group Discussion Guides...................................................................................................................................94 Annex 26: Completed Project Monitoring Plan ............................................................................................................................96 Annex 27: Additional findings on school feeding-related infrastructure......................................................................................98 Annex 28: Additional findings on schoolgardens .......................................................................................................................100 Annex 29: Additional findings on school water and sanitation-related infrastructure ...............................................................101 Annex 30: Additional findings from school survey .....................................................................................................................106 Annex 31: Additional benefits of school feeding ........................................................................................................................108 Annex 32: Additional findings from household survey...............................................................................................................110 6 List of Acronyms ACRC All Children Reading Cam bodia BTB Battam bang Province CARD Council for Agriculture and Rural Developm ent CDF Com m une Developm ent Fund CIP Com m une Investm ent Plan CDP Com m une Developm ent Plan CO (WFP) Country Office CP Country Program m e CRF Corporate Results Fram ework CSP Country Strategic Plan DAC (OECD) Developm ent Assistance Com m ittee DEQAS (WFP) Decentralized Evaluation Quality Assurance System DID Difference -in-difference DOE District Office of Education DSFC District School Feeding Com m ittee DTMT District Training and Monitoring Te ams EGR Early Grade Reading EGRA Early Grade Reading Assessm ent EMIS Education Managem ent Inform ation System EQAD Education Quality Assurance Departm ent ESP Education Strategic Plan ET Evaluation Team FAO (United Nations) Food and Agriculture Organization FAS Foreign Agricultural Service FGD Focus Group Discussion FY Financial Year GCNF Global Child Nutrition Forum GDI Gender Developm ent Index GDP Gross Dom estic Product GEEW Gender em powerm ent and equality of wom en GII Gender Inequality Index GIZ Deutsche Gesellschaft für Internationale Zusam m enarbeit GPE Global Partnership for Education HDI Hum an Developm ent Index HGSF Hom e -grown school feeding HQ (WFP) Headquarters IDS Institute of Developm ent Studies IRL Indochina Research Lim ited ISPA Inter-agency Social Protection Assessm ent KAPE Kam puchean Action for Prim ary Education KII Key Inform ant Interview KTM Kam pong Thom Province LMIC Lower-Middle -Incom e Country LRP Local and Regional Procurem ent LSFC Local School Feeding Com m ittee MAFF Ministry of Agriculture, Forestry and Fisheries MDG Millennium Developm ent Goal MEF Ministry of Econom y and FInance MGD McGovern-Dole MoEYS Ministry of Education, Youth and Sport MoH Ministry of Health MoI Ministry of Interior MoU Mem orandum of Understanding MoWA Ministry of Wom en’s Affairs MPI Multi-dim ensional Poverty Index MT Metric ton MTR Mid-term Review NCE No Cost Extension NER Net Enrolm ent Rate NGO Non-Governm ental Organization NSFP National Strategic Developm ent Plan NSFSN National Strategy for Food Security and Nutrition NSPPF National Social Protection Policy Fram ework OECD Organization for Econom ic Cooperation and Developm ent PCC Program m e Coordination Com m ittee PMP Perform ance Monitoring Plan PDoE Provincial Departm ent of Education PRISM Platform s for Real-tim e Inform ation SysteMs PSFC Provincial School Feeding Com m ittee PSM Propensity Score Matching PTA Parents and Teachers Association PTTC Provincial Teacher Training College RBB (WFP) Regional Bureau in Bangkok rCSI Reduced Coping Strategies Index RF Results Fram ework RS Rectangular Strategy SABER System s Approach for Better Education Results SBCC Social Behaviour Change Com m unication SDG Sustainable Developm ent Goal SEANUTS South East Asian Nutrition Survey SF School Feeding SFTF School Feeding Taskforce SMP School Meal Program m e SO Strategic Objective SRP Siem Reap Province SSC School Support Com m ittee SY School Year THR Take Hom e Ration ToR Term s of Reference UN United Nations UNDAF United Nations Developm ent Assistance Fram ework UNDP United Nations Developm ent Program m e UNEG United Nations Evaluation Group UNESCO United Nations Educational, Scientific and Cultural Organization UNICEF United Nations Children’s Fund UNSWAP United Nations System s-Wide Action Plan USD United States Dollar (currency) USAID United States Agency for International Developm ent USDA United States Departm ent of Agriculture WASH Water, sanitation and hygiene WEI World Education, Inc. WFP World Food Program m e WHO (United Nations) World Health Organization WV World Vision [WFP Cam bodia ] [Link to the website ] [Cam bodia , Septem ber 2019 download/ https://docs.wfp.org/api/docum ents/cdcbe4be57914c65b0dbcbd201196ab6/ and Year, Report num ber] Decentralized Evaluation Endline Evaluation of United States Department of Agriculture (USDA) McGovern-Dole Grant Food for Education Programme for WFP Cambodia FY 2017-2019 FINAL Evaluation Report: Volume 2 - Annexes January 2020 Commissioned by WFP Cambodia Evaluation Manager: Bunthang Chhe Prepared by Sophia Dunn, Team Leader Jean-Pierre Silvéréano, Evaluator Sovith Sin, Evaluator Prem Bhandari, Evaluator DECENTRALIZED EVALUATION FOR EVIDENCE-BASED DECISION MAKING ii List of Annexes Annex 1: Evaluation Terms of Reference ........................................................................................................................................1 Annex 2: Key evaluation report users...........................................................................................................................................14 Annex 3: USDA McGovern-Dole Results Frameworks...................................................................................................................15 Annex 4: Description of project activities.....................................................................................................................................17 Annex 5: Maps of WFP activities in Cambodia from 2017-2019...................................................................................................19 Annex 6: Key primary education indicators..................................................................................................................................21 Annex 7: Recommendations from the baseline (2017) and MTR (2018)......................................................................................22 Annex 8: Internal and external evaluation stakeholders..............................................................................................................23 Annex 9: Key evaluation questions...............................................................................................................................................26 Annex 10: Evaluation Matrix - Cambodia McGovern-Dole FFE Programme.................................................................................27 Annex 11: Full evaluation timeline................................................................................................................................................37 Annex 12: Summary of the quantitative field mission schedule...................................................................................................38 Annex 13: Qualitative field mission schedule ...............................................................................................................................39 Annex 14: Endline survey documents received from WFP CO (as at 30 September) ...................................................................44 Annex 15: List of agencies.............................................................................................................................................................45 Annex 16: WFP school feeding interventions in sampled case schools since baseline.................................................................51 Annex 17: Map of target (treatment) provinces and comparison (control) districts....................................................................53 Annex 18: Full list of case and comparison schools for endline evaluation..................................................................................54 Annex 19: 2017-2019 baseline methodology for selection of comparison schools......................................................................59 Annex 20: Geographic breakdown of sampled schools and households for the quantitative survey..........................................62 Annex 21: School assessment .......................................................................................................................................................63 Annex 22: Teachers questionnaire................................................................................................................................................73 Annex 23: Household questionnaire.............................................................................................................................................75 Annex 24: Key informant guides...................................................................................................................................................93 Annex 25: Focus Group Discussion Guides ...................................................................................................................................94 Annex 26: Completed Project Monitoring Plan ............................................................................................................................96 Annex 27: Additional findings on school feeding-related infrastructure......................................................................................98 Annex 28: Additional findings on schoolgardens........................................................................................................................100 Annex 29: Additional findings on school water and sanitation-related infrastructure................................................................101 Annex 30: Additional findings from school survey......................................................................................................................106 Annex 31: Additional benefits of school feeding ........................................................................................................................108 Annex 32: Additional findings from household survey ...............................................................................................................110 1 Annex 1: Evaluation Terms of Reference Introduction These Terms of Reference (TOR) are for the evaluation of United States Department of Agriculture (USDA) McGovern￾Dole International Food for Education and Child Nutrition Programme supported school feeding activities in Cambodia. The activity evaluation is commissioned by WFP Country Office (WFP Cambodia CO) and will cover the period from September 2017 to August 2019. These TOR were prepared by the WFP Cambodia CO based upon an initial document review and consultation with stakeholders and following a standard template. The purpose of the TOR is to provide key learning themes, programme scope, and other key information which can guide evaluation team to come with appropriate proposal for conducting the evaluation. Additionally, the purpose is to involve stakeholders early on, keep them informed of progress, and provide opportunities for inputs to secure their support and commitment. The evaluation process within WFP will be managed by an evaluation manager (WFP - EM) appointed by the WFP Cambodia Country Office (CO) who will be the main focal point for day to day contact during the evaluation period. An outside firm will be contracted to carry out the actual evaluation and will appoint their own evaluation manager. This evaluation will provide an evidence-based, independent assessment of the results of the programme to enable WFP-Cambodia, government and Cooperating Partners (CPs) to demonstrate results and learning to feed into future programmes in particular the government led and managed School Feeding Programme (SFP) while also making it possible to quantify the impacts of the programme. Reasons for the Evaluation The reasons for the evaluation being commissioned are presented below. Rationale The evaluation is being commissioned for the following reasons: • The WFP Cambodia CO is commissioning an endline evaluation for the FY 2017-2019 McGovern-Dole supported WFP Education Support activities in Cambodia to assess the performance of programme operations and associated interventions for the purposes of accountability and learning for programme strengthening. • This endline evaluation will also fulfil a requirement of the McGovern-Dole funded projects by critically and objectively evaluating the results of the FY 2017-2019 school feeding programme. The SY 2018/2019 evaluation will also determine whether recommendations made during the baseline survey and the mid￾term review were integrated into programme implementation and if so, whether these recommendations were successful in strengthening the programme. Objectives Evaluations in WFP serve the dual and mutually reinforcing objectives of accountability and learning. This evaluation is conducted for accountability purposes to USDA as a donor while having a learning purpose for WFP, partners – including government and other stakeholders to feed into future programme design and also for the key government counterpart the Ministry of Education, Youth and Sport (MoEYS) to take forward as they assume full management of the school feeding programme. • Accountability – The evaluation will assess and report on the performance and results of the McGovern-Dole funded activities during the funding period. • Learning – The evaluation will determine the reasons why certain results occurred or not to draw lessons, derive good practices and pointers for learning. It will provide evidence-based findings to inform operational and strategic decision-making. Findings will be actively disseminated, and lessons will be incorporated into relevant lesson sharing systems For learning, the final evaluation is to critically and objectively review and take stock of the program participant’s implementation experience and the implementing environment. For accountability, the evaluation is to assess whether targeted beneficiaries have received services as expected, assess whether the project is on track to meeting 2 its stated goals and objectives, evaluate the results frameworks and assumptions, and discuss necessary for future programme managements and designs modelled for government owned SFP. Stakeholders and Users A number of stakeholders both inside and outside of WFP have interests in the results of the evaluation and some of these will be asked to play a role in the evaluation process. Table 1 below provides a preliminary stakeholder analysis, which should be deepened by the evaluation team as part of the Inception phase. Accountability to affected populations, is tied to WFP’s commitments to include beneficiaries as key stakeholders in WFP’s work. As such, WFP is committed to ensuring gender equality and women’s empowerment (GEEW) in the evaluation process, with participation and consultation in the evaluation by women, men, boys and girlsfrom different groups through individual interview and group discussion with teachers and students as part of fieldsurvey. The primary users of this evaluation are detailed in table one below. Table 1: Preliminary Stakeholders’ analysis Stakeholders Interest in the evaluation and likely uses of evaluation report to this stakeholder INTERNAL STAKEHOLDERS Country Office Cambodia (CO) The WFP Cambodia country office has direct stake in decision-making, notably related to programme implementation and/or design, Country Strategy and partnerships, scaling up of activities or interventions Regional Bureau Bangkok (RB) Responsible for both oversight of COs and technical guidance and support, the RB management has an interest in an independent/impartial account of the operational performance as well as in learning from the evaluation findings to provide strategic guidance, programme support, and oversight. The Regional Evaluation Officers supports CO/RB management to ensure quality, credible and useful decentralized evaluations. WFP School Feeding Nutrition Units HQ and WFP HQ technical units are responsible for issuing and overseeing the rollout of normative guidance on corporate programme themes, activities and modalities, as well as of overarching corporate policies and strategies. They also have an interest in the lessons and may use evaluations for wider organizational learning and accountability, as many may have relevance beyond the geographical area of focus. Relevant HQ units should be consulted from the planning phase to ensure that key policy, strategic and programmatic considerations are understood from the onset of the evaluation. Office of Evaluation (OEV) OEV has a stake in ensuring that decentralized evaluations deliver quality, credible and useful evaluations respecting provisions for impartiality as well as roles and accountabilities of various decentralized evaluation stakeholders as identified in the evaluation policy. The office may use the evaluation findings, as appropriate, to feed into evaluation syntheses as well as for annual reporting to the Executive Board. WFP Executive Board (EB) The WFP governing body has an interest in being informed about the effectiveness of WFP programmes. This evaluation will not be presented to the Board, but its findings may feed into thematic and/or regional syntheses and corporate learning processes. EXTERNAL STAKEHOLDERS Beneficiaries Asthe ultimate recipients of food assistance, beneficiaries have a stake in WFP determining whether its assistance is appropriate and effective. As such, the level of participation in the evaluation of women, men, boys and girls from different groups will be determined and their respective perspectives will besought. 3 Government (MoEYS, MEF, MoH other) The Government has a direct interest in knowing whether WFP activities in the country are aligned with its priorities, harmonized with the action of other partners and meet the expected results. Issues related to capacity development, handover and sustainability will be of particular interest. MoEYS might use evaluation findings for decision making related to programme implementation and/or design, country strategy and partnerships. MoH, MAFF and CARD might also use these findings for their learning and implementation of programmes in the future. UN Country team The UNCT’s harmonized action should contribute to the realization of the government developmental objectives. It has therefore an interest in ensuring that WFP programmes are effective in contributing to the UN concerted efforts. Various agencies are also direct partners of WFP at policy and activity level. NGOs: WV, Plan and WEI NGOs are WFP’s partners for the implementation of some activities while having their own interventions. The results of the evaluation might affect future implementation modalities, strategic orientations and partnerships. Donors: USDA WFP operations are voluntarily funded by a number of donors. They have an interest in knowing whether their funds have been spent efficiently and if WFP’s work has been effective and contributed to their own strategies and programmes. USDA will use evaluation findings to inform project strategy, results frameworks, and critical assumptions. Other education, nutrition and social protection partners and stakeholders in Cambodia (USAID, UNICEF, NGOs, etc) Results from this evaluation will be used to inform the direction of government and WFP work on school meals moving forward. As part of the baseline, a reference groups comprising key stakeholdersin country wasformed to provide inputs and contribute to the related evaluation processes. Context Current economic growth has allowed Cambodia to attain lower-middle-income country (LMIC) status, with GDP per capita reaching USD 1,1591 . Over the past two decades, Cambodia has seen a significant decline in poverty rate, from 50 percent in 1992 to 13.5 percent in 20142 . Ranked 146 out of 189 countries on the UNDP 2017 Human Development Index, Cambodia's growth is expected to remain robust at around seven percent, driven by solid performances in garment manufacture, construction, tourism, and production of food and cash crops. Cambodia achieved the World Bank’s LMIC status in mid-2016, though it is recognized that human capital development and economic sustainability lag behind, thus delaying the graduation from the UN’s least developed country (LDC)rating. Despite economic growth and current development in urban areas, rural development lags behind. Rural communities, which make up 79 percent of the population, account or most of the country's poor3 . A significant proportion of Cambodians lives on the brink of poverty; it has been estimated that losing just USD 0.30 a day per person in income would double the poverty rate4 . This means that natural hazards such as storms, floods, droughts or serious illness could cause profound setbacks to fragile livelihoods. Food poverty reduced from 20 percent in 1993 to 4.1 percent in 2010 and zero in 2014, surpassing Cambodia’s Millennium Development Goal (MDG) 1 target. However, the newly proposed Sustainable Development Goal (SDG) indicators, undernourishment and dietary diversity, suggest that 14 percent of households continued to consume less than the minimum dietary energy requirement while 11.6 percent had inadequate dietary diversity5 . Thus, work remains to be done to end food insecurity and hunger for all. The 2014 Cambodia Demographic Health Survey (CDHS) found that the stunting rate fell from 49.2 percent in 2010 to 32.4 percent in 2014; approximately half a million Cambodian children under five are stunted while wasting remains unacceptably high at 9.6 percent. While micronutrient deficiencies appear to be reducing, iodine deficiency in 1 WB. World Bank Open Data: http://data.worldbank.org/ 2 MoP. Poverty Estimate in 2014 in Cambodia 3 Cambodia Inter-Censal Population Survey, 2013 4 WB Policy Note on Poverty Monitoring and Analysis, October 2013 5 Cambodia Socioeconomic Survey, 2014, National Institute of Statistics, Ministry of Planning 4 increasing, which impacts growth and cognitive development. Two out of three children aged 6-23 months do not have access to timely, appropriate, nutritionally adequate and safe complementary food. Total mortality rate of children under five is 35 per 1,000 live births of which malnutrition contributes 12.25 percent. It is important to note that while under-nutrition continues to play an important role in determining population wellness and productivity, over-nutrition is on the rise; while 14 percent of women between 15 and 49 years of age have a body mass index (BMI) below 18.5 (thin), 18 percent are overweight (BMI>25). This double burden is indicative of economic shifts and predicts greater challenges in future, including those associated with non-communicable diseases, unless addressed in a timely manner. The Royal Government of Cambodia is very committed to improving educational standards while aiming to embed programmessupported by development partners, such asthe school meals and take-home rations, within its national strategies. The national decentralization and deconcentration reforms, place greater responsibility with subnational authorities to take ownership of planning and delivery of basic services, including education. The roles of commune councils and community committees are also closely tied into this. Further, the introduction of greater social accountability, aims to mobilize communities as active participants in local social and economic development. In education, Cambodia has made good strides in improving primary education programs in rural areas. The net primary school enrolment figure increased from 81 percent in 2001 to 97.7 percent in 2016-2017. Dropout is pervasive in some regions, particularly towardsthe end of the primary school cycle asstudents become more likely to leave school rather than repeat a year; though not captured at aggregate level, attendance and absenteeism are of particular concern. Available national statistics don’t show substantial differences between boys and girls Particularly at primary school level where gender at national level in school year 2017-18 is composed of 52% for boys and 48% for girls. The percentage of gender composition is also the same for urban and rural area. The ASEAN integration in 2015 and the desire of Cambodia to be a middle-income country by 2030 require Cambodia to invest in its physical and human capital and adopt reforms to enable sustained and inclusive growth. Under MoEYS, USAID is currently implementing its new education strategy (2016-21), with a focus on improving early grade reading through their partners including Kampuchea Action for Primary Education (KAPE) and World Education International (WEI). WFP through KAPE and World Education International (WEI), will work closely with USAID and UNICEF to support early grade reading programme under the national education strategy and child friendly school policy framework. GIZ is working closely with school health department to support food safety and health in schools under a newly endorsed national school health policy. WFP is also working closely with Plan International, who works in the area of education, and World Vision, who work in the area of community development including education, to support MoEYS at national and subnational to provide SMP and THR which is under the national social protection strategy framework. The collaboration is also formed to provide infrastructure building and/or rehabilitation and other school supportintervention. School feeding programme is a major component of the ongoing WFP Cambodia’s Country Strategic Plan (CSP), and currently implementing in 8 out of the Cambodia’s 26 provinces. The programme in school year 2018-19 provides multiple food assistance modalities including school meals (SMP) benefiting 277,400 including 136,000 girls, and food take home rations (THR) benefiting 8,400 including 4,750 girls while cash scholarships were phased-out in 2018. A Home-Grown School Feeding (HGSF) model is being implemented in almost 300 schools under the SMP, utilizing locally procured commodities. A daily school meal (breakfast) is provided to all pre-and primary children attending morning classes, aimed to encourage student’s enrolment, attendance and completion of their primary education, and also to reduce short-term hunger and improve their concentration in the classroom. Food scholarships or take￾home ration is provided to the poor children in grade 4 to 6, aimed to further facilitate their progression in grades 4 to 6 and provide more equitable access to the Government’sscholarship programme for the poor in grade 7 to 9. The Persistent gender inequality is measured in the Gender Inequality Index; Cambodia ranks 93 out of 149 countries on the Global Gender Gap Index 20186 . However, women are increasingly income generators, migrating from rural areas to urban areas to work or starting small businesses from their homes. The number of women having primary occupation in the private sector is higher than men in many provinces7 , particularly in the garment sector. Women are typically employed at lower levels and paid less. It is estimated that on average women are paid thirty percent less than men on commensurate work8 . 6 Human Development Report,2015, UNDP 7 Commune Database 2013, Ministry of Planning 8 CSO report on Cambodian gender issues. 2009 5 Subject of the Evaluation WFP’s FY2017-2019 McGovern-Dole Programme of US$15 million grant implements a school meals (SMP), Hybrid Home Grown School Feeding programme (HGSF), food take home rations (THR) modalities and also supports complementary activities focused on improving literacy, and health and dietary practices in three of the eight WFP￾supported provinces in the country: Siem Reap, Battambang and Kampong Thom. The hybrid HGSF model is being implemented by utilizing locally procured commodities to complete USDA supported food commodity. The SMP and the HGSF cover 585 schools and benefits to 150,600 children including 73,000 girls while THR, which prioritize girls in areas where gender gap is high, covers 433 schools and benefits 8,400 children including 4,750girls. The Government of the United States of America, through the USDA McGovern-Dole Program, has been a trusted partner of the World Food Programme in Cambodia, dating back to 2001. Since its inception, this partnership has ensured that more than 3.6 million children have benefited from School Feeding Programmes. The 2017-2019 McGovern-Dole funded operation is a continuance of the 2013-2016 McGovern-Dole Program; implemented by WFP in partnership with World Education, KAPE, PLAN International, World Vision and Government ministries; covering the same provinces in Kampong Thom, Siem Reap and Battambang; and continuing the SMP programme in largely the same schools while discontinuing THR in Battambang from school year 2017-18 (please modalities of programme in different school years on map in annex 1) . The approach and transfer, under these grants, is largely the same with some schools receiving the SMP, some schools receiving the THR Programme and other schools receiving both programmes, depending on the detailed targeting criteria within WFP, which will be available for the evaluation team during inception phase if needed. WFP anticipates a full handover of THR schools and a gradual handover of SMP schools to Government in the years after the end of the 2017-2019 grant of 15 million USD; hence, the current grant is focussing strongly on the Government’s programme management and accelerate of handing over of these programme responsibilities to the Government. Additionally, the current grant is focused on partnership to achieve maximum impact, with a particularly strong focus on literacy improvements, especially among the grade 1 and 2 students whereas the previous round funding focussed on literacy of grade 6students. Recognizing the importance of sustainability of the school feeding programme moving forward, WFP’s overarching vision is to oversee the transition from a WFP-led school feeding programme in Cambodia to a government-led programme by 2021, as outlined in the School Feeding Roadmap which was agreed and signed between WFP and the Ministry of Education, Youth and Sports (MoEYS) in May 2015. WFP’s strategy is to utilize MGD commodities and capacity building funds to increase the readiness of MoEYS, preparing them for adopting the SMP and absorbing the THR beneficiaries into the national cash scholarship programme. As with previous cycle grants, the FY 2017-2019 McGovern Dole funded operation also requires undertaking a baseline survey, mid-term review and final evaluation. The baseline survey and mid-term review were conducted in 2017 and 2018 respectively, and the final evaluation will be conducted in 2019 with indicative date for each evaluation activities in annex 2: Evaluation schedule. The recommendations made during the baseline and mid-term were taken on and support activities have been adjusted or developed as manifest in management response. As in 2.3, the reports were shared with all user for their learning including decision making. The evaluation process within WFP will be managed by an evaluation manager appointed by the WFP Cambodia CO who will be the main focal point for day to day contact during the evaluation period with technical support and oversight from regional bureau. An external firm will be contracted to carry out the actual evaluation and will appoint their own evaluationmanager. The USDA-McGovern-Dole International Food for Education and Child Nutrition Program has two strategic objectives (as per the results framework outlined in Annex 10): Improved literacy of school-age children and increased use of health and dietary practices. To support literacy objective, a range of activities are designed to produce intermediate results of improving student attendance, quality of literacy instruction, and attentiveness. Similar to literacy, to support health and dietary practice objective, a range of activities are conducted to produce intermediate results of Improved Knowledge of Health and Hygiene Practices, Increased Knowledge of Safe Food Prep and Storage Practices, Increased Knowledge of Nutrition, Increased Access to Clean Water and Sanitation Services, Increased Access to Preventative Health Interventions and Increased Access to Requisite Food Prep and Storage Tools andEquipment. Evaluation Approach Scope The 2017-2019 endline evaluation will cover the WFP Cambodia School Feeding USDA McGovern-Dole International Food for Education and Child Nutrition Programme FFE-442-2016/015-00, including all activities and processesrelated to their formulation, implementation, resourcing, monitoring, evaluation, and reporting relevant to answer the 6 evaluation questions. This evaluation, commissioned by the WFP Cambodia Country Office, will cover three school feeding years9 of implementation of the McGovern-Dole funded operation for FY 2017-2019. The final evaluation will assess the impact of the program against the two strategic objectives: improve the literacy of school-age children and increase the use of improved health and dietary practices The final evaluation will assess progress from the beginning of the project period (referencing baseline and mid-term results) and will document lessons learned; assess relevance, effectiveness, efficiency, impact and sustainability efforts to date; and discuss and recommend for future programme. As such, the evaluation is focused on outcome and output results and their affecting factors, partnerships, implementation arrangements and systems, national ownership readiness, programme and information management approach, and community engagement. The final evaluation will be primarily for accountability and learning purposes. It will assess the progress of the indicators in the project agreement and Performance Monitoring Plan, and the recommendations of the baseline survey and the mid-term review. Evaluation Criteria and Questions Evaluation Criteria: Following the MTR evaluation criteria, the evaluation will apply the international evaluation criteria of Relevance, Effectiveness, Efficiency, Impact, Sustainability10. The selected criteria are well aligned with criteria agreed by McGovern-Dole in Table 2 and set in evaluation plan i.e., relevance is aligned with appropriateness; effectiveness, efficiency and impact are aligned with results of the programme; and sustainability is aligned with “Why and how the operation produced the observed results” and “Sustainability of the project moving forward”. The evaluation should analyse how GEEW objectives and GEEW mainstreaming principles were included in the intervention design, and whetherthe object has been guided by WFP and system-wide objectives on GEEW. The GEEW dimensions should be integrated into all evaluation criteria as appropriate. Evaluation Questions: Aligned with the evaluation criteria, the evaluation will address the following key questions, which will be further developed by the evaluation team during the inception phase. Collectively, the questions aim at highlighting the key lessons and performance of the USDA McGovern Dole Grants FFE, which could inform future strategic and operational decisions. Table 2: Criteria and evaluation questions Criteria Evaluation Questions Appropriateness Areas for analysis will include the extent to which the objectives, targeting, choice of activities and of transfer modalities were: Appropriate to the needs of the target population on both females and males; Aligned with relevant stated national policies, including sector policies and strategies and seek complementarity with the interventions of relevant humanitarian and development partners Aligned with WFP strategies, policies and normative guidance. 9 In school year 2016-17 the current grant ran in parallel with a former USDA McGovern-Dole grant for the same programme. Under the current grant, literacy activities were introduced since the start of the school year 2016-17; THRs were distributed from August 2017; meals were provided from November 2017; and infrastructure, WASH, training and capacity development activities were implemented from November 2017. 10 For more detail see: http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm and http://www.alnap.org/what-we-do/evaluation/eha 7 Results of the programme While ensuring that differences in benefits between boys and girls from different groups are considered, the evaluation will analyze: The efficiency of the program (attainment of the planned outputs, cost factors, logistics and pipeline performance); The effectiveness of the program (the contribution of outputs to selected program objectives); The intended and unintended impacts of the program (comparing to non-program schools/areas against selected program objectives). MoEYS capacity to manage the school feeding programme moving forward: Has the Government of Cambodia adequately staffed and resourced MoEYS to effectively assume management of the school feeding programme in USDA supported areas? Is the design of the programme suitable to government control? Is there a functioning government reporting and monitoring and evaluation system? Are community feedback mechanisms in place? Why and how the operation produced the observed results The evaluation will generate insights into the main internal and external factors that caused the observed changes and affected how results were achieved. The inquiry is likely to focus, amongst others, on: Internally (factors within WFP’s control): the processes, systems and tools in place to support the operation design, implementation, monitoring/ evaluation and reporting; the governance structure and institutional arrangements (including issues related to staffing, capacity and technical backstopping from RB/HQ); the partnership and coordination arrangements; etc. Externally (factors outside WFP’s control): the external operating environment; the funding climate; external incentives and pressures; etc. The implications for this as the programme transitions to MoEYS Sustainability of the project moving forward With a roadmap for national ownership of the school feeding programme already being implemented, the government will begin to gradually assume ownership of the programme within period of the roadmap and beyond. As such, the evaluation will have a strong focus on USDA-supported implementation models tested over the course of the project period. It should draw conclusions about the following: Cost effectiveness, efficiency and practicality of each of the models tested Government buy in on each model tested Recommendations on which model may be most appropriate given the findings Evidence that activities are likely to be sustained or scaled up beyond the project life The evaluation will also assess the success of USDA supported initiatives to improve information systems within MoEYS to ensure transparency and cost effectiveness of the programme. Data Availability The main sources of information available to the evaluation team are outlined in annex 7. These are not exhaustive and additional information can be provided as needed and availability. Concerning the quality of data and information, the evaluation team should: assess data availability and reliability as part of the inception phase expanding on the information provided in section 4.3. This assessment will inform the data collection systematically check accuracy, consistency and validity of collected data and information and acknowledge any limitations/caveats in drawing conclusions using the data. 8 Methodology The endline evaluation will replicate the previous endline and baseline approach and methodology11 . The detailed methodology will be developed by the evaluation team during the inception phase. It will: • Employ the evaluation criteria of relevance, effectiveness, efficiency, impact, and sustainability • Demonstrate impartiality and lack of biases by relying on a cross-section of information sources (stakeholder groups, including beneficiaries, etc.) The selection of field visit sites will also need to demonstrate impartiality. • Use mixed methods(quantitative and qualitative) to ensure triangulate information from difference methods and sources to enhance the reliability and credibility of the findings. The methods are conducted in parallel and used to triangulate information from difference methods and sources to enhance the reliability and credibility of the findings. Qualitative approach includes focus group discussions and key informant interviews while quantitative approach involves reviewing and collecting quantitative data from the existing monitoring data and a cross-sectionalsurvey of a sub-sample ofschool feeding schools visited in the previous baseline survey. During the inception phase, the service provider should provide a detailed methodology of how they intend to conduct the endline evaluation. Qualitative methods such as focus group discussions and key informant interviews, will be used where relevant to highlight lessons learned and case studies representative of the interventions; • Apply an evaluation matrix geared towards addressing the key evaluation questions considering the data availability challenges, the budget and timing constraints; • Ensure through the use of mixed methodsthat women, girls, men and boysfrom differentstakeholder groups participate and that their different voices are heard and used; • Partnership with local research firmsis encouraged. Thisincludesthe use of local enumeratorsfor any survey work, ensuring that culturally and political sensitivities are addressed and that the enumeration teams have the local language expertise to elicit the needed information from beneficiaries • The methodology will be GEEW-sensitive, indicating what data collection methods are employed to seek information on GEEW issues and to ensure the inclusion of women and marginalised groups. • The following potential risks to the methodology have been identified. School year will be finish by August; hence, the data collection should be done prior and/or during this month to get all information from difference kind of respondents such as teachers, cooks, storekeeper, parents and students. Language and culture are also barriers for the evaluation; hence, evaluation team should be aware of and take pre-emptive action before going down to the filed. Quality Assurance and Quality Assessment WFP’s Decentralized Evaluation Quality Assurance System (DEQAS) defines the quality standards expected from this evaluation and sets out processes with in-built steps for Quality Assurance, Templates for evaluation products and Checklists for their review. DEQAS is closely aligned to the WFP’s evaluation quality assurance system (EQAS) and is based on the UNEG norms and standards and good practice of the international evaluation community and aims to ensure that the evaluation process and products conform to best practice. DEQAS will be systematically applied to this evaluation. The WFP Evaluation Manager will be responsible for ensuring that the evaluation progresses as per the DEQAS Process Guide and for conducting a rigorous quality control of the evaluation products ahead of their finalization. WFP has developed a set of Quality Assurance Checklists for its decentralized evaluations. This includes Checklists for feedback on quality for each of the evaluation products. The relevant Checklist will be applied at each stage, to ensure the quality of the evaluation process and outputs. 11 Quasi-experimental design was used in the endline and baseline in 2017. Before/after comparison will be done through use of the same sampling strategies followed in the 2017 baseline survey. 9 To enhance the quality and credibility of this evaluation, an outsourced quality support (QS) service directly managed by WFP’s Office of Evaluation in Headquarter provides review of the draft inception and evaluation report (in addition to the same provided on draft TOR), and provide: • systematic feedback from an evaluation perspective, on the quality of the draft inception and evaluation report; • recommendations on how to improve the quality of the final inception/evaluation report. The evaluation manager will review the feedback and recommendations from QS and share with the team leader, who is expected to use them to finalise the inception/ evaluation report. To ensure transparency and credibility of the process in line with the UNEG norms and standards[1], a rationale should be provided for any recommendations that the team does not take into account when finalising the report. This quality assurance process as outline above does not interfere with the views and independence of the evaluation team, but ensures the report provides the necessary evidence in a clear and convincing way and draws its conclusions on that basis. The evaluation team will be required to ensure the quality of data (validity, consistency and accuracy) throughout the analytical and reporting phases. The evaluation team should be assured of the accessibility of all relevant documentation within the provisions of the directive on disclosure of information. This is available in WFP’s Directive CP2010/001 on Information Disclosure. All final evaluation reports will be subjected to a post hoc quality assessment by an independent entity through a process that is managed by OEV. The overall rating category of the reports will be made public alongside the evaluation reports. Phases and Deliverables The evaluation will proceed through the following phases. The deliverables and deadlines for each phase are as follows: Figure 1: Summary Process Map 1. Prepare 2. Inception •Inception Report 3.Collect data • Debriefing PPT 4. Analyze data and Report •Evaluation Report 5.Disseminate and follow-up Timeline: The timeline of for the whole endline evaluation is from April to November 2019, covering planning/preparation, inception, data collection, data processing and data analysis and report, and dissemination (see details in endline evaluation planning and deliverable timelines below). For the evaluation team, the timing will start from May, with any primary data collection to begin no later the first week of August. The specific timetable is shown in annex 2. Deliverable timelines: The main deliverables table are detailed in annex 6. Organization of the Evaluation & Ethics Evaluation Conduct The evaluation team will conduct the evaluation under the direction of its team leader and in close communication with the WFP CO evaluation manager. The team will be hired following agreement with WFP on its composition. The team will conduct and report on the evaluation according to McGovern-Dole and WFP standards as follows: • Must be financially and legally separate from the participant's organization; • Must have personal and professional integrity; [1] UNEG Norm #7 states “that transparency is an essential element that establishestrust and builds confidence, enhancesstakeholder ownership and increases public accountability” 10 • Must respect the right of institutions and individuals to provide information in confidence and ensure that sensitive data cannot be traced to its source. Evaluators must take care that those involved in evaluations have a chance to examine the statements attributed to them; • Must be sensitive to beliefs, manners and customs of the social and cultural environments in which they work; In light of the United Nations Universal Declaration of Human Rights, evaluators must be sensitive to and address issues of discrimination and gender inequality; and Evaluations sometimes uncover evidence of wrongdoing. Such cases must be reported discreetly to the appropriate investigative body. Also, the evaluators are not expected to evaluate the personal performance of individuals and must balance an evaluation of management functions with due consideration for this principle. The evaluation team will not have been involved in the design or implementation of the subject of evaluation or have any other conflicts of interest. Further, they will act impartially and respect the code of conduct of the evaluation profession. For the WFP CO evaluation manager, s/he will not take any role in the independent evaluation team and has no direct involvement in the implementation of the subject of the evaluation. Team composition and competencies The evaluation team is expected to include 3-5 members, including the team leader and other team members as necessary to ensure a complementary mix of expertise in the technical areas covered by the evaluation for both national and international (exclude field enumerators). To the extent possible, the evaluation will be conducted by a gender-balanced, geographically and culturally diverse team with appropriate skills to assess gender dimensions of the subject as specified in the scope, approach and methodology sections of the ToR. At least one team member should have WFP experience. The team will be multi-disciplinary and include members who together include an appropriate balance of expertise and practical knowledge in the following areas: • Institutional capacity development (with a focus on handover process, cost-efficiency analysis, supply chain management, logistics) • School feeding, education, nutrition and food security • Agro-economics/rural development • Knowledge management • Gender expertise / good knowledge of gender analysis • Survey, sampling, and statistical skills All team members should have strong analytical and communication skills, evaluation experience and familiarity with Cambodia country and/region. All team members should have strong skills in oral and written English. Given that local counterparts and beneficiaries may have limited English, partnership with local organization/firm for field work in Khmer will berecommended. The Team leader will have technical expertise in one of the technical areaslisted above as well as expertise in designing methodology and data collection tools and demonstrated experience in leading similar evaluations. She/he will also have leadership, analytical and communication skills, including a track record of excellent English writing and presentation skills. Her/his primary responsibilities will be: i) defining the evaluation approach and methodology; ii) guiding and managing the team; iii) leading the evaluation mission and representing the evaluation team; iv) drafting and revising, as required, the inception report, the end of field work (i.e. exit) debriefing presentation and evaluation report in line with DEQAS. The team members will bring together a complementary combination of the technical expertise required and have a track record of written work on similar assignments. Team members will: i) contribute to the methodology in their area of expertise based on a document review; ii) conduct field work; iii) participate in team meetings and meetings with stakeholders; iv) contribute to the drafting and revision of the evaluation products in their technical area(s). 11 Security Considerations Security clearance: Security concern is not much regardless men and women to work in Cambodia context though precaution is made to snatching in city at night-time. However, security clearance where required is to be obtained from the Cambodia CO. As an independent supplier’ of evaluation services to WFP, the evaluation company is responsible for ensuring the security of all persons contracted, including adequate arrangements for evacuation for medical or situational reasons. The consultants contracted by the evaluation company do not fall under the UN Department of Safety & Security (UNDSS) system for UN personnel. However, to avoid any security incidents, the Evaluation Manager is requested to ensure: • The WFP CO registers the team members with the Security Officer on arrival in country and arranges a security briefing for them to gain an understanding of the security situation on the ground. • The team members observe applicable UN security rules and regulations –e.g. curfews etc. Ethics WFP's decentralised evaluations must conform to WFP and UNEG ethical standards and norms. The contractors undertaking the evaluations are responsible for safeguarding and ensuring ethics at all stages of the evaluation cycle (preparation and design, data collection, data analysis, reporting and dissemination). This should include, but is not limited to, ensuring informed consent, protecting privacy, confidentiality and anonymity of participants, ensuring cultural sensitivity, respecting the autonomy of participants, ensuring fair recruitment of participants (including women and socially excluded groups) and ensuring that the evaluation results in no harm to participants or their communities. Contractors are responsible for managing any potential ethical risks and issues and must put in place in consultation with the Evaluation Manager, processes and systems to identify, report and resolve any ethical issues that might arise during the implementation of the evaluation. Ethical approvals and reviews by relevant national and institutional review boards must be sought where required. Roles and Responsibilities of Stakeholders The WFP Cambodia Country Office: • The WFP CO Management will take responsibility to: • Assign an evaluation manager for the evaluation: Mr Bunthang Chhe, Programme Policy Officer (Programme Support). • Compose the internal evaluation committee and the evaluation reference group • Approve the final ToR, inception and evaluation reports. • Ensure the independence and impartiality of the evaluation at all stages, including establishment of an evaluation committee and of an evaluation reference group (see below and TN on Independence and Impartiality). • Participate in discussions on the evaluation design and the evaluation subject, its performance and results with the evaluation manager and the evaluation team • Organise and participate in two separate debriefings, one internal and one with externalstakeholders • Oversee dissemination and follow-up processes, including the preparation of a management response to the evaluation recommendations The Evaluation Manager: • Manages the evaluation process through all phases including drafting this TOR • Ensures quality assurance mechanisms are operational • Consolidates and shares comments on draft TOR, inception and evaluation reports with the evaluation team • Ensures expected use of quality assurance mechanisms (checklists, quality support • Ensures that the team has access to all documentation and information necessary to the evaluation; facilitates the team’s contacts with local stakeholders; sets up meetings, field visits; provides logistic support during the fieldwork; and arranges for interpretation, ifrequired. • Organises security briefings for the evaluation team and provides any materials asrequired 12 An internal Evaluation Committee has been formed as part of ensuring the independence and impartiality of the evaluation. The role and responsibility of committee members will be detailed in annex 2. An internal evaluation committee chaired by the Country Director (CD) will approve Terms of Reference, budget, evaluation team, inception and evaluation reports, which helps to maintain distance from influence by programme implementers. An Evaluation Reference Group has been formed, as appropriate, with representation from WFP country office, Regional Bureau, Government partners, UN agencies and NGO partners. Please refer to annex 4 where list of members is available. The ERG members will review and comment on the draft evaluation products and act as key informants in order to further safeguard against bias and influence. The Regional Bureau: the RB will take responsibility to: • Advise the Evaluation Manager and provide support to the evaluation process where appropriate. • Participate in discussions with the evaluation team on the evaluation design and on the evaluation subject as required. • Provide comments on the draft TOR, Inception and Evaluation reports • Support the Management Response to the evaluation and track the implementation of the recommendations. • While the Regional Evaluation Officer will perform most of the above responsibilities, other RB relevant technical staff may participate in the evaluation reference group and/or comment on evaluation products as appropriate. Relevant WFP Headquarters divisions will take responsibility to: • Discuss WFP strategies, policies or systems in their area of responsibility and subject of evaluation. • Comment on the evaluation TOR, inception and evaluation reports, asrequired. • Other Stakeholders (Government, NGOs, UN agencies) will perform the roles and responsibilities of evaluation reference group since they are members of the group. • The Office of Evaluation (OEV). OEV, through the Regional Evaluation Officer, will advise the Evaluation Manager and provide support to the evaluation process when required. It is responsible for providing access to the outsourced quality support service reviewing draft ToR, inception and evaluation reports from an evaluation perspective. It also ensures a help desk function upon request. USDA FAS • Provide inputs and comment on draft endline evaluation ToRs. • Participate in an introduction teleconference with the selected independent evaluator prior to evaluate field work for the endline evaluation. • Provide comment on the endline evaluation inception report. • Participate in discussions of findings and recommendations that suggest changes in the project strategy, results frameworks and critical assumptions. • Provide comment on the endline evaluation report Communication and budget Communication To ensure a smooth and efficient process and enhance the learning from this evaluation, the evaluation team should place emphasis on transparent and open communication with key stakeholders. These will be achieved by ensuring a clear agreement on channels and frequency of communication with and between key stakeholders and by producing clear deliverables that are written in both English and Khmer. The Evaluation manager will submit all final deliverables to the WFP CO for pre-approval. Upon pre-approval of deliverables, the WFP CO will forward the deliverablesto WFP’s Washington Office with the Bangkok Regional Bureau in copy. WFP’s Washington Office will transmit deliverables to the USDA FAD for comments and inputs. All communication with USDA will be transmitted via WFP’s Washington Office including invitations to the FAD programme staff to participate in teleconferences to discuss CO management responses to evaluate findings and recommendations. 13 The service provider will deliver an evaluation report. USDA comments on final draft report will be taken into consideration by the evaluation team in addition to comments from external stakeholders in the evaluation reference group. The evaluation team will produce an excel file indicating all commentsreceived and how these were addressed. Exit debriefings will follow all field visits. A final presentation on the overall findings will be delivered to theCO. The Communication and Learning Plan should include a GEEW responsive dissemination strategy, indicating how findings including GEEW will be disseminated and how stakeholders interested or those affected by GEEW issues will be engaged. As part of the international standards for evaluation, WFP requires that all evaluations are made publicly available. To ensure maximum use of the lessons learned for national partners, the resulting reports will be translated into Khmer language. Importantly, this will facilitate learning amongst government, as technical staff often do not speak or read English Budget Funding Source: The endline evaluation will be funded by the WFP Cambodia Country Office using the M&E budget allocation in the McGovern-Dole grant funds. The service provider will outline their budget in a financial proposal to WFP as part of their response to the Request for Proposals (RfP) (Annex 2: Evaluation schedule indicated number of days which help evaluation team to estimate the budget). For the purpose of this evaluation the service provider will: Include budget for international and domestic travel and for all relevant in-country data collection (both qualitative and quantitative) Hire and supervise any and all technical and administrative assistance required (including in-country). The final budget and handling will be determined by the option of contracting that will be used and the rates that will apply at the time of contracting. Follow the agreed rates for decentralized evaluations as provided for in the Long-Term Agreement (LTA) with WFP 14 Annex 2: Key evaluation report users • WFP Cambodia CO and its main implementing partner, the Ministry of Education, Youth and Sport (MoEYS), notably with respect to decision-making related to programme implementation and/or design, country strategy and partnerships. The Royal Government of Cambodia is expected to take over the management and monitoring of the school feeding programme, therefore information on whether the programme is yielding the desired results is of primary importance. • Ministry of Health (MoH), Ministry of Agriculture, Forestry and Fisheries (MAFF), and the Council for Agriculture and Rural Development (CARD) (as appropriate). • Implementing partners including Plan International, World Education, KAPE and World Vision International as well as others involved in programme design, including schoolcommittees. • Findings will also be shared with education development partners, including USAID and other key education, nutrition and health stakeholders. • USDA will use the evaluation findings to inform project strategy, results frameworks and critical assumptions. • WFP’s Regional Bureau in Bangkok (RBB) is expected to use the evaluation findings to provide strategic guidance, programme support, oversight, and to extract lessons for sharing across the region. • WFP Headquarters (HQ) may use evaluations for wider organizational learning andaccountability. • WFP’s Office of Evaluation (OEV) may use the evaluation findings, as appropriate, to feed into evaluation syntheses. • Other WFP Regional Bureaux and COs under their oversight may also benefit from the findings, which can contribute to corporate learning on implementation of capacity development interventions. • WFP’s governing body, the Executive Board, has an interest in being informed about the effectiveness of WFP operations. The endline evaluation report will not be presented to the WFP Executive Board but its findings may feed into annual syntheses and into corporate learning processes. 15 Annex 3: USDA McGovern-Dole Results Frameworks 16 17 Annex 4: Description of project activities 1. Capacity building: Local, regional, national level WFP works in close collaboration with the MoEYS to strengthen institutional capacities in the ministry, its subnational authorities and local communities to take full ownership of the school feeding programme in accordance with the roadmap signed between WFP and the MoEYS in May 2015. WFP assists the MoEYS with identifying the most appropriate school meals model and integrating school meals strategies into relevant national policies, development plans and associated legislation. WFP also assists the MoEYS in establishing strategies for the financing, organizational structure and management of the school meals programme. 2. Provide school meals WFP, in collaboration with the MoEYS, provides a daily hot breakfast to primary school children in Grades 1-6 WFP use the USDA donated fortified rice and vegetable oil in the meals and complement this with pulses, animal protein and condiments (salt) sourced from other donors. WFP partners also provide training, coaching and mentoring to school staff and cooks on minimum food handling and preparation standards, as per the MoEYS guidelines. WFP and its partners are responsible for monitoring the distribution of daily meals. 3. Provide take home rations WFP provides food scholarships/THRs of 10kg of rice and 0.91 kg of oil two times during the school year to vulnerable children in Grades 4-6 who are at risk of dropping out of school. The intention of the THR is to encourage vulnerable children, particularly girls, to continue to attend school. The distributions are managed by the local school feeding committees (LSFC), with oversight and monitoring by WFP and its partners along with district and provincial authorities. 4. Establish school gardens WFP’s partners purchase vegetable seeds and distributes them to schools with adequate capacity and prerequisite inputs for on-site gardening. WFP and the MoEYS use the produce derived from the gardens in school meal preparation. WFP, the MoEYS and other partners also use the school gardens as a basis for life skills training, and environmental, agricultural and nutritional awareness raising and skills development. WFP's partners provide technical support and oversight following Food and Agriculture Organization (FAO) curriculum guidance. 5. Training: Commodity management WFP uses existing training materials, which are endorsed by the MoEYS, to train members of the LSFC in standard practices of supply management. WFP places focus on schools affected by staff turn-over or changes in the LSFC structure. WFP also provides on-the-job routine coaching and mentoring. 6. Training: Food preparation and storage practices WFP provides training to LSFCs on storing, handling and preparing food, management of cooking activities, and cleanliness and prevention of infestation, following a standard training guide developed by WFP and the MoEYS. WFP provides routine mentoring and coaching on the training subjects during periodic school monitoring visits. WFP also works with the MoEYS to introduce district and provincial level cooking competitions and share successful recipes and enhanced cooking practices. 7. Training: School administrators In collaboration with the MoEYS, WFP’s partner World Education provides training for school administrators and school principals focusing on literacy coaching. The training follows the minimum 18 standards established by the MoEYS. WFP also includes the same content in other trainings under this project (e.g. food preparation and storage management training) when and where possible. 8. Training: Teachers WFP partners work with the MoEYS Teacher Training Department to train and mentor teachers, school administrators and provincial and district education authorities to ensure the effective use of new national benchmarks for Early Grade Reading (EGR) in Grades 1, 2 and 3. Partners conduct EGR assessments for assisted schools, provide four-day start-up trainings, and offer regular refresher sessions to Grades 1-3 teachers, school directors and librarians. Partners use phone calls and in-person visits to schools to provide ongoing monitoring and mentoring support to school staff. 9. Establish activities to promote literacy Following an annual assessment of school conditions and performance, WFP's partners provide all EGR target schools with reading materials and test kits to introduce new literacy education approaches in accordance with new national benchmarks. Partners also provide toolkits to the school libraries so that they may be used by children during the library break or borrowed by parents to use at home. The toolkits include guidance materials for teachers which outline the reading benchmarks for students in Grades 1-3 (to assist in determining their progress); reading materials (classroom learning); and games and books (break and home learning). In schools receiving the EGR support, partners also train school staff to carry out meetings to encourage parents and community members to take a more active role in schools and organize activities in the library that reinforce reading skills acquired in the classroom. 10. Building/Rehabilitation: Latrines WFP's partners build or rehabilitate latrines, ensuring that these are connected to a hand washing station, and provide guidance on their use. WFP also builds or rehabilitates latrines in schools where the latrine-to￾student ratio is below the national standard of one latrine per 50 students and where it is satisfactorily shown that schools are willing to maintain the latrines. Where possible, WFP arranges separate latrines for boys and girls. WFP requires local communities to contribute building or rehabilitationmaterials. 11. Building/Rehabilitation: Wells and water stations/systems Following the WFP's annualschool assessment, WFP and itssub-recipients construct and rehabilitate wells, rainwater harvesting and water purification systems in schools that are most in need and in which the community and school management demonstrate greatest commitment to the maintenance of the systems. Through its sub-recipients, WPP also provides training for the appropriate use and maintenance of the wells, rainwater harvesting and water purification systems. 12. Building/Rehabilitation: Kitchens 13. Provide energy efficient stoves 14. Training: Good health and nutrition practices WFP and partners work closely with the School Health Department of the MoEYS to provide materials and training on good health and nutritional practices to members of every LSFC. WFP trains the LSFC on nutrition and hygiene practicesto facilitate discussion and awarenessraising among students, their parents and the larger community. WFP also conducts cooking demonstrations in conjunction with trainings in commodity management, storage and food preparation for all cooks in participating schools. 15. Raising awareness on the importance of education In collaboration with the MoEYS and other partners, WFP works with LSFCs to provide key messages on the value of education to students, their parents and communities. WFP delivers messages at events such as take-home ration distributions and focus group discussions. WFP promotes the roles and responsibilities of parents in encouraging children to study at home during after school hours, parental participation in and contribution to their children's education, as well as hygiene and nutrition. 19 Annex 5: Maps of WFP activities in Cambodia from 2017-2019 20 21 Annex 6: Key primary education indicators In the three provinces where the USDA FFE programme is implemented, in SY2018/19 the primary education completion rate was as follows: • Primary education completion rates by gender (2018-2019) in USDA supported provinces(%) Source: EMIS data, MoEYS. Dropout accelerates toward the end of the primary school cycle, as shown in the graph below. This means that interventions such as scholarships are particularly important in late primary grades, to help prevent drop-out and enable student to access scholarships in lower-secondary grades. • Dropout 2017/2018 through primary education cycle (%) Source: EMIS data, MoEYS. In the three provinces where the USDA FFE programme wasimplemented, in SY2017/2018 dropout waslowest in Siem Reap, and lower for girls than for boys in all three provinces. • Dropout 2017-2018 in USDA supported provinces(%) Source: EMIS data, MoEYS. Battambang Siem Reap Kampong Thom 4.1 2.9 5.3 4.6 4.6 5.4 8 6 4 2 0 Total Girls Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 3.2 3.4 3.0 4.0 5.0 4.2 4.4 3.5 3.2 3.5 5.7 6 4 2 0 6.2 8 Total Girls Battambang Siem Reap Kampong Thom 50 0 75.75 84.74 77.80 89.91 77.26 100 84.79 Girl Boy 22 Annex 7: Recommendations from the baseline (2017) and MTR (2018) Recommendations from the baseline (2017) 1 Continue programme implementation as per the current agreement (2017-2019). 2 The WFP CO and literacy partners (World Education and KAPE) should ensure that all USDA supported schools receive training and support to implement the new literacy curriculum by the end of the 2017- 2019 programme cycle. 3 The WFP CO and the other implementing partners (World Vision and PLAN International) should aim to have a minimum package of infrastructure facilities including kitchens, energy efficient stoves, storerooms, handwashing facilities and separate latrines for girls and boys in all USDA supported schools with the school meals programme. 4 The WFP CO should focus on capacity building of the Royal Government of Cambodia and review the Roadmap benchmarks as per WFP’s current plan and the findings of the 2013-2016 endline evaluation. Recommendations from the MTR (2018) 1 WFP should strengthen and monitor the connectedness of all activities implemented under the USDA McGovern-Dole FFE programme with reference to both the rationale of the programme and USDA guidance. 2 WFP and the MoEYS should ensure that detailed transition planning take place as soon as possible to enable the development of a realistic timeframe for transitioning to national ownership. 3 Ensure that the currentschool meals/ HGSF guidelines are revised and ready for use if/when the MoEYS HGSF proposal is funded. 4 WFP together with the MoEYS and other ministries as appropriate should carry out a detailed costing exercise of HGSF to enable the MoEYS and the Ministry of Economics and Finance (MEF) to accurately budget for national transition. 23 Annex 8: Internal and external evaluation stakeholders Stakeholder Interest in the programme Involvement in evaluation and likely use Who Internal (WFP) stakeholders WFP Cambodia Country Office • Responsible for the country level planning and programme implementation, the CO is the primary stakeholder of the endline evaluation. • The COhas a directstake in the evaluation and interest in learning and receiving feedback on how the CO will move forward. • The CO is also called upon to account internally as well asto its beneficiaries and partners for the performance and results if its operation. • Development of the ToR • Provision of documents, reports, information and data to the team • Initial briefing and overview of WFP work in Cambodia, strategic thinking, and planned responses for the future • Participate as key informants • Admin and logistics support • Participate in debriefings and provide feedback on preliminary findings and conclusions • Evaluation will show where decision-making related to programme implementation and/or design, country strategy and partnerships is needed. • Evaluation Focal Point • Country Director • Head of Programme • Head of Education Unit • Head of Sub-Office Siem Reap • CO programme and support staff as required Regional Bureau in Bangkok • Responsible for both oversight of CO and technical guidance and support. • The RB management has an interest in an independent account of the operational performance as well as in learning from the evaluation findings to apply this learning to other COs • Participate as key informants • Evaluation will provide strategic guidance, programme support, oversight, and to extract lessons for sharing across the region • Regional School Feeding Advisor • Regional Evaluation Officer • Regional Nutrition Advisor Office of Evaluation in Rome • OEV has a stake in ensuring that this approach is effective in delivering quality, useful and credible evaluations • Provides clear guidance on standards and expectations (DEQAS) for evaluation manager and ET • OEV may use the evaluation findings, as appropriate, to feed into evaluation syntheses. • OEV WFP Executive Board • WFP’s governing body is interested in the effectiveness of WFP operations. • The EB has an interest in being informed about the effectiveness of WFP operations. • Members of the Executive Board External stakeholders Beneficiaries • As the ultimate recipients of food assistance, beneficiaries have a stake in WFP determining whether its assistance is appropriate and effective. • Participation of women, men and school children into the evaluation will therefore be critical. • Principal source of information on the relevance, effectiveness, and sustainability of activities. • KIIs and FGDs on programme performance • Feedback on programme implementation • Give perspective on future focus for WFP activities • School children (boys & girls) enrolled in schools • Parents, PTA members, and school staff (School Directors, teachers, storekeepers & cooks) • Schools include WFP direct assisted schools as well as 24 comparison schools with no WFP support. Royal Government of Cambodia • The Government has a direct interest in knowing whether WFP activities in the country are aligned with its priorities, harmonized with the action of other partners and meet the expected results. • Issues related to capacity development, handover and sustainability (Roadmap 2021) will be of particular interest. • Key informants on programme context and performance: • MoEYS • MoH on health and nutrition issues • MAFF on food production and types of food availability, etc. • MoI regarding ongoing decentralization efforts • CARD on coordination of Food Security and Social Protection and how this supports food security and school feeding. • Feedback on how WFP’s work contributed to nationalstrategies, how it contributed to national strategies, on capacity building, and on appropriateness of activities and targeting • Recommendations for future programs and collaboration opportunities • Participate in debriefings and provide feedback on preliminary findings and conclusions • Ministry of Education, Youth and Sport including multiple departments within it. • Ministry of Health • Ministry of Agriculture, Forestry and Fisheries • Ministry of Interior • Council for Agriculture and Rural Development • District Governors • Commune Chiefs • Provincial Governors • Provincial Directors of Education • District Directors of Education • Local authorities as appropriate United Nations agencies • The UN agencies aim to implement harmonized action to should contribute to the realization of government development objectives. • Other UN agencies therefore has an interestin ensuring that WFP operations are effective in contributing to the UN concerted efforts. • Key informants on: • Coordination with other UN agencies on the UNDAF • Collaboration with WFP on the FFE Programme. • Complementarity of WFP activities in relation to other agencies providing assistance. • Targeting activities, and synergies • Recommendations for future programs and collaboration opportunities • United Nations Agencies working in Cambodia especially including: • UNICEF • FAO • UNESCO • World Bank • WHO • UNDP Non￾Governmental Organizations (NGOs) • WFP partners with NGOs for the implementation of some activities while atthe same time having their own interventions. • The results of the evaluation might affect future implementation modalities, strategic orientations and partnerships. • Key informants in evaluation interviews and data sharing: • Feedback on operational effectiveness and appropriateness of activities • Provide perspective of future focus for WFP activities and possibilities for collaboration • Plan International • World Education • KAPE • World Vision Civil Society • Civil society groups work within the same context in which WFP operates and have an interest in areas related to WFP interventions. • Their experience and knowledge will inform the evaluation and they will be interested in the evaluation findings – especially those related to partnerships • Community members • Local School Feeding Committees • School Support Committees 25 Donors • The FFE programme is funded by USDA, MGD and USAID. • Donors have an interest in knowing whether their funds have been spent efficiently and if WFP’s work has been effective and contributed to their own strategies and programs. • Key informants on: • Understanding the funding climate • Providing recommendations for future programme funding and areas of collaboration • Appropriateness of targeting and WFP response • Source of information on priorities and challenges • USDA • Japanese Embassy • KOICA • Any other donors supporting school feeding in Cambodia 26 Annex 9: Key evaluation questions Evaluation areas Evaluation questions The appropriateness of the programme Areas for analysis will include the extent to which the objectives, targeting, choice of activities and of transfer modalities were: • Are the activities and transfers appropriate to the needs of the target population and communities? • Are the activities and transfers aligned with relevantstated local and national policies, including sector policies and strategies and do they seek complementarity with the interventions of relevant humanitarian and development partners [as well as with other CO interventions in the country, if relevant]? • Are the activities and transfers aligned with WFP strategies, policies and normative guidance? The results of the programme While ensuring that differences in benefits between boys and girls from different groups are considered, the evaluation will analyse: • The efficiency of the programme (attainment of the planned outputs, cost factors, logistics and pipeline performance); • The effectiveness of the programme (the contribution of outputs to selected programme objectives); • The intended and unintended impacts of the programme as per the PMP/RFs • MoEYS capacity to manage school feeding moving forward • Does the Government of Cambodia have adequate institutional and human resource capacity to assume responsibility for future school meals and scholarshipactivities? • Is the design of the programme suitable to government management? • Which procurement models are feasible for government and how can they designed in such a way to ensure accountability and transparency? • Is there a functioning government reporting and monitoring and evaluation system? • Are community feedback mechanisms in place? Why and how the operation produced the observed results The evaluation should generate insights into the main internal and external factors that caused the observed changes and affected how results were achieved. The inquiry is likely to focus, amongst others, on: • Internally (factors within WFP’s control): the processes, systems and tools in place to support the operation design, implementation, monitoring/evaluation and reporting; the governance structure and institutional arrangements (including issues related to staffing, capacity and technical backstopping from RB/HQ); the partnership and coordination arrangements; etc. • Externally (factors outside WFP’s control): the external operating environment; the funding climate; external incentives and pressures; etc. • The implications for this as the programme transitions to MoEYS Sustainability of the project moving forward With a roadmap for national ownership of the school feeding programme already being implemented, the government will begin to gradually assume ownership of the programme over the next 5 years. As such, the evaluation should have a strong focus on various implementation models tested over the course of the project period. It should draw conclusions about the following: • Cost effectiveness, efficiency and practicality of each of the modelstested • Government buy in on each model tested • Recommendations on which model may be most appropriate given the findings • Evidence that activities are likely to be sustained or scaled up beyond the projectlife The evaluation will assess the success of USDA supported initiatives to improve information systems within MoEYS to ensure transparency and cost effectiveness of the programme. 27 Annex 10: Evaluation Matrix - Cambodia McGovern-Dole FFE Programme The following matrix will be utilized by the team members to focus questioning with respect to questions highlighted in the ToR. McGovern-Dole Results Framework: Improved Literacy of School-Age Children (SO1) Key Question 1: How appropriate is the operation? Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Relevance Is McGovern-Dole-supported assistance appropriate to the needs of the target population on both females and males? Attendance rate in targeted areas Drop-out rate in targeted areas Literacy rates in targeted areas (Reading capacity as per RF indicators) WFP CO, SFTP/MOEYS and cooperating partners M&E data and reports Desk review Secondary data triangulated with quantitative analysis, including comparison between baseline and endline Strong Is there a difference between these rates for boys and girls? Coherence Is McGovern-Dole FFE programme aligned with relevant stated national policies? Mention/integration of SF in national development strategy documents NSDP 2014-2018, ESP 2014-2018, NSPPF 2016- 2025 Desk review Qualitative analysis – secondary document review triangulated with key informant interviews Strong Does McGovern-Dole FFE programme seek complementarity with interventions ofrelevant humanitarian and development partners? Compliance with UNDAF outcomes UNDAFs for Cambodia 2016-2018 & 2019-2023 Desk review Qualitative analysis – secondary document review triangulated with key informant interviews Strong Is McGovern-Dole FFE programme aligned with WFP strategies, policies and normative guidance? Compliance with WFP Strategic Plan and School Feeding Strategy WFP Strategic Plan 2017- 2021 SDG targets 2.1 and 2.2, SDG 4 WFP School Feeding Policy 2013 Desk review Qualitative analysis – secondary document review triangulated with key informant interviews Strong 28 Key Question 2: What are the results of the operation? Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Efficiency Has the programme been sufficiently funded? Budget data, budget revisions Planned vs. actual budget and expenditure WFP Financial and operational information Desk review Interviews with WFP Finance and other support staff Qualitative and quantitative analysis – review of budget data and triangulation between multiple key informants. Strong Have the McGovern-Dole SF￾supported activities been undertaken and completed in a timely manner? Food delivery data School supplies & materials delivery data WFP CO, MOEYS and cooperating partners M&E data and reports Desk review Quantitative analysis Qualitative analysis – triangulation between multiple key informants Medium Are appropriate human resources allocated to implementation in USDA supported areas? HR data and organization chart in WFP CO, MOEYS, cooperating partners WFP CO, MOEYS and cooperating partners M&E data and reports Desk review Qualitative analysis – WFP HR information triangulated with key informant interviews. Medium How efficient are the arrangements for monitoring implementation? Presence of sufficient programme monitoring data at both output & outcome levels WFP CO M&E data and reports (including PRISM) ACR reports Desk review Qualitative analysis – triangulation between multiple key informants Medium What were the roles and institutional strengths/weaknesses of the WFP CO, government ministries, cooperating partners and beneficiary groups in McGovern-Dole FFE programme implementation? Establishment of SF units and coordination committees at central and decentralized levels WFP CO and SFTF/MOEYS Desk review Meetings with key informants in WFP CO, MoEYS, PoEs, DoEs Qualitative analysis – triangulation between multiple key informants Medium Are the monitoring systems in place adequate to show impact over a longer period? Analysis produced by WFP CO WFP CO M&E data and reports, VAM surveys, BFM reports Desk review Qualitative analysis – triangulation between multiple key informants + review of what monitoring data is collected. Medium 29 Effectiveness (continued) To what extent have SF activities improved student attendance, student attentiveness, quality of literacy instruction, and contributed to improved literacy of school-age children Attendance rate Drop-out rate Promotion rate Results of EGR tests Is there a difference between these rates for boys and girls? WFP CO M&E data and reports MOEYS statistics (EMIS) CPs surveys and reports Desk review Field visits Quantitative analysis - Comparison between baseline and endline Qualitative analysis – Secondary data review and triangulation between multiple key informants Medium Key Question 2: What are the results of the operation? (continued) Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Effectiveness (continued) Are the criteria for targeting appropriate? How has targeting been done? Adequate evidence of inclusion of education rates, poverty rates, etc. Number of revisions of CFSVA or similar surveys and changes introduced in SF targeting accordingly MOEYS statistics (EMIS) WFP MERVAM surveys and reports Desk review Meetings with WFP CO and MOEYS staff Quantitative analysis - Comparison between baseline and endline Qualitative analysis – triangulation between multiple key informants Medium Are the outputs levels sufficient? No. of beneficiaries receiving food and non-food items vs. planned. Is there a difference between boys and girls? Tonnage of food distributed vs. plans No. of school supplies & materials distributed vs. planned No. of schools assisted vs. planned WFP CO M&E data and reports SPRs, ACR Desk review Quantitative analysis Medium How effective is UN inter-agency coordination in meeting theeducation needs in the targeted implementation areas? Number and type of joint initiatives to implement the "essential package" in WFP assisted schools Joint work plans with UN sister agencies Desk review Meetings with cooperating partners Qualitative analysis – triangulation between multiple key informants Medium 30 Sustainability Is there an exit strategy outlining the timing, allocation of responsibilities on handover to the government and/or other agencies? WFP 2013 SF Policy Goals Number of delivery models nationally owned Education NCI MOEYS policy and strategy documents MOEYS/WFP SF Roadmap Desk review Meetings with donors Qualitative analysis – triangulation between multiple key informants and secondary document review. Strong To what extent are the programme activities and local structures likely to be sustained after the completion of donor-funding? Existence of a SF line in the MOEYS budget Number of sustainable delivery models taken over by government Number and type of initiatives taken by PTAs and community at large to support SF activities MOEYS policy and strategy documents Key government informants at central and decentralized levels Desk review Meetings with key informants in WFP CO, MOEYS, PoEs, DoEs Focus group meetings with programme participants Qualitative analysis – triangulation between multiple key informants Medium Key Question 3: Why and how has the operation produced the observed results? Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Internally (factors within WFP's control) To what extent has there been effective cooperation and coordination in the design and implementation of the SF activities between WFP, government, cooperating partners and beneficiary communities? Perception of management strengths/difficulties by WFP staff, government staff, and cooperating partners LoU, MoU, FLA, etc. Number of meetings from a Programme coordination committee WFP staff, government staff, cooperating partners, programme participants Interviews with implementing partners (WFP staff, government staff at national and decentralized levels, and cooperating partners) Focus group meetings with programme participants Qualitative analysis – triangulation between multiple key informants Medium Externally (factors outside WFP's control) To what extent do the criteria for targeting influence results? Food insecurity, poverty, low educational, nutrition and gender indicators EMIS, Commune Database (CDB) on poverty, CFSVA Meetings with key informants in WFP CO, MOEYS, PoEs, DoEs Qualitative analysis – triangulation between multiple key informants Medium To what extent does the environment of targeted schools influence results? Teacher/pupils ratio, school infrastructure and equipment WFP CO data, EMIS Meetings with key informants in WFP CO, MOEYS, PoEs, DoEs Qualitative analysis – triangulation between multiple key informants Medium 31 McGovern-Dole Results Framework: Increased Use of Health and Dietary Practices (SO2) Key Question 1: How appropriate is the operation? Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Relevance To what extent are the objectives, targeting, choice of activities and transfer modalities appropriate to the needs of the targeted population on both females and males? Poverty rates, food security rates, combined with education indicators. Reduction in health-related absences Have the needs of both boys and girls been taken into consideration? WFP CO, SFTP/MOEYS and cooperating partners M&E data and reports Desk review Review of WFP assessment information Qualitative analysis – triangulation between multiple key informants Strong Coherence Is McGovern-Dole FFE programme aligned with relevant stated national policies? Compliance with stated aims and direction of relevant government policies (food security, nutrition, school health etc.). Government policies on school feeding, nutrition, school health, safety nets etc. Desk review Key informant interviews with Government personnel Qualitative analysis – triangulation between multiple key informants Strong Does McGovern-Dole FFE programme seek complementarity with other interventions of WFP in Cambodia and with those of relevant humanitarian and development partners? Compliance with stated aims and direction of relevant policies of other development actors such as UN agencies and NGOs. UNDAFs for Cambodia 2016-2018 & 2019-2023 Other policies and strategies of other development actors. Desk review Key informant interviews with non-WFP development actors Qualitative analysis – triangulation between multiple key informants Strong Is McGovern-Dole FFE programme aligned with WFP strategies, policies and normative guidance? Compliance with stated aims and direction of relevant WFP policies and strategies including corporate guidance, and regional strategies as appropriate. Review of relevant WFP policies e.g. School Feeding Policy, Safety nets Policy, Gender Policy, Nutrition Policy etc. Desk review Key informant interviews with WFP Regional Bureau personnel and HQ staff as appropriate Qualitative analysis – triangulation between multiple key informants Strong 32 Key Question 2: What are the results of the operation? Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Efficiency Has the programme been sufficiently funded? Budget data, budget revisions Planned vs. actual budget and expenditure WFP Financial and operational information Desk review Interviews with WFP Finance and other support staff Quantitative review of budget data triangulated with qualitative information from multiple key informants Strong How much of overall funding has been spent on direct programme costs compared with indirect costs? Budget data, budget revisions Planned vs. actual budget and expenditure WFP Financial and operational information Desk review Interviews with WFP Finance and other support staff Qualitative and quantitative analysis Medium Have the McGovern-Dole SF￾supported activities been undertaken and completed in a timely manner? Food delivery data Non-food delivery information (seeds, inputs for school gardens etc.). WFP CO, MOEYS and cooperating partners M&E data and reports Desk review Key informant interviews with school personnel, parents, implementing partners, and WFP Qualitative analysis – triangulation between multiple key informants Medium Are appropriate human resources allocated to implementation? HR data and organization chart in WFP CO, MOEYS, cooperating partners WFP CO, MOEYS and cooperating partners M&E data and reports Desk review Key informant interviews with school personnel, parents, implementing partners, and WFP Qualitative analysis – triangulation between multiple key informants Medium How efficient are the arrangements for monitoring implementation? Presence of sufficient programme monitoring data at both output and outcome levels WFP CO M&E data and reports CPs reports Desk review Key informant interviews with school personnel, parents, implementing partners, and WFP Qualitative analysis – triangulation between multiple key informants Medium 33 Key Question 2: What are the results of the operation? (Continued) Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Efficiency (continued) What were the roles and institutional strengths/weaknesses of the WFP CO, governmentministries, cooperating partners and beneficiary groupsin McGovern￾Dole FFE programme implementation? Establishment of SF units and coordination committees at central and decentralized levels WFP CO and SFTF/MOEYS Desk review Key informant interviews with school personnel, parents, implementing partners, and WFP Qualitative analysis – triangulation between multiple key informants Medium Are the monitoring systems in place adequate to show impact over a longer period? Presence of outcome data. Analysis produced by WFP CO WFP CO M&E data and MERVAM reports Desk review Qualitative analysis – triangulation between multiple key informants Medium Effectiveness Are the outputs levels sufficient? No. of beneficiaries receiving food and non-food items vs. planned. Is there a difference between boys and girls? Tonnage of food distributed vs. plans Number of school gardens established vs planned Number of teachers trained Number of cooks trained vs planned No. of storekeepers trained vs planned Number of schools assisted vs. planned WFP CO M&E data and reports WFP SPRs Desk review Quantitative analysis - Comparison between baseline and endline Strong Is there a difference in the effectiveness of SMP vs SMP+THR? Education indicators (as per SO1) Food consumption score Household Dietary Diversity Score Individual Dietary Diversity Score Coping Strategies Index Household Hunger Scale Is there a difference between boys and girls? School Assessment Teacher Assessment Household Questionnaire Quantitative surveys Quantitative Analysis Comparison between intervention schools Medium 34 Key Question 2: What are the results of the operation? Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Effectiveness (continued) How effective has the programme been at reducing health-related absences? Number of health-related absences Is there a difference between boys and girls? School Assessment Quantitative surveys Quantitative analysis - Comparison between baseline and endline Medium How effective has the programme been at improving knowledge of health and hygiene practices? Percent of schools with soap and water at handwashing stations commonly used by students School Assessment Quantitative surveys Quantitative analysis - Comparison between baseline and endline Medium Number/ percent of schools using improved sanitation facilities (latrines) Qualitative analysis – triangulation between multiple key informants Is there a difference between boys and girls? How effective has the programme been at increasing knowledge of safe food preparation and storage practices? Percent of households in target schools that store food off the ground Percent of schools in target communities that clean cooking and eating equipment, consistent with accepted standards School Assessment Quantitative surveys Quantitative analysis - Comparison between baseline and endline Qualitative analysis – triangulation between multiple key informants Medium How effective has the programme been at increasing knowledge of nutrition? The percentage of students and parents who can identify at least one local source of information on proper diets and nutrition. Is there a difference between boys and girls? The percentage of cooks and storekeepers that have a passing grade on good nutrition and dietary practices Household Questionnaire Quantitative surveys Quantitative analysis - Comparison between baseline and endline Qualitative analysis – triangulation between multiple key informants Medium 35 Key Question 2 What are the results of the operation? Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Effectiveness (continued) How effective has the programme been at increasing access to preventive health services? Percentage of children dewormed within the past 6 months. Is there a difference between boys and girls? The number of target schools with at least one-month supply of soap (hand and dish soap) Number of students receiving daily school meals with micronutrient fortified commodities. Is there a difference between boys and girls? Household Questionnaire School Assessment Quantitative surveys Quantitative analysis - Comparison between baseline and endline Qualitative analysis – triangulation between multiple key informants Medium How effective has the programme been at increasing access to clean water and sanitation services? The ratio of latrines (boys/girls) to students at target schools The percentage of schools withyear-round access to safe and clear watersource The number of target schools that have latrines of sufficient quality that are in good repair School Assessment Quantitative surveys Quantitative analysis - Comparison between baseline and endline Qualitative analysis – triangulation between multiple key informants Medium How effective has the programme been at increasing access to requisite food preparation and storage tools and equipment? Number of functioning kitchens Number of kitchens using fuel efficient stoves No. of dedicated food storerooms Number of food storerooms using appropriate food storage techniques School Assessment Quantitative surveys Quantitative analysis - Comparison between baseline and endline Qualitative analysis – triangulation between multiple key informants Medium Sustainability To what extent are the results and benefits derived from WFP’s interventions likely to endure? MoEYS Handover (Roadmap) Actions undertaken by PTC, parents and other stakeholders WFP CO, SFTP/MOEYS and cooperating partners M&E data and reports Desk review Meetings with key informants in WFP CO, MOEYS, PoEs, DoEs, committee members, other parents Qualitative analysis – triangulation between multiple key informants Medium 36 Key Question 3: Why and how has the operation produced the observed results? Sub Question Measure/Indicator Main Sources of information Data Collection Methods Data Analysis Methods Evidence quality Internally (factors within WFP's control) What internal factors have contributed to the success or otherwise of the FFE programme? Positive or negative external issues mentioned during interviews or FGDs WFP staff, staff, partners, participants government cooperating programme Interviews and FGDswith WFP personnel, implementing partners and programme participants Qualitative analysis – triangulation between multiple key informants Medium What internal problems and constraints have been faced during the implementation of the response and how did the operation deal with these? Number of internal problems/constraints which have been appropriately addressed vs. Number of ongoing problems/constraints WFP staff, staff, partners, participants government cooperating programme Interviews and FGDswith WFP personnel, implementing partners and programme participants Qualitative analysis – triangulation between multiple key informants Medium Externally (factors outside WFP's control) What external factors have contributed to the success or otherwise of the FFE programme? Positive or negative external issues mentioned during interviews or FGDs WFP staff, staff, partners, participants government cooperating programme Interviews and FGDswith WFP personnel, implementing partners and programme participants Qualitative analysis – triangulation between multiple key informants Medium What external problems and constraints have been faced during the implementation of the response and how did the operation deal with these? Number of external problems/constraints which have been appropriately addressed vs. Number of ongoing problems/constraints WFP staff, staff, partners, participants government cooperating programme Interviews and FGDswith WFP personnel, implementing partners and programme participants Qualitative analysis – triangulation between multiple key informants Medium 37 Annex 11: Full evaluation timeline Phases, Deliverables and Timeline Timeframe Deliverable submissions Phase 2 – Inception Briefing TOR to evaluation team 10 June 2019 Remote desk review of a draft inception report (IR) 10-21 June Submission of the draft Inception Report to WFP 21 June WFP shares the draft IR with DEQAS and ERG, RB, and relevant WFP HQ divisions for comments 24-28 June ET (1) Reviewing and revising the draft IR based on comments received, (2) submitting the revised IR to the WFP internal evaluation committee for approval and (3) sharing the revised IR with key stakeholders 1-5 July WFP shares the revised IR with USDA for comments 8-19 July ET revision of draft IR based on stakeholder comments 22-26 July Submission of final Inception Report to WFP 26 July Phase 3 – Data collection Quantitative survey data collection (IRL) 22 July – 9 August Briefing of KonTerra evaluation team at CO 29 July Qualitative data collection 30 July-14 August Debriefing of evaluation team at CO 15/16 August PowerPoint presentation of preliminary findings from fieldwork (internal stakeholder) 15 August PowerPoint presentation of preliminary findings from fieldwork (external stakeholders) 16 August Phase 4 - Analyse data and report Drafting of evaluation report (ER) 20 August–22 September Submission of draft evaluation report to WFP 30 September WFP CO initial review of draft evaluation report (one day) 30 September ET revision of initial draft (one day) 1 October WFP sharing the draft ER with DE QS and ERG, RB, and relevant WFP Headquarters divisions for comments (one week) 2-9 October ET reviewing and revising the draft ER based on comments received 10-13 October ET re-submission of revised ER to the internal evaluation committee for approval 14 October WFP sharing the draft ER with key stakeholders for review (one week) 14-21 October ET revise the drafted ER based on stakeholder comments 21-28 October Submission of revised evaluation report to WFP 28 October WFP sharing the revised ER with USDA for comments (30 days) 28 Oct ET revision the draft ER based on stakeholder comments 28 November - 2 December Submission of final evaluation report to WFP 2 December 38 Annex 12: Summary of the quantitative field mission schedule Dates Team member Actions and Locations Stakeholders 8 to 12 July 2019 Sophia Dunn Dr. Prem Bhandari Phnom Penh - Visit to WFP Work with IRL on survey planning KonTerra, IRL and WFP Cambodia 15-19 July IRL Team Phnom Penh - Training of quantitative survey field teams IRL with support from KonTerra as required 13 July to 9 August Dr. Prem Bhandari Remote - Support IRL regarding survey methodology, data collection and cleaning KonTerra 22 July to 9 August IRL Team Conduct quantitative surveys in Battambang, Kampong Thom, Siem Reap, Preah Vihear, Kratie and Kampong Cham Provinces School administrators, teachers & staff, and households 22 to 26 July Mr Bunyeth Ho Unscheduled spot checking of the provincial survey work School administrators, teachers & staff, and households 29 July to 16 August Sophia Dunn Jean-Pierre Silvéréano Dr. Sovith Sin Conduct qualitative field mission in Phnom Penh, Battambang, Siem Reap, Kampong Thom. WFP CO, Government ministries, United Nations agencies, NGO partners, Schools, and various beneficiary groups. 39 Annex 13: Qualitative field mission schedule Tentative Field Mission Schedule for USDA FFE Evaluation Mission Phnom Penh, Cambodia Date Time Stakeholder Activities/people to meet Meeting Location/ Accommodation ET1 ET2 Notes Status KonTerra arrival Phnom Penh 08:15- 8:30 WFP CO Security briefing WFP office: House#108, St.63, Sangkat Boeung Raing, Khan Daun Penh SP &SV JP 08:30- 09:30 Brief meeting with WFP CO personnel: Country Director, and key staff from education, finance, M&E, logistic and social protection SP &SV JP 29-Jul-19 10:00- 11:00 MEFF Meeting with MEFF: Chan Narith, PhD.Secretary-General, National Social Protection Council, Cambodia, Mobile: +855-12 280 288 Email: chan_narith@mef.gov.kh Uy Chanimol, Director of Social Assistance General Secretariat for National Social Protection Council Email: uy_channimol@yahoo.com, Mobile: +855 17977783 Mr. Sovadh and his colleague are available for the meeting, Tele: (855) 96 5677 677, Email: isean81@gmail.com Ministry of Economy and Finance #92, S/K Wat Phnom, K/H Daun Penh, 5th floor (new Building) SP &SV JP Confirmed Monday 13:30- 14:15 WFP CO Meeting with Activity 4: Krsina and Phalla SP &SV JP Confirmed 14:45- 15:45 MoEYS, Primary Education Department Meeting MoEYS: PED H.E. Puth Samith, Director General H.E. Chan Sophea, Dir. of PED (012211336) Mr. Chun Ramy, Deputy Dir. of PED; Mr. Ven Thol, Chief of Scholarship Office (017682226); MoEYS Building: #168, Preah Norodom Blvd, Sangkat Boeung Keng Kang - In front Panhasatra university SP &SV JP Confirmed 40 15:45- 17:00 WFP CO Meeting Kannitha, Sokrathna, Vatanak and Sokhemarey Education: WFP office: House#108, St.63, Sangkat Boeung Raing, Khan Daun Penh SP &SV JP Confirmed 08:30- 09:30 WFP CO Meeting Supply Thomas, Kunakar, Savuth, Tipo, Hong, Sokheng changes: WFP office: House#108, St.63, Sangkat Boeung Raing, Khan Daun Penh JP Confirmed 10:00- 11:00 MoEYS Secretary of State, H.E. Nath H.E. San Vathana, Under Secretary MoEYS, vathana20@yahoo.com MOEYS Bunroeun of State, At his office SP &SV Confirmed 30-Jul-19 Tuesday 8:30- 9:30 GIZ Meeting with Rin Thavy, Program Advisor, Regional Fit Programme GIZ, WASH GIZ: for School MoEYS building, Street 380, BKK 1, Phnom Penh,Cambodia Mobile: +855 12 920 858 JP Confirmed 10:00- 11:00 School Health Department Meeting MoEYS: Ms. Chhay Kimsotheavy, Director Ms. So Chhorvirath, Deputy Ms. Slad Chenda, Dep. Chief of technicaloffice SHD of SHD of SHD MoEYS New Building: Street 380, Boeung Keng Kang I (close to BKK Market) 4th Floor SP &SV Confirmed 14:00- 15:00 KOICA Meeting with KOICA Pen Vuthida, tel: 012 697117, thydakoica@gmail.com, Jeong Yun Gil, Country Director, Kim Jiyoon, june0619@koica.go.kr Koyena, koyena614@koica.go.kr No. 445, Preah Monivong (St. 93), corner of St. 232, 12th floor, 12258 Phnom Penh, x x Confirmed 15:30- 17:00 WFP CO Meeting with Activity Kurt, Indira, Yav, Seanglay and Chanvibol 5: WFP office: House#108, St.63, Sangkat Boeung Raing, Khan Daun Penh 31-Jul-19 08:00- 9:00 Quality Assurance Department Meeting with QAD: Ung Chinna, Tel: 012883552, chinnaung@gmail.com MoEYS New Building: Street 380, Boeung Keng Kang I (close to BKK Market) 4th Floor JP Confirmed Wednesday 09:30- 10:30 CARD Meeting with CARD: SP, FS&N Dr. Say Ung, CARD, 012 92 6789,sayungcard@gmail.com Office of the Ministers #41, Russian Blvd, Council of Federation SP &SV JP Confirmed 41 11:00- 12:00 WVI Mr. Ravuth Thea, Senior Program Officer Tel: 012 546 007,ravuth_thea@wvi.org) WFP office: House#108, St.63, Sangkat Boeung Raing, Khan Daun Penh SP &SV JP Confirmed 13:30- 14:15 WFP CO Meeting Finance: Synlin and Ly Muy WFP office: House#108, St.63, Sangkat Boeung Raing, Khan Daun Penh SP &SV JP 16:00- 17:00 PLAN Mr. Chhun Kimheang, country grand coordinator, Kimheang.Chhun@plan-international.org, Tel: 012264326, WFP office: House#108, St.63, Sangkat Boeung Raing, Khan Daun Penh confirmed ET conduct interview in Kampong Thom Province: 1-2 August 2019 08:00- 09:00 Meeting with World Education Mr. Dara Kim, Director of (WE) dara_kim@kh.worlded.org, 081300999 Mr. Sieng Heng, sieng_heng@kh.worlded.org Mr. Sao Vanna, Director of KAPE (012342322) WFP office: House#108, St.63, Sangkat Boeung Raing, Khan Daun Penh x x Confirmed 09:30- 10:30 MoEYS, Policy Department Meeting MoEYS: PD Mr. Dy Khamboly, Director of PD, Tel:017883967, khamboly12@gmail.com; Mr. Hem Kunthea, Deputy Director, hamkunthea1@gmail.com Mr. Kang Sophannna, Chief office, kangsophanna@yahoo.com MoEYS Building: #168, Preah Norodom Blvd, Sangkat Boeung Keng Kang - In front Panhasatra university x x Confirmed 05-Aug-19 Monday 10:30- 11:30 Primary Education Department Meeting MoEYS: PED Mr. Venthol, Chief of scholarship Office, Tel:017682226, venthol16@gmail.com; MoEYS Building: #168, Preah Norodom Blvd, Sangkat Boeung Keng Kang - In front Panhasatra university Confirmed 11:00- 12:00 Embassy of Japan (Donor) Meeting with Janpan Embassy: Mr. Masoda, Second Secretary 194 Preah Norodom Boulevard, PNP; Contact: japan.embassy@pp.mofa.go .jp x x Confirmed 13:30- 14:30 FAO Meeting with FAO: Iean Russel (Programme), Iean.Russell@fao.org FAO office: # 5 street 370, Phnom Penh SP &SV Confirmed 42 14:00- 15:00 Teacher Department Meeting MoEYS: TD Mr. Ngor Penglong, Director, 016825533 or 077982983 Mr. Yit Sopheak, 012611848 Mr. Leang Sun Heang, Chief of Admin, 012915417: sunleangheang@gmail.com MoEYS Building: Preah Norodom Blvd, Adjacent to National Institute of Education. JP Confirmed 15:30- 17:00 USAID Meeting with USAID: Sereisatya ROS, Education Project Management Specialist USAID Cambodia, Email: sros@usaid.gov, Cell: 855 17 666 727 Laura Cizmo, Deputy Director, office of food security and environment, email: lcizmo@usaid.gov, tel: (855) 78999203 US Embassy, 1 Christopher Howes (96), Phnom Penh, Contact: Sao Samsak, Tele: 012997009 SP &SV JP Confirmed 8:30- 9:30 CARD Meeting with CARD: SP, FS&N HE Sok Silo , Deputy Secretary General of CARD, 012 451111, soksilo@gmail.com Office of the Council of Minister SP &SV Confirmed 06-Aug-19 Tuesday TBD UNICEF Meeting with UNICEF: Ms. Katheryn Bennett, kbennett@unicef.org Mr. Chum Channra, cchum@unicef.org Rasika Sridhar Sethi(Education).rsridharsethi@unicef.org Meeting at Exchange Square - UNICEF Office ( behind Vattanac Tower) SP &SV Tele conf 11:00- 12:00 UNESCO Meeting with UNESCO: Sardar Umar Alam, UNESCO Director, Email: su.alam@unesco.org, Mobile: +855 12 543 000 Mr. Santosh Khatri, Chief of education, Email: s.khatri@unesco.org, mobile: +855 95 948 137 UNESCO Office JP Confirmed Depart for BTB ET conduct interview in BTB & SRP: 7-10 August 2019 12-Aug-19 8:30- 9:30 MoI/CARD Meeting CARD and MoI: H.E. Ngy Chanphal, Vice Chairman (CARD) & Secret. of State at MoI, mrdngycp@online.com.kh, 012909879 Contact Mr. Ma Nirith, nirith_ma@yahoo.com Assistant 012771878 Ministry of Interior,275 Preah Norodom Blvd. (41), Room 303. SP &SV JP Confirmed Monday 10:00- 11:00 USDA in Ho Chi Minh Megan (Davidow) Francic Agricultural Attaché U.S. Consulate General Ho Chi Minh City Megan.Francic@fas.usda.gov Vo Thanh Dinh Hoai x x 43 11:00- 12:00 WFP Regional Bureau in Bangkok RBB: Luna Kim, RBB, Monitoring and evaluation, luna.kim@wfp.org Kimberly, RBB, kimberly.deni@wfp.org Yumiko; Regional Evaluation Officer at yumiko.kanemitsu@wfp.org Insa; Evaluation: insa.deimann@wfp.org x x 1:30- 2:30 MAFF Meeting MAFF: Mr. Mak Soeun, Deputy Director General of General Directorate of Agriculture, MAFF, tel: 012826617, Email: Maksoeun168@gmail.com No 54B/49F, St. 395-656, Sangkat Toeuk Laak 3, khan Tuol Kok, Phnom Penh. SP &SV JP Confirmed 4:00- 5:00 WFP CO Meeting with Francesca WFP office: House#108, St.63, Sangkat Boeung Raing, Khan Daun Penh 08:00- 09:00 USDA Washington Rummery, Molly - FAS Molly.Rumery@fas.usda.gov Booked 13-Aug-19 Tuesday 8:00- 17.00 Preparation of debrief presentations 14-Aug-19 Wednesday 8:30- 9:30 Curriculum Department Meeting MoEYS: CD Ms. Mao Thannavy; Tel: 012559685 MoEYS New Building: Street 380, Boeung Keng Kang I (close to BKK Market) 4th Floor Confirmed 8:00- 17:00 ET finalize presentation 15-Aug-19 Thursday 10:00- 12:00 Internal preliminary debriefing presentation to WFP 16-Aug-19 Friday 10:00- 12:00 External preliminary debriefing presentation to partners 16-Aug-19 ET’s Departure 44 Annex 14: Endline survey documents received from WFP CO (as at 30 September) In addition to the large volume of files already shared with the Evaluation Team during the 2017 Baseline Survey and the 2018 Mid-term Review, the Country Office hasshared the following documentsfor the 2019 Endline Evaluation. Data Upload for Endline Evaluation 1 MTR inception report and final report of USDA McGovern, 2017-2019 2 QPR report from partner, PLAN: Q1, Q2 & Q3 3 QPR report from partner, WVC: Q1, Q2 & Q3 4 QPR report from partner, WE: Q1 , Q2 & Q3 5 WEI success story FFE Y3 Q2 6 USDA Semi-annual reports from: 1). Oct 2018- March 2019 and 2). Apr 2018- Sept 2018 7 UDSA schools support summary SY2018-19 8 Annual Country Report 2018 9 School List SY18-19 (USDA) 10 WFP_KHM_EDU_USDA_Targeted_Area_2017_2018 11 WFP_KHM_EDU_USDA_Targeted_Area_2018_2019 12 EMIS data for 2018-19 13 School List SY18-19 (USDA) Case and Compare schools 14 080719 - List of case and comparison schools 15 List of case schools programme [SY2016-2019] 16 ESP 2019-23- incorporated the SM/HGSF -latest draft- the English final signed version not yet available. 17 Annex 1 for the Letter of Understanding with MoEYS – ongoing process for signature 18 HGSF concept note from MoEYS to MEF (Social protection Council) – in Khmer only 19 Draft HGSF implementation framework – in Khmer - ongoing prepare 20 Draft HGSF operational guideline –in Khmer - ongoing prepare 21 Draft of food safety materials – almost finalize, including video (Cannot upload videos as big file) 22 Translated Nutrition Materials for USDA 23 Output progress-By June 2019_Edu (USDA)-26 Jul 2019- final 24 20-MoEYS letter to GCNF 18 Jan 2019 25 Draft of Main Report EC -English (19032019) 26 Equity-Focus-Scholarship-Policy-Eng-May2019 27 Social protection committee 28 ProposalAttachments_2019-0113 (USDA- LRP proposal) 29 ProposalAttachments_2019_Cambodia (USDA McGovern-Dole) 30 PRISM PowerPoint (shown to us by Krisna and Phalla) + Other documents 31 Annual SF Workshop in KCG 18-19 March 2019-Final 32 WFP HGSF Preliminary Results FINAL (HGSF supplier consultation 2018) 33 Nutrition reports 34 Guideline on Food Safety in Schools-May 2019 35 Cambodia Funding Proposal - Russian Federation_08 May 36 Graphic of Primary of Kingdom (2018-2019) - official 37 HGSF guidance-Khmer version 38 2019-05-31-Scholarship Evaluation Report-Vol-I-final 39 2019-05-31-Scholarship Evaluation Report-Vol-II-final 40 Cambodia mission report - January 2017 45 Annex 15: List of agencies Number Community representatives 14 Donors 9 Government representatives 49 NGO implementing partner personnel 24 Other UN agency personnel 3 Parents 64 School children 32 School personnel 53 WFP personnel 29 TOTAL 277 46 47 48 49 50 51 Annex 16: WFP school feeding interventions in sampled case schools since baseline School Code Provinces District School Name School meals interventions provided SY 2016/17 (Baseline) SY 2017/18 SY 2018/19 (Endline) 6010505077 Kampong Thom Baray Prey Dom SMP SMP+THR SMP+THR 6010611038 Kampong Thom Baray Praing Samrong SMP SMP+THR SMP+THR 6011701064 Kampong Thom Baray Tnot Chum SMP SMP+THR SMP+THR 6020303059 Kampong Thom Kampong Svay O Ta Sek SMP HGSF-Hybrid HGSF-Hybrid 6020411021 Kampong Thom Kampong Svay Anlung Krasaing SMP HGSF-Hybrid HGSF-Hybrid 6040106026 Kampong Thom Prasat Balaing Kruos SMP SMP+THR SMP+THR 6040305004 Kampong Thom Prasat Balaing Kranhoung SMP SMP+THR SMP+THR 6040602020 Kampong Thom Prasat Balaing Trach SMP SMP+THR SMP+THR 6050112006 Kampong Thom Prasat Sambo Prasat SMP SMP+THR SMP+THR 6050113011 Kampong Thom Prasat Sambo Veal Veng SMP SMP+THR SMP+THR 6050201013 Kampong Thom Prasat Sambo Kampong Chvea SMP SMP+THR SMP+THR 6050406029 Kampong Thom Prasat Sambo Choam Boeung SMP SMP+THR SMP+THR 6050407031 Kampong Thom Prasat Sambo Tumnup SMP SMP+THR SMP+THR 6050508033 Kampong Thom Prasat Sambo Kampong Chheu Teal SMP SMP+THR SMP+THR 6060510022 Kampong Thom Sandann Phtorl Rumpos SMP SMP+THR SMP+THR 6060701033 Kampong Thom Sandann Danghet SMP SMP+THR SMP+THR 6060707039 Kampong Thom Sandann Svay SMP SMP+THR SMP+THR 6070101001 Kampong Thom Santouk Boeung Lovea SMP SMP+THR SMP+THR 6070408015 Kampong Thom Santouk Chi Meak SMP SMP+THR SMP+THR 6070503022 Kampong Thom Santouk Trapaing Pring SMP SMP+THR SMP+THR 6070806032 Kampong Thom Santouk Chambak Khang Cheung SMP SMP+THR SMP+THR 6070906037 Kampong Thom Santouk Chhouk Rumduol Choam Thna SMP SMP+THR SMP+THR 6080103002 Kampong Thom Staung Beng SMP SMP HGSF-Hybrid 6080109007 Kampong Thom Staung Sla Kor SMP SMP+THR HGSF-Hybrid+THR 52 6080201011 Kampong Thom Staung Preah Naingkoal SMP SMP+THR HGSF-Hybrid+THR 6080202079 Kampong Thom Staung Sampor SMP SMP+THR HGSF-Hybrid+THR 6080203014 Kampong Thom Staung Srey Rongeut SMP SMP+THR HGSF-Hybrid+THR 6080303021 Kampong Thom Staung Trapaing Choa SMP SMP+THR HGSF-Hybrid+THR 6080913054 Kampong Thom Staung Ta Treal SMP SMP+THR HGSF-Hybrid+THR 6081004057 Kampong Thom Staung Leuk Cheah SMP SMP+THR HGSF-Hybrid+THR 6081007058 Kampong Thom Staung Prasat SMP SMP+THR HGSF-Hybrid+THR 6081214069 Kampong Thom Staung Lovea SMP SMP HGSF-Hybrid 53 Annex 17: Map of target (treatment) provinces and comparison (control) districts Source: KonTerra Evaluation Team based on data provided by WFP Cambodia Country Office 54 Annex 18: Full list of case and comparison schools for endline evaluation No. SCHOOL ID PROVINCE SCHOOL NAME SCHOOL TYPE 1 2130307024 Battambang Prey Sinh Case 2 2130402007 Battambang Kouk Poun Case 3 2130404009 Battambang Tuol Mates Case 4 2130408025 Battambang Boeung Raing Case 5 2130411012 Battambang Samrong Case 6 6010505077 Kampong Thom Prey Dom Case 7 6010611038 Kampong Thom Praing Samrong Case 8 6011701064 Kampong Thom Tnot Chum Case 9 6020303059 Kampong Thom O Ta Sek Case 10 6020411021 Kampong Thom Anlung Krasaing Case 11 6040106026 Kampong Thom Kruos Case 12 6040305004 Kampong Thom Kranhoung Case 13 6040602020 Kampong Thom Trach Case 14 6050112006 Kampong Thom Prasat Case 15 6050113011 Kampong Thom Veal Veng Case 16 6050201013 Kampong Thom Kampong Chvea Case 17 6050406029 Kampong Thom Choam Boeung Case 18 6050407031 Kampong Thom Tumnup Case 19 6050508033 Kampong Thom Kampong Chheu Teal Case 20 6060510022 Kampong Thom Phtorl Rumpos Case 21 6060701033 Kampong Thom Danghet Case 22 6060707039 Kampong Thom Svay Case 23 6070101001 Kampong Thom Boeung Lovea Case 24 6070408015 Kampong Thom Chi Meak Case 25 6070503022 Kampong Thom Trapaing Pring Case 26 6070806032 Kampong Thom Chambak Khang Cheung Case 27 6070906037 Kampong Thom Chhouk Rumduol Choam Thna Case 28 6080103002 Kampong Thom Beng Case 29 6080109007 Kampong Thom Sla Kor Case 30 6080201011 Kampong Thom Preah Naingkoal Case 31 6080202079 Kampong Thom Sampor Case 32 6080203014 Kampong Thom Srey Rongeut Case 33 6080303021 Kampong Thom Trapaing Choa Case 34 6080913054 Kampong Thom Ta Treal Case 35 6081004057 Kampong Thom Leuk Cheah Case 36 6081007058 Kampong Thom Prasat (Lekchaes) Case 37 6081214069 Kampong Thom Lovea Case 38 17030201025 Siemreap Tuol Kruos Case 39 17030305007 Siemreap Ta Koh Case 40 17030604012 Siemreap Sras Khvav Case 41 17030604027 Siemreap Ta Pen Case 42 17040209006 Siemreap Wat Kandal Case 43 17040510016 Siemreap Khnar Thnung Case 44 17040516020 Siemreap Kilometer Ta Chhim Case 45 17040707030 Siemreap Kbal Kduoch Case 46 17040907071 Siemreap Chup Tnot Case 47 17040910077 Siemreap Sop Mong Case 48 17041001038 Siemreap Chamreun Rath Case 55 49 17041002039 Siemreap Boeung Case 50 17041003040 Siemreap Spean Touch Case 51 17041106046 Siemreap Damrei Chhlang Case 52 17041107047 Siemreap Kanseng Case 53 17041211052 Siemreap Lovea Case 54 17060403030 Siemreap Lork Ta Ma Case 55 17060405011 Siemreap Prasat Khnar Case 56 17060603013 Siemreap Damnak Kchas Case 57 17060717028 Siemreap Sela Rumduol Case 58 17060806029 Siemreap Prey Longeang Case 59 17060903019 Siemreap Sranal Case 60 17061003023 Siemreap Hun Sen Ta An Case 61 17061007025 Siemreap Teuk Chum Case 62 17070207066 Siemreap Prasat Char Case 63 17070209004 Siemreap Peam Case 64 17070307006 Siemreap Kdei Run Case 65 17070803025 Siemreap Trakeat Case 66 17071006027 Siemreap Ta Tork Case 67 17071010030 Siemreap Chambak He Case 68 17071106032 Siemreap Prasat Case 69 17071109033 Siemreap Pradak Case 70 17090508019 Siemreap Pongror Case 71 17110114045 Siemreap Prasat Sanloang Case 72 17110601024 Siemreap Thlat Case 73 17110604025 Siemreap Kauk Sangker Case 74 17110807052 Siemreap Popel Kandal Case 75 17111001039 Siemreap Boeung Ngot Case 76 17111002063 Siemreap Punleuvichea Chup Sman Case 77 17111004041 Siemreap Phka Rumchek Case 78 17111005050 Siemreap Champei Case 79 17111008042 Siemreap Boeung Vean Case 80 17130107025 Siemreap Sre Robang Case 81 17130201012 Siemreap Kantuot Case 82 17130202005 Siemreap Hun Sen Khnar Krao Case 83 17130305006 Siemreap Anlung Thom Case 84 17140309020 Siemreap Khnar Phtaul Case 85 17140311022 Siemreap Trapaing Krasaing Case 86 17140404015 Siemreap Russey Tauch Case 56 No. SCHOOL ID PROVINCE SCHOOL NAME SCHOOL TYPE 87 2010203048 Battambang Phnom Chi Hang Comparison 88 2010205020 Battambang Kampang Keut Comparison 89 2010505045 Battambang Bos Khnor Comparison 90 2010602022 Battambang Kampeuv Comparison 91 2010703029 Battambang Sneung Comparison 92 2010704057 Battambang Chak Angkam Comparison 93 2020102035 Battambang Wat Thmar Kaul Comparison 94 2020303023 Battambang Veal Trea Comparison 95 2020306026 Battambang Prey Dach Comparison 96 2020411043 Battambang Tuol Preah Khe Comparison 97 2020602016 Battambang Nikum Comparison 98 2020804018 Battambang Ta Meak Chraneang Comparison 99 2021003042 Battambang Saray Andet Comparison 100 2050404020 Battambang Rohal Suong Comparison 101 2050407022 Battambang Bak Rotes Comparison 102 2050601010 Battambang Prey Chas Comparison 103 2060115091 Battambang Boh Pouy Comparison 104 2060404046 Battambang Chrey Run Comparison 105 2060409087 Battambang Svay Bopha Comparison 106 2060502002 Battambang Angkrorng Comparison 107 2060802065 Battambang Kilometer Lekh 4 Comparison 108 2060901039 Battambang Bung Bei Comparison 109 2060902040 Battambang Kuoy Chik Dei Comparison 110 2070206011 Battambang Kandal Stung Comparison 111 2070207037 Battambang Tvak Comparison 112 2070303016 Battambang Boeung Khtum Comparison 113 2070401007 Battambang Treng Comparison 114 2070403043 Battambang Chumnap Phas Komathsu Comparison 115 2070506027 Battambang Kiri Chum Comparison 116 2080407033 Battambang Ta Pon 1 Comparison 117 2080602047 Battambang Sala Trav Comparison 118 2080905021 Battambang Dambauk Khpuos Comparison 119 2080906022 Battambang Tuol Lovieng Comparison 120 2110101001 Battambang Sovann Kiri Comparison 121 2110103021 Battambang O Chrab Comparison 122 2110106010 Battambang Sre Andaung 2 Comparison 123 2110605024 Battambang Phoum Tnot Comparison 124 2110606022 Battambang Spong Chreuv Krom Comparison 125 2120205008 Battambang Chambak Comparison 126 2120402011 Battambang Kandoal Comparison 127 2120403019 Battambang O Thmar Comparison 128 2120506014 Battambang BoeungTrakuon Comparison 129 3020105003 Kampong Cham Bos Khnor Comparison 130 3020110028 Kampong Cham Phoum 35 Comparison 131 3020207007 Kampong Cham Preus Meas Comparison 132 3020304011 Kampong Cham Trapaing Russey Comparison 133 3020407015 Kampong Cham Phoum 3 Comparison 134 3020504030 Kampong Cham Por Preng Comparison 135 3020610038 Kampong Cham Sre Preal Comparison 136 3020701040 Kampong Cham Ta Ong Comparison 57 137 3130402019 Kampong Cham Baray Comparison 138 3130606025 Kampong Cham Phoum Thmei Comparison 139 3130707027 Kampong Cham Memeang Comparison 140 3130812058 Kampong Cham Kampong Samnanh Comparison 141 3131001036 Kampong Cham Sosen Comparison 142 3131309049 Kampong Cham O Ta Thok Comparison 143 3150102001 Kampong Cham Areak Tnot Comparison 144 3150405005 Kampong Cham Chheu Teu Comparison 145 3150902014 Kampong Cham Prek Kak Comparison 146 3151201032 Kampong Cham Sopheas Comparison 147 10030303012 Kratie Kampong Kor Comparison 148 10030505020 Kratie Prek Praing Comparison 149 10030508023 Kratie Prek Prasap Leu Comparison 150 10040206039 Kratie Tonsong Thleak Comparison 151 10040601017 Kratie Pa Khle Comparison 152 10040603038 Kratie Sre Chhouk Comparison 153 10040704044 Kratie Koh Real Comparison 154 10040904029 Kratie Rovieng Comparison 155 13030104003 Preah Vihear Phoum Veal Po Comparison 156 13030106024 Preah Vihear Bun Rany Hun Sen Trapaing Comparison 157 13030214034 Preah Vihear Sen Techo Morokot Comparison 158 13030506017 Preah Vihear Choam Sre Comparison 159 13030601005 Preah Vihear Kantuot Comparison 160 13060101001 Preah Vihear Chamroeun Comparison 161 13060503014 Preah Vihear Trapaing Raing Comparison 58 59 Annex 19: 2017-2019 baseline methodology for selection of comparison schools The comparison group for the 2017-2019 baseline survey is built by using Propensity Score Matching (PSM). Schools in the treatment group will be paired to schools located in districts in the table below where no school feeding programme is implemented. The independent variables included in the PSM is the percentage of poor households. Thus, effects in schools with poorer students will be compared to those in schools with poorerstudents. Districts from which control schools are selected Province District Battambang Banan Ek Phnom Maung Russey Ratanak Mondul Samlaut Sampeuv Loun Sangker Thmar Kaul Preah Vihear Sangkom Thmei Chom Ksan Kratie Prek Prasap Sambo Kampong Cham Chamkar Leu Prey Chhor Steung Trang As a new baseline, this survey employs a comparison group constructed by matching techniques (propensity score matching), selected from schools in the above districts in which no support is provided to any schools. It will serve to examine differences at baseline and next endline phases (through the difference-in-differences test), to quantify the change in key outcomes in the presence and in the absence of the intervention. The construction of the Propensity Score for matching comparison schools to treatment schools, as computed with the R software (MatchIt function), has been based on a logit model (see Figure 1). The algorithm applied for matching was the nearest neighbour, with replacement (given the small number of available control schools compared to treatment schools, one comparison school can be paired to several treatment schools). The independent variable was the percentage of poor households12. Graphs and tables (see Figure 2) below show how matching has resulted in a control sample in which schools are comparable to treated ones in terms of the independent variables: - Matched data QQ Plots show that control and treatment groups follow a similar distribution (more than all data QQ plots do). - Histograms for Matched control groups are similar to those of Matched treated groups (showing satisfactory regions of common support) - Balance improvement indicators are generally high, all above 67% 12 Perc.Poor HH, as it appears in following graphs and tables. 60 Logistic regression for propensity score estimation !"#$% = 1.0627 ∗ !/"0. !##" 11 − 0.2765 Note: logistic regression. The dependent variable (Proba) is 1 if the school is targeted by WFP intervention, and 0 otherwise. The coefficient represents the contribution of variable percentage of poor households (Perc. Poor HH) to the probability that the school istargeted. Deviance Residuals: Min 1Q Median 3Q Max -1.285 -1.127 -1.078 1.221 1.294 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -0.2765 0.1286 -2.150 0.0315 * Perc.Poor HH 1.0627 0.7066 1.504 0.1326 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: 1582.8 on 1143 degrees of freedom Residual deviance: 1580.5 on 1142 degrees of freedom AIC: 1584.5 Number of Fisher Scoring iterations: 3 Figure 2. Propensity Score Matching Percent balance improvement Propensity Score X..IDPoor Sample sizes All 602 542 Matched 316 542 Unmatched 286 0 Discarded 0 0 Mean Diff. eQQ Med eQQ Mean eQQ Max 99.8356 68.9951 67.4969 75.2708 99.8340 68.9767 67.5573 75.4431 Control Treated 61 Effect size and error estimations Three provinces Girls / boys Universe Number of schools 580 Number of students (HH) 131,201 Sample Sample size for schools 90 Sample size for HH (6 per school) 540 270 Effect size13 detected at school level 7.368 Effect size14 detected at HH level 8.700 12.4 Sample error15 at HH level 5.160 7.3 13 At school level, effect calculations are based on a paired t-test on the differences in outcome variable (the change Δ = ystart - yend). The Outcome variable proxy used is promotion rate. The estimation is based on a power of 80 %, an error of 5% (Threshold probability for rejecting the null hypothesis, Δ =0). The Standard Deviation of change is estimated through the formula: S(Δ )= S*(2(1-rwithin))1/2, S being the standard deviation of the outcome (promotion rate) in the treatment school population at baseline and rwithin the within-subject correlation of the outcome. For the three provinces together, S(Δ )=14.41, as S= 10.38481915 and rwithin = 0.0367 14 At HH level, effect calculations are based on a Chi-squared statistic (or z test), to compare proportions with a dichotomous outcome between two samples. The estimation is based on a power of 80 % and a 5% error. As outcome variable, we used “Percent of student (girls/boys) who, by the end of grade 6 demonstrate reading comprehension equivalent to their grade level as defined by national standards at USDA supported schools” and its value at baseline for the treatment group, 56%. 15 For each population group, error calculations are based on the estimation of the variance of the dichotomous variables’ estimatesfrom a single survey, i.e. variance of the estimate of the proportion of the population that verifies a particular characteristic. This is done assuming a 95 % confidence interval, and a probability of 50%, which is most conservative value for the true proportion of the population that verifies a particular characteristic. At household levels, the design effect of the cluster sampling is estimated at 1.5. 62 Annex 20: Geographic breakdown of sampled schools and households for the quantitative survey Province USDA supported schools (N u m b e r ) Comparison sc h o o l s (Number) Total schools SMP only (& HGSF Hybrid) (%)# SMP + THR (& HGSF￾Hybrid+ THR) (%)## Battambang 5 46 51 (29.8) Kampong Cham 20 20 (11.7) Kampong Thom 4 28 32 (18.7) Siem Reap 8 41 49 (28.7) Kratie 9 9 (5.3) Preah Vihear 10 10 (5.8) Sub-Total (%) 17 69 85 171 (100.0) Total 86 85 171 • Distribution of surveyed households by province, supporting programme, and sex of sampledchild Province USDA supported schools Comparison Total Schools (%) SMP only (& HGSF Hybrid) SMP + THR (& HGSF Hybrid+ THR) Battambang 30 0 252 282 (29.4) Siem Reap 46 246 0 292 (30.5) Kampong Thom 24 168 0 192 (20.09) Kampong Cham 0 0 108 108 (11.3) Kratie 0 0 42 42 (4.4.0) Preah Vihear 0 0 42 42 (4.4) Genderbreakdown of surveyed children Boys Girls Boys Girls Boys Girls Boys Girls 50 50 207 207 222 222 494 469 (50.0) (50.0) (50.0) (50.0) (50.0) (51.3) (48.7) Sub-Total (%) 100 414 444 (10.4) (43.2) (46.3) 958 Total 514 444 63 b. Female: Annex 21: School assessment School Assessment Questionnaire SY 2018-2019 Food for Education Programme I. GENERAL INFORMATION 1. Date: ………………… Start time: ……… End time: 2. Interviewer ID: 3. Interviewee name: ………………… …………… 4. Sex: ○1. M ○2. F 5. Interviewee is: 6. Interviewee Tel: ○School director/Deputy Director ○School Administrator ○ Head Teacher 7. Province: …………………………………………………………………… 8. District: …………………………………………………………………… 9. Commune: ………………………………………………………………… 10. Village: …………………………………………………………… ………………………………………… ……………………… 11. School name: ………………………………………… ………………… 12. School code: ………………………………………… …………………… 13. School type: ○ 1. SMP alone ○ 2. SMP+THR ○ 5. HGSF-Hybrid ○ 6. HGSF￾Hybrid + THR ○ 4. Control Please provide information on school mapping 14. Number of primary-school-age (6-11) children in the school’s catchment area during this school year, 2018-2019 *confirm with school census record *the number of eligible students in catchment area may be more or less than the actual number of enrolled students 14.a. Number of students aged 6 to 11 years in this school. *confirm with school records together with interviewee 14.b. Number of students aged over 11 years in this school during school year(2018-2019) *refer to document review (to be done by 2 enumerators after interview) 14.c. Total Number of school days during school year 2018-2019 Total: ……………… … Total: ……………… … Total: ……………… … Female: ………………… Female: ………………… Female: ………………… *refer only to students are present in school 14.d Total number of school days missed by all student during school year (2018-2019) *refer to document review (to be done by 2 enumerators after interview) *refer to Nov 2018 – June 2019 Total: Number of days: Number of days: RECORD IN SEPARATE SHEET 15. Total Number of school days in June 2019 *refer to document review (to be done by 2 enumerators after interview) Number of day: Number of day: 16. Total number of school days missed by all students in June 2019 Number of days: RECORD IN SEPARATE SHEET a. 64 17. Number of students in this school year, 2018-2019 by grade and gender 17.a. Grade 17.b. Number of enrolled student 17.c. Number of promoted student 17.d. Number of repeated student 17.e. Number of dropped out student Total Female Total Female Total Female Total Femal e 1 2 3 4 5 6 18. Disability Data information 1 2 3 4 5 6 II. IMPROVED QUALITY OF LITERACY INSTRUCTION 20. Number of teachers and contractual teachers for grade 1-6 in this school year, 2018-2019 (exclude school director, even though s/he also taught) Total teachers from grade 1-6: Female: ………………… ………………… 21. Number of contractual teacher if any Total: ………………… Female: ………………… 22. Number of full time equivalent teaching staff in this school year, 2018-2019 (Include school director if s/he also taught) (can be administrator that can teach sometime) 3.a. Total teachers from grade 1-6: Female: ………………… ………………… 23. Does the school have attendance sheets/ books in this school year, 2018-2019? (Please observe or ask school director to show it) 23.1. Number of classroom that have attendance sheets/books in this school year, 2018-2019? ○ 1. Yes ○ 2. No …………………………….classrooms 24. Total number of studying days for this school year, 2018-2019 (preparation days of teacher) ………………… days 25. Attendance of grade 1-6 Teachers in this school year Nov 2018- June 2019 (LOGIC: check with Q20) 25.a. Teacher (one teacher per line) 25.b. Sex (1. M and 2. F) 25.c. Number of teaching days 25.d. Number of absent days 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 65 16 17 18 19 20 21 22 23 24 25 26. Does the school have: (Multiple answers) □ 1. School development plan □ 2. Safe school environment (school gate, fence, and fence around pond (if applicable) at a minimum. Additional features may include slope/ ramp for disabled students, and playground) □ 3. Record and reporting system □ 4. Don’t have any above 27. Number of teachers using the national literacy curriculum and the related instructional materials during school year 2018-2019. a. Total: b. Female: (Need to check name of curriculum) III. IMPROVED SCHOOL INFRASTRUCTURE (Please observe and check with relevant people) 28. Number of Grade 1-6 classrooms available in use in this school year, 2018-2019 ………………… classroom(s) 29. How many Grade 1-6 classrooms have sufficient literacy instructional materials for effective instruction in this school year, 2018-2019? Note for enumerator: - Alphabet poster (reading related material) - Numeric (mathematics related material) - Picture with description (ie.Human structure, animal) - Science related poster (observe the class) ………………… classroom(s) 30. Did the school receive school materials or learning package in this school year, 2018-2019? ○ 1. Yes ○ 2. No ○ 99. N/A 31. Did the school receive stationery package in this school year, 2018-2019? (Folders, hole-punchers, calculators, whiteboards, and other non-food items…) ○ 1. Yes ○ 2. No ○ 99. N/A 32-39. USE OBSERVATION SHEET IV. INCREASED ENGAGEMENT OF LOCAL AND COMMUNITY GROUPS 40. Does the school have a Local School Feeding Committee (LSFC), or School Support Committee (SSCs)? ○ 1. Yes ○ 2. No (If no, skip to Q 41) 40.a. Is it functioning and contributing to the school? ○ 1. Yes ○ 2. No ○ 99. N/A 40.b. Number of parents who are members of LSFC, or SSCs Total: ………………… Female: ………………… 40.c. Has there been any training for LSFC/SSC on the importance of education? ○ 1. Yes ○ 2. No 40.d. How many times were awareness-raising events/trainings conducted for school year 2018-2019? ○ 1. 1 time ○ 2. 2 times ○ 3. More than 2 times ○ 4. None (Skip to Q 41) 40.e. When were awareness-raising events/trainings conducted? (Multiple answers) □ 1. At beginning of the school year □ 2. At village meeting □ 3. Other, specify ……………… 41. Are there public-private partnerships formed in this school? (eg: farmer association HGSF) ○ 1. Yes ○ 2. No (If no, skip to Q 42) 41.a. If yes, how many members? 41.a.1. Number of groups: …………. 66 41.a.2. Total: …………… Female: …………… 42. How much did community/ parents contribute to the school in this school year, 2018-2019? 42.a. In cash:.............................................................................. USD/ year 42.b. In kind: ..............................................................................USD/ year V. INCREASED USE OF HEALTH DIETARY PRACTICES 43. Does the school have soap and water at a hand washing station in this school year, 2018-2019? ○ 1. Yes ○ 2. No (If no, skip to Q 45) 43.a. Did students use hand washing station? How often? ○ 1. Yes, always ○ 2. Yes, often ○ 3. Yes, sometimes ○ 4. Yes, rarely ○ 5. Did not use ○ 98. Don’t know 44. How long does the school have current soap supply in stock (hand and/or dish soap)? ○ 1. Less than 1 week ○ 5. 1 month to 3 months ○ 2. 1-2 weeks ○ 6. 4 months to 6 months ○ 3. 2-3 weeks ○ 7. Whole school year ○ 4. 3-4 weeks ○ 8. No soap 44.a. Who provided soaps for hand washing to school? (Multiple answers) □ 1. . Programme budget □ 5. Other NGOs (PB) □ 6. Companies □ 2. WFP/PLAN/WVC □ 7. Other, specify ………………… □ 3. Community □ 4. Charity persons 44.b. How frequently does the school buy or receive soap? ○ 1. Weekly ○ 5. Every 1-3 months ○ 2. Every 2-3 weeks ○ 6. Every 4-6 months ○ 3. Every 3-4 weeks ○ 7. Annually ○ 4. Monthly ○ 8. Never 45. Did the school receive kitchen utensil packages? ○ 1. Yes ○ 2. No (If no, skip to Q 45c) 45.a. If yes, who provided kitchen utensil packages? (Multiple answers) □ 1. Programme budget □ 5. Other NGOs (PB) □ 6. Companies □ 2. WFP/PLAN/WVC □ 7. Other, specify ………………… □ 3. Community □ 4. Charity persons 45.b. If yes, what are they? And How many? □ 1. Cooking pots:…… □ 5. Knife:…….. □ 2. Serving pots:……... □ 6. Spoon and Plat:…….. □ 3. Cutting board:……. □ 7. Cooking equipment:……. □ 4. Storage □ 8. Other, specify ………………… equipment:……. 45.c. If no, why? (IF CONTROL SCHOOL, NOT APPLICABLE) ……………………………………………………………………… 45.d. Does this school arrange meals distribution directly from cooking pot? ○ 1. Yes ○ 2. No 45.e. Does this school arrange meals distribution by class by stainless steel pots ○ 1. Yes ○ 2. No 45.f. Does this school arrange meals distribution by class by plastic pots ○ 1. Yes ○ 2. No 45.g. In June 2019, what types of plate do children use and what percentage? (Multiple answers) ( 0 if none) □ 1. Ceramic:……...% □ 3. stainless steel ............% □ 2. plastic:………...% □ 4. plastic bags.................% 46. Did the school receive hygiene packages for a yearly supply? ○ 1. Yes ○ 2. No (If no, skip to Q 47) 46.a. If yes, who provided hygiene packages for yearly supply? (Multiple answers) □ 1. PB budget □ 5. Other NGOs □ 2. WFP/PLAN/WVC □ 6. Companies □ 3. Community □ 7. Other, specify ………………… □ 4. Charity persons 46.b. If yes, what are they? □ 1. Soap □ 6. Toothpastes and brushes 67 Note: Don’t read answers (Multiple answers) □ 2. Water filters □ 7. Hand towel □ 3. Bowls □ 8. Nail cutter □ 4. Combs □ 9. toilet brushes □ 5. long brooms □ 10. Other, specify ………………… 47 Did the school receive the training on food preparation and storage practices in the school year 2018-2019? ○ 1. Yes ○ 2. No 48. What did the school implement for food preparation and storage practices? Note: Don’t read answers (Multiple answers) □ 1. Clean cooking area □ 2. Store food at the appropriate temperatures (not in plastic pan, petrol tank) □ 3. Cover cooked food and store in safe place □ 4. Wash hand before cooking □ 5. Other, specify ………………… 49. Do teachers/ stakeholders know about proper food storage? (Meat, vegetable, cooked meal, etc.) ○ 1. Yes ○ 2. No 50. Did the school receive the training on good health and nutrition practices? ○ 1. Yes ○ 2. No 51. Please name the 3 food groups? Note: Don’t read answers (Multiple answers) □ 1. Energy (Carbohydrates. Ex: rice, sugar, potato, oil) □ 2. Building (Protein. Ex: meat, fish, peanuts, eggs) □ 3. Protective foods (Vitamins & minerals. Ex: green leaves, pumpkin, banana, mango) □ 888. Don’t know Solid waste disposal 52. Does the school have bins or other equipment for managing solid waste? ○ 1. Yes ○ 2. No 53. Is solid waste collected daily and safely disposed of? ○ 1. Yes ○ 2. No ADDITIONAL SUPPORT PROGRAM 54.15.Is there any support programme being implemented /implemented in this school year, 2018-2019? 1. Yes 2. No (skip to 54.16) 54.15.a. If yes, what are they? o 1.Building (Construction, rehabilitation, repair) o 2. Material for Study/ Office supply o 3.Morning meal program or meal program o 4. Others (specify)…………………………... 54.16. Has this school ever been received any support programme in the last school year, 2017-2018? 1. Yes 2. No 54.16. a. If yes, what are they? o 1.Building (Construction, rehabilitation, repair) o 2. Material for Study/ Office supply o 3.Morning meal program or meal program o 4. Others (specify)…………………………... 68 School Assessment Questionnaire SY 2018-2019 Food for Education Programme I. GENERAL INFORMATION 1. Date: ………………… Start time: ……… End time: 2. Interviewer ID: 3. Interviewee name: ………………… …………… 4. Sex: ○1. M ○2. F 5. Interviewee is: ○School director/Deputy Director ○School Administrator ○ Head Teacher 6. Interviewee Tel: ………………………………………………………………… 7. Province: …………………………………………………………………… 11. School name: …………………………………………………………… 8. District: …………………………………………………………………… 12. School code: ……………………………………………………………… 9. Commune: ………………………………………………………………… 13. School type: ○ 1. SMP alone ○ 2. SMP+THR ○ 5. HGSF-Hybrid ○ 6. HGSF-Hybrid + THR ○ 4. Control 10. Village: …………………………………………………………… OBSERVATION FORM III. IMPROVED SCHOOL INFRASTRUCTURE (Please observe and check with relevant people) 32. Does the school have latrines? ○ 1. Yes ○ 2. No (If no, skip to Q 33) 32.a. Number of latrines in the school grounds ( 0 if no non-functioning or functioning latrine) 32.a.1. Functioning: ………………… 32.a.2 Non-functioning: …………… 32.b. Number of latrines rehabilitated or constructed in this school year, 2018-2019 ( 0 if no latrine rehabilitated or constructed and skip to Q 32.c) 32.b.1. Rehabilitated:………. 32.b.2. Supported by:  1. WFP/PLAN/World Vision  2. Other, specify 32.b.3. Constructed:………... 32.b.4. Supported by:  1. WFP/PLAN/World Vision  2. Other, specify 32.c. Are the functioning latrines separated for teachers and students? ○ 1. Yes ○ 2. No (If no, skip to Q 32e) 32.d. If 32c yes, how many functioning latrines for teachers? Number:………………… 32.e. Are the functioning latrines for students separated for boy and girl students? ○ 1. Yes ○ 2. No (If no, skip to Q 32.g) ○ 99. N/A 32.f. If 32e yes, how many? 32.f.1. Functioning latrines for boy students: …………… 32.f.2. Functioning latrines for girl students: …………… 32.g. Functioning latrines non-separated Number:………………… 32.h. Does the school have latrines accessible for students with disability? ○ 1. Yes ○ 2. No (If no, skip to Q 32.i) 32.h.1. If yes, how many latrines for students with disability? Functioning latrines for disable students:…………………. 32.h.2. Facilities of latrine for students with disability (Multiple answers) □ 1. Suitable size for students with disability □ 2. Slope (ramp) for students with disability □ 3. Handle for students with disability □ 4. Other, specify: …………………………… □ 5. Don’t have any above 32.i. What are the current conditions of functioning latrines? (skip if all latrines are non-functioning Q 32j) (Multiple answers) □ 1. Clean and well maintained □ 2. Dirty, not well maintained □ 3. Broken but still usable □ 4. Other, specify: …………………………… 32.j. What are the current conditions of non-functioning latrines? (skip if all latrines are functioning Q 32.k) (Multiple answers) □ 1. Door is broken □ 2. Commode is broken □ 3. Pit latrines is broken or full □ 4. Washbasins are broken □ 5. Other, specify: …………………………… 69 32.k. How do you manage and maintain the latrines? □ 1. Train students and take turn to clean latrines sometimes 70 (Multiple answers) □ 2. Keep soap/hand washing facilities within or near the toilets sometimes □ 3. Lock latrines during school vacation □ 4. Ensure washbasin is full of water □ 5. Propose users to leave shoes outside of latrine stalls □ 6. Other, specify: …………………………… 33. Does the school have kitchen? (Ask kitchen key) ○ 1. Yes ○ 2. No (If no, skip to Q 34) 33.a. Number of kitchens rehabilitated or constructed in this school year, 2018-2019 ( 0 if no kitchen rehabilitated or constructed and skip to Q 33.b) 33.a.1. Rehabilitated:……... 33.a.2. Supported by:  1. WFP/PLAN/World Vision  2. Other, specify 33.a.3. Constructed:……… 33.a.4. Supported by:  1. WFP/PLAN/World Vision  2. Other, specify 33.b. If yes, what are the current conditions of the kitchen? (Multiple answers) □ 1. Good condition □ 2. Lacking kitchen utensils □ 3. Clean cooking and eating equipment □ 4. Leaking roofs □ 5. Flooded during rainy season □ 6. Using rocks as stove □ 7. Other, specify: …………………………… 34. Does the school have energy-saving stoves? ○ 1. Yes ○ 2. No (If no, skip to Q 35) 34.a. Number of energy-saving stoves rehabilitated or constructed in the school year, 2018-2019 ( 0 if no energy-saving stove rehabilitated or constructed and skip to 34.b) 34.a.1. Rehabilitated:………………….. 34.a.2. Supported by: ○ 1. WFP/PLAN/World Vision ○ 2. Other, specify 34.a.3. Constructed:……………… 34.a.4. Supported by: ○ 1. WFP/PLAN/World Vision ○ 2. Other, specify 34.b. If yes, what is the current condition of the energy-saving stoves? (single answer) ○ 1. Good condition and function well ○ 2. Poor condition but still work ○ 3. Broken, not functioning ○ 4. Other, specify: …………………………… 35. Does the school have a storeroom (or place to store food)? ○ 1. Yes ○ 2. No, please specify where the food stored: ………………………………………………………… (and, skip to Q 36) 35.a. Number of storerooms (for food) rehabilitated or constructed this school year, 2018-2019 (0 if no storeroom rehabilitated or constructed and Skip to Q 35.b) 35.a.1. Rehabilitated:………………… 35.a.2. Supported by: ○ 1. WFP/PLAN/World Vision ○ 2. Other, specify 35.a.3. Constructed:………………… 35.a.4. Supported by: ○ 1. WFP/PLAN/World Vision ○ 2. Other, specify 35.b. If yes, what are the current conditions of the storerooms? (Multiple answers) □ 1. Well cleaned □ 2. Floor is dry □ 3. Pallets for food storage □ 4. Door is locked well □ 5. Security guard at night time/ during school vacation □ 6. Foods are stored in order □ 7. Leaking roofs □ 8. Broken windows/door □ 9. Damaged walls □ 10. No walls □ 11. Food was stored off ground □ 12. Storeroom had ventilation □ 13. Other, specify: …………………………… 36. Doesthe school have drilled wells/waterstations?(Drinkable water) ○ 1. Yes ○ 2. No (If no, skip to Q 37) 71 36.a. Number of drilled wells/water stations rehabilitated or constructed in this school year, 2018-2019 (0 if no drilled wells/water station rehabilitated or constructed and kip to Q 36.b) 36a.1. Rehabilitated:……………… 36.a.2. Supported by: ○ 1. WFP/PLAN/World Vision ○ 2. Other, specify 36.a.3. Constructed:……………… 36.a.4. Supported by: ○ 1. WFP/PLAN/World Vision ○ 2. Other, specify 36.b. Number of functioning drilled wells or water station installed on the school grounds? (0 if doesn’t have and skip to Q 36.d) 36.b.1 Functioning drilled well: ………………… 36.b.2. Functioning water station: ………………… 36.b.3. Other, specify:………………………………: ……………………… 36.c. What are the current conditions of the functioning drilled wells/ water station? (Multiple answers) □ 1. Functioning well year-round □ 2. Water is used for human consumption □ 3. Platform is clean □ 4. System to clean the wells □ 5. Other, specify: …………………………… 36.d. Number of non-functioning drilled wells or waterstation installed on the school grounds? (0 if doesn’t have and skip to Q 37) 36.d.1. Non-functioning drilled well: ………………… 36.d.2. Non- Functioning water station: …………… 36.d.3. Other, Specify: ……………………………: ……… 36.e. What are the current conditions of the non-functioning drilled wells/ water station? (Multiple answers) □ 1. Functioning only during rainy season □ 2. Water is used for animals only □ 3. Arsenic (poisonous) □ 4. Hand pump/ rain water station was broken □ 5. Other, specify: …………………………… 37. Does the school have year-round access to a clean and safe water source? ○ 1. Yes (whole school year) ○ 2. No (some months not available) ○ 3. No (no clean water) 38. Does the school have hand washing station in the school? ○ 1. Yes ○ 2. No (If no, skip to Q 39) 38.a. If yes, how many hand washing station in the school? 38.a.1. Fix hand washing station:…………….. 38.a.1.1. Single Student: ………… 38.a.1.2. Multiple Student: ………… 38.a.2. Mobile hand washing station:…………….. 38.a.2.1. Single Student: ………… 38.a.2.2. Multiple Student: ………… 38.b. If yes, what are the current conditions of the hand washing station? Description Fix hand washing station Mobile hand washing station 38.b.1. Good condition & function well year-round ○ ○ 38.b.2. Good condition & function well only during rainy season ○ ○ 38.b.3. Poor condition but still work year round ○ ○ 38.b.4. Poor condition but still work only during rainy season ○ ○ 38.b.5. Broken, not functioning ○ ○ 38.b.6. Other, specify ………… ………… 38.c. Number of hand washing stations rehabilitated or constructed in this school year, 2018-2019 (0 if no hand washing station rehabilitated or constructed and skip to Q 39) 38.c.1. Rehabilitated:.. 38c.2. Supported by: ○ 1. WFP/PLAN/World Vision ○ 2. Other, specify 38.c.3. Constructed:… 38.c.4. Supported by: ○ 1. WFP/PLAN/World Vision ○ 2. Other, specify 39. Does the school have vegetable gardens? ○ 1. Yes ○ 2. No (If no, end observation) 39.a. Were any vegetable gardens rehabilitated or established i this school year, 2018-2019? ○ 1. Yes ○ 2. No (If no, end observation) 72 39.b. If yes, what were the purposes for establishing the gardens? □ 1. Practicing life skills □ 2. Supplementing SMP recipe □ 3. Both 39.c. How many hours a week were children mentored on school gardens? ○ 1. 1 – 2 hours ○ 2. 3 – 5 hours ○ 3. > 5 hours 39.d. How did schools get vegetable seed? □ 1. Purchased using PB □ 3. Purchased using School Improvement Grant □ 2. Donated by PLAN/World Vision □ 4. Donated by WFP 39.e. Which months did you grow vegetable in the gardens in this school year, 2018-2019? (Multiple answers) □ 10. Oct □ 11. Nov □ 12. Dec □ 1. Jan □ 2. Feb □ 3. Mar □ 4. Apr □ 5. May □ 6. Jun □ 7. July □ 8. August □ 9. September □ 13. Don’t grow vegetable 39.f. How did you manage and maintain the vegetable garden? (Multiple answers) □ 1. Children are mentored by trained teachers and community □ 2. Children from grade to grade are assigned to each plot of land □ 3. Prevent animals from entering school compound by repairing schools’ fence once per year □ 4. Other, specify: …………………………… 73 Annex 22: Teachers questionnaire 1. GENERAL INFORMATION School Assessment Questionnaire for Education Programme (Teacher) USDA McGovern Dole 1.1. Date: 1.2. Interviewer ID: 1.3.School name: 1.5. District: 1.7. Village: 1.8. Teacher Name 1.4. Province: 1.6. Commune: 1.9. Teaching Grade: 1. Grade 2 , 2. Grade 4 , 3. Grade 6  19.1 Shift: All interviews must be morning shift 1.8.1. Contact Number: 2. IMPROVED STUDENT ATTENTIVENESS 2.1. Number of absent students in your class on 9th July (Check records on attendance sheet together with teacher) a.Total: b. Female: 2.3. Total Number of students in your class *note to interviewer: current students in month of July 2.4 Percentage of students in your class eat breakfast at school (School provided) *note to interviewer: clarify that this is EAT, not just offered. Short-term hunger a. Total: b. Female: %: 2.5. Please estimate the number of children being hungry during classes and frequency of this happening this July. LOGIC: Daily + Sometimes = Total 2.5.1. Does it vary by month? 2.5.2. If yes, please specify the month when most students come to school hungry. 2.5.3 Does it also vary by shift? a. Total: a. Daily: 1. Yes:  2. No:  Month: 1. Yes:  2. No:  b. Female: b. Sometimes: 2.5.4. In which shift are more children hungry during classes? 1. Morning:  2. Afternoon:  2.6. Please estimate the number of children who are inattentive (sleepy, inactive) during classes and frequency of this happening this July. LOGIC: Daily + Sometimes = Total 2.6.1. Does it vary by month? 2.6.2. If yes, please specify the month when most students are inattentive? 2.6.3 Does it also vary by shift? a. Total: a. Daily: 1. Yes: Month: 1. Yes:  2. No:  b. Female: b. Sometimes: 2. No: 2.6.4. In which shift are more children inattentive (sleepy, inactive) during classes? 1. Morning:  2. Afternoon:  74 Signature of the teacher: 75 MAIN RESPONDENT WILL BE THE HOUSEHOLD HEAD (OR ADULT MALE OR FEMALE); ANSWERS FOR SOME MODULE WILL BE TAKEN FROM THE MOTHER OR MAIN FEMALE OF THE HOUSEHOLD AND SCHOOL GOING CHILD (WHO HAVE BEEN SELECTED FROM SCHOOL FOR HOUSEHOLD SURVEY). INTERVIEWER MUST RECORD WHICH MODULE IS ANSWERED BY WHICH RESPONDENT. Annex 23: Household questionnaire Household Survey Questionnaire McGovern-Dole School Feeding and Take-home Rations Endline Survey 2019 My name is ……………. and I work for ……………….. (name) and my colleague is ………………….. andworks for................We are part of a team carrying out a survey to gather information on the Impact of WFP’s interventions in this commune. We would like to ask you some questions about your family. The interview usually takes around 1 hour to complete. Any information that you provide will be kept strictly confidential and will not be shown to other people. This is voluntary and you can choose not to answer any or all of the questions if you want. However, we hope that you will participate since your views are important. Do you have any questions? May we begin now 1. Yes⎕↓ (go to the following questions) 2. No ⎕ →Refused Outcome of interview 1. Completed 2. Partially completed 3. Interview postponed 4. Others SECTION AA – BASIC QUESTIONNAIRE INFORMATION, QUALITY CONTROL AND DATA ENTRY QUESTIONNAIRE INFORMATION AA01.Questionnaire Number: AA02. Location: Name: Code: Province District Commune Village AA03. Date: | | | / | | | / 2019 (Day/Month /Year) AA04. Start time End time AA05. ID of enumerator QUALITY CONTROL AA06. ID of team leader AA07. Date of checking: | | | / | | | / 2019 (Day/Month /Year) AA08. Remark: 76 SECTION AB – BASIC INFORMATION OF INTERVIEWEE AB01. Name of FIRST interviewee AB02. Gender of interviewee (circle) 1 = Male 2 = Female AB03 Relationship to children Refer to Codelist – A03 AB04. Phone number (if any) | | | | | | | | | | | AB05. Name of interviewee (Mother/Main Female) Can be the same as AB01 if the same respondent AB06. Gender of interviewee (circle) 1 = Male 2 = Female AB07 Relationship to children Refer to Codelist – A03 AB08. Phone number (if any) | | | | | | | | | | | AB09. Sample type (circle) 1= Treatment 2= Control AB10. Name of child (sample selection) AB11. Type of benefit received (For treatment only) 1 = SMP+THR 3 = SMP 4 = HGSF-Hybrid 5 = HGSF-Hybrid + THR AB12. Name of primary school AB12a. Code of primary school | | | | | | | | | | | | | 76 PART I: Respondent: The Household head or an adult who knows educational information of all school-age children in the household SECTION A: HOUSEHOLD COMPOSITION (All household members who currently live and eat at the household including the respondent) INTERVIEW: RECORD SINGLE ANSWER PER COLUMN PER ROW ID Code Name Relationshi p to household head (See Code below) Sex 1. Male 2. Female Age (years) Marital Status 1. Single 2. Married 3. Widow/ widower 4. Divorced/ separated 5. Desert Age at First Marriag e Can [Name] read and write? 1. Can read and write 2. Can sign only 3. Can read only 4. Cannot read and write Education (highest class completed) (See Code below) Currently attending school? 1. Yes 2. No (skip to A12a) If yes, is this school included in the same selection? (The interviewer will ask the name of the school, and then fill this answer by corroborating the name with the list provided to him/her of sample schools) 1. Yes 2. No Current two main occupation s 1st 2nd A01 A02 A03 A04 A05 A06 A07 A08 A09 A10 A11 A12 a A12 b 01 HHH 02 03 04 05 06 07 08 09 10 11 12 13 77 Exchange Rate & Gold price • 1 USD = 4,000 Riels • 1 Thai Baht = 132 Riels • Gold (kilo) = 159USD/Chi Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION B1: EDUCATIONAL CHARACTERISTICS OF SCHOOL AGE CHILDREN (Children aged 6-11 years old OR anyone in Grades 1-6) Code of A03. Relationship to household head Code of A09. Education (highest class completed) Code of A12a-A12b. Occupation 1. Household head 2. Husband/wife 3. Son/daughter 4. Brother/Sister 5. Father/mother 6. Uncle/aunt 7. Father/mother-in-law 8. Grandson/daughter 9. Niece/nephew 10. Cousin 11. Other relatives 12. Permanent servant 13. Other non-relatives 14. Son /Daughter-in￾law 15. Grand father/mother 99. Never attended school 98. Still in grade 1 97. Preschool class (general) 1. Completed grade1 . . . 12. Completed grade12 14. Vocational trainings 16. BA/BSc pass 18. MA/MSc and above 1.Farming activities 2.Pastoral activities 3.Fishing/ activities 4.Agricultural labour (daily wage) 5.Non-agricultural labour (daily wage) 6.Tailor/potter/blacksmith/goldsmith/hair cutter/ cobbler/carpenter/mason/plumber/electrician/motor mechanic 7.Government officer 8.Private sector employee/worker (monthly / bi-monthly salary) 9. NGO worker 10.Driver 11. Other salaried worker 12. Doctor/engineer/ lawyer 13. Teacher 14. Religious worker 15. Midwife/nurse 16.Food processing 17. Handicrafts 18.Sand harvester 19. Charcoal production 20.Brewing 21. Petty trader/ Vendor 22.Business/shop 23.Medium/ large scale trader 24.Contractor 25.Student 26.Housewife 27.No occupation 28. Other (specify) 78 TO BE FILLED FOR ALL HOUSEHOLD MEMBERS who are 6-11 years old, or in Grades 1-6 (Please copy I.D. code from family list in SECTION A carefully) ID Code At what age (in years) did [NAME] start school? (write 999 and skip to B1.10a if never attended school) Can [Name] read or write a simple sentence? 1. Yes 2. No If [NAME] is currently attending school If not currently attending school If [NAME] never attended school Highest completed grade # of days attended school last week Did [NAME] receive any external (non￾household) support for this school year? If yes, name the most important (See Code below, allow multiple) Can you give 2 important reasons why [NAME] attend this school (See Code below) Do you plan on enrolling [NAME] in school next year? 1. Yes 2. No Reason for stopping school (name up to 2 reasons) (See Code below) Can you give 2 main reasons why [NAME] never attended school? (See Code below) 1st 2nd 1st 2nd 1st 2nd B1.1 B1.2 B1.3 B1.4 B1.5 B1.6 B1.7a B1.7b B1.8 B1.9a B1.9b B1.10a B1.10b Code of B1.6. Name of external support Code of B1.7a-B1.7b. Reasons of attend school Code of B1.9/1.10a-B1.9/1.10b. Reasons for stopping/never attending school 1. Financial 1.Good teachers 1.The child doesn’t want to study 2.Food 2.Parents’encouragement 2. The child did not do well in school and dropped out 3.Other in-kind 3.The child is very devoted to studies 3. No suitable school available/school is too far 4. No support at all 4.Meal is available at school 4. No teacher/Supplies 888. Don’t know 5. The teachers teach very well and help my child in studying 5. Cannot afford the cost of schooling due to poverty 6.Boarding facilities in the school 6. The child must engage in paid work to contribute to household income 7.Good academic performance of the child 7. The child must help with household chores (taking care of children/elderly 8.The school has very good facilities (classrooms, toilets, drinking 8. The child is disable and not eligible to attend a regular school water etc.) 9. The child has been suffering from long term illness (over 3 months) 9.Take home rations (eg. rice, oil, bean, etc.) 10. No meal is offered at school 11. I want my child to get an education 11. Already completed desired schooling 12. The school is located near to my house 10.Other 12. The child got married and dropped out of school 13. Going to school is not safe for the child (e.g. risk of being harassed on way to school) 14. Other (specify) 79 Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION B2: EDUCATIONAL CHARACTERISTICS OF SCHOOL AGE CHILDREN (Continue) B2.1. Now we will ask you if you have considered the following factors when you made a decision about your child’s schooling (ask about the sample child). 1. Very important 2. Important 3. Not important 4. I did not consider this factor INTERVIEWER: USE SHOWCARDS Reasons Answer B2.1.1. The school is close to my house B2.1.2. Good quality of education at the school B2.1.3. Good quality of infrastructure at the school (e.g. classroom, toilets, drinking water) B2.1.4. Good future career/livelihood prospect of the child if s/he finishes the school B2.1.5. My child need to do household chore rather than going to school (e.g. taking care of siblings/elderly at home) B2.1.6. Scholarships received for continuing school ( B2.1.7. Hot breakfast at school (school feeding program) B2.1.8. Costs of schooling (e.g. fees, uniforms and books) B2.1.9. Concern about security of the child when traveling to school (e.g. personal security risks as a result of ethnic conflict, civil disturbances, physical violence (harassment, rape, corporal punishment at school B2.1.10. Long term illness/disability of the child B2.1.11. Others (please specify) B2.2. Please identify 3 benefits of primary education? *Do not read the options first. Record the answers. (Please probe as much as possible) 1) My child will learn to read and write 2) Primary education will make my child a good humanbeing 3) Primary education will help him/her to continue studying in upper classes 4) Primary education will provide my child with valuable life skills 5) Primary education will help to get a good job 6) Primary education will help him/her to be goodfarmer 7) Primary education will help my child to become a better father/mother when they are adult 8) Primary education will open up more opportunities in the future 9) Others (specify) 98. Don’t know 99. No benefit 1. 2. 3. B2.3. How does your child travel to primary school (the most often)? SA 1. Foot 2. Bicycle 3. Any carts 4. Motorcycle 5. Koyun (tractor) 6.Others (specify) B2.4. How long does it take to go to school from home if the child walks? 1. Less than 15min 2. 15min to 29 min 80 3.30min to 44min 4. 45min to 1hr 5. More than 1 hr B2.5 How much does it cost to go to school (one way) if the child uses public transport (most often)? SA **If respondent never uses public transport, identify the most common public transport in this area. In USD (99 = N/A if the school is too close to use a public transport) Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION C1 – HOUSEHOLD INCOME Now I would like to know about income earned by household members in the last 12 months (Respondent: main adult or HHH). Member ID Income In the past 12 months, Which How much was earned Code (from activity how many months was months? List from this activity? (total section A) (see income earned from this months in of 12 month, in US codes below) activity? below col (see code Dollar) C1.1 C1.2 C1.3 C1.4 C1.5 Total 81 C1.2. Code of income activities C1.4. code of months 1. Farming activities 2. Pastoral activities (pig, cow, chicken) 3. Fishing/ activities 4. Agricultural labour 5. Non-agricultural labour 6. Tailor/potter/blacksmith/goldsmith/ cutter/cobbler/carpenter/mason plumber/electrician/motor mechanic 7. Government officer 8. Private sector employee 9. NGO worker 10. Driver 11. Other salaried worker 12. Doctor/engineer/lawyer 13. Teacher 14. Religious worker hair 15. Midwife/nurse 16. Food processing 17. Handicrafts 18. Sand harvester 19. Charcoal production 20. Brewing 21. Pretty trader 22. Business/shop 23. Medium/ large scale trader 24. Contractor 1 = January 2 = February 3 = March 4 = April 5 = May 6 = June 7 = July 8 = August 9 = September 10 = October 11 = November 12 = December In the last 12 months, how much did your household members receive from the following activities Activity C1.6 Annual Cash Income (in US Dollars) C1.6.1. Agricultural product sale C1.6.2. Animal sale C1.6.3. Animal product sale (milk, eggs…) C1.6.4. THR sale C1.6.5 total Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION C1 – HOUSEHOLD INCOME (continue) In the last 12 months (from July 2018-June 2019) did your household receive any income/assistance from the following sources, in addition to your salary/household production sales? Description Estimated Earning C1.7. Income in Cash C1.8. Non￾Cash 1.Remittance from foreign country 2.Remittance within home country 3.Rice and cash from WFP (THR) 4. Assistance (cash or kind) from International organization/NGO 5. Assistance from government 6. Interest earning/periodic payment received from collective 7. Interest earning from Credit (money lent to others) 8. Rice and cash from others 9. total 82 Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION C2 – HOUSEHOLD ASSETS Please list # of the following assets owned by the household (m2= 0.0001 ha, Ar= 0.01 ha, Kong=0.09 ha, 1Rai = 0.16 ha) Productive Assets Non-Productive Assets C2.1 Assets C2.2 1=Yes 2= No C2.2 # of assets C2.1 Assets C2.2 1=Yes 2= No C2.2 # of assets owned 01. Up land (in m2) 19. Television 02. Rice land (in ha) 20. Radio 03. Rice miller 21. Bicycle 04. Hand hammer 22. Motorbike 05. Plough 23. Car 06. Hoe 24. Cell phone 07. Axe 25. Bed 08. Cart 26. Table 09. Hand tractor 27. Chair 10. Tractor 28. Metal cooking pot 11. Water pump 29. Water collecting bin 12. Sewing machine 30. Water storage bin 13. Buffalo 31. Jewelry 14. Cow 32. Gold 15. Pig 33. Others (specify) 16. Chicken 17. Cash savings (in USD) 18. Generator Others C.2 Describe the main materials used in the house: (INTERVIEWER TO OBSERVE AND RECORD BELOW) 2 Wall RECORD CODE [1] Hay/straw/jute stick/palm leaf/plastic 3 Roof RECORD CODE [2] Bamboo [3] Mud [4] Taly/tiles [5] Tin [6] Cement/brick /rod 4 Floor RECORD CODE [7] Wood 83 Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. MODULE D: HOUSEHOLD EXPENDITURES ON EDUCATION Now I would like to ask you about the expenses related to education. Please tell me how much your household spent on each of the listed items in last one day/month/year? (All children in the household) D1. Serial D2. Items D3. Amount (US Dollar) D4. Frequency of expense (see code) Codes [1] Daily [2] Monthly [3] Quarterly [4] Yearly [5] One time 1. Admissions / Registration 2. Materials and books 3. Extra Tuition fees 4. Travel to/from School 5. Food, snacks, water at school 6. Uniforms 7. Other expenses related to education Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION E1 – FOOD AND SMALL NON-FOOD EXPENDITURES How much did your household approximately spend on the following items in the last 30 days? *Record as 0 if none; if bought product for whole year: divide by 12 Items Approximate value in US Dollars (in cash) Not include your own product Approximate value in US Dollars (in credit) E1.a E1.b E1.c E1.1 Paddy / Rice E1.2 Other cereals & staples E1.3 Pulses/beans/nuts E1.4 Vegetables E1.5 Fruits E1.6 Meat, fish, eggs E1.7 Cooking oil E1.8 Other food items E1.9 Firewood /cooking fuel E1.10 Energy (e.g., battery, gas) E1.11 Cigarettes/Alcohol E1.12 Drinking water E1.13 Personal care (e.g. soap, toothpaste, razor, sanitary napkins, hair cut) E1.14 Communication (cell phone, phone card) E1.15 Total 84 Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION E2 – NON-FOOD EXPENDITURES How much did your household approximately spend on the following items in the last 6 months? Items Approximate value in US Dollars in cash) Not include your own product Approximate value in US Dollars (in credit) E2.a E2.b E2.c E2.1 Healthcare for adults and children 5 years or older E2.2 Healthcare for children less than 5 years old E2.3 Transportation (maintenance and repair, gasoline and diesel for own transportation, moving fee) E2.4 Clothing and footwear E2.5 Debt repayment E2.6 Sending remittances E2.7 House construction/maintenance including electricity & water E2.8 Shop/trade/commerce E2.9 Farming (seeds, fertilizers, labor costs…), Livestock breeding (vaccines, fodder…) E2.10 Fish breeding, fishing E2.11 Celebrations/social events/donation E2.13 Total E3.1 Have you ever encountered difficulties covering the expense? 1. Yes, usually 2. Yes, sometimes 3. No, never Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION F1 – REDUCED COPING STRATEGIES INDEX During the last 7 days, how many days did your household have to employ one of the following strategies to cope with a lack of food or money to buy it? (READ OUT EACH STRATEGY) Freq uenc y (# of days from 0 to 7) F1.1 Relied on less preferred, less expensive food | | F1.2 Borrowed food or relied on help from friends or relatives | | F1.3 Reduced the number of meals eaten per day | | F1 4 Reduced portion size of meals | | F1.5 Reduction in the quantities consumed by adults/mothers for young children | | 85 Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION F2 – LIVELIHOOD COPING STRATEGIES (SHOWCARD) F.2. In the past 30 days, have there been times when you did not have enough food or enough money to buy food? 1= Yes  CONTINUE 2= No  SKIP TO F3 IF YES: In the past 30 days, did you do any of the following things in order to get food or money to buy food? 1= Yes 2 = No F2.1 Sold household goods (e.g. radio, furniture, refrigerator, television, jewelry, clothes, utensils etc.) | | F2.2 Sold productive assets or means of transport (e.g. sewing machine, wheelbarrow, bicycle, ploughing tools, seeds etc.) | | F2.3 Reduced essential non-food expenditures such as education, healthcare, etc. | | F2.4 Spent savings | | F2.5 Borrowed money / food from a formal lender / bank or informal sources | | F2.6 Sold house or land | | F2.7 Withdrew children from school | | F2.8 Illegal income activities (e.g. theft, prostitution, etc.) | | F2.9 Sent an adult household member to seek work elsewhere (e.g. outside of the usual seasonal migration) | | F2.10 Begged | | Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION F3 – HOUSEHOLD HUNGER SCALE F3.1In the past 30 days, was there ever no food to eat of any kind in your house because of lack of resources to get food? 0 = No (Skip to F3.2) 1 = Yes F3.1a How often did this happen in the past 30 days? 1 = Rarely (1–2 times) 2 = Sometimes (3–10 times) 3 = Often (more than 10 times) F3.2 In the past 30 days, did you or any household member go to sleep at night hungry because there was not enough food? 0 = No (Skip to F3.3) 1 = Yes F3.2a How often did this happen in the past 30 days? 1 = Rarely (1–2 times) 2 = Sometimes (3–10 times) 3 = Often (more than10 times) F3.3 In the past 30 days, did you or any household member go a whole day and night without eating anything at all because there was not enough food? 0 = No (Skip to Section G) 1 = Yes F3.3a How often did this happen in the past 30 days? 1 = Rarely (1–2 times) 2 = Sometimes (3–10 times) 3 = Often (more than 10 times) 86 Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. MODULE G: BENEFITS OF SCHOOL FEEDING Benefits of the child G1. How many days in the last month did your child attend MORNING SESSIONS in school? Insert number of days (answer should be less than 30 days) IF 0 = SKIP TO G3 G2. Did your child receive a meal every day during each of these sessions? 1 = yes 2 = No G3. Does your child bring part of the food from 1 = Yes, always 2 = Most days, 3-4 days per week 3 = Sometimes, 1-2 days per week 4 = Rarely 5 = Never school to share with the other members of the household when he/she received SMP/HGSF￾Hybrid? (SHOWCARD) G4. Does school feeding benefit your child? (Record all mentioned, Showcard) 1 = Child gets food 2 = Child is more active/attentive 3 = Child is learning better 4 = Child is healthier 5 = Child has more opportunity 6 = Illness related absence is reduced 7 = No, it does not benefit my child 8 = Other (specify) G5. Does school feeding benefit the Households (Record all mentioned) (showcard) 1 = No benefit 2 = School feeding saves time for parents 3 = School feeding saves food/money for household 4 = School feeding saves time for household skipping morning cooking 5 = Other (specify) Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. SECTION H: OTHER EFFECTS OF SCHOOL FEEDING H01.When your children go to school, does anyone in your household save time? 1. Yes 2.No (Skip to H02) H01a.If yes, who? 1. Men 2. Women; 3. Both H01b. If yes, from which activity? (More than 1 answer) 1. Preparing food; 2. Taking care of children 3. Both; 4. Other (specify) H01c. If yes, how much time do you save? (Hours per day) H01d. If yes, how do you use this time? (You can choose maximum 2 activities) 1 = Household chores 2 = Rest/Leisure 3 = Income-earning activity 4 = Farm/livestock work 5 = Child care 6 = Other (specify) 1st 2nd H02.When your children attend school, is it time consuming for anyone in your household? 1. Yes 2.No (Skip to Module I) H02a.If yes, who? 1. Men; 2. Women; 3. Both H02b. If yes, what other activities must be done? (choose 2 activities) 1. Taking the child to school 2. Helping the child with the home work 1st 2nd 87 3. Meeting with the teachers/school staff 4. Preparing school material (books/clothes) 5. Doing tasks that are usually done by the child 6. Others (specify) H02c.If yes, how much time do you consume? (Hours per day) Respondent will be HHH (or adult male or female). Please write the ID code of the respondent. MODULE I: PARENTS/COMMUNITY PARTICIPATION IN THE SCHOOL FEEDING PROGRAM I1. Are you a member of the PTA (Parent Teacher Association)? 1 = yes 2 = No I2. Are you a member of the School Feeding Committee (SFC)? 1 = yes 2 = No I3. Are you a member of the School Management Committee (SMC)? 1 = yes 2 = No I4. Are you a member of the Mothers’ Committee (MC)? 1 = yes 2 = No I5. How many times did you visit the school during this school year 2018-2019 (excluding bringing the child to school)? # of times Part-2: Respondent will be mother or main female of the household. Respondent will be mother or main female of the household. Please write the ID code of the respondent. PART 2: RESPONDENT: The main female member of the household or an adult who was involved in the household food preparation and present and ate food together with the household members in the past week (starting from yesterday) SECTION K1 – FOOD CONSUMPTION PART 1(Yesterday) K1.1 K1.2 K1.3 # of meal Frequency This # compared to usual time over the last 6 months Quantity eaten per meal compared to usual time over the last 6 months Codes for K1.2 & K1.3: 1= Less 2= Same 3= more 01. How many meals were eaten by adults (aged >=15) living in your household yesterday? (e.g. breakfast, lunch, dinner, supper) 02. How many meals were eaten by children (aged less than 15) living in your household yesterday? 88 Respondent will be mother or main female of the household. Please write the ID code of the respondent. SECTION K2 – FOOD CONSUMPTION PART 2 Please tell me how many days in the past week (beginning from yesterday) your household has eaten the following foods and what was the source of these foods (includes meals prepared or bought by household and food taken away from home by all or most of household members such as having food at restaurant, wedding party etc. and NOT including school meals eaten by the child at school). Record “0” for items not eaten over the last 7 days. Record “99” for second source if only one source. NB: If less than 15g of fish or meat shared by household, record as Condiments codes for K2.2 & K2.3 1= Own production 6= Exchange of items forfood 2= Fishing, hunting, 7= Received as gift gathering 3= Purchase 8= Food aid as part ofthe SMP/HGSF-Hybrid /THR 4= Borrowed 9= Other (specify) 5= Exchange of labour for food K2.0. Food items K2.1. # of days eaten over the last 7 days K2.2. Main source (Majority of the source) K2.3. Second source 01. Rice Maximum 7 02. Maize 03. Bread 04. Cassava 05. Sweet potato, potato, yam 06. Bean, groundnut, other pulse 07. Fish 08. Other aquatic animal (frog…) 09. Meat (beef, pork, chicken) 09.a. Organ meat or blood￾based foods (Liver, kidney, heart or other organ meats or blood-based foods) 10. Wild meat 11. Egg 12. Vegetables (any vegetable incl. leafy) 12.a. Green leafy vegetable (Dark green/leafy vegetables, including wild ones + locally available leaves such as amaranth, cassava leaves, broccoli, kale, spinach etc.), 12.b. Orange vegetable (carrot, pumpkin, etc.) 13. Fruit 13.a Orange fruits (e.g. ripe mangoes, cantaloupe, ripe papaya, other orange or yellow fruits) 14. Sugar/sweet 15. Vegetable oil, animal fat 16. Milk product 16.a. Condensed milk 17.Prahok 18. Condiments or seasoning 89 Respondent will be mother or main female of the household. Please write the ID code of the respondent. SECTION K3: DIETARY DIVERSITY For Control Group only K3.1. Do your children have breakfast every day? 1. Yes 2. No K3.1a. If yes, what does [SAMPLED CHILD] usually eat for breakfast? SHOWCARD 1=rice; 2 = bread 3= fresh fish; 4=dried fish 5=meat; 6=vegetables 7=porridge; 8=noodles 9 = sweet desserts; 10 = others (specify) (allow for multiple) For Treatment Group only (SMP or SMP+THR or HGSF-Hybrid or HGSF Hybrid + THR) K3.2. During June this year, did SAMPLED CHILD] have breakfast when it was not offered at the school? 1. Yes 2. No K3.2a. If yes, SHOWCARD what did SAMPLED CHILD] eat? 1=rice; 2 = bread 3= fresh fish; 4=dried fish 5=meat; 6=vegetables 7=porridge; 8=noodles 9 = sweet desserts; 10 = others (specify) (allow for multiple) Respondent will be mother or main female of the household. Please write the ID code of the respondent. Benefit Received (for treatment group only, and if you are asking control group please skip to section L) (Complete below table if respondent receive benefit from WFP) – CASE ONLY [Respondent: Head of the household or mother of the child who was picked in school (1st￾6thgrade) through the random sampling process] K3.3. How many years have you received school meals or THRs? (#of years) (0 if none) K3.3a. SMP/HGSF-Hybrid K3.3b. THR K3.3c. How many times did you receive THR from November 2018 to June 2019? .............................times [Only if they receive THR in school year 2018-2019] How many household members receive THR? (#) Quantity of food received each time (taking into account THR only) (in kg) Who do you share your THR with? (See Code below) Do you sell any of your THR? (See Code below) If you sell your THR, what do you use that money for? (See Code below) Do you use iodized salt for your family? 1. Yes 2. No If yes, do you have it now? 1. Yes 2. No Do you use vitamin A fortified oil for your family? 1. Yes 2. No If yes, do you have it now? 1. Yes 2. No Rice Oil Beans K3.4 K3.5a K3.5b K3.5c K3.6 K3.7 K3.8 K3.9 K3.10 K3.11 K3.12 90 Code of K3.6. Who do you share your THR? Code of K3.7. Do you sell any of your THR? Code of K3.8. If you sell your THR, what do you use that money for? 1. Other household members 2. Family outside the household 3. Friends/neighbours/other 4. I don’t share my THR 1. Yes, usually 2. Yes, sometimes 3. No, never 1. To buy food 5. To invest in a 2. To buy other productive activity household/personal 6. To meet education items expenditures 3. To buy clothes 7. Other (specify) 4. To meet health related expenditures Respondent will be HHH or mother. Please write the ID code of the respondent. SECTION L: CHILD HEALTH Respondent: head of household, or mother, referring to child who was picked in school by random sampling process We would now like to know health status and illnesses ID code (Copy from SECTION A) Age (in years) Has the child been dewormed in the past 6 months? 1. Yes, at school 2. Yes, at the health center 3. Yes, bought it 4. No Has the child suffered from any illness in the last 2 weeks? 99. No illness 1. Diarrhea 2. Vomiting 3. Fever 4. Cough 5. Measles 6. Other (specify) How many days in the last two weeks has the childbeen unable to attend school due to illness? (number of days) L01 L02 L03 L04 L05 Respondent will be mother of the child. Please write the ID code of the respondent. [The following questions are to be answered by Mother of the child selected through the sampling process or head of household] L1. Have you ever heard of night blindness (local term)? 1.Yes 2. No L2. Does any of your children have difficulty seeing in dim light (For instance at dusk or in a dark place)? 1.Yes 2. No L3. Have you ever received a vitamin A capsule for your child? 1.Yes 2. No (Skip to L5) L4. If yes, how many months ago did your child take the last capsule? 1. Less than 6 months 2. More than 6 months 98. Don’t know L5. What kind of toilet facility does your household use? (showcard) 1. Flush latrine/toilet with water 2. Traditional pit latrine 3. Partly open pit (no roof or wall) 4. River/pond side 5. Bush/open field 6. Other (specify) L6. What is the household’s main source of drinking water? (showcard) 1. Pond/river/canal 2. Open ring well 3. Closed ring well 4. Open spring 5. Hand pump 6. Tapped water 7. Rain water 8. Bought water 91 9. Hand dug (no ring) 10. Other (specify) L7. Can you name three important health or hygiene practices? (don’t read the answer) 1.Bathing 2. Hand washing 3. Wound care 4.Clean toilets 5. Clean water 6. Clean food before cooking 7. Eating healthy food 8. Don’t know 9. Others (specify) 1. 2. 3. L8. Can you tell me your main source of information on good health practices? (don’t read the answer) (choose only one answer) 1. Poster 2. TV 3. Radio 4. Health center 5. School 6. Don’t know 7. Other (specify) Respondent will be mother or main female of the household. Please write the ID code of the respondent. MODULE M: MORBIDITY (Questions to be asked about members aged 6-11 years OR anyone in Grades 1-6: M1. M2. First Name M3. Has M4. M5. How many M6. How Illness Code: 1= Fever 2= Cough or colds 3= difficulty in breathing 4= Diarrhoea 5= Fever with chills like malaria 6=Worm 7= Skin infections 8=Stomach-ache 9= Measles 10. Others (Specify) ID (Please bring all [NAME] What days in the last many days members from suffered from did 1 month has [NAME] Module A any illness in [NAME] [NAME] been unable between 6 and 11 the last 1 suffer suffered from to go to yrs old or in Grades 1-6) month? from? this illness? school? Part-3: Respondent will be the school going child (the sample child) of the household. Respondent will be the school going child. Please write the ID code of the respondent. SECTION N3 – FOOD CONSUMPTION PART 3 (DIETARY DIVERSITY) Please tell us the number of meals that you ate yesterday during day and night and even you eat outside. Please start with morning meal. N3.1. Please, insert day of week for yesterday (see codesbelow): 1- Monday 2- Tuesday 3- Wednesday 4- Thursday 5- Friday 6- Saturday 7- Sunday Was the food they ate part of SMP/HGSF-Hybrid or THR? N3.2. Id code of child (from SECTIONA01): Source N3.3a. Breakfast 1. Yes 2. No N3.3b. Snack 1. Yes 2. No N3.3c. Lunch 1. Yes 2. No N3.3d. Snack 1. Yes 2. No N3.3e. Dinner 1. Yes 2. No N3.2a. Did the child eat this yesterday? (If No, Skip N3.2.1 & N3.2.2) N3.2.1. it was not part of SMP/HGSF-Hybrid or THR N3.2.2. It was part of SMP/HGSF-Hybrid or THR 92 After finishing answering the above question, please fill info about group of food depending on the above answer. For group of food that is not mentioned please ask the question: N3.4 Did the child eat this kind of food yesterday? Code 1. Yes (it was not part of SMP/HGSF-Hybrid or THR) 2. Yes (it was part of SMP/HGSF-Hybrid or THR) 3. Yes (Both SMP/HGSF-Hybrid and THR) 4. Yes (SMP/HGSF-Hybrid/THR and Family food) 5. No 98. Don’t know Food Group Description N3.4 N3.4.1. Cereals Corn/maize, rice, wheat or any other grains or foods made from these (e.g., bread, noodles, porridge or other grain products or pastes or other locally available grains) N3.4.2. Vitamin A rich vegetables and tubers Pumpkin, carrots or sweet potatoes that are orange inside + other locally available vitamin-A rich vegetables, yellow vegetable N3.4.3. White tubers and roots White potatoes, white sweet potatoes, taro, white yams, white cassava, or other foods made from roots N3.4.4. Dark green leafy vegetable Dark green/leafy vegetables, including wild ones + locally available vitamin-A rich leaves such as amaranth, cassava leaves, kale, spinach etc. N3.4.5. Other vegetable Other vegetables (e.g. tomato, onion, eggplant...), including wild N3.4.6. Vitamin A rich fruits Ripe mangoes, cantaloupe, ripe papaya, other orange or yellow N3.4.7. Other fruits f O ru th it e s r fruits, bananas, including wild fruits N3.4.8. Organ meat (ironic) Liver, kidney, heart or other organ meats or blood-based foods N3.4.9. Flesh meats Beef, pork, lamb, goat, rabbit, dog, wild game, chicken, duck, or N3.4.10. Eggs Chicken, duck, guinea hen or any other egg N3.4.11. Fish Fresh or dried fish or smoke fish, seafood or shellfish, snail, frog N3.4.12. Legumes, nuts, and Beans, peas, lentils, tofu, nuts, seeds or foods made from these s N e 3 e . d 4 s .13. Milk and milk Milk, cheese, yogurt or other milk products N3.4.14. Oils and fats Oil, fats or butter added to food or used for cooking N3.4.15. Sweets Sugar, honey, sweetened condensed milk, sweetened soda or sugary foods such as chocolates, candies, cookies and cakes N3.4.16. Spices, condiments, beverages Spices (black pepper, salt, chili powder), condiments (soy sauce, hot sauce, prahok), coffee, tea, alcoholic beverages OR local product: sour palm juice Respondent will be the school going child. Please write the ID code of the respondent. Benefit Received (for treatment group only) [Only if they receive SMP/HGSF-Hybrid] Answer N3.5. Do you eat all your school meal every school day (morning sessions)? (The respondent is the child that eats hot meal at school) 1. Yes 2. No. It’s not available every day during morning sessions 3. No, it’s not offered to me 4. No, I don’t like it 5. No, I don’t have time to eat 6. No, I’m not hungry 7. No, I like to take some of it home to my family 8. Other (specify)………………… N3.6. How often do you bring home your school meal (not THR) to your family) when you receive it? 1. Everyday 2. 3-4 days a week 3. 1-2 days a week 4. Rarely 5. Never 93 Annex 24: Key informant guides • Interviews with Province and District Level Government Officials Priorschool visits, in each province it will be necessary to requesttwomeetings, one with the Province Education administration, the other with the District Education Office. Province Education Office • The meeting at province level is likely to be more a courtesy visit than a substantial meeting. It can nevertheless be an opportunity for ET members to ask questions about: • The role of the province administration within the MGD FFE process; • An appreciation about the partnership between the province administration and WFP regarding MDG FFE activities; • An appraisal of the programme outcomes; • Evidences in support to this assessment. District Education Office • The head of district office staff, the district officer in charge of primary education, one district office staff in charge of school feeding activities (focal point) should participate in the meeting, which should be organized according to two sequences: • One presentation by district officers of the general educational context of the district, and challenges, with an emphasis on primary education. • One Q&A session. The following questions are provided as a framework for the Q&A session: 1. What is the total primary school age population in the district (gender disaggregated)? 2. What is the total number of children enrolled at school in the district for the present school year (gender disaggregated)? 3. How many primary schools, private and public, are there in the district? 4. How many primary schools benefit from a school feeding programme? 5. According to you, is there a need for school feeding in this district? If yes, for whatreasons? 6. According to you, what is the overall goal of the MGD FFE programme? Is it targeted at education, health and nutrition, social/family support? 7. Isthere a body/system at district level where partners(representative of diverse district offices, NGOs, WFP Field Office, other UN agencies, private sector, etc.) discuss education issues, school feeding included? 8. When did the MGD FFE programme start in the district? Is it still ongoing? 9. What are/have been programme modalities? Can you provide operational details about programme implementation? 10. Have you been involved in the design of the MDG FFE programme prior to its launching in thedistrict? 11. Did you benefit from a specific SF training? How long was it, and given bywhom? 12. What is your exact role/duty within the MGD FFE process? 13. Are you in charge of reporting on SF activity? If yes, what are the topics covered by your reports, the rate of submission of reports, and to whom do you report? 14. Are you facing difficulties to complete your duty regarding SF? 15. Do you think you would need some refresher training? If yes, on whattopics? 16. According to you, to what extent is the MGD FFE programme successful? What is the major benefit for the children? 17. Can you provide objective evidences in support of your assessment? 18. Is there a need for improvement? What would you suggest? In addition to the meeting with district education office staff, the ET will collect data from the 3 (4 whenever possible) previous school years showing enrolment, attendance, drop-out, per primary school grade, disaggregated by gender. Where possible, it will be wise to send a message to the District Education Officer prior to the school visit so that he/she has the time to collect the data and is ready to provide them upon ET arrival. Question 94 Question Annex 25: Focus Group Discussion Guides • Interviews with Parents The questionnaire below presents a set of questions that could be asked at school level during focus group discussions with representatives of the schoolchildren, PTA and individual parents. Based on the SABER approach16 and although covering a broader scope, these questions will serve the main purpose of assessing the community participation and ownership in schools where the MGD FFE programme is implemented. To prevent possible bias, it will be made clear since the beginning that the head teacher and teachers do not attend the meeting with parents. The interview format will follow a standard introduction of the team and explanation of the evaluation purpose. The team’s independence, neutrality and confidentially of responses will be noted, as well as the approximate time (40-50 minutes) of the meeting/interview. 1. When did school feeding activities start in thisschool? 2. According to you, why is there school feeding in thisschool? 3. Have you been consulted before the launching of the SF activity in the school? Bywhom? 4. Have you been involved in deciding which commodities are provided? 5. Is there a LSFC in the school? 6. Who participate in the LSFC? 7. How many women are members of the LSFC? 8. What kind of activities do LSFC members must conduct each day? Please describe a typical SFday. 9. When and by whom is the food delivered to the school? Has it always been done in duetime? 10. How does the transporter know the exact quantity of food he should deliver to theschool? 11. Who does the unloading of the truck? 12. Are representatives of the LSFC present during truck unloading? 13. Who controls the quantity of food delivered and signs the waybill? 14. Where is the food stored? 15. Who is responsible for food storage? 16. Is there a book to keep record of food storage and daily distribution? Who is responsible forit? 17. Do parents contribute some additional food in-kind or cash resources for the food of thechildren? 18. What other contribution do parents make to the SF activity, if any? 19. According to you, to what extent is the programme successful? What is the major benefit for the children and their family? 20. Is there a need for improvement? What would you suggest? 16 Systems Approach for Better Education Results 95 Question • Interviews with Teachers The meeting with parents will be followed by a separate meeting with the head teacher and teachers that will focus on the following main areas: • Role and responsibility of the head teacher and teachers in the day-to-day SF operation within the school; • Record keeping and reporting; • Qualitative and quantitative assessment of SF outcomes and impact. 1. How long have you been director/head teacher in thisschool? 2. According to you, why is there school feeding in thisschool? 3. Who (which institution) presented the school feeding programme to you? 4. Have you been involved in the design of the SF activity? 5. Did you benefit from a SF training? How long was it, and given bywhom? 6. What is your exact role/duty related to SF activity at the school? 7. Have you been provided with SF monitoring tools? If yes, by whom (which institution)? 8. Are you in charge of reporting on SF activity? If yes, what are the topics covered by your reports, the rate of submission of reports, and to whom do you report? 9. Are you facing difficulties to complete your duty regarding SF? 10. Do you think you would need some refresher training? If yes, on whattopics? 11. According to you, to what extent is the SF programme successful? What is the major benefit for the children? 12. Is there a need for improvement? What would you suggest? 96 Annex 26: Completed Project Monitoring Plan By result Indicator Target Baseline Mid-term Endline % Achieved Strategic Objective 1: Improve literacy of school age children Improved literacy of school-aged children Number of individuals benefiting directly from USDA-funded interventions 226,200 0 187,800 279,039 123% Number of individuals benefiting indirectly from USDA-funded interventions 241,300 0 191,600 319,387 132% Percent of students who, by the end of two grades or primary schooling, demonstrate that they can read and understand the meaning of grade level text 75% N/A17 23.8% 30.5% 55.5% Improved literacy instructional materials Number of textbooks and other teaching and learning materials provided as a result of USDA assistance 163,404 50,018 154,540 159,148 97.4% Increased skills and knowledge of teachers Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance. 1,540 0 1,371 1.699 110% Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance 2,200 1,16719 1,900 3,212 146% Increased skills and knowledge of school administrators Number of school administrators in target schools who demonstrate use of new techniques or tools as a result of USDA assistance. 413 0 188 484 137.5% Number of school administrators and officials trained or certified as a result of USDA assistance 591 0 509 714 117% Increased access to food (school feeding) Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance 38,945,000 0 14,887,000 32,895,842 85.7% Number of individuals receiving take-home rations as a result of USDA assistance 22,500 0 15,004 28,855 128% Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance. 189,600 0 150,570 175,397 92.5% Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance 281,500 0 185,500 249,414 99.7% 17 A 16.1 percent score at baseline has been communicated to the ET. For methodological reason, this figure cannot be used for comparison. Nevertheless, this percentage provides an indication about the level of performance at the starting point of the programme. 18 As per World Education baseline figures (July 2017). 500 National Standard reading books have been provided. Since then (2018) the indicator has been revised to include all the components of the learning package. Hence the much larger numbers at MTR. 19 As per World Education figures as at July2017 97 By result Indicator Target Baseline Mid-term Endline % Achieved Improved student attendance Number of students regularly (80%) attending USDA supported schools 127,600 195,400 167,600 192,723 >100% each year Improved school infrastructure Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/ constructed as a result of USDA assistance 866 0 422 1,395 161% Strategic Objective 2: Improved use of health and dietary practices Increased knowledge of nutrition Number of individuals trained in children health and nutrition as a result of USDA assistance 3,600 0 2,370 4,026 112% Increased access to clean water and sanitation Number of schools using an improved water source 500 433 568 572 114% Number of schools with improved sanitary facilities 590 574 583 576 98% Foundational Results Improved policy and regulatory framework Number of child health and nutrition policies, regulations or administrative procedures in the following stages of development as a result of USDA assistance 1 1 School Health Policy at Stage 3 School Health Policy at Stage 5 100% Number of educational policies, regulations or administrative procedures in the following stages of development as a result of USDA assistance 4 0 ESP 2019- 2023: Stage 1 3 EGR standards developed ESP 2019-2023 Stage 5 (with SM/HGSF incorporated) 3 EGR standards developed 100% achieved Increased engagement or local organizations and community groups Number of Parent-Teacher Associations (PTAs) or similar school governance structures supported as a result of USDA assistance 590 590 585 590 100% 98 Annex 27: Additional findings on school feeding-related infrastructure • Kitchen facilities in USDA-supportedschools Total schools # with kitchen Endline All USDA supported schools 86 84 SMP only ^ 17 16 SMP + THR^^ 69 68 Comparison sc h o o l s 85 1 Baseline Total USDA supported schools 86 82 SMP only 57 54 SMP + THR 29 28 Comparison schools 86 0 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR Note: Only one comparison school has a kitchen. • Condition of kitchens in USDA-supported schools(%) • Use of energy-efficient stoves in USDA-supported schools Total # using E-E stove Endline All USDA supported schools 86 62 SMP only ^ 17 11 SMP + THR^^ 69 51 Comparison sc h o o l s 85 0 Baseline All USDA supported schools 86 52 SMP only 57 39 SMP + THR 29 13 Comparison sc h o o l s 86 0 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR 80 100 57 0 20 40 60 47.7 10.5 14 1.2 2.3 4.7 Good condition Flooded during rainy season No wall Using rocks asstove Leaking roofs Clean cooking and eating equipment Lacking kitchen utensils 99 • Condition of energy efficient stoves in USDA-supported schools (%) (n=62) • Condition of food storerooms in USDA-supported schools (%) (n = 58) No walls Leaking roofs Security guard at night time/ during schoolvacation Storeroom had ventilation Foods are stored in order Pallets for food storage Well cleaned Floor is dry Door is locked well 0 20 40 60 80 100 0 20 40 60 80 100 Good condition and function well 52.3 18.6 Broken, not functioning 1.2 Poor condition but still work 100 Annex 28: Additional findings on schoolgardens • Schools with vegetable gardens Total # of schools with garden % of schools with vegetablegarden % of rehabilitated vegetable garden (n=90) Endline All USDA supported schools 86 74 86 95 SMP only ^ 17 8 47 75 SMP + THR^^ 69 66 96 94 Comparison sc h o o l s 85 16 19 75 Baseline All USDA supported schools 86 73 85 90 SMP only 57 50 88 92 SMP + THR 29 23 79 87 Comparison sc h o o l s 86 14 16 57 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR • Source of vegetable seeds (%) (n=90) All USDA supported schools SMP only ^ SMP + THR ^^ Comparison sc h o o l s Purchased using PB 77 88 76 69 Purchased using School Improvement Grant 88 88 88 6 Donated by PLAN/World Vision 7 0 8 13 Donated by WFP 30 13 32 0 Dong Dukor Organization 1 0 2 0 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR • School garden maintenance (%) Comparison schools All USDA supported schools Children from grade to grade are assigned to each plot of land Children are mentored by trained teachers and community Prevent animals from entering school compound by repairing schools’ fence once per year 0 20 40 60 80 100 101 0 20 40 60 80 100 Comparison schools (n=73) All USDA supported schools (n=82) Other System to clean the wells Water is used for human consumption Platform is clean Functioning well year-round 0 20 40 60 80 100 Comparison schools (n=73) All USDA supported schools (n=82) Water is used for animals only Arsenic (poisonous) Functioning only during rainy season Other Hand pump/ rain water station was broken Annex 29: Additional findings on school water and sanitation-related infrastructure • Number and percent of functioning drilled wells/ watercatchments Total drilled wells Total water catchments % Functioning drilled wells % Functioning water catchments Endline All USDA supported schools 112 195 61 91 SMP only ^ 9 43 50 87 SMP + THR^^ 103 152 64 92 Comparison sc h o o l s 77 202 55 78 Baseline Total USDA supported schools 104 160 70 87 SMP only 66 109 75 87 SMP + THR 38 51 59 86 Comparison sc h o o l s 89 155 67 68 ^ including HGSF-Hybrid; ^^ including HGSF-Hybrid+ THR Note: Percent functioning drilled wells and percent functioning water catchments were based on schools with drilled wells/catchment stations • Condition of functioning drilled well/water catchments(%) Note: Other reasons = pipe is broken, hole at the bottom, rust in the water, no water flow out, ring is broken, well broken. • Condition of non-functioning drilled well/water catchments(%) 102 • School access to year-round clean water % of schools with year-round access to a clean water source Yes (whole school year) No (some months not available) No (no water) Endline All USDA-supported schools 66 17 16 SMP only ^ 65 6 65 SMP + THR^^ 67 20 13 Comparison sc h o o l s 68 13 19 Baseline All USDA-supported schools 64 13 23 SMP only 58 16 26 SMP + THR 76 7 17 Comparison sc h o o l s 43 10 47 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR • Schools with fixed hand and mobile hand washing stations(%). % of schools with hand washing station(N=171) % fixed handwashing (n=163) % mobile handwashing (n=163) Endline All USDA supported schools 99 53 47 SMP only ^ 94 66 34 SMP + THR^^ 100 51 50 Comparison sc h o o l s 92 58 42 Baseline All USDA supported schools 86 60 58 SMP only 82 56 51 SMP + THR 93 69 72 Comparison sc h o o l s 57 40 31 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR • Condition of fixed (non-mobile) hand washing stations (%) 0 20 40 60 80 100 Comparison schools (n=62) All USDA supported schools (n=68) Good condition & function well year-round Good condition & function well only during rainy season Broken, not functioning Poor condition but still work year-round 103 • Condition of mobile hand washing stations(%) • Functional and non-functional latrines in surveyed schools(%). % of functional latrines! % of non￾functional latrines! # of functional latrines per school! # non￾functional latrines per school! Endline All USDA supported schools 90 10 6*** 0.7 SMP only ^ 92 8 4** 0.4 SMP + THR^^ 90 10 6 0.8 Comparison sc h o o l s 87 13 4 0.7 Baseline All USDA supported schools 90 10 5.1 0.6 SMP only 91 9 4.8 0.5 SMP + THR 88 12 5.7 0.8 Comparison schools 80 20 3.6 0.9 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR !*** (t-test, two-tailed) p<=.001; ** p<=.01; * p<.05 (endline comparisons) • Presence of functioning latrines for teachers # Schools with separate functional latrine for teachers # of functioning latrines for teachers % of functional latrines for teachers! # of teachers per functional staff latrine!! Endline All USDA supported schools 49 80 32 4 SMP only ^ 10 19 43 3 SMP + THR^^ 39 61 29 5 Comparison sc h o o l s 51 79 38 6 Baseline All USDA supported schools 67 105 24 5 SMP only 41 61 23 6 SMP + THR 26 44 27 5 Comparison sc h o o l s 51 81 27 7 ^ including HGSF Hybrid ^^ including HGSF-Hybrid+ THR Comparison schools (n=42) Good condition & function well only during rainy season All USDA supported schools (n=58) Good condition & function well year-round Broken, not functioning 0 20 40 60 80 100 104 ! For schools with functional latrines for teachers. This number is calculated by dividing number of functional latrines for teachers by the total number of functional latrines in school. !! For schools with functional latrines for teachers. This is calculated by dividing total number of teachers in school by total number of functional latrines. • Presence of functioning latrines for boys and girls # Schools with separate functioning latrines for boys and girls % of non-separate latrines for students! # of boys per separate latrine for boys!! # of girls per separate latrine for girls!! Endline All USDA supported schools 79 9 68 59 SMP only ^ 15 13 71 63 SMP + THR^^ 64 8 68 58 Comparison sc h o o l s 48 44 102 95 Baseline All USDA supported schools 66 21 97 87 SMP only 42 23 97 90 SMP + THR 24 15 96 84 Comparison sc h o o l s 46 36 209 183 ^ including HGSF Hybrid ^^ including HGSF-Hybrid+ THR ! All schools. This number is calculated by dividing number of non-separated functional latrines for students by the total number of latrines for students (which is the total of functioning latrines for boys, functioning latrines for girls, and functional latrines non￾separated). !! Schools with separate functional latrines for boys and girls (n=127). This number is calculated by dividing total number of boys (girls) in school by the total number of functional latrines for boys (girls). • Presence of functioning latrines for disabled students # of schools with latrine for disabled students # latrines for disabled students! % latrines fordisabled students!! # disabled students per latrine for disabled students!!! Endline All USDA supported schools 23 28 10 1.5 SMP only ^ 3 3 5 2 SMP + THR^^ 20 25 11 1.4 Comparison sc h o o l s 9 14 3 0.8 Baseline All USDA supported schools 37 40 10 10 SMP only 24 25 10 10 SMP + THR 13 15 9 10 Comparison sc h o o l s 7 7 2 27 ^ including HGSF Hybrid ^^ including HGSF-Hybrid+ THR ! Schools with disabled latrine only (n=32). 105 0 10 20 30 40 50 60 70 80 90 100 Broken but still usable Dirty, not well maintained Clean and wellmaintained Comparison schools All USDA supported schools 0 20 40 60 80 100 Comparison schools All USDA supported schools Others Wash basins broken Pit latrines broken or full Commode broken Door broken 0 20 40 60 80 100 Comparison schools All USDA supported schools Wear shoes in the toilet Propose users to leave shoes outside of latrinestalls Ensure washbasin is full of water Lock latrines during school vacation Keep soap/hand washing facilities within or near thetoilets… Train students and take turn to clean latrines sometimes !! Among all schools (n=171). !!! Among schools with latrine for disabled students (n=32). Total number of disabled students divided by total number of latrines for disabled students. • Design of latrine facilities for students with disabilities(%) • Condition of functioning latrines (%) (n=171) • Condition of non-functioning latrines (%) (n=171) Note: Others –1. Have only some toilet bowls as the constructor left before completion, 2. New toilet only built, 3. Old toilets locked so that new ones are used, 4. Toilet collapsed. • Latrine maintenance (%) (n=171) 0 20 40 60 80 100 Handle Slope (ramp) Suitable size Comparison schools (n=9) All USDA supported schools (n=23) 106 Annex 30: Additional findings from school survey • Teaching staff in surveyed schools (SY 2016/17) # Total teachers (Grades 1-6) # of Contract teachers Total Female % Total Female % Endline All USDA supported schools 565 63 288 67 SMP only ^ 98 59 55 64 SMP + THR^^ 467 64 233 68 Comparison sc h o o l s 598 62 105 67 Baseline All USDA supported schools 516 62 220 66 SMP only 325 58 121 63 SMP + THR 191 68 99 70 Comparison sc h o o l s 532 63 47 60 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR • Gender and attendance of full-time teachers (Grades 1-6) & teacher attendance All USDA supported schools SMP only ^ SMP + THR ^^ Comparison schools Endline Percentage of male teachers (%) 39 46 37 40 Percentage of female teachers (%) 61 54 63 60 Number of school days in a year (mean) 220 218 220 217 Number of days teachers being absent (mean) 4.7 5.4 4.5 6.6 Percentage teacher attendance during official SY 2018/19 97.9 97.5 98.0 97.0 Baseline Percentage of male teachers (%) 38 42 32 37 Percentage of female teachers (%) 62 58 68 63 Number of school days in a year (mean) 213 213 214 210 Number of days teachers being absent (mean) 5.3 5.9 4.9 5.2 Percentage teacher attendance during official SY 2016/17 97.5 97.2 97.7 97.6 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR 107 • Full time teachers (Grades 1-6) # of Full-time teachers per School % of Schools with attendance sheet Endline All USDA supported schools 7.0 100 SMP only ^ 6.71 100 SMP + THR^^ 7.00 100 Comparison sc h o o l s 8.86 100 Baseline All USDA supported schools 7 100 SMP only 6 100 SMP + THR 7 100 Comparison sc h o o l s 7 100 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR 108 Annex 31: Additional benefits of school feeding • Household income and expenditure Source: Endline evaluation ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR • Household expenditure % of expenditure on food % of expenditure on education % of expenditure on non-food items % of expenditure paid in credit Endline All USDA supported schools 44.5** 8.4** 47.1*** 14.8*** SMP only ^ 43.2 7.9 48.9 14.6 SMP + THR^^ 44.8 8.5 46.7 14.8 Comparison sch o o l s 48.2 10.2 41.7 9.7 Baseline All USDA supported schools 39.5 7.8 52.7 18.9 SMP only 41.0 6.5 52.5 15.2 SMP + THR 38.5 9.7 51.8 21.2 Comparison schools 43.9 8.7 47.4 13.2 Source: Endline evaluation ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR ! t-test (two tailed) *** p<=.001; ** p<=.01; * p<.05 • Benefits of school feeding for the child, as perceived by parents of USDA-supportedchildren Benefits to the child Gets food Healthier Learns better More attentive in class Has more opportunities No benefit Endline SMP only ^ 74.0 62.0 47.0 21.0 39.0 0.0 SMP + THR^^ 87.2 70.3 60.1 56.3 44.4 0.2 Baseline SMP only 75.4 67.0 51.2 35.6 31.9 1.2 SMP+THR 88.0 71.3 51.7 33.1 31.7 0.8 Source: Endline evaluation ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR All USDA supported schools SMP only ^ SMP + THR^^ Comparison schools $20.00 $15.00 $10.00 $5.00 $0.00 Household income/day (US$) Household expenditure / day (US$) 109 • Benefits of school feeding for the household, as perceived by parents of USDA-supported children Benefits to the household Saves HH food/money Saves time for parents Saves morning cooking time No benefit Endline SMP only ^ 79.0 38.0 65.0 1.0 SMP + THR^^ 80.4 59.9 73.9 1.2 Baseline SMP only 78.9 69.2 51.8 4.0 SMP+THR 87.8 73.9 65.9 3.2 Source: Endline evaluation ^ including HGSF -Hybrid; ^^ including HGSF-Hybrid+ THR • Percentage of parents reporting the use of saved time for various activities Source: Endline evaluation NB. Households provided multiple answers, so the sum of all answers does not equal 100% ^ including HGSF -Hybrid; ^^ including HGSF-Hybrid+ THR All USDA supported schools SMP only ^ SMP + THR^^ Comparison schools 40.3 43.6 45 43.2 1.4 11 10.4 3.1 8.2 9.7 2 7 7 2.9 8 9.1 0.5 1.9 4 1.4 Household chores (%) Farm/ livestock work (%) Income- earning activity (%) Child care (%) Rest/ Leisure (%) 110 Annex 32: Additional findings from household survey • Demographic information of surveyed households (N=958) HH Size (Number) Dependency ratio (Number)# ##Household head marital status (%) Single Married Widowed Separated/ divorced All USDA supported schools 5.04 1.28 1.0 81.7 14.6 2.7 SMP only ^ 5.33 1.12 1.0 87.0 11.0 1.0 SMP + THR^^ 4.97 1.31 1.0 80.4 15.5 3.2 Comparison sc h o o l s 5.36 1.21 0.7 85.4 12.4 1.6 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR # Number of children below 18 years old plus elderly people of age 65 years and over divided by number of individuals between the age of 18-64. ## Of the total 958 cases, 89.8% reported themselves as household head, 5.4% husband/wife, 1.8% grandfather/mother, 1.7% father/mother and rest 1.3% reported their relationship with the household head as father/mother in law, brother/sister, uncle/aunt and other relatives. • Education and literacy information of heads of household of surveyed households (N=958) Highest class completed (%) Literacy (%) Never Primary Secondary High school or above Can read and write Can sign only Can read only Cannot read or write All USDA supported schools 33.9!*** 48.4 11.1 6.6 52.5!*** 6.2 4.7 36.6 SMP only ^ 33.0 48.0 13.0 6.0 57.0 7.0 5.0 31.0 SMP + THR^^ 34.1 48.6 10.6 6.8 51.4 6.0 4.6 37.9 Comparison schools 19.8 59.0 15.5 5.6 64.6 3.4 3.4 28.6 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR ! Chi-Square test of association between literacy and USDA-supported and comparison schools *** p<=.001; ** p<=.01; * p<.05 • Number of school-aged children (6 - 11 years) per household, and their school starting age (N=958) Number children (6-11 years) in household! Average age of starting school! All USDA supported schools 1.18 6.0 SMP only ^ 1.22 6.0 SMP + THR^^ 1.17 6.0 Comparison sc h o o l s 1.13 6.1 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR !*** (t-test, two-tailed) p<=.001; ** p<=.01; * p<.05 111 • Household main source of water – USDA supported households only (n=514) • Household toilet facilities – USDA supported households only (n=514) • Household's main source of livelihood USDA supported schools (%) SMP Only^ SMP + THR^^ Comparison (%) 0 20 40 60 80 100 Fishing/ activities Tailor/potter/blacksmith NGO worker Doctor/engineer/lawyer Business/shop Housewife Pastoralism Non-agricultural labour Private sector employee Other salaried worker Petty trader Contractor Farming activities Agricultural labour (daily wage) Government officer Driver Teacher Medium/large scale trader No occupation All USDA supported schools SMP only (& HGSF – HYBRID) SMP + THR (& HGSF-HYBRID+ THR) Bush/open field/river/ pond site (%) Traditional pit toilet/ partly open pit(%) Flush toilet/ latrine with water (%) 0% 20% 40% 60% 80% 100% 0 20 40 60 80 100 All USDA supported schools 49.4 50.6 SMP only (& HGSF – HYBRID) 37 63.1 SMP + THR (& HGSF-HYBRID+ THR) 52.4 47.6 Improved (%) Unimproved (%) 112 • Household's secondary source of livelihood • Average number of meals consumed in the household the day before the survey by age group All USDA supported schools SMP only^ SMP + THR ^^ Comparison schools ENDLINE Adults 2.67 2.73 2.66 2.64 Children 2.88* 2.92 2.87 2.81 BASELINE Adults 2.63 2.68 2.59 2.65 Children 2.62 2.38 2.80 2.77 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR #*** (t-test, two-tailed) p<=.001; ** p<=.01; * p<.05 • Percentage of households reporting consumption of each food group in last seven days USDA supported schools (%) SMP Only^ SMP + THR^^ Comparison (%) 100 90 80 70 60 50 40 30 20 10 0 Fishing/ activities Tailor/potter/blacksmith NGO worker Midwife/nurse Brewing Medium/large scale trader No secondary occupation Farming activities Pastoralism Agricultural labour (daily wage) Non-agriculturallabour Government officer Private sector employee Driver Other salaried worker Food processing Charcoal production Petty trader Business/shop Contractor Housewife 113 • Reduced Coping StrategiesIndex All USDA supported schools (Mean)! SMP only ^ SMP + THR^^Comparison schools (Mean) ENDLINE 9.74 7.62 10.26 9.02 BASELINE 8.95 6.98 10.23 10.31 ^ including HGSF Hybrid; ^^ including HGSF-Hybrid+ THR !(t-test, two-tailed)*** p<=.001; ** p<=.01; * p<.05 A t-student test was applied to Reduced CSI between USDA supported schools and comparison schools to test the hypotheses that th e a v e ra g e RC S I is equal across samples. Note: Reduced CSI is calculated so that the lower the number, the better the household is coping. CSI=0 when the HH does not go through any difficulties • Percentage of parents reporting to save time from each activity, as an effect of school attendance Preparing food (%) Taking care of children (%) Both (%) ENDLINE (n=958) All USDA supported schools 26 2 34 SMP only ^ 24 2 37 SMP + THR^^ 27 2 33 Comparison schools 11 14 34 BASELINE All USDA supported schools 29 23 15 SMP only 26 18 27 SMP+THR 32 26 8 Comparison schools 20 29 12 ^ including HGSF Hybrid ^^ including HGSF-Hybrid+ THR 0 20 40 60 80 100 Milk and milk products Cassava, sweet potato, potato,tubers Fruits Beans, peas, ground nuts, cashews Sugar and sugar products Oils, fat Vegetables, green leaves Tea, spices, condiments Beef, pork, poultry, eggs,fish Maize, porridge, rice, pasta,bread Comparison schools All USDA supported schools