Decentralized evaluation for evidence-based decision making USDA Mc-GOVERN DOLE FY17 BASELINE STUDY IN BANGLADESH Inception Report of Baseline Evaluation [FY 14-16] Inception Report of End-line Evaluation May 2018 WFP Country Office Bangladesh Mr.Ezas Nabi, WFP Bangladesh Evaluation Manager Prepared by DECENTRALIZED EVALUATION FOR EVIDENCE-BASED DECISION MAKING End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 2020-2021 Final Report March, 2021 World Food Programme Evaluation Manager: Geophrey Sekei Prepared by Team Leader: Rahul Agrawal Gender Specialist: Mrinalini Mazumdar Research Manager: Aditi Chordia Decentralized Evaluation Acknowledgements The NRMC evaluation team wishes to acknowledge the guidance, support, and cooperation received from all the participants in the evaluation. NRMC takes this opportunity to extend sincere thanks to the distinguished Government officials from Ministry of Primary and Mass Education, Directorate of Primary Education at National and District and Upazila level, Bangladesh National Nutrition Council (BNNC), Ministry of Health and Family Welfare and Department of Agricultural Extension, Ministry of Agriculture for their time and precious inputs. The NRMC Evaluation Team expresses its gratitude to Mr. Rezaul Karim, Mr. Ezaz Nabi, Mr. Geophrey Sekei, Ms. Allen Amanya Mr. Antonio Battista, Ms. Katelyn Runyan-Gless, Ms. Farzana Akter, Mr. Md. Abdullah-Al-Mamun Patwary, Ms. Sneha Lata and Mr. George Suman Karmaker for their valuable suggestions and guidance for the evaluation. We would also like to thank the staff of Room to Read, and RIC who took time out to speak to us and provide their views on the school feeding programme and facilitate the logistics during the survey. We are thankful to the team from Data Management Aid for their partnership with the NRMC team during the evaluation, particularly on data collection. Last but not the least, the evaluation team wishes to acknowledge the cooperation received from all informants, including school head teachers, teachers, parents, cooks, storekeepers, and SMC members, during the primary survey. Disclaimer The opinions expressed in this report are those of the Evaluation Team, and do not necessarily reflect those of the World Food Programme. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. Decentralized Evaluation Table of Contents LIST OF TABLES............................................................................................................................................................................II LIST OF FIGURES ..........................................................................................................................................................................II EXECUTIVE SUMMARY.................................................................................................................................................................I 1. INTRODUCTION...................................................................................................................................................................1 1.1. EVALUATION OVERVIEW .............................................................................................................................1 1.2. OVERVIEW OF THE EVALUATION SUBJECT.................................................................................................1 1.3. CONTEXT.......................................................................................................................................................3 1.3.1 Food Security and Nutrition................................................................................................................................................. 3 1.3.2 Health............................................................................................................................................................................................ 4 1.3.3 Education..................................................................................................................................................................................... 4 1.3.4 Water, Sanitation and Hygiene ........................................................................................................................................... 5 1.3.5 School Feeding Needs.............................................................................................................................................................. 5 1.3.6 Gender Analysis ......................................................................................................................................................................... 6 1.3.7 Government Policies and Programmes............................................................................................................................ 6 1.3.8 Development Assistance in Bangladesh........................................................................................................................... 7 1.3.9 COVID-19 Situation in Cox’s Bazar .................................................................................................................................... 9 2. EVALUATION METHODOLOGY AND LIMITATIONS..............................................................................................11 2.1 Approach and Methodology ........................................................................................................................................................11 2.1.1 Data Triangulation .....................................................................................................................................................................14 2.1.2 Secondary Data.............................................................................................................................................................................15 2.1.3 Primary Data .................................................................................................................................................................................15 2.1.4 Site Mapping ..................................................................................................................................................................................16 2.1.5 Sampling..........................................................................................................................................................................................17 2.1.6 Data Analysis.................................................................................................................................................................................18 2.2 Gender dimensions of the end-term evaluation...................................................................................................................18 2.3 Limitations and Risks.....................................................................................................................................................................19 2.4 Ensuring Quality and Adherence to Ethical Standards....................................................................................................20 3. EVALUATION FINDINGS.................................................................................................................................................20 3.1. RELEVANCE.................................................................................................................................................................21 3.2. EFFECTIVENESS..........................................................................................................................................................24 3.3. EFFICIENCY .................................................................................................................................................................37 3.4. IMPACT ........................................................................................................................................................................39 3.5. SUSTAINABILITY.........................................................................................................................................................41 3.6. IMPACT OF COVID-19 ON THE PROGRAMME.......................................................................................................43 4. CONCLUSIONS....................................................................................................................................................................44 5. RECOMMENDATIONS......................................................................................................................................................47 ANNEX A: MAP OF THE INTERVENTION AREA...............................................................................................................51 ANNEX B: PROJECT-LEVEL RESULTS...................................................................................................................................53 ANNEX C: PLANNED OUTCOMES OF WFP BANGLADESH MCGOVERN DOLE -FY17 AWARD ...........................56 ANNEX D: ACTIVITY WISE GRADUATION TIMELINE.....................................................................................................63 ANNEX E: MCGOVERN DOLE TARGET BENEFICIARIES AND FUNDING FOR WFP SCHOOL FEEDING PROGRAMME...............................................................................................................................................................................65 ANNEX F: ROLE OF PARTNERS..............................................................................................................................................66 ANNEX G: STAKEHOLDER ANALYSIS ..................................................................................................................................68 ANNEX H: SUMMARY OF THE METHODOLOGY PRESENTING THE METHOD OF DATA COLLECTION, THE TARGET GROUPS/SOURCES OF INFORMATION, THE NATURE OF THE INTERVIEWS, AND THE INDICATIVE OUTPUTS .............................................................................................................................................................72 ANNEX I: CONCEPTUAL FRAMEWORK FOR THE END-TERM EVALUATION..........................................................73 ANNEX J: SAMPLING PROTOCOL ..........................................................................................................................................75 ANNEX K: LIST OF SAMPLE SCHOOLS .................................................................................................................................76 ANNEX L: EVALUATION MATRIX..........................................................................................................................................78 ANNEX M: DATA COLLECTION TOOLS................................................................................................................................88 ANNEX N: EVALUATION MISSION SCHEDULE ................................................................................................................135 ANNEX O: DOCUMENTS GATHERED..................................................................................................................................137 ANNEX P: ETHICAL CONSIDERATIONS.............................................................................................................................141 ANNEX Q: NRMC’S INTERNAL PROTOCOLS.....................................................................................................................142 ANNEX R: TEAM COMPOSITION AND SPECIFIC TASKS...............................................................................................143 ANNEX S: THE TRAINING SCHEDULE OF THE DATA COLLECTION TEAM............................................................144 ANNEX T: END-TERM VALUES OF KEY INDICATORS...................................................................................................145 ANNEX U: TERMS OF REFERENCE FOR END-TERM EVALUATION..........................................................................177 ANNEX V: TERMS OF REFERENCE FOR EVALUATION REFERENCE GROUP.........................................................185 ANNEX W: BIBLIOGRAPHY...................................................................................................................................................188 ANNEX X: LIST OF ACRONYMS.............................................................................................................................................190 List of Tables Table 1: MGD FY17 end-term evaluation questions .........................................................................................................................12 Table 2: Sampling .............................................................................................................................................................................................17 Table 3: Enrolment in sample intervention and comparison schools ......................................................................................29 Table 4: Parents trained by WFP who are aware and demonstrate handwashing practices at critical times........32 Table 5: Planned outcomes and annual targets ..................................................................................................................................56 Table 6: Activity wise Graduation Timelines .......................................................................................................................................63 Table 7: Role of partners ...............................................................................................................................................................................66 Table 8: Stakeholder analysis......................................................................................................................................................................68 Table 9: Summary of the Evaluation Methodology ...........................................................................................................................72 Table 10: Evaluation matrix.........................................................................................................................................................................78 Table 11: Mapping of the indicators with the type of tools and the questions .....................................................................88 Table 12: Evaluation mission schedule................................................................................................................................................135 Table 13: Documents Gathered and Reviewed ................................................................................................................................137 Table 14: Team Composition and Responsibilities ........................................................................................................................143 Table 15: Training Schedule .....................................................................................................................................................................144 List of Figures Figure 1: Perception of teachers on impact on students’ attendance if provision of biscuits stopped......................31 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 i Executive Summary Introduction 1. The end-term evaluation (ETE) of the School Feeding Programme (SFP), being implemented during the period January 2018-June 2021 in Cox’s Bazar district, supported by McGovern-Dole International Food for Education and Child Nutrition Grant through the US Department of Agriculture (USDA) has been commissioned by WFP Country Office, Bangladesh (WFP-CO). The evaluation considers the evaluation questions specifically for the end-term of FY17 award and was conducted during the period June 2020– March 2021, wherein the data collection was done during the period, 20th December 2020 to 7th January 2021. 2. The USDA McGovern-Dole FY-17 project was expected to benefit 47,689 schoolchildren of Grades I-V in 146 schools by providing approximately 9.3 million micro-nutrient fortified biscuits per year and supporting complimentary education interventions in two upazilas (Ukhiya and Kutubdia) of Cox’s Bazar. The project was also expected to aid the formulation and operationalization of the first National School Meal Policy (NSMP) and help mainstream Government of Bangladesh’s National School Feeding in Poverty Prone Areas Programme (NSFPPA) into the revenue budget. 3. The ETE serves several critical purposes intended for accountability and learning. The aim of the evaluation is to assess the achievement on project results, explore the strength of the exit strategy with a focus on achieving program sustainability and compare the end-term values with the baseline and mid-term values, through the lens of relevance, efficiency, impact, effectiveness and sustainability. For the end-term evaluation, while equal weightages on each of the OECD-DAC criteria have been given, the focus has been on impact and sustainability. The evaluation also strives to underline the key lessons learnt that can inform strategic and operational decisions for future programmes. 4. There are various stakeholders that will be interested in the results of this evaluation including WFP Country Office, Regional Bureau, and WFP Washington Office, USDA including the Food Assistance Division in Washington, DC and the Government of Bangladesh, specifically the Ministry of Primary and Mass Education (MoPME), Directorate of Primary Education (DPE), Local Government Division (LGD), Department of Public Health and Engineering (DPHE), Ministry of Health and Family Welfare. In addition, NGOs and international organizations Room to Read, RIC, BRAC, GAIN (Global Alliance for Improved Nutrition), UNICEF and FAO also have a stake in the evaluation. The primary users of this evaluation will be WFP-Bangladesh, USDA, NGOs, the GoB, WFP Regional Bureau, WFP HQ, the Office of Evaluation as well as other COs for learning purposes. 5. The evaluation attempts to highlight beneficiaries’ perspective and experiences (excluding students) with regards to the programme activities and results yielded. Additionally, gender equity and inclusion has been mainstreamed throughout the evaluation and the unintended impact of the programme on gender dimensions have also been assessed. Wherever appropriate1, gender dimensions have been factored into the sub-questions/key information areas for each evaluation question. 6. In accordance with the circular of GoB, which restricts the access to teacher’s attendance to district and upazila officers, evaluation of indicators that required teachers’ attendance has been excluded from the scope of the evaluation. Furthermore, given the context of the global COVID-19 pandemic and its associated restrictions, students were excluded as a respondent category for the end-term evaluation to prevent risk of exposure and ensure adherence to safety protocols. For the indicators for which primary data collection could not be done, values from the monitoring reports and the MTE have been used for the purpose of reporting. Methodology 7. The end-term evaluation adopted a mixed-method approach for primary data collection. The difference in status of indicators from baseline to end-term was examined through a comparison group vis-a-vis intervention schools supported by the FY-17 project. 8. Factoring the situation caused due to COVID-19, the data collection was scheduled in December 2020. The entire data collection process was carried out by ensuring that all safety guidelines were followed so that no respondents were subjected to risk for exposure to COVID-19. To ensure the 1 SFP by design does not have a specific focus on gender related issues and therefore the evaluation highlights gender dimensions wherever appropriate. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 ii safety of all the stakeholders involved, DMA (NRMC’s local partner in Bangladesh) carried out the data collection under the virtual guidance of the core evaluation team in New Delhi. 9. Following the simple random sampling approach adopted during baseline and mid-term, 50 Government Public Schools (GPS) were identified as sample (30 intervention and 20 comparison). The schools were selected randomly in the same proportion as the number of programme schools in the districts. The sample size was calculated at the programme level using the ‘differences method’ formula with a finite population2. One parent of a student from each grade was selected per school (total 250 parents from 50 schools). From each school, one head teacher, one teacher, and one storekeeper were also interviewed. Overall 20 FGDs were conducted, 10 for each category i.e. mothers and SMC members. 10. Considering that the Difference-in-Difference analysis was not possible during the baseline, an activity evaluation was conducted wherein comparative monitoring data from the comparison schools was collected. Accordingly, for school feeding indicators, a direct comparison between the intervention and comparison schools has been carried out. Limitation 11. The implementation of the programme activities commenced from October 2018. Due to closure of schools since February 2020 and postponement of activities and re-alignment of implementation strategy due to COVID-19 prior to the commencement of data collection (December 2020) for ETE, the evaluation may not highlight or report significant change in the impact of the programme’s activities. 12. Given that students were not sampled for the end-term evaluation, data on significant indicators had to be captured through parents. Questions on oral fluency and comprehension could not be asked as the EGRA tool was not administered. The reporting on indicators for literacy outcomes therefore, has been done based on the data captured during the MTE and supplemented through the findings of the qualitative discussions. Key Findings Relevance 13. Findings suggest that the programme is well aligned with the national government’s priorities, policies and strategies. The programme is aligned with the priorities of the NEP (2010), PEDP4, NPAN-2 as well as the NSSS (2015). 14. Given the persistent challenge of quality education in Bangladesh (especially in Cox’s Bazar), the programme’s strategy which included providing trainings to teachers and headmasters on new teaching techniques, effective school management and establishing reading corners to improve Bangla reading comprehension, was relevant and responsive to the challenges. The biscuit distribution intervention finds its relevance given its role in improving attentiveness, attendance and enrolment of students. Community participation events such as “Read-play festival” and “Grade I reception day” celebrated as part of SFP also contribute to the relevance of the programme since they aim to address challenges related to awareness around attendance, enrolment and literacy and provide platforms to encourage understanding of and demand for quality of education. 15. The pandemic gave rise to new challenges, especially with regard to food insecurity and poor food consumption. In this context, SFP’s implementation strategy including i) distribution of fortified biscuits, ii) awareness generation on cooking and consumption of nutritious food and iii) promotion of vegetable gardens contributes to the relevance of the programme as it aims to address challenges of hunger, food insecurity and poor food consumption. 16. In the challenging context of the pandemic, WFP’s adaptation of its implementation strategy to emphasise on promoting health and hygiene practices, capacity building of teachers and SMC members, include more home visits, door-to-door distribution of biscuits and use of digital platforms to enable last mile delivery was responsive to the changing context and evolving needs of the community. SFP’s efforts to ensure continuation of students’ education at home through home visits, distribution of story cards and utilisation of online platforms such as Facebook and TV channels, made the interventions responsive and highly relevant. However, 2 Confidence interval of 1.96 and estimated difference set at 5%, as per Cochran (1977), Cochran, W. G. (1977) Sampling Techniques. 3rd Edition, John Wiley & Sons, New York End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 iii despite such efforts, a large proportion of students remain unreached with online education (through TV, online and SMS platforms) owing to a lack of access to devices and internet for online classes and communication. 17. SFP was also found to be complementing initiatives made by other donors and the government, through its provision of fortified biscuits to students and complimentary education interventions, further proving its relevance. Therefore, the evaluation team found that the implementation strategy adopted by the programme, adequately targets the right people with the right type of assistance and was therefore, found to be relevant. Effectiveness 18. The evaluation findings indicate that the existence of classroom libraries for each grade and building the capacity of book captains has led to developing and encouraging reading habits, resulting in an improvement in the reading and comprehension skills of students, as reported by parents and teachers. In addition to establishing classroom libraries, interventions such as Read Play Festival and Supplementary Reading Material (SRM) period have also been found to be effective in promoting reading and comprehension abilities. These efforts have also been instrumental in making students and parents aspire towards greater educational outcomes. Further, with regard to improving attentiveness of students, discussions with teachers revealed that biscuit distribution has been a key enabler to ensure the same as it reduced distractions caused by hunger. 19. Discussions with parents in intervention schools suggests that there is an increased understanding and acknowledgement of the value of quality education. Given the importance placed on education by parents, especially for girls, and the reported improvement in students’ attentiveness and reading and comprehension abilities, the SFP has been effective in promoting literacy outcomes. 20. With regard to the capacity building of teachers and headmasters, trainings for new teaching learning techniques were postponed due to closure of schools since March 2020 (as a result of the pandemic). Thus, the progress made on the indicators with regard to trainings fell short of their targets. However, an increase in demonstration of learnings from these trainings have been self￾reported by teachers, head teachers and storekeepers. 21. A comparison of the end-term findings with the baseline figures indicated a 37 percent increase in average enrolment per sample school (in intervention schools) and a decrease of 2.5 percent in comparison schools. Discussions in Ramu revealed that parents and teachers felt that the provision of free school bags and dry food/biscuits would considerably help improve enrolment, indicating towards the effectiveness of the SFP activities towards improving the enrolment and attendance of students in the intervention schools. The project has not only achieved its targets of promoting regular attendance but also exceeded it in case of both boys and girls. 22. WFP’s efforts to promote improved nutrition and health included establishing vegetable gardens, training Little Agriculturalists and sensitizing teachers through various capacity building activities. The end-term findings indicate that 67 percent of the schools in Ukhiya and 58 percent in Kutubdia have vegetable gardens. However, no new vegetable gardens were established3 during the period April 2020 to September 2020, as schools were closed and some of the training of Little Agriculturalists and teachers were postponed due to the same reason. Despite the operational challenges, the trainings on nutrition, health and hygiene previously conducted with teachers as well as parents and the increased emphasis on messages related to health and hygiene during the pandemic helped improve health and nutrition practices. The status of demonstration of child health and nutrition practices by parents were assessed using two parameters: handwashing at critical times4 and dietary diversity. 23. The dietary diversity score for students for end-term was calculated based on responses from parents. The mean DDS during the end-term evaluation witnessed a decrease from the mid-term in both sample intervention (4.96 to 4.85) and comparison (5.49 to 5.04) schools. This decrease may be attributed to the ongoing COVID-19 pandemic wherein affordability and availability of diverse nutrition-rich food has been a major challenge. However, it is interesting to note that the reduction in the mean dietary diversity score is lesser in the schools in Ukhiya and Kutubdia as 3 Semi Annual Report April to September 2020 4 The numbers reported here are for those who practice handwashing on 4 or more out of 6 critical times End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 iv compared to Ramu. This could be attributed to WFP’s continued messaging on promotion of health, hygiene and nutrition practices during the pandemic through partnerships with organizations such as Sesame Workshop Bangladesh. Such initiatives towards constant reinforcement and promotion of health, hygiene and nutrition practices led to the motivation of parents and students in intervention areas to prioritise a diverse and nutrition rich diet, despite challenges posed by the COVID-19 pandemic, further demonstrating its effectiveness. Efficiency 24. Efficiency of processes in a programme is paramount to its success. One of the key components of the SFP that required considerable coordination was the biscuit distribution component. Findings from the end-term also suggest that all sampled schools in Ukhiya and Kutubdia received biscuits on a daily basis, till before closure of schools due to COVID-19. Moreover, 10 schools (out of 18 sampled schools) in Ukhiya and one in two schools in Kutubdia reported delivery of stock before any request for replenishment. It was also reported that there were barely ever any instances of stock out during the grant period. Based on the findings of the primary survey, that reflect (i) only one instance of stock out and; (ii) regular distribution of biscuits to students, it can be inferred that the process of inventory planning, demand forecasting and biscuit distribution is efficient. 25. The efficiency of the programme can also be determined by (i) the use of an online database/MIS by MoPME for timely reporting of data and its utilisation for planning and timely decision making; (ii) timely reporting of progress on programme implementation by the implementing partners and utilisation of data by WFP. WFP has provided technical support to DPE for real-time monitoring of the National SFP through the SF online-database. Managed by the MoPME and a PMU, this MIS-driven system and processes for reporting were observed to be efficient. It can be inferred, that regular reporting and monitoring of data led to ensuring timely completion of activities thereby reflecting efficiency of processes. Further, timely response to the programme needs, measures taken to ensure smooth implementation of biscuit distribution activities during and pre-COVID-19 along with WFP and its implementing partners’ proactive engagement with the government, school administration and the community reflects the efficiency in processes. Furthermore, capacity building activities with the relevant officials at GoB took place as per plan in a timely manner, showcasing efficiency of the programme. This was also reflected in the discussions held with GoB representatives, who reported that the activities took place in a seamless and coordinated manner. Impact 26. It is important to understand the impact of the SFP in light of the limitations imposed on the programme as a result of the COVID-19 pandemic. Information on certain indicators is limited given that the perspective of direct beneficiaries (i.e. students) could not be captured during the ETE. Despite facing challenges in implementation in the last year of the programme, the programme exceeded its targets on various indicators such as improved learning abilities (fluency and comprehension), enrolment and attendance of students. Contributing further to the overall objective of improving literacy and education outcomes, the programme was able to garner support amongst parents as well as SMC members. This was evident in the enhanced value placed on education, reported prioritization of quality education, and the proactiveness demonstrated by parents and SMC members to ensure continuation of education for their children and students. This showcases the impact of the education interventions undertaken by the programme. 27. With regard to improving the health and nutrition outcomes, there has been an increase in the adoption of health and hygiene practices in the sample intervention schools and a noteworthy improvement in the number of functional and usable toilets, number of separate toilets for boys and girls and demonstration of handwashing practices amongst parents. The project has also been successful in improving awareness about nutrition, health and hygiene as reflected in the increase in DDS from baseline to end-term. Considering an increased awareness amongst and adoption of health and dietary practices by the parents and their children, it can be inferred that the project has made an impact towards increasing the use of health and dietary practices. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 v 28. The success of the programme and the impact of the engagement created through it, is evident given the ownership showcased by the community in terms of providing support to the implementing partners to ensure continuity of the programme interventions, be it door-to-door biscuit distribution, encouraging students to maintain reading habits or participating in various programme activities. This not only showcases the impact of the programme on the community but also makes a strong case for the sustainability of the programme efforts. Sustainability 29. As a result of the ongoing engagement of WFP with regard to providing technical assistance to GoB and continued advocacy efforts, the National School Meal Policy was approved by GoB. The Government currently allocates USD $76 million annually towards the implementation of the School Feeding Project in Poverty Prone Areas (SFPPPA). The anticipated approval of the Primary School Meal Project (PSMP), which envisions providing school feeding to all primary school children, provides further evidence of the success of the programme and the government’s commitment and willingness to take ownership of the programme. 30. WFP’s work with the National Curriculum and Textbook Board (NCTB) to ensure that health and nutrition information is mainstreamed into grade-level science curricula in pre-primary grades through grade 5 further provides evidence for GoB recognizing the need for prioritizing and mainstreaming nutrition, ensuring sustainability of the SFP interventions. 31. WFP’s support to MoPME and DPE in operationalizing an online database for school meals has also enabled the GoB officials to independently plan their requisitioning, forecasting, budgeting and expenditure. The adoption and subsequent utilization of the online database for planning by GoB showcases the sustainability of this mechanism and the programme’s efforts aimed at an evidence based planning and an efficient takeover of the programme at the national level. 32. With regard to the capacity of the School management Committees and the community, the findings of the end-term evaluation suggest that their capacities have been built towards ensuring community’s involvement in school feeding and education activities. This was especially evident during the COVID-19 crisis, when community engagement in the SFP activities witnessed a significant boost. Discussions with SMC members and parents highlighted various instances where they took initiatives to ensure smooth door-to-door distribution of biscuits and continuation of education for all students, while the schools remained closed. However, to sustain this momentum, there is a need for a community engagement strategy within PSMP. 33. Further working towards ensuring the sustainability of the impact achieved by SFP during the grant period, WFP has worked in collaboration with the Cabinet Division, Economic Relations Division (ERD) and Ministry of Primary and Mass Education (MoPME) to ensure the SFP’s inclusion in government plans; as a result of which the school feeding programme has been mentioned in the 8th 5 Year Plan of the Government of Bangladesh. This makes a strong case for institutionalization of the SFP to ensure sustainability. Impact of COVID-19 34. The challenging period of implementation during the COVID-19 pandemic, sought with uncertainties, was met with re-alignment in the programme’s activities, to ensure that the needs of the students and communities were met in a responsive manner. The programme also attempted to address the changing needs of beneficiaries that emerged as a result of the pandemic, while carrying out its regular activities. Wherever feasible, a shift to remote modalities and direct support to students’ homes to deliver the programme activities was made. WFP along with GoB and implementing partners, collectively worked towards minimizing disruption to students’ learning and nutrition. Some new mechanisms were created and adaptations were made to respond to new challenges and explore new possibilities. These included home delivery of biscuits to address the nutritional needs of children, designing of book cards and leveraging online platforms to ensure continuation in improvement of educational outcomes while also encouraging involvement of all the stakeholders to provide a supportive environment to the beneficiaries. Despite such efforts, literacy gains were inequitable as there were constraints with regard to access to online platforms and digital devices in hard to reach areas, especially in Kutubdia. However, the impact created by the programme over the last 3 years culminated into a strong ownership of the programme by the government and the community alike, bolstering the sustainability of the programme. Drawing learnings from this experience, WFP End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 vi can create more resilient programmes in the future and factor in uncertainties in planning to create even greater impact. Conclusions 35. The evaluation findings suggest that SFP activities have been able to address the felt need for enhancing children’s interest in education related activities and improving their performance on literacy outcomes. Further, it has also been able to address challenges related to awareness around attendance, enrolment and literacy and provide platforms to encourage the same while also contributing towards strengthening GoB’s existing initiatives. Since SFP adequately targets the right people with the right type of assistance, it was therefore found to be relevant. 36. The findings indicate that the technical assistance provided by WFP enhanced GoB’s understanding on the merits of implementing nutrition sensitive interventions. SFP’s efforts to sensitize representatives of different ministries of GoB on the relevance of school feeding programme and its activities, and build consensus at the GoB level on minimum nutritional requirement for school meals contributed to the formulation and approval of the National School Meal Policy 2019. 37. At the programme level, the higher end-term values as compared to baseline values indicates that the programme has been able to achieve the intended results (for certain indicators, see Annex T for details). However, the pandemic caused postponement of certain training activities, due to which the targets have not been able to be met. 38. The evaluation findings highlight an enhanced ownership and commitment shown by the community and students to continue their education despite the closure of schools and consume healthy diets despite the challenges brought about by the pandemic. This serves as evidence that the programme has been effective and has been able to achieve its strategic outcome on improving literacy of school age children and increased use of health and dietary practices by the end of the programme. 39. The evaluation brought to light a lack of gender disaggregated data in the monitoring of activities and the findings also point towards a need to mainstream gender within the programme to increase the participation of women in SMCs as well as to involve mothers towards enabling quality education and enhanced nutrition for both boys and girls. 40. The findings suggest that the program processes have been effective in (a) ensuring that all the students in the class get to read story books as per their abilities (b) motivating and influencing students to adopt and enhance reading habits. The parents have proudly expressed an intent to support their children in pursuing quality education. The enhanced understanding of the importance of education has translated in increased attendance and enrolment of students in intervention schools as compared to baseline. 41. The end-term evaluation found that the distribution of biscuits continues to be a major influencing factor for parents to send their children to school and for students to attend the school, further contributing to the increased demand for quality education. 42. The SMC’s active involvement in the implementation of activities during the pandemic was found to be instrumental towards ensuring smooth functioning of the programme interventions during the pandemic. However, the gender balance in the SMC as well the participation of women continues to be skewed in favour of men. Evidence on enhanced community engagement in SFP activities was also found to be an enabling factor towards the sustainability of the programme outcomes. 43. The approval of NSMP and allocation of budget by GoB demonstrates its commitment and willingness to take the ownership of the programme and can be considered as an achievement of important milestone towards ensuring sustainability of SFP. 44. Regardless of the challenges posed by the COVID-19 pandemic, the project has been successful in improving awareness about nutrition, health and hygiene. Considering an increased awareness amongst and adoption of health and dietary practices by the parents and their children, it can be inferred that the project has made a significant impact towards increasing the use of improved health and dietary practices. 45. The evaluation findings also suggest that the technical assistance provided by WFP enabled the government to design a policy framework towards ensuring inclusive and equitable quality education and improved nutrition for primary school children. Further, the capacity building support has facilitated MoPME in identification of capacity gaps, preparation and End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 vii implementation of capacity building plan to independently undertake implementation of NSMP. Recommendations Policy Advocacy (National level) 46. Since the School Meal Policy has been approved, WFP should now provide technical assistance to MoPME and DPE in operationalization and coordination of activities under NSMP. It should also provide technical assistance to MoPME in establishing a Research and Development (R&D) Centre that can conduct research on applied nutrition in the local context. The evidence and knowledge generated can be utilized by GoB officials at national and local level for informed decision making. 47. The parents’ perception on experience with the adoption of new techniques by the teachers is positive. However, shortage in number of teachers in some schools and absence of mainstreaming of these methods and modules in the existing GoB curriculum acts as a barrier for success. Therefore, there is a need for increased engagement and advocacy for mainstreaming of new techniques and methods within the existing curriculum prescribed by GoB and adoption of these techniques at scale. Towards this, WFP should consider dissemination of evidence on benefits of adoption of new teaching techniques generated through this programme at the national level (MoPME, DPE), thereby making a case for consideration by GoB. 48. There is also a need for WFP to explore alternate packaging of biscuits and minimise generation of plastic waste. Waste to Art events/competitions may also be considered in schools to serve the dual purpose of raising awareness about fortified biscuits as well as ensuring waste recycling. Actionable (Specific to SFP implementation in Cox’s Bazar) 49. SFP has been able to demonstrate its relevance and impact in rural areas. Going forward, WFP should provide technical assistance to MoPME and other relevant ministries in replication of SFP/meals programme in the schools in urban areas. 50. WFP also needs to now enhance focus on improving resilience of the communities in addressing inequalities with regard to (i) access to learning opportunities; (ii) food and nutritional security and responding to crisis such as COVID-19, natural calamities. Creation of digital versions of libraries along with ensuring access to devices and internet as well as exploring ways to promote home gardening can be the first steps. 51. WFP should consider developing a menu for nutritious food in schools that states nutritional information and serving sizes to promote healthy meals. In addition, it should also scale up the efforts on training of cooks and include aspects such as disinfecting dishes, cookware and utensils, maintaining cleanliness in kitchen, managing waste and storing perishable and non-perishable commodities. This would help sustaining the efforts of the SFP and ensure success of the implementation of NSMP. 52. Given the increased community ownership of programme activities, WFP must strengthen community engagement for their enhanced participation in supporting implementation of activities by building their capacities and supporting MoPME in defining contours for community engagement. 53. Lastly, WFP now needs to further promote enhanced participation of women in SMC activities and incorporate more direct gender mainstreaming in future programming. More trainings and gender sensitization workshops with SMCs and parents may be considered. An overall mainstreaming of gender within the programme should also be considered in order to strengthen the impact of the programme. This can be done through inclusion of gender specific indicators for monitoring as well as sensitization activities conducted with the programme staff as well as the beneficiaries. 54. Over the course of three years, SFP interventions have proven to be effective and impactful in the promotion and improvement of literacy as well as health and dietary practices to a large extent. It is worth mentioning that WFP's strategy to work in close collaboration with the government has worked well and resulted in a high perceived value of the SFP amongst government stakeholders, significantly contributing to the eventual development and approval of the NSMP. Going forward, it is important to incorporate learnings from the implementation of the SFP in the last three years. Therefore, WFP must continue to leverage upon its strength of constantly liaising with governments to ensure sustainability. Drawing from achievements of this SFP, there is also a need to focus more on encouraging community engagement, capacity building and ownership as that proves to be the most effective in times of unforeseen risks and crises. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 1 1. Introduction 1.1. Evaluation Overview 55. This document is the end-term evaluation (ETE) report of the School Feeding Programme (SFP), being implemented during the period January 2018-June 2021 in Cox’s Bazar district, supported by McGovern-Dole International Food for Education and Child Nutrition Grant through the US Department of Agriculture (USDA). The ETE is commissioned by WFP Country Office, Bangladesh (WFP-CO). The ETE considers the evaluation questions specifically for the end-term of FY17 award. This evaluation was conducted during the period June 2020– March 2021, wherein the data collection was done during the period, 20th December 2020 to 7th January 2021. 56. The ETE serves several critical purposes intended for accountability and learning with an equal weightage to both as per the ToR. The aim of the evaluation is to i) measure the outcome-level results in order to understand what and how certain factors contributed to achieving the results, ii) understand the extent to which needs have been met, through a timely and transparent process, iii) understand the impact of interventions and explore the strength of the exit strategy with a focus on achieving program sustainability and iv) compare the end-term values with the baseline and mid-term values for all performance indicators as outlined in the Terms of Reference (ToR), and v) to underline the key lessons learnt that can inform strategic and operational decisions for future programmes. The evaluation will be used to highlight the results of the programme in achievement of indicators for McGovern Dole’s two strategic objectives as well as the highest-level results that feed into the strategic objectives. Furthermore, the ETE provides key insights regarding the progress made by the project during the implementation period, draws lessons from the interventions and helps understand the factors that affected the performance of the project. The evaluation on the basis of relevance, effectiveness, efficiency, sustainability, and impact aims to provide evidence to showcase the results achieved and draw out lessons learnt during the implementation period of the programme. 1.2. Overview of the Evaluation Subject 57. The subject of the evaluation is the WFP implemented USDA-funded SFP in Bangladesh from 2017-2021. Over the course of three years, WFP has been carrying out project activities aimed at (i) increasing student enrolment and student-teacher attendance through the provision of school meals and recognition of high-performing teachers, (ii) improving literacy among school￾aged children and the quality of education through early grade reading and other learning interventions, (iii) improving the health and dietary practices of students by improving water systems and latrine facilities and through water, sanitation and hygiene education initiatives, (iv) increasing government ownership through capacity building activities, and (v) increasing awareness regarding the importance of education among parents and community members through ongoing advocacy campaigns. 58. WFP has been providing micronutrient-fortified biscuits to around 47,689 primary-school-aged children per year in two upazilas in Cox’s Bazar district - Ukhiya and Kutubdia. The programme targeted 146 government- schools in the host community and 11 madrasahs in Ukhiya. The programme intervention map is presented in Annex A. 59. An end line evaluation of the 2015-17 SFP highlighted the need to strengthen the functioning and role of School Management Committees (SMCs) (especially women members) in the SFP, improve teaching and learning environment in schools, improve training (and re-training) of teachers on new teaching techniques and methods, and strengthen coordination and partnerships with the Ministry of Primary and Mass Education (MoPME), Government of Bangladesh (GoB). The poorly accomplished outputs on training in safe food preparation and storage practices was also elaborated upon. Teacher’s shortage and inadequate school infrastructure were identified as external factors affecting the literacy outcome. The SFP-FY17 award incorporated all the End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 2 recommendations (in the programme design) from the previous evaluation except addressing the shortage of teachers. This required direct intervention from GoB. 60. The scope of the evaluation is to address the proposed key evaluation questions including activities and processes related to the programme’s formulation, implementation, resourcing, monitoring, and evaluation through the lens of the OECD-DAC criteria. The evaluation report will be reviewed by a group of key internal and external evaluation stakeholders that constitute Evaluation Reference Group (ERG). The Terms of Reference for the ERG is presented in Annex V. The stakeholders that will be interested in the results of this evaluation include WFP Country Office, Regional Bureau, and WFP Washington Office, USDA including the Food Assistance Division in Washington, DC and the Government of Bangladesh, specifically the Ministry of Primary and Mass Education (MoPME), Directorate of Primary Education (DPE), Local Government Division (LGD), Department of Public Health and Engineering (DPHE), Ministry of Health and Family Welfare. In addition, NGOs and international organizations Room to Read, RIC, BRAC, GAIN (Global Alliance for Improved Nutrition), UNICEF and FAO also have a stake in the evaluation. The primary users of this evaluation will be WFP-Bangladesh, USDA, NGOs, the GoB, WFP Regional Bureau, WFP HQ, the Office of Evaluation as well as other COs for learning purposes. 61. In accordance with the circular of GoB, which restricts the access to teacher’s attendance to district and upazila officers5, evaluation of indicators based on teacher’s attendance has been excluded from the scope of evaluation. Furthermore, due to the travel restrictions imposed due to COVID-19 and to adhere to safety protocols, students were excluded as a respondent category for the end-term evaluation. Indicators for which primary data collection could not be done, values from the monitoring reports and the MTE have been used for the purpose of reporting. 62. Objectives, Outcomes and Activities: The strategic objectives and foundational results and expected outcomes have been presented in the results framework in Annex B. Key activities under each of the strategic objectives of the programme and indicators are presented in Annex C. 63. Planned Outputs and Beneficiaries: The USDA McGovern-Dole FY-17 project was expected to benefit 47,689 schoolchildren of Grades I-V in 146 schools by providing approximately 9.3 million micro-nutrient fortified biscuits per year and supporting complimentary education interventions in two upazilas (Ukhiya and Kutubdia) of Cox’s Bazar. The project was also expected to aid the formulation and operationalization of the first National School Meal Policy (NSMP) and help mainstream Government of Bangladesh’s National School Feeding in Poverty Prone Areas Programme (NSFPPA) into the revenue budget. Details of the planned outcomes have been provided in Annex C. The overall targeted beneficiaries and regions in line with the GoB and USDA priorities are presented in Annex E. 64. Partners: WFP has established strategic partnerships and leveraged support from government institutions, other UN agencies and NGOs towards achieving the McGovern-Dole goals. The list of partners along with their responsibilities is presented in Annex F. 65. Resource Requirements and Funding of USDA McGovern-Dole SFP: WFP has allocated $17,119,720.00 for donations of commodities, transportation, and financial assistance through the USDA McGovern-Dole SFP. WFP is providing financial support to its implementing partners for covering the direct programme costs and administration expenses. 66. Gender Dimensions of intervention: SFP by design, does not have a specific focus on gender related issues. However, WFP aims to encourage the equal participation of women and girls in all project activities. Distribution of biscuits is done in a gender equitable manner, wherein all boys and girls in the target schools are provided biscuits. 5 The circular came into effect before the midterm evaluation. It is not related to COVID-19. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 3 1.3. Context 67. With a population of over 160 million, Bangladesh has increased its HDI value by 60 percent during the period 1990 to 2019, putting itself in the medium human development category6. Gross National Income (GNI) per capita has also increased by 266.9 percent during the same period. However, despite significant gains in macro-economic growth and human development over the past decade, Bangladesh’s Human Development Index (HDI) of 0.632 is below the average of 0.649 for countries in the medium human development group. The Multidimensional Poverty Index (MPI)7 which includes education, health, and standard of living also highlights widespread income poverty in Bangladesh. The MPI i.e. the share of the population that is multi-dimensionally poor adjusted by the intensity of the deprivations, is 0.198. The contributions of deprivation from education, health, and standard of living to overall poverty are 29.2, 23.5, and 47.3 percentage points respectively. 1.3.1 Food Security and Nutrition 68. Bangladesh has made impressive gains with regard to improving the state of food security over the past few decades, wherein the prevalence of undernourishment has declined from 32 percent in 1990 to 13 percent in 20198. However, there are still 26 million people who are food-insecure. 69. The Food Security and Nutrition Surveillance Project reports shows that while food insecurity has declined for all the quintiles, it has declined relatively slowly for the poorer ones. In 2010, the proportions of food-insecure households were found to be 68 percent for the bottom two quintiles and 20 percent for the top two quintiles. By 2014, these proportions had come down to 43 percent among the bottom two quintiles and 6 percent among the top two9. These figures suggest that the rate of progress was much slower for the poorer groups10. Moreover, existing household-level food insecurity is exacerbated by poor access to food and limited dietary diversity, in turn worsening undernutrition. 70. The concerns about food security and nutrition are emerging as a result of climate change. About half of the population already suffers from iron and zinc deficiency and there exists, a genuine concern that rising carbon dioxide in the atmosphere will exacerbate the problem of micronutrient deficiencies. Increased salinity due to the inflow of sea water into groundwater in the coastal areas, especially in districts like Cox’s Bazar, poses the threat of altering the micronutrient content of staple foods, including rice. 71. The Joint Response Plan (JRP) for Rohingya Humanitarian Crisis 201911 by the Food and Agriculture Organization (FAO) suggests that 695,000 people are severely food-insecure in Cox’s Bazar. The JRP also reports that Cox’s Bazar has borne significant negative impacts on the environment leading to a reduction in agricultural production. A study conducted in Bangladesh by FAO on gender and food security12 highlighted the gender differences in access to food and related resources. 6 United Nations Development Programme. (2019). Human development indices and indicators: 2018 Statistical update. 7 Human Development Report 2016-Human Development for Everyone: Briefing note for countries on the 2016 Human Development Report Note: The HDR 2019 Statistical Update does not contain the complete and updated MPI due to missing information on some indicators such as nutrition and will be available in due course. If made available, the same shall be updated in the evaluation report. 8 https://knoema.com/atlas/Bangladesh/topics/Health/Nutrition/Prevalence-of-undernourishment 9 Helen Keller International and James P. Grant School of Public Health). (2015). State of Food Security and Nutrition in Bangladesh: 2014. BRAC University: Dhaka 10 Ahmed, Akhter. (2017). Strategic Review of Food Security and Nutrition in Bangladesh. 10.13140/RG.2.2.24673.71529. 11 http://www.fao.org/3/CA3252EN/ca3252en.pdf 12 Alston, M., & Akhter, B. (2016). Gender and food security in Bangladesh: the impact of climate change. Gender, Place & Culture, 23(10), 1450-1464. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 4 72. Findings from the Second Rapid Assessment of Food and Nutrition Security in the Context of COVID-19 in Bangladesh (May to July 2020) by FAO13 revealed that the COVID-19 pandemic has increased hunger and malnutrition, with greater numbers of young people and their families facing food insecurity. About 36.4 percent of the respondents reported moderate or severe food insecurity post the COVID-19 pandemic, which was higher than the national average (31.5 percent) before the COVID-19 pandemic. Additionally, it was found that less than one-third of the respondents had minimum dietary diversity during the reporting period. 1.3.2 Health 73. With a current expenditure of 2.4 percent of its Gross Domestic Product (GDP) on health14, Bangladesh spends well below the already-low South Asia regional average of 3.5 percent15 and global average of 9.9 percent16. 74. While the Human Development Report (HDR) 2020, indicates improvement in life expectancy at birth from 58 to 72.6 years during 1990-2020, the under-five mortality rate per 1,000 live births is 30.2.17 In case of SFP intervention upazilas (in Cox’s Bazar), the under-five mortality rate in 2015 was 47.4 and 5.6 for Kutubdia and Ukhiya respectively18 . 75. All 64 districts in Bangladesh are endemic with parasitic diseases, with 78 million (55%) infected with roundworm19. The most vulnerable to such infections are the 5-14 years age group as roundworm causes impaired growth and cognition development among children. To address this, GoB’s Ministry of Health and Family Welfare (MoHFW) initiated a deworming programme in 2010. It also initiated the “Little Doctor” initiative, with an aim to increase drug coverage (for deworming) as well as knowledge on important health matters among students, especially on personal hygiene. Deworming is now conducted for children aged 5-12 years in all primary level schools in the country. 1.3.3 Education 76. With 1.3 percent of GDP and 9.3 percent of total government expenditure being spent on education20, there has been a significant improvement in the status of children’s education. The country’s net enrolment rate (NER) at the primary school level has increased from 80 to 95 percent during 2000-2017. In 2017, female NER was 93 percent and male NER 89 percent in primary schools. Furthermore, as per the World Health Organization, the Gender Parity Index (GPI)21 for gross enrolment ratio in primary education for Bangladesh was 66.2 in 2015. 77. The quality of education, however, has not been able to match the level of enrolment rate. According to the World Development Report 2018, 35 percent of grade III students scored too low to even be tested on reading comprehension in Bangla, and only 25 percent of grade V students in the country passed the minimum threshold in mathematics. National learning assessments by the GoB also show poor literacy and numeracy skills among students wherein only 25 to 44 percent of the students in Grades V-VIII have mastery over Bangla, English and Mathematics. Further, the GDI indicates that the mean years of schooling for girls is lower (5.2) as compared to boys (6.7). Studies reveal that key factors behind this are likely to be lack of access to early childhood development programs, low quality of teaching practices, challenges related to poor school 13 https://reliefweb.int/sites/reliefweb.int/files/resources/cb1018en_compressed.pdf 14 2019 Health SDG Profile: Bangladesh 15 https://data.worldbank.org/indicator/SH.XPD.CHEX.GD.ZS?locations=8S 16https://data.worldbank.org/indicator/SH.XPD.CHEX.GD.ZS?end=2017&name_desc=false&start=2000&view=chart 17 http://hdr.undp.org/en/indicators/57506 18 http://app.dghs.gov.bd/localhealthBulletin2016/publish/publish.php?org=10000924&year=2016&lvl=1 19 Rahman, M. M. (2017). Biannually school-based deworming by Mebendazole 500mg has reduced the worm load drastically in Bangladesh. EC Bacteriology and Virology Research, 2, 113-114. 20https://www.ceicdata.com/en/bangladesh/education-statistics/bd-government-expenditure-on-education-total--of￾government-expenditure 21 The GPI is the ratio of girls to boys enrolled at primary level in public and private schools. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 5 management, and low levels of overall spending on public education.22 According to the latest Human Development Report 2019, the percentage of primary school teachers trained to teach in Bangladesh were only 50 per cent.23 78. It is important to highlight that among all districts, Cox’s Bazar’s performance on indicators such as primary education NER and dropout rates is lowest in the country. The NER in Cox’s Bazar is 72.6 percent for boys and 69 percent for girls, compared to the national average of 95 percent. Further, the net attendance ratio for primary school age children is 85.9 percent24. Similarly, while the national average dropout rate for boys and girls is 14 percent and 11 percent respectively, it is 39.6 percent (for boys) and 23 percent (for girls) in Cox’s Bazar. 1.3.4 Water, Sanitation and Hygiene 79. According to the Joint Monitoring Programme (JMP) Report 2017, the proportion of population with access to safely managed water is 56 percent while the proportion of population with access to ‘at least basic’ sanitation is 48 percent. 80. The preliminary findings of the Joint Multi-Sector Needs Assessment Report (October 2020)25 suggest that while a vast majority of households reported using improved drinking water sources, 4 per cent of households experienced loss or diminished access to clean water and sanitation as a result of the COVID-19 pandemic. With regard to sanitation, almost all participants reported dysfunctional sanitation facilities and lack of funds to repair them. This was reported to have resulted in open defecation practices sometimes. 81. With regard to the WASH situation in schools, the latest JMP report on WASH in schools26 reveals that as of 2019, 69 percent of schools had a basic drinking water service (defined as improved source of water with water available), 63 percent of schools had a basic sanitation service (defined as improved single sex facility that is usable) and 57 percent of schools had a basic hygiene service (defined as handwashing facility with water and soap available). 82. Furthermore, as per the WASH Sector Cox’s Bazar Situation Report27 (January 2019) by the Inter Sector Coordination Group, 1.05 million people in Cox’s Bazar are still in need of immediate WASH support. 1.3.5 School Feeding Needs 83. Malnutrition estimates from the Global Nutrition Report (2018) show that wasting in Bangladesh is at 14.4 percent among children under five years and “has not shown much progress/worsened” against the global nutrition targets of 201928. Similarly, about one out of three children (32.6%) under five years are underweight. In case of Cox’s Bazar, 15 percent of children under five are under weight, while 43 percent29 of children are stunted. In Cox’s Bazar, 44 percent of under 5 in the refugee community were found to be stunted as of March 2018.30 84. The WFP-CO’s Vulnerability Assessment and Mapping (VAM) Unit assessed priority districts and sub-districts by using a weighted index combining food insecurity, nutrition, poverty, and 22 http://www.xinhuanet.com/english/2019-02/27/c_137855105.htm 23 https://thefinancialexpress.com.bd/views/improving-quality-of-education-main-challenge-1580309749 24 Multiple Indicator Cluster Surveys (MICS) 2018-19. 25https://www.humanitarianresponse.info/en/operations/bangladesh/document/j-msna-refugee-and-host￾communities-preliminary-findings-october-2020 26 https://data.unicef.org/topic/water-and-sanitation/wash-in-schools/ 27https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/documents/files/cxb_wash_sect or_operation_presence_map_january_2019_0.pdf 28 2018 Nutrition country profile accessed from www.globalnutritionreport.org Source: UNICEF/WHO/World Bank Group: Joint child malnutrition estimates. 29 Hasib, Md & Hassan, Md & Hasan, Mehedi & Khan, md. shafiqul islam. (2020). Effect of nutritional status on Rohingya under-five children in Bangladesh. International Journal of Public Health Science (IJPHS). 09. 101-106. 10.11591/ijphs.v9i4.20546. 30 https://data2.unhcr.org/en/documents/download/72273 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 6 education indicators. The report pointed to a high need of intervention in Cox’s Bazar district. The district has a poverty rate of 22 percent and extreme poverty rate of 11.3 percent. Further, it is highly food insecure with over 34 percent of the population living below the food consumption poverty line31. The region is also experiencing a Level 2 emergency with the influx of over 900,000 Rohingya refugees from Myanmar which is further exacerbating food insecurity. 1.3.6 Gender Analysis 85. Although the constitution of Bangladesh guarantees equal rights, women and girls in Bangladesh still lag behind men in many aspects of life. Discrimination against women and male dominance is a prevalent feature of society, especially in rural areas. 86. The World Economic Forum’s Global Gender Gap Reports (2017 and 2018)32 revealed that Bangladesh has progressed significantly from its position as 72nd in 2016, 47th (in a tally of 144 countries) in 2017, and 48th amongst 149 countries in 2018, leading South Asia in terms of gender equality. The recent Global Gender Gap 2020 report reveals that it is now ranked at the 50th position resulting in a 2 point dip from 2018. The country is ranked 139 in labour force participation, 98 in wage equality, 131 in earned income, and 139 in legislators, professionals/technical workers, thereby concealing the continued struggle with violence against women, wage disparity and the participation of women in workforce. 87. While poverty rates in Bangladesh are decreasing, vulnerability to poverty continues to have concrete gender dimensions. Significant disparities in employment and wage rates persist (female GNI 2,873; male GNI 7,031)33 which, combined with considerable gaps in asset ownership, seriously limit women’s economic opportunities. With a large population mostly engaged in informal labour, women are faced with lower income, greater insecurity, and unfavourable working conditions. 88. Gender-based capability poverty continues to be a key issue in the health and education sectors despite significant improvements in recent years, and is reflected in poor nutrition, maternal mortality and child mortality indicators. For every 100,00034 live births, 176 women die from pregnancy-related causes. While the adolescent fertility rate has shown improvement since 2006, it still remains an area of concern. 1.3.7 Government Policies and Programmes 89. The Government of Bangladesh envisions transforming Bangladesh into a middle-income country by 2021 and a prosperous country by 2041. The government has translated this vision into an actionable agenda by formulating the Perspective Plan (2010-2021) and two Five Year Plans (6th and 7th FYPs)35 by integrating Sustainable Development Goals (SDGs) into it. Given the comprehensiveness and cross-cutting nature of SDGs, policy coherence is very critical. 90. The government has been loosely implementing the National Education Policy since 2010. Efforts to increase the educational participation (enrolment, attendance, persistence etc.) are being made through the Primary Education Stipend Project (PESP). 91. The recent Fourth Primary Education Development Programme (PEDP4), GoB’s flagship programme in the education sector for the duration 2019 to 2023, aims at providing children of pre-primary to grade V quality education with key emphasis on efficiency, inclusivity and 31Terms of Reference (TOR) of Fiscal Year (FY) 2017 – FY2020 McGovern-Dole International Food for Education and Child Nutrition Grant (McGovern-Dole) 32 The Global Gender Gap Report benchmarks countries on their progress towards gender parity across four thematic dimensions: Economic Participation and Opportunity, Educational Attainment, Health and Survival, and Political Empowerment. 33 United Nations Development Programme. (2020). Human development indices and indicators: 2019 Statistical update. 34 Islam, M. S., & Hasan, M. N. (2017). Bangladesh. In The World Guide to Sustainable Enterprise (pp. 36-41). Routledge. 35 https://info.undp.org/docs/pdc/Documents/BGD/UNGA_Booklet_2017.pdf End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 7 equity. The programme aims at deploying quality teachers who would be provided with continuous professional development.36 92. The MoHFW embarked on a sector-wide approach (SWAp) modality for the health, nutrition and population (HNP) sectors in 1998. Assessments indicate that the MoHFW has made substantial progress in health outcomes and health systems strengthening37. 93. However, undernutrition costs Bangladesh more than 7,000 crore BDT (US$ 1 billion) in lost productivity every year38, and even more in healthcare costs39. The Second National Plan of Action for Nutrition (NPAN-2), an integrated and multi-sectoral framework for improving the nutritional status in the country, acknowledges that ensuring effective investments in nutrition is estimated to lead to economic gains in Bangladesh, through an estimated increased productivity, exceeding 70,000 crore BDT by 202140 and possibly even more41. 94. GoB has also formulated a comprehensive National Social Security Strategy (NSSS) to integrate all safety net programmes. The coverage of social safety net recipients is 27.80 percent and the allocation has been increased from 1.9 percent of GDP in FY09 to 2.3 percent in FY17 (Household Income and Expenditure Survey, 2016) and 2.5 percent of GDP in FY19.42 95. The GoB also approved the National School Meal Policy (NSMP) in August 2019. According to the NSMP, the government aims to provide nutritious meals five days per week and fortified biscuits one day per week to nearly 15 million primary school students with the aim to increase attendance and reduce dropout rates. While this is still being finalized, the Primary School Meal Project includes a provision for nutritious meals for three days and fortified biscuits for three days in the week. Minimum dietary diversity will be ensured by preparing meals with fortified rice, lentils, fortified refined oil, locally grown fresh vegetables, and eggs43. The policy aims to ensure the minimum nutritional requirements in school meals for every primary school student by 2023.44 1.3.8 Development Assistance in Bangladesh 96. Bangladesh receives development assistance from various sources. The World Bank is implementing a “Transforming Secondary Education for Results (TSER)” project to support the government’s Secondary Education Development Programme, which is expected to benefit 13 million students from Grades VI-XII. It is expected to enhance the quality of teaching and learning as well as improve access and retention of students, especially girls and children from poor households. 97. Further, aiming to reach over 18 million children studying in pre-primary level to grade V, World Bank is implementing the Quality Learning for All Programme (QLEAP) across the country. The programme will help improve the quality of education and ensure equitable access to primary education. The programme aims to bring about one million out-of-school children to learning centres that would follow national primary education curriculum, and thus help them integrate with the formal education system. The project will also build close to 95,000 36 http://documents.worldbank.org/curated/en/857071529206219039/pdf/BANGLADESH-QLEAP-PAD-05252018.pdf 37 Ahsan, K. Z., Streatfield, P. K., Ijdi, R. E., Escudero, G. M., Khan, A. W., & Reza, M. M. (2015). Fifteen years of sector-wide approach (SWAp) in Bangladesh health sector: an assessment of progress. Health policy and planning, 31(5), 612-623. 38 FAO, WFP and IFAD. 2012. The State of Food Insecurity in the World 2012. Economic growth is necessary but not sufficient to accelerate reduction of hunger and malnutrition. Rome, FAO. 39 UN Renewed Efforts Against Child Hunger (REACH), Undernutrition in Bangladesh A Common Narrative. 2014 40 Howlader, et al. (2012). Investing in Nutrition Now: A Smart Start for Our Children, Our Future. Estimates of Benefits and Costs of a Comprehensive Program for Nutrition in Bangladesh, 2011– 2021. PROFILES and Nutrition Costing Technical Report. Washington, DC: Food and Nutrition Technical Assistance III Project (FANTA), FHI 360 41 Ministry of Health and Family Welfare, Government of Bangladesh, 2017. Second National Plan of Action for Nutrition (2016-2025) 42https://www.worldbank.org/en/news/feature/2019/04/29/social-safety-nets-in-bangladesh-help-reduce-poverty￾and-improve-human-capital 43At least once in a week 44 https://www.thedailystar.net/country/cabinet-okays-national-school-meal-policy-2019-1.5-crore-children-1787272 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 8 classrooms, teachers’ rooms, and multipurpose rooms to ensure international standard students-to-classroom ratio. It will also build 80,000 WASH blocks and 15,000 safe water sources with special emphasis on facilities for girl students and female teachers. Moreover, hygiene promotion will be integrated in the revised curricula and textbooks. 98. Bangladesh also receives development assistance from the US Government and its agencies. USDA is providing assistance through Winrock International as part of the Food for Progress programme. This programme aims to improve agricultural productivity and expand the trade of agricultural products in Bangladesh.45 99. Additionally, the Feed the Future programme (part of the US Government-funded Global Hunger and Food Security Initiative) helps more than 700,000 farmers to increase their production of food, commercial crops, fish, and livestock, gain access to markets, and improve the nutrition of their families.46 100. USAID is also providing support to GoB towards enhancing agricultural productivity, diversifying production to improve nutrition, building capacities of farmers to upgrade to new technologies, improving market systems and contributing to women’s empowerment in agriculture. 47 101. Further, a relevant US Government Foreign Assistance supported programme is the MaMoni Maternal and Newborn Care Strengthening activity, implemented by Save the Children for the period of 2018 – 2032. This programme aims to contribute towards providing quality maternal and newborn health care and nutrition services through technical capacity building of human resources, improvement of health information systems, development of appropriate guidelines and protocols, assistance in policy and strategy development, and strengthening of governance at national and sub-national levels.48 102. Towards education, USAID is assisting the GoB in improving reading skills amongst primary school students through teacher trainings, provision of supplementary reading materials and creation of community reading camps. Additionally, in collaboration with BRAC, it is also helping in increasing access to schooling among out-of-school children in urban slums. 49 103. In the WASH sector, Water Aid has also launched the WASH4UrbanPoor project to impact the lives of 450,000 staying in slums and low-income urban communities through sustainable WASH services in slums, schools, healthcare centres and public places.50 104. The World Bank is implementing the Multipurpose Disaster Shelter Project in Bangladesh with the objective of reducing the vulnerability of the coastal population across selected districts of Bangladesh to natural disasters. These shelters will also serve as schools and allow for continued use and upkeep of the infrastructure in non-disaster periods. 105. In addition, NGOs are a major performer working on diverse areas like healthcare, education and safe drinking water. For instance, BRAC in Bangladesh has been providing education in a cost-effective way which has boosted both school attendance and retention. BRAC schools have initiated two measures; separate toilets and two free sets of school uniform for girls. 106. UNICEF is also contributing to the development of Bangladesh in the sector of health, nutrition, education and WASH. 107. The United Kingdom's Department for International Development has provided more than £80 million to UNOPS for addressing challenges related to the humanitarian context of Cox’s Bazar district. UNOPS is working closely with 15 implementing partners to extend support 45 https://www.fas.usda.gov/programs/food-progress 46 https://www.feedthefuture.gov/country/bangladesh/ 47 https://www.usaid.gov/bangladesh/agriculture-and-food-security 48 https://www.foreignassistance.gov/explore/country/Bangladesh 49 https://www.usaid.gov/bangladesh/education 50https://www.wateraid.org/bd/media/wateraid-launches-the-wash4urbanpoor-project-aiming-to-serve-450000- urban-poor-with-water End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 9 through improving access to food, education, health, sanitation and jobs. These partners include Action Against Hunger, BRAC, Christian Aid, Handicap International, HelpAge International, International Organization for Migration, Oxfam in Bangladesh, Save the Children International, Solidarités International, UNICEF, United Nations High Commissioner for Refugees, UNFPA, UN Women, World Food Programme and World Health Organization. 108. On 25th September 2020, the World Bank pledged 200 million USD to help Bangladesh improve access to safe water and sanitation services in rural areas under a project called the Bangladesh Rural Water, Sanitation and Hygiene for Human Capital Development Project.51 This project will help 600,000 people avail safe and clean water through large and small piped water schemes in rural areas and provide access to improved sanitation services to over 3.6 million rural people. 109. In mid- 2020, the World Bank also approved $202 million in additional financing for the Modern Food Storage Facilities Project to increase the storage capacity of Bangladesh’s national strategic grain reserves by 535,500 tons for 4.5 million households. This project was aimed to help Bangladesh address food insecurity not only for the frequent climate-induced disasters but also for crises situations like the current COVID-19 pandemic.52 110. In the specific context of COVID-19, many international actors such as World Bank, ADB and IMF have disbursed large amounts of funds to Bangladesh in order to provide assistance in combating the COVID-19 crisis. Development partners disbursed $7.2 billion throughout FY 2019-20 as compared to $6.5 billion during FY 2018-19, indicating towards a disbursement growth of 11 percent according to the preliminary report of the Economic Relations Division (ERD).53 1.3.9 COVID-19 Situation in Cox’s Bazar 111. There were 543,717 COVID-19 confirmed cases in Bangladesh as on 23rd February 202154. 5,608 cases have been found in Cox’s Bazar as on 15th December 202055. 112. The area of Cox’s Bazar (especially islands like Kutubdia) is known to be seasonally prone to both landslides and flash floods. Towards this, the GoB implements annual preparatory mitigation measures for the monsoons and the challenges associated with it. However, this time the annual monsoon preparations were impacted as disaster risk reduction (DRR) efforts were suspended and delivery of supplies through road transport was affected due to the COVID-19 related “lockdown”.56 This also affected safe access to food items and nutrition which is critical to battle a crisis like the COVID-19.57 113. This vulnerability was further enhanced by restrictions on internet usage since September 2019 that made it difficult to communicate reliable information about the virus with the host and refugee communities in Cox’s Bazar.58 Realising the importance of unrestricted access to information via mobile and internet communication for slowing the transmission of the disease, a group of 50 international organizations had written a letter59 to Bangladesh’s honourable prime minister requesting her to immediately lift this mobile and internet ban in 51https://www.worldbank.org/en/news/press-release/2020/09/25/world-bank-helps-bangladesh-ensure-safe￾water-and-sanitation-in-rural-areas 52https://www.lightcastlebd.com/insights/2020/12/how-dependent-is-bangladesh-on-development￾aid#:~:text=In%20the%20beginning%20of%20FY,from%20the%20Covid%2D19%20fallout. 53https://www.lightcastlebd.com/insights/2020/12/how-dependent-is-bangladesh-on-development￾aid#:~:text=In%20the%20beginning%20of%20FY,from%20the%20Covid%2D19%20fallout. 54 https://covid19.who.int/table 55 https://data.humdata.org/dataset/district-wise-quarantine-for-covid-19 56https://www.unhcr.org/news/briefing/2020/4/5e9ea77e4/covid-19-unhcr-warns-severe-implications-annual￾monsoon-response-bangladesh.html 57https://docs.wfp.org/api/documents/WFP-0000114547/download/?_ga=2.208620311.404038719.1588218681- 1607365947.1560326616 58 https://www.unhcr.org/news/stories/2020/4/5e9ead964/covid-19-arrives-camp-devastating.html 59 https://www.hrw.org/sites/default/files/supporting_resources/200402_joint_letter_covid19.pdf End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 10 Cox’s Bazar, among other things. These organizations included Amnesty International, ASEAN Parliamentarians for Human Rights, Human Rights Watch and more.60 ‘ 114. Other reasons why the Cox’s Bazar refugee camps are particularly vulnerable include housing without potable water and running water for toilets, lack of access to adequate healthcare and a potential shortage in medical supplies and testing capabilities. 61 On 14th January, a fire erupted in the Nayapara camp in Cox’s Bazaar gutting 550 shelters and 150 shops. While there were no casualties, this unfortunate incident increased the vulnerability of the refugees in terms of food security, WASH, healthcare and education. WFP provided emergency food assistance including hot meals to families in need. 115. On 24th March 2020, the Refugee Relief and Repatriation Commissioner (RRRC) (based in Cox’s Bazar) released a statement defining the essential and critical activities in all 34 camps in Cox’s Bazar for COVID-19. The statement defined education as a non-essential activity. This resulted in severely restricted access for UN and NGO staff and the closure of learning facilities. This closure means that the education sector closed nearly 6,000 education facilities across the camps for the foreseeable future, disrupting learning of over 325,000 children (of which, 49 per cent are girls and adolescents aged 3 to - 24 years), and cutting off their access to education. 116. To address the above challenges during this crisis, the United Nations High Commissioner for Refugees is monitoring for a potential outbreak and Bangladesh’s Health Ministry has completed and validated a national response plan to contain the virus. Humanitarian agencies working with the GoB’s Refugee Relief and Repatriation Commissioner (RRRC) are finalizing a multi-sector plan that would focus on prevention and readiness (e.g. preparing testing kits and extra beds for quarantine), raising awareness around best-practices for hygiene, increasing facilities and supplies for hand washing, and increasing the water supply during crucial period.62. 117. Additionally, on the request of the GoB, WHO facilitated a COVID-19 Intra Action Review (IAR) in the first week of February 2021. The purpose was to analyse the ongoing COVID-19 response in Cox’s Bazaar, share lessons learnt and agree on next steps to further improve and strengthen the continued response to the pandemic. 63 The Ministry of Local Government, Rural Development and Cooperatives (MoLGRD&C) has also drafted a ‘National WASH Sector Strategic Paper 2020-22 for Response to COVID-19 Outbreak through Water, Sanitation and Hygiene Interventions’ with technical assistance from UNICEF and in close collaboration with other relevant ministries. 118. In terms of funds for COVID-19 response, a global network of aid agencies have launched a new COVID-19 aid fund to anticipate and respond to critical virus-related humanitarian needs in low-income countries. This fund is being kick started with a donation from IKEA Foundation for €1.5M and is being led by Start Network, which has 50 members, many with expertise in the prevention and control of infectious diseases across the globe. This fund is being used by local organizations to help vulnerable communities across Bangladesh, including Rohingya refugees as well as the host community in Cox’s Bazar. 64 119. Furthermore, GoB, UNICEF and Save the Children have created Education Sector Strategy (for Cox’s Bazar)65 that aims at supporting Education in Emergency (EiE) partners with ongoing coordination and monitoring of activities to ensure efficiency, adequate reach to vulnerable children and to avoid duplication and also to identify and pursue areas of advocacy and 60https://www.hrw.org/news/2020/04/01/joint-letter-re-restrictions-communication-fencing-and-covid-19-coxs￾bazar-district 61 https://reliefweb.int/report/bangladesh/rohingya-refugees-cox-s-bazar-brace-covid-19-pandemic 62 https://reliefweb.int/report/bangladesh/rohingya-refugees-cox-s-bazar-brace-covid-19-pandemic 63https://www.who.int/bangladesh/news/detail/16-02-2021-cox-s-bazar-covid-19-intra-action-review-a-year￾without-precedent-in-review 64https://ikeafoundation.org/press-release/new-covid-19-fund-launched-to-help-vulnerable-communities-in-low￾income-countries/ 65 Cox’ Bazar Education Sector- Cox’ Bazar-Bangladesh COVID 19 Response Strategy,2020 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 11 collaboration. The Education Sector strategy is based on a timeframe that includes the immediate term (4-6 weeks), mid-term (6-8 weeks) and long term (8-12 weeks) putting timelines to the implementation of the different proposed activities. The activities will be implemented through dedicated Working Groups: ECD Working Group, Youth Working Group, and the Technical Working Group. The Working Groups and their Chairs are responsible to the Education Sector Coordinators for the timely and satisfactory implementation and completion of the activities. 120. In the context of a lockdown, hunger, malnutrition, and other problems that have always plagued Bangladesh are poised to intensify66. Moreover, challenges related to education and gender may also arise. As per a study conducted with adolescents in the Rohingya and Bangladeshi communities by UNHCR in December 2020, food insecurity emerged as one of the most severe consequences of the pandemic. 23 per cent Bangladeshi and 18 per cent Rohingya adolescents said they felt hungrier now than prior to COVID-1967. This was validated by findings in the Joint Multi-Sector Needs Assessment (J-MSNA) 2020, which found that the percentage of households with a poor food consumption score jumped from 5% to 15% in refugee communities and 3% to 8% in host communities from 2019 to 2020.68 121. Economic hardships caused by the crisis will have spill-over effects as families consider the financial and opportunity costs of educating their daughters. Dropout rates amongst girls increased during the Ebola virus and girls’ learning at home was limited, as shown by Plan International’s analysis. This may also prove to be a potential threat due to COVID-19 in Bangladesh. Therefore, while funds for immediate relief for COVID-19 affected areas are important, Bangladesh is also likely to face challenges with regard to maintaining equitable access to basic social services like health and education. 2. Evaluation Methodology and Limitations 2.1 Approach and Methodology 122. The evaluation methodology was guided by the ToR and the concept note, underpinned by the results framework of the FY17 project. It presented an approach that included the key thematic and functional areas that the programme interventions worked upon. These interventions can be categorised under four domains: i) Individual (school going Bangladeshi children), ii) Institutional (school teachers, school administrators, and store keepers), iii) Social (parents and community), and iv) External environment (policy, government support, civil society). In light of the results framework, the conceptual framework aimed to examine the outcomes and results of the programme.69 The framework further strengthened the methodology by analysing cross cutting factors such as gender and WASH; issues that may have acted as triggers or barriers, influencing the actors and structures in the programme and determining the extent of effectiveness of the programme. This conceptual framework has been described in detail in Annex I. 123. The evaluation methodology accorded equal weightage to accountability and learning and adopted a mixed-method approach (quantitative and qualitative tools, administered through virtual engagement70) for primary data collection. The difference in status of indicators from baseline to end-term was examined through a comparison group (schools in Ramu Upazila as 66 https://thediplomat.com/2020/04/the-covid-19-catastrophe-in-bangladesh/ 67 https://www.unhcr.org/blogs/covid-19s-impact-on-rohingya-and-bangladeshi-adolescents-in-coxs-bazar/ 68https://www.humanitarianresponse.info/en/operations/bangladesh/document/j-msna-refugee-and-host￾communities-preliminary-findings-october-2020 69 Of the four domains of the intervention, information about the first one i.e. Individual (school going children) was collected indirectly (through parents and teachers) during the end term evaluation as meeting with students was not possible due to the COVID-19 crisis. 70 Virtual platforms such as Skype/Zoom Call/Whatsapp with the help of our local partners. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 12 identified during the baseline study) vis-a-vis (intervention) schools supported by FY-17 project (provision of fortified biscuits and trainings). 124. The end-term evaluation has been conducted during the period June 2020 - March 2021. The evaluation covered the MGD FY17 programme, including activities and processes related to its implementation, resourcing, monitoring and reporting relevant to answer the evaluation questions for the end-term evaluation. 125. Factoring the situation caused due to COVID-19, the data collection was scheduled in December 2020. The entire process of data collection was designed and carried out in a manner to ensure that respondents were not subjected to risk for exposure to COVID-19. This included following up on all the relevant guidelines for protection as advised by World Health Organisation (WHO) and GoB. To ensure the safety of all the stakeholders involved, DMA (NRMC’s local partner in Bangladesh) carried out the data collection under the virtual guidance, involvement and close monitoring of the core evaluation team (based in New Delhi). This was facilitated by the availability of all necessary permissions for data collection from the district/upazila administration. 126. Considering that the schools were closed, for the purpose of sampling of respondents, the contact details of respondents (parents, teachers, headmasters and district level officials) was shared with the core evaluation team (NRMC). NRMC ensured that the privacy of data provided by WFP and its partners was protected and there was no breach of confidentiality. 127. In order to ensure that the students were not exposed to risk from COVID-19, students were excluded from the respondent’s category (for interviews and FGDs). Given their exclusion from the data collection process, for certain indicators, data from the MTE has been considered for the purpose of reporting in the end-term evaluation. These indicators have been highlighted in the table in Annex M. The other categories of respondents namely, parents, school teachers, headmasters and district level officials were interviewed during the primary data collection phase. 128. Evaluation Criteria and Questions: The evaluation through the lens of relevance, efficiency, impact, effectiveness and sustainability attempts to (a) assess and report on the performance and results of the FY17 project (with reference to the baseline and mid-term values) and (b) draw lessons for learning – identifying reasons why certain results occurred or not. It is important to highlight here that while doing so, the evaluation has considered the impact of COVID -19 on programme implementation and achievement of desired outputs. 129. The evaluation also attempted to highlight beneficiaries’ perspective and experience (excluding students) with regards to the programme activities and results yielded. Gender equity and inclusion was mainstreamed throughout the evaluation and the unintended impact of the programme on gender dimensions were also assessed. The evaluation matrix (Annex L) highlights the key information areas mapped with the end-term evaluation questions, the approach for data collection and data sources. Wherever appropriate71, gender dimensions have been factored into the sub-questions/key information areas for each evaluation question. Table 2 presents the key questions that were examined through the end￾term evaluation. Table 1: MGD FY17 end-term evaluation questions Key Questions Relevance  Did the project reach the intended beneficiaries with the right mix of assistance?  Is the project aligned with national government’s policies and strategies including education, school feeding, safety net and national five years plan? Considering the COVID-19 71 SFP by design does not have a specific focus on gender related issues and therefore the evaluation will highlight gender dimensions wherever appropriate. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 13 context, did the project adapt /realign its activities to support GoB in responding to the challenges?  Is the project perceived as relevant in addressing the needs of target population? Is the project perceived to be relevant during a pandemic such as COVID-19?  Does the project complement other donor-funded and government initiatives? Effectiveness and Efficiency  To what extent did the outputs lead to the realization of the expected results and outcomes – were the set targets achieved?  What were the contributing and impeding factors in achieving the expected results?  In what ways was the project affected as a result of COVID-19?  What effect did the COVID-19 crisis have on utilisation of resources (financial as well as human capital)? What was the effect of reallocation (if any) on the project’s implementation and results?  What was the output and were all the project activities carried out as planned? Reasons for deviation, if any  What was the efficiency of the programme, in terms of transfer cost, cost/beneficiary, logistics, and timeliness of delivery?  How efficient were the operations and approach in terms of capacity building of government towards eventual handover? Impact  How did the project contribute to observed impacts?  Have there been any unintended outcomes, either positive or negative? Have the outcomes been specifically affected by COVID-19?  What internal and external factors affected the project’s ability to deliver impact?  What are the spill-over effects of the project?  Are local communities fully involved in and contributing towards the school feeding programme? Sustainability  What remains to be done in order to achieve a full handover of the National School Meals Programme?  How is the exit strategy affected by COVID-19?  In the context of a pandemic, how does WFP plan to ensure sustainability of the project?  What changes (if any) have been made to the timelines and modalities in the process of handing over to GoB, after the advent of the pandemic? General  What are lessons learned from the project? What were the learnings specifically from implementing a project during a global pandemic?  How can WFP improve future programming, in the context of these lessons learned? 130. To assess the relevance of the programme, the evaluation gathered evidence on whether the project was aligned with the national priorities, strategies and the national five-year plans. Additionally, the evaluation examined whether the design and interventions of the project were responsive to the local needs of the most vulnerable groups and gender inclusive. The evaluation also examined if and how the project adapted/realigned its activities to support GoB in responding to the challenges presented as a result of COVID-19. The evaluation further assessed the complementarity of interventions with other donor funded and government initiatives. Through primary and secondary data, the evaluation also identified whether local gender issues were integrated into the implementation strategy and whether the right mix of assistance was provided to the beneficiaries. 131. To assess the effectiveness of the programme, the evaluation team examined the extent to which the outputs led to the realization of the expected results and outcomes and the key End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 14 enablers and barriers (internal and external) that led to the same. The targets for performance indicators have been presented in Annex B. 132. Towards this, the evaluation assessed and compared the values on certain indicators such as enrolment, attendance, attentiveness, availability of school infrastructure (toilets and water facility) and health-related absenteeism by comparing the baseline and end-term values. However, considering the exclusion of students from the respondent category, for indicators such as attentiveness, the mid-term values have been considered for end-term evaluation. 133. Considering that the schools were closed on account of COVID-19 (since mid-February 2020), for indicators such as attendance and biscuit distribution, data was collected for the months of September and November 2019, as those months had the least number of holidays72. 134. The impact of the project was also assessed in terms of improved learning outcomes, improved dietary diversity and adoption of better health and hygiene practices. Furthermore, unintended impacts of the project were assessed and the internal and external factors that led to the impact were also be examined. On indicators such as demonstration of reading and understanding by students (Grade III), a comparison of the baseline and MTE values was considered since it was not possible to conduct the Early Grade Reading Assessment (EGRA), in the absence of children as a respondent category. The mapping of indicators along with the source of information is provided in Annex M. 135. The evaluation assessed the efficiency of the project with regards to timely delivery of biscuits and processes of implementation including the capacity building activities and fund utilization based on submission of periodic monitoring reports. The effect of COVID-19 on utilisation of resources (financial as well as human capital) and implementation of project’s activities were also assessed. 136. To assess sustainability, the evaluation highlighted the ownership and commitment of GoB and school officials to independently take forward various interventions of the programme. The evaluation attempted to understand if and how the exit strategy has been affected by COVID-19. 137. For the end-term evaluation, while there were equal weightages on each of the OECD-DAC criteria, the focus has been on impact and sustainability. 138. Through the use of qualitative tools, the evaluation identified reasons that led to the progress or affected the performance of the project. Further, it highlighted the perceptions of the programme beneficiaries (such as parents and SMC members) and their overall experience with regards to the services provided as part of the project. The perception of parents and experience of school authorities with regard to biscuit distribution and the essential learning package (ELP) which includes activities such as school gardens, little doctors, read play festival were also be examined. This helped the evaluation team understand the enablers and barriers for the achievement of desired outcomes. 139. The baseline and mid-term evaluations had made recommendations that specifically related to designing and strengthening a community engagement strategy for improving community participation; supporting GoB to plan for implementation of National School Meal Policy and strengthening of the local capacities especially that of SMC and the community. The end-term evaluation therefore, assessed the extent to which the recommendations from the last evaluations had been implemented. 2.1.1 Data Triangulation 140. The methodology entailed a desk review of the project and other related documents and a primary survey which included both quantitative and qualitative (key informant interviews and focus group discussions using virtual engagement) methods. An attempt was made to triangulate data from the primary survey (comparison of perspectives of different 72 As indicated by the school headmasters and district level officials during the baseline and mid-term evaluation, in the month of December students appear for final exams and attend schools only on the day of examination. In January, the attendance is not regular due to new enrollments and delay in joining by the students, post session break. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 15 stakeholders on the same issue) to assess the reliability and validity of the data (refer evaluation matrix in Annex L). However, considering that (i) students as a respondent category were not covered; (ii) there were no school visits and classroom observations, triangulation of data was only possible in a limited manner. Considering that the schools had been closed for over six months and that the reporting on the students’ absence (by parents) was based on recall (September 2019 and November 2019), triangulation with the school attendance data was not possible. Hence, complementary observations and comments made by all stakeholders were taken into account in the final analysis and the preparation of the evaluation report. The findings were synthesised to determine the achievement of the performance indicators, identify the enabling factors and barriers, understand the perceptions and experiences of the beneficiaries towards the project services, examine contribution towards government’s agenda, innovations, learnings, good practices and key considerations for scaling up. 141. A summary of the methodology presenting the method of data collection, the target groups/sources of information, the nature of the interviews, and the indicative outputs is presented in the table below. 142. The structured interviews with parents, school heads, storekeepers and teachers was conducted by the DMA staff (with the virtual presence of the NRMC team). In case of personal interactions between the respondents and our local partners, adoption of all COVID-19 related safety measures and precautions was ensured. The data collection team strictly followed WHO guidelines and those issued by the local government, practicing social distancing and going through regular temperature screen tests. They were also provided with a COVID-19 protection kit for data collection, which included masks, hand sanitizers, face shields and gloves. The FGDs/KIIs with parents, SMC members and KIIs with government staff and implementing partners was conducted by the NRMC core team, virtually (from Delhi). The DMA team assisted NRMC core team in connecting with the participants virtually, through a video call. A summary of the evaluation methodology has been attached in Annexe H. 2.1.2 Secondary Data 143. A systematic review of literature was carried out to build the context of the SFP and its results. Additionally, the evaluation team referred to WFP’s COVID-19 related guidelines and strategy for the project to understand the changes in the implementation strategy of the project, induced by COVID-19. Review of WFP’s Strategic Plan 2017-20, the NPAN-2, GoB’s Draft National School Feeding Policy and WFP’s Gender Policy allowed the evaluation team to identify linkages and alignment of the SFP with WFP’s mandate and the government’s priorities. Combined with these, the desk review also examined the findings from the FY17 baseline and mid-term evaluation, national-level secondary data as well as all WFP monitoring data to build the contribution of the programme around MGD SO1 and SO2. 2.1.3 Primary Data 144. The primary data collection for the end-term evaluation was done partly through virtual engagement, wherein the qualitative discussions with government stakeholders and implementing partners was done over virtual platforms and the quantitative data collection and FGDs was done physically by our partner (DMA) in Bangladesh. 145. The quantitative questionnaires presented in Annex M covers key programme outcomes such as literacy enrolment, disease-related absenteeism of students, dietary diversity score, and status of availability of WASH facilities in schools. 146. The qualitative tools developed for the survey are also presented in Annex M. The qualitative tools focused on understanding beneficiaries’ experience (directly and indirectly) with the project activities as well as getting their views on the impact of the programme. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 16 147. As stated earlier, the evaluation team collected data (both qualitative and quantitative) from only parents, head teachers, teachers, SMCs and government stakeholders. During the mid￾term evaluation, direct interactions with students provided their personal perspective about biscuits, illness-related absenteeism, home and class learning environment and the programme’s effect on their education and nutrition. For the purpose of the end-term evaluation, these perspectives were explored indirectly through interactions with parents and teachers. The structured tools were administered with the following respondents: a. Parents: Parents of the sampled school going children were interviewed to collect data on receipt of biscuits, their awareness about project activities and contribution to the project, the dietary diversity of the children, the educational environment at home, their child’s and their own awareness on child health and nutrition practices, adoption of these practices. Both quantitative and qualitative tools were administered with parents. b. School-based respondents: A questionnaire for the school was developed which gathered information through discussions with the headmaster on aspects such as enrolment data, attendance data (for students for the months- September 2019 and November 2019), infrastructure, maintenance, biscuit utilisation, and specific questions on the provision of biscuit. In addition, questions related to classroom libraries, school gardens, little doctors and WASH were also been included. Further, a questionnaire was developed for the headmaster to capture the roles and responsibilities played by the school administrators. The other respondents from schools included school teachers, and storekeepers. The mapping of the questionnaire with the indicators is provided in the form of a table in Annex M along with the tools. Discussions were also undertaken with SMC members to understand their involvement in school related decisions. c. KIIs were held with government officials at the national and district level, focusing on their engagement with the implementation of the FY 17 project and SFP, the contribution of WFP in augmenting their capacities to implement SFP, the effect of COVID-19 on the FY-17 project as well as the National School Meals Programme and the support required to ensure implementation of activities independently and in a sustained manner. The discussion guide for the capacity building component with the representatives, GoB is provided in Annex M. d. Representatives, WFP CO: In order to further understand the challenges faced during the implementation (especially in context of COVID-19), adaptations and modifications in the implementation plan, success stories, and learnings, in-depth discussions with representatives of WFP CO were conducted during the data collection phase. These were carried out virtually by the NRMC core team. The discussion guide for the capacity building component with the representatives, WFP is provided in Annex M. e. KIIs with representatives from RtR and RIC were also conducted virtually to better understand their role and engagement in the programme. 2.1.4 Site Mapping 148. The end-term evaluation covered the diversity of geographical locations of the schools. The advantage of this approach was that the intensity of the evaluation vis-à-vis community response in diverse geographies increased, adding value to the data. Moreover, the information gathered was more nuanced and retained the robustness of the evaluation design. This ensured upholding both WFP’s mandate and the government’s concerns. There were no foreseen drawbacks to this approach. Considering that the data collection was undertaken through a virtual engagement approach, identification of schools were based on (i) availability of internet connectivity; (ii) willingness of the school headmaster to participate in the data collection process; (iii) prevalence categorisation of location (affected by COVID￾19) as green or high-risk red zones. 149. The sample for end-term evaluation was spread across two sub-districts (Ukhiya and Kutubdia) of Cox’s Bazar. Following the approach adopted during baseline and mid-term, 50 schools were sampled. Of these, 30 were intervention schools. The sample of the schools were End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 17 spread across the districts in the same proportion as the number of programme schools present in the districts, thus ensuring adequate statistical representation of the sample. The schools were selected using simple random sampling, in line with the methodology followed during baseline and MTE. The list of sample schools is presented in Annex K. 150. For the comparison group, 20 schools were sampled from Ramu (same district, different sub￾district/upazila), with their consent to participate. 2.1.5 Sampling 151. The sample size was calculated at the project level, using the ‘differences method’ formula with a finite population (confidence interval of 1.96, estimated difference set at 5%). The overview of the sampling has been provided in Table 3 below and a detailed sampling protocol is presented in Annex J. 152. As also mentioned above, to limit the exposure of risk to students, the evaluation team did not interact with children directly and relied on the responses of other stakeholders to measure change. 153. By allocating schools as per their probability proportionate to size (i.e. the total number of GPS in the two Upazilas), 12 schools in Kutubdia and 18 schools in Ukhiya were selected randomly. In each school, the school administrator (through DMA) was requested to inform the teachers and parents in advance about the virtual survey to ensure availability of logistical support and their consent. 154. From amongst the sampled schools, 20 percent of the schools were identified for the purpose of conducting FGDs. FGDs were conducted in open spaces following all COVID-19 protection measures. Table 2: Sampling Quantitative (Structured interviews) Details Intervention Comparison No. of schools 30 20 Parents 5 per school 150 100 Total (End-term) 180 120 Quantitative (KII) School Head 1 per school 30 20 School Teacher 1 per school 30 20 Store keepers 1 per school 30 Total (End-term) 90 40 Qualitative – Key Informant Interviews (KII)  Director, Field Service Wing, Department of Agricultural Extension (DAE), Ministry of Agriculture 1  Senior Trainer, BIRTAN 1  Project Director (Additional Secretary), School Feeding Programme in Poverty Prone Areas, Directorate of Primary Education (DPE) 1  WFP staff at Dhaka (Country Office) 4  WFP staff at Cox’s Bazar 2  Project Manager, Room to Read 1  Project Coordinator, RIC 1  District and Upazila Education Officers 4 Total (End-term) 15 Qualitative- Focus Group Discussions (FGD) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 18 Mothers 6 4 SMC members 6 4 Total (End-term) 12 8 2.1.6 Data Analysis 155. Primary quantitative and qualitative data was analysed using the questions set out as per the evaluation matrix and the relevant themes identified under the conceptual approach. The qualitative data has been analysed using content analysis, wherein the focus has been on providing the reasons for the changes observed. The content of the FGDs and the IDIs was classified under the thematic areas of the evaluation and their associated indicators. Within each category, sub￾categories dependent on the qualitative information obtained from the field were generated. Emerging trends were then noted from the analysis of the main and sub-categories and assessed in response to the evaluation questions. The evaluation also aimed to highlight the best practices, challenges, and learnings to inform the programme. 156. The quantitative data was analysed using SPSS software post a rigorous review of any errors in the data. In addition, secondary data from relevant sources and qualitative findings was used to validate the findings from the quantitative data. 157. Overall, a triangulation of the quantitative data, qualitative data and project documents was conducted to present conclusions on overall trends and patterns. Wherever important, viewpoints were illustrated as quotations from relevant stakeholders. 158. The data analysis aimed to highlight how the values established during the baseline study and mid￾term evaluation changed, the progress made and the results achieved. The data analysis also provides a descriptive analysis of sex disaggregated data setting the benchmark values in treatment and comparison schools and allows for ensuring the end-term evaluation to estimate the extent of change that is attributable to the project. 159. The raw data obtained from the field has been checked by the data analyst for consistency errors, duplicity of cases and missing data. Most of these errors were already minimised at the stage of software development process for Computer Aided Personal Interview (CAPI) enabled data collection. Outliers in the quantitative data were triangulated with the qualitative information to assess the validity of the data point in the outlier. These outliers were noted and highlighted during the analysis along with the associated qualitative observations. 160. The analysis of qualitative data helped in understanding the reasons that have contributed to changes in progress indicators established during the baseline study. The qualitative analysis has also been used to understand the perceptions of beneficiaries about the services they are receiving under the programme, the value they see in these services, their views on what is missing from the school feeding programme and their overall experience with it. Perceived enablers and barriers by the SMCs and school administrations in implementing the programme objectives have also been highlighted through this analysis. Furthermore, best practices and learnings have also been captured through KIIs and FGDs. 2.2 Gender dimensions of the end-term evaluation 161. For the evaluation, gender analysis was considered as a prerequisite to ensure that the design, planning, implementation, monitoring and evaluation of the programme take into account the existing gender disparities. The evaluation has incorporated inputs from the Technical Note on Integrating Gender in WFP Evaluation, 2019 and Gender Equality and Empowerment of Women (GEEW) framework to adequately focus on the gender dimensions within the programme. The design of the evaluation (methodology and the analytical framework) has taken into consideration the United Nations Evaluation Group (UNEG) indicators, specifically relating to gender equality and human rights. Additionally, separate FGDs with mothers and fathers was conducted. It was ensured that only female enumerators interacted with the women being interviewed so as to End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 19 respect the cultural norms and ensure sensitivity. Primary data on progress against outputs and outcomes has been disaggregated by gender. 162. The evaluation is guided by WFP’s latest Gender Policy (2015-20). Aligning with these principles, we understand that development programmes affect women, men, boys, and girls differently. Hence, the evaluation emphasises on testing the programme’s coherence with the national policy on gender, and operates inclusive strategies and implementation based on a sound gender analysis. The evaluation focusses on data disaggregated by sex and age, providing empirical insights that can be used and built upon by the users. However, it is important to point out that the SFP targeted girls and boys equally but did not have any specific indicators to measure the impact of the programme on women and girls. The evaluation did consider the impact of the programme on women and girls specifically and analysed the affect it had in terms of girls’ attendance, enrolment, access to toilets and other facilities, literacy outcomes as well as women’s participation in SMCs. 2.3 Limitations and Risks 163. The implementation of the programme activities commenced from October 2018. Due to closure of schools since February 2020 and postponement of activities and re-alignment of implementation strategy due to COVID-19 prior to the commencement of data collection (December 2020) for ETE, the evaluation may not highlight/report significant change/impact of the programme’s activities. 164. The evaluation measures the attentiveness of children based on information from their school teachers, which may be biased based on the teacher’s perception of the child and may be affected due to recall bias. Attentiveness of a student is a function of different variables such as short-term hunger, poor teaching methods, and lack of interest in the subject matter being taught in class, and incidents at home, among others. Given that the teachers did not have direct interactions with students over a significant period of time, the accuracy of the data could not be validated. 165. Given that students were not sampled for the end-term evaluation, data on significant indicators had to be captured through parents. Questions on oral fluency and comprehension could not be asked as the EGRA tool was not administered. The reporting on indicators for literacy outcomes therefore, has been done based on the data captured during the MTE and supplemented through the findings of the qualitative discussions. 166.Due to the unavailability of the teacher’s attendance data, in accordance with the circular from GoB, the evaluation does not report any findings on the related indicator (custom indicator 1.1.1). This was discussed and agreed upon with WFP. 167. In order to ensure availability of respondents, students’ attendance records and biscuit distribution records, the visit plan was shared with schools prior to the commencement of survey. There is a possibility that with prior information, schools were better prepared and the observations that the evaluation team made may not reflect the reality. Triangulation, where appropriate, by comparison of perspectives of different stakeholders both quantitatively and through FGDs helped in addressing this limitation. 168. Though the SFP does not have a specific focus on gender related issues, an attempt has been made to integrate Gender Equality and Empowerment of Women (GEEW) throughout the evaluation. However, unlike the usual practice of sampling of students to ensure equal representation of boys and girls, only the available school-level data was disaggregated by sex. Parents of equal number of boys and girls were sampled to ensure equal representation in the absence of students as a respondent category. The study also examined the status of participation of women during SMC meetings. Data obtained for the gender-related questions has been analysed comprehensively to report on the gender dimensions of the evaluation to the extent possible. Data on attainment of outputs and outcomes has also been disaggregated by gender. 169. Given the precarious context in which the data collection had to be done, there was a huge risk of contracting or transmitting the COVID-19 virus. In order to limit exposure and prevent this from happening, all safety protocols and guidelines as advised by WHO was followed strictly in the field. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 20 2.4 Ensuring Quality and Adherence to Ethical Standards 170. WFP is closely aligned with UNEG norms, standards and good practices of the international evaluation community to ensure that the evaluation process and products conform to the same. These standards and guidelines guided the evaluation team at NRMC. The quality assurance expert at NRMC ensured that the prescribed quality protocols were integrated within the data collection process to obtain reliable data and ensure the quality of deliverables. Some of the key steps taken to ensure quality at various stages of the survey have been mentioned in Annex Q. 171. The evaluation team made sure that no data was collected without the consent of the participants or approval of the concerned authorities. For this, a verbal consent was sought from all respondents. Further, the participants were informed about a) their choice to decline their participation in the interview at any time and b) about their rights to confidentiality, anonymity, prevention from any harm and access to information about the evaluation. Special care was sought in ensuring ethical behaviour and understanding in data collection. Respondents, especially women were spoken to in a manner that was sensitive to their culture and did not cause intimidation, for which the enumerators were specially trained. It was ensured that all women (mothers) were interviewed by women enumerators to make them comfortable. During the survey, it was ensured that the evaluation team did not ask any questions or pose any cross￾questions that were personal or sensitive, or that might physically, mentally or emotionally harm the respondent. Moreover, informal/casual discussions were held with parents, teachers, SMC members and headmasters prior to administering any tool, to build rapport and ensure their comfort and ease. Considering that the evaluation team was interacting with people from the host communities (given the huge influx of Rohingya in Cox’s Bazar), all matters related to access, availability and affordability of goods and services were discussed keeping the sensitivity of the context in mind. All interactions took place in Bangla so as to reduce the possibility of data loss due to language barriers and ensure ease of discussions. The evaluation team ensured that all the enumerators spoke the local language and understood the local dialect for the same reasons. The evaluation team also consisted of Bengali speaking members and the discussions with the government officials also took place in Bangla to avoid any issues in understanding and articulation due to language barriers. 172. Some other measures to taken to ensure quality data collection were as follows: a. The data collection team comprised of personnel well-experienced in collecting and collating both quantitative and qualitative information. Separate teams were made for each tool/questionnaire to ensure that the tasks were clearly assigned and the teams were well￾versed with their respective tools. KIIs with the government officials, WFP field office staff and partners were conducted by the NRMC core team (details provided in Annex Q) to ensure the quality of discussions with these stakeholders. b. As part of quality control and to ensure timeliness of collection of data, NRMC developed a detailed field movement plan in advance of the survey and strictly followed it with the help of DMA. There was no delay in the planned data collection despite the restrictions posed DUE to COVID-19. This was further ensured by the virtual platforms leveraged for the timely completion of the data collection process. c. A robust monitoring process was followed for quality assurance during data collection. At least one member of the core evaluation team of NRMC was present virtually in the field during the entire period of data collection. d. Regular updates regarding the progress of the evaluation were provided to the Evaluation Manger, WFP – CO by the evaluation team. The team members followed quality norms prescribed by WFP. 3. Evaluation Findings 173. This section presents the field findings and assesses the relevance, effectiveness, efficiency, impact, and sustainability of the programme with gender equity and inclusion mainstreamed throughout. The findings have been reported based on the data collected from both primary End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 21 survey and secondary review. For the primary survey, quantitative and qualitative data from the stakeholders were triangulated before converging into the overall findings. Wherever appropriate, gender dimensions were factored into the sub-questions, judgement criteria, and indicators for each evaluation question. Snapshot of various co-curricular activities undertaken as part of SFP Book Captains: Students selected to issue books to their peers from the Classroom Libraries provided by Room to Read. They receive a badge, cap and t-shirt in order to distinguish them from other students. Little Doctors: Students selected to assist in de-worming drives and health check-up (weight, height and vision test) including disseminate health and hygiene related messages to their peers in assemblies, classrooms etc. They receive a white apron in order to distinguish them from other students. Vegetable Garden: A garden established in the school premises where vegetables and fruits are grown. The purpose is to teach students about the importance of nutrients and promote nutritious dietary practices through practical demonstration. Little Agriculturalists: Students selected to be in-charge of the vegetable gardens and disseminate nutrition and safe storage related messages to their peers. Read Play Festival: A community event to encourage game-based learning. Students play word making, reading and spelling games and so on. Healthy Meal Preparation Day: A community event to demonstrate nutritious and healthy cooking practices. Mothers of the school students cook for this event using vegetables from the school vegetable garden. Discussions around nutritious properties of food are also held. Grade 1 Reception Day: An event hosted by schools to welcome Grade 1 students and felicitate their parents for enrolling their child in the school. The aim is to encourage enrolment through recognition. 3.1. Relevance Evaluation Question 1: Did the project reach the intended beneficiaries with the right mix of assistance? Evaluation Question 2: Is the project aligned with the national government’s policies and strategies including education, school feeding, safety net and national five years plan? Evaluation Question 3: Is the project perceived as relevant in addressing the needs of target population? Is the project perceived to be relevant during a pandemic such as COVID-19? Evaluation Question 4: Does the project complement other donor-funded and government initiatives? 174. Considering the poor performance of Cox’s Bazar on indicators of NER and dropout rates73, fortified biscuits are one of the means to improve attendance and attentiveness of students. This was validated by the views of parents during FGDs, where it was clearly articulated that distribution of biscuits acts as one of the triggers for students to attend school regularly. Furthermore, during discussions, parents and SMC members in sampled intervention schools cited various instances of community members and other stakeholders coming together to aid the implementation of the SFP activities, especially for door to door biscuit distribution. The fact that the community members took ownership and contributed to the SFP activities during the COVID￾19 pandemic (when standard procedures could not be followed) gives testimony to their perceived value and relevance of the project. Discussions with Upazila Education Officers (UEOs) 73 Details presented in the section on context End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 22 of Ukhiya and Kutubdia also confirmed that they found the initiative to be useful and relevant. It can therefore, be inferred that the biscuit distribution intervention is relevant. 175. Despite the improvement in enrolment rates at the national level, quality of education continues to remain a challenge (as indicated in the World Development Report 201874), even in 2020.75 The key influencing factors affecting the poor results in Bangla (including reading and comprehension) and Mathematics are (i) lack of access to early childhood development programmes, (ii) low quality of teaching practices, (iii) challenges related to poor school management, and (iv) low levels of overall spending on public education. 176. In this context, SFP’s implementation strategy that comprises of (i) provision of trainings to teachers and headmasters on new teaching techniques, library management and effective school management and (ii) establishment of reading corners to improve Bangla reading comprehension, find its relevance. Community participation events such as “Read-play festival” and “Grade I reception day” celebrated as part of SFP also contributes to the relevance of the programme since it aims to address challenges related to awareness around attendance, enrolment and literacy and provides platforms to encourage the same. 177. Discussions with parents revealed that there is a felt need for interventions to increase children’s interest in education related activities, to improve their performance and to ensure an enabling learning environment in the schools. Through interventions such as the Read Play festival as well as Classroom Libraries, parents believe that the programme is addressing the need for improving the quality of education. Such interventions were appreciated as it was not only contributing towards the enhancement of children’s reading fluency and comprehension skills but also increasing their interest in education. 178. Bangladesh also has a budding private tutoring industry.While the government attempts to outlaw private tutoring, this trend continues to grow especially in the context of COVID-19 when schools are closed. This can often place children from low-income households at a disadvantage since their parents are unable to afford such services76. Therefore, SFP’s efforts to i) build capacities of teachers on new teaching learning techniques, ii) provide teaching learning materials and aids and iii) ensure continuation of education at home during COVID-19 through home visits, online platforms such as Facebook and TV channels are found to be relevant as they reduce dependence on private tutors and ensure equal access to learning for all students. 179. However, according to the Interim Education Watch Report 2020-21 (released on 18th January 2021)77, 57.9 percent of students did not join online classes due to a lack of devices or access to the internet. This was also validated during discussions with SMC members where they mentioned that many students discontinue their education because their parents cannot afford a smartphone or TV. Therefore, while SFP’s efforts to promote students’ engagement with online education through home visits (for monitoring) are helping some students continue their education during COVID-19, a large proportion of students remain unable to access digital platforms. 180. The SFP’s efforts towards rehabilitation and construction of water systems and toilets in schools address the challenge of availability of functional toilets in schools, as highlighted in the baseline study as well as the latest JMP report.78 Sensitisation of teachers, parents and students on health and hygiene practices planned under SFP further creates awareness and influences the target population to adopt good health and hygiene practices. Hence, the implementation strategy adopted by the programme, adequately targets the right people with the right type of assistance and was therefore, found to be relevant. 74 Details mentioned in the section on context 75 https://thefinancialexpress.com.bd/views/improving-quality-of-education-main-challenge-1580309749 76 https://wenr.wes.org/2019/08/education-in-bangladesh 77 https://www.ucanews.com/news/digital-divide-forces-poor-bangladeshi-students-out-of-school/91122# 78 Details presented in the section on context End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 23 181. In addition to this, the SFP’s geographical focus on Ukhiya and Kutubdia is found to be relevant, keeping in mind the context of the Rohingya refugee influx in Cox’s Bazar, high rates of malnutrition in the district of Cox’s Bazar as well as the findings of the study by WFP’s Vulnerability Assessment and Mapping (VAM) Unit79. 182. While Bangladesh has made significant progress in reducing malnutrition and ensuring food security80, the advent of the COVID-19 pandemic has posed new challenges in this regard. Food insecurity and poor food consumption have proved to be pressing concerns during this crisis, as also indicated in the study conducted by UNHCR as well as the Joint Multi-Sector Needs Assessment (J-MSNA) 2020.81 In this context, SFP’s implementation strategy including i) distribution of fortified biscuits, ii) awareness generation on cooking and consumption of nutritious food and iii) promotion of vegetable gardens contributes to the relevance of the programme as it aims to address challenges of hunger, food insecurity and poor food consumption. 183. In the context of the COVID-19 pandemic, the needs of the community have also evolved and changed. In order to address these changing needs, WFP modified its implementation strategy to include more home visits82, door-to-door distribution of biscuits and capacity building of teachers and SMC members to aid the implementation of the programme and enable last mile delivery. This adaptation of the implementation strategy was responsive to the evolving needs of the community and therefore helped reach the intended beneficiaries, ensuring the relevance and responsiveness of the programme to changing contexts. 184. SFP’s technical support to the Little Doctor initiative contributes towards strengthening GoB’s initiative by augmenting the capacities of little doctors to distribute deworming tablets, conduct height, weight and eye check-ups and disseminate health and hygiene related messages to peer students. 185. The National Education Policy (NEP), 2010 articulates GoB’s approach to support ‘education for all’, the eradication of illiteracy and improvements in the quality of education. SFP’s activities such as (i) creating awareness on the importance of education, (ii) developing new teaching techniques and tools and providing training to teachers on the same, (iii) establishment of class room libraries to promote a reading habit among students and; (iv) providing support to GoB in strengthening school governance structures are aligned with Bangladesh’s NEP (2010) and supports MoPME in achieving its objectives. 186. GoB’s flagship programme in the education sector (PEDP4) aims at providing pre-primary to grade V students quality education, with key emphasis on efficiency, inclusivity and equity from 2019- 2023. Towards this, the support provided through SFP in terms of providing training to teachers on new teaching techniques is aligned with PEDP4. 187. The NPAN-2 has highlighted the need for and focus towards (i) social protection programmes that include school meals and school feeding, (ii) strengthening multi-sectoral programmes towards ensuring nutrition and increasing joint efforts and coordination among ministries with regard to social safety nets, education, water, sanitation and hygiene, (iii) strengthening/integrating nutrition education in regular curricula of primary schools and scaling up school health, school feeding and school gardening programmes. Therefore SFP’s activities on (i) distribution of fortified biscuits; (ii) supporting vegetable gardens; and (iii) creating awareness on health, hygiene and nutrition, aligns with NPAN-2. Furthermore, the SFP also supports MoPME in reporting its contribution in the joint efforts of various ministries for implementation of NPAN-2. 188. Additionally, the NSSS (2015) acknowledged the need to scale-up and increase outreach, access and coverage of existing safety net programmes. The implementation strategy of adopting a saturated approach in the two upazilas ensures improved coverage of SFP. 79 Details presented in the section on context 80 Shoba Suri, “Breakout Nation: The Nutrition Transformation of Bangladesh”, ORF Issue Brief No. 323, November 2019, Observer Research Foundation. 81 Details presented in the section on context 82 Home visits followed all WHO COVID-19 protocols and guidelines to ensure the safety of students and families. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 24 189. In light of the findings above, it can be inferred that the programme is well aligned with the national government’s priorities, policies and strategies. 190. Besides WFP, other development partners such as UNICEF, UNHCR, USAID, BRAC and Save the Children are also implementing programmes in Cox’s Bazar. These partners are primarily working for Rohingya refugees on aspects such as food distribution, health, child protection, water and sanitation, among others. World Bank is providing support to scale up access to energy, water, sanitation services and disaster-resilient infrastructures for the Rohingya as well as the surrounding host communities. For the host community, UNICEF is supporting the creation of child friendly schools and early childhood development centres (for preschool children aged 3-6 years). Provision of nutrition support services is one of the important support measures to early childhood development centres. SFP, through its provision of fortified biscuits to students and complimentary education interventions, extends the development partners’ support to GoB from early child development centres to primary schooling. The SFP activities were therefore, found to be complementing other donor-funded and government initiatives. 3.2. Effectiveness Evaluation Question 4: To what extent the outputs led to the realization of the expected results and outcomes – were the set targets achieved? Evaluation Question 5: What was the output and were all the project activities carried out as planned? Evaluation Question 6: In what ways was the project affected as a result of COVID-19? As per the Project Level Results Framework (attached in Annex B), the end line evaluation reported on a total of 35 indicators. The targets for all of these indicators have been indicated in Annex C as well as in Annex T along with the project’s progress from baseline to end-term. Out of the 35 indicators identified, 1 indicator i.e. Percent of teachers in target schools who attend and teach school at least 90 percent of scheduled school days per school year was excluded from the effectiveness section83. Additionally, owing to methodological limitations due to COVID-19 (refer section 2.3) data from the MTE has been considered for the purpose of reporting on 6 indicators in the end-term evaluation. These indicators have been highlighted in the table in Annex M. Of the remaining 29 indicators, it was found that the target was achieved for 14 indicators and was not achieved for 15 indicators. However, of these 15 indicators, there were 7 indicators where more than 80 percent of the target was achieved. Therefore, it is inferred that in a case where COVID-19 restrictions were not present, the programme may have been able to achieve its targets on 7 more indicators. A colour coded representation of the status of target achievement for each indicator has been presented in Annex T. Support Improved Literacy and Improve Student Enrolment and Attendance Number of textbooks and other teaching and learning materials provided as a result of USDA assistance 191. The survey findings indicate that all sampled intervention schools have received textbooks from WFP as a result of USDA assistance. The periodic monitoring report84 highlights that 2,60,728 textbooks and learning materials have been provided to all intervention schools. Additionally, WFP is also providing training to teachers and headmasters with an objective of improving literacy outcomes in students. Number of classroom libraries (classroom bookshelves with books) established as a result of USDA assistance 83 Refer to Section 2.3 Limitations and Risks 84 WFP Semi-annual report April 2020-September 2020; Source: WFP. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 25 192. Existence of classroom libraries for each grade was reported in all the sampled intervention schools. In case of shared classrooms, separate libraries for each grade were observed. These classroom libraries have books that are colour-coded based on the difficulty levels of reading. 193. The presence of classroom libraries and the system of book captains was confirmed through discussions with parents. Parents also reported their children (students) reading stories out to them at home. In fact, some parents mentioned that students were learning more from story books than from their regular text books. They also expressed happiness about the fact that their children could now read better and even write stories of their own due to exposure to these story books. 194. During the period of SFP, WFP a) built the capacities of book captains and teachers on library management, b) provided supplementary books and uniforms for book captains, c) facilitated regular monitoring of students’ reading and comprehension abilities. Discussions with the headmasters and teachers provided an affirmation on such support received by the intervention schools. In addition to this, WFP initiated a separate Supplementary Reading Material (SRM) period for children, wherein teachers and sometimes Literacy Facilitators (LFs) interacted with students, and helped them read books available in the library. 195. The analysis of findings at a disaggregated level (on the basis of gender) cannot be presented as the primary data from students could not be collected85. 196. Discussions with parents confirmed issuing of supplementary reading materials by students from the classroom libraries on a regular basis (weekly) in the intervention schools. During discussions, parents in Ukhiya were able to better highlight the functioning of classroom libraries and the roles and responsibilities of book captains as compared to Kutubdia. In the comparison schools of Ramu where no classroom libraries were established through USDA assistance, discussions with parents revealed that while pre-primary students had access to story books in their schools, students of Grade 3, 4 and 5 did not engage in reading books other than their textbooks at all. When asked, parents mentioned that there is definitely a need for story books for primary students as well, as they too may learn better with the help of pictures and stories. A clear need for supplementary approaches to aid learning of students emerged from these discussions in Ramu. This further establishes the effectiveness of initiatives under SFP such as establishing classroom libraries to promote reading and comprehension abilities. Number of school administrators and officials trained or certified as a result of USDA assistance Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance 197. The survey findings indicate that 20 head teachers and 113 teachers in the sampled intervention schools have been trained on new teaching techniques as a result of USDA assistance in the last year of the programme. At a disaggregated level (based on gender), 15 male head teachers and 5 female head teachers were trained. 43 male teachers and 70 female teachers have been trained in the sampled schools as a result of USDA assistance. At a project level86, 97 head teachers and 550 teachers have been trained on new teaching techniques as a result of USDA assistance in the last year of the programme. This falls short when compared to the target set out for training of head teachers (146) and teachers (730). This may be on account of closure of schools since March 2020 (as a result of the pandemic), due to which trainings had to be postponed. Number of school administrators and officials in target schools who demonstrate the use of new techniques or tools as a result of USDA assistance Number of teachers/educators/teaching assistants in target schools who demonstrate the use of new and quality teaching techniques or tools as a result of USDA assistance 85 To prevent exposure of children to COVID-19. 86 The numbers reported are estimates based on extrapolation of survey findings. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 26 198. 18 head teachers and 102 teachers trained through USDA assistance in the sampled intervention schools reported the use of new teaching techniques or tools. At the project level87, 88 head teachers and 496 teachers have demonstrated the use of new techniques/tools. This falls short of the project targets (116 head teachers and 620 teachers) on these indicators. A Rapid Education and Risk Analysis88 suggests that inadequate building facilities in schools of Cox’s Bazar tend to adversely impact the quality of education. Limited access to power was highlighted as a major challenge, restricting teachers’ ability to use digital content as a new technique/tool for enhancing the quality of teaching. When compared with the mid-term values89, the values for end-term indicates an increase in demonstration of new teaching techniques and tools by 13 percent for head teachers and 47 percent for teachers. During the end term evaluation, this increase could not be validated through discussions with students.90 Therefore, it is important to note that a higher adoption of new teaching techniques and tools has been self-reported by the head teachers and teachers and may not be a representation of an actual increase/improvement. During discussions with parents in sampled intervention schools, parents too expressed satisfaction on the teaching quality and methods used. On the other hand, parents in Ramu pointed towards the need for further training of teachers to help them enhance their teaching skills in order to ensure a joyful and interactive environment in the classroom. This showcases the effectiveness of the trainings on new and quality training techniques and tools conducted by WFP and the implementing partners. Percentage of parents in target communities who can name at least three benefits of primary education 199. 71 percent parents in intervention schools and 77 percent in comparison schools could name at least three benefits of primary education. However, there was no significant difference91 observed between the end-term and baseline values. The three most common responses from parents (both intervention and comparison schools) on benefits of primary education were that it a) improves future opportunities of work for children and b) helps child’s skill development and c) helps break the cycle of poverty. This findings of the evaluation indicate that interventions focussing on increasing the perceived importance of education (such as interactions with PTA and SMC members and community events such as Read Play Festival and Grade 1 Reception Day) may need to be revisited and improved so as to generate awareness on the importance of education for children. The higher percentage of parents naming at least three benefits of primary education in comparison schools may be attributed to previous programmes undertaken in schools in Ramu. Schools in Ramu received support from USAID in the form of READ programme, implemented by Save the Children as was observed during the MTE. 200. Discussions with parents in intervention schools suggest that they consider primary education to be very valuable and important. According to them, a student’s education is needed to gain knowledge, be independent and confident, to be employed (at a job or a business), to travel, to ensure a secure future and/or help the parents out. Parents in Ukhiya also mentioned that education is important as it helps their children learn about cleanliness, discipline, morals and values as well as the skill of communicating effectively. Some parents also mentioned that according to them, a girl’s education was more important than a boy’s as the girl would become a mother in the future and ensure that her children get educated too. A parent in Ukhiya also 87 The numbers reported are estimates based on extrapolation of survey findings. 88 https://www.eccnetwork.net/sites/default/files/media/file/Rapid-Education-and-Risk-Analysis-Cox039s-Bazar￾Final-Report.pdf 89 The classroom visits conducted during the mid-term evaluation validated the use of new teaching techniques, wherein the use of participatory techniques (storytelling and role play) by trained teachers were observed across all the sampled intervention schools. However, during the end-term evaluation, classroom observations could not be conducted due to the methodological limitations (mentioned in section 1.4). 90 Discussions with students could not be conducted during the end term evaluation due to methodological limitations (mentioned in section 1.4). 91 Non-significant difference (@5% level of significance) between midline and baseline values End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 27 mentioned that in a case where the girl child has to be married off early due to poverty, parents often take a commitment from her in-laws that the child will continue her education after marriage. Similarly, in comparison schools, parents perceived education to be very important for their children. Some of the advantages of education they highlighted during discussions were that the students will be able to gain employment, be vigilant and not be fooled easily, as well as be able to help in their children’s education in future. While the reasons cited for educating their daughters’ education may seem unidimensional (from a gender lens), wherein their role as a future mother who educates their child is highlighted, it is encouraging to learn that the parents do see the value of quality education (even inter-generationally) and are willing to educate both sons and daughters. This does point towards the success of the programme in inculcating an understanding of the value of education through various interventions amongst parents. However, going forward, it may be important for future programmes to highlight gender aspects and promote an understanding amongst parents such that they are able to envision an alternative future for their daughters, other than one where education is seen as a means for a daughter to fulfil the role of a mother. Percentage of students in classrooms identified as attentive by their teachers 201. Given that no students were sampled for the end-term92, data on perception of teachers on the attentiveness of (sampled) students was not captured. The mid-term values for this indicator (75% in intervention schools and 71% in comparison schools) has been considered as the end￾term value. There was a significant increase93 in the proportion of students identified as attentive by their teachers (75%) in the intervention schools during the mid-term evaluation as compared to baseline (70%). On the other hand, in case of comparison schools, there was a significant decrease94 in the proportion of students identified as attentive by their teachers (71%) during the mid-term evaluation as compared to the baseline (89%). During discussions with teachers during the mid-term evaluation, biscuit distribution emerged as the key enabler for improving attentiveness amongst students. During the end-term evaluation, this was validated through discussions with parents and SMC members as well. Percentage of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text 202. Due to similar reasons as mentioned above, an assessment on reading and comprehension abilities of grade 3 students could not be conducted for the purpose of the end-term evaluation. In the absence of data on this indicator, the mid-term values have been considered for assessing the change in the reading and comprehension abilities of students. 203. The MTE revealed a significant difference95 between mid-term (28%) and baseline values (6.9%) for demonstration of reading comprehension abilities (combination of fluency and comprehension) among students in intervention schools. For comparison schools, the difference from baseline (15.4%) to mid-term (17%) was not statistically significant. Furthermore, at a disaggregated level, the MTE revealed that 32 percent girls and 25 percent boys in sample intervention schools could fluently read and understand grade II level text. Within the sample intervention schools, Ukhiya had a higher proportion of students (30%) demonstrating proficiency in reading and comprehension skills as compared to Kutubdia (27%). Considering that the mid-term value was 7 percent short of the final target of 35 percent and that COVID-19 adversely affected the learning curve of all students in the final year of implementation, it may be 92 Methodological limitations due to COVID-19 (refer section 1.4) 93 Significant difference (@5% level of significance) between baseline and midline values for student attentiveness in intervention schools 94 Significant difference (@5% level of significance) between baseline and midline values for student attentiveness in comparison schools 95 Significant difference at 95% confidence interval (2-tailed) between baseline and midline value of sample intervention schools for demonstration of reading and comprehension abilities End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 28 inferred that the programme falls short of fulfilling its targets due to the closure of schools and inaccessibility of education facilities during the pandemic. However, efforts to ensure continuity of students’ education as well as the encouraging response of parents on their children’s reading and comprehension levels suggests that the programme sustained its efforts to improve literacy outcomes. 204. Discussions with the parents (conducted for the end-term evaluation) further confirms the improvement in reading and comprehension abilities of their children. They attributed this to initiatives such as establishing classroom libraries and providing supplementary reading materials. In their view, the reading activities conducted in school also led to an increase in fluency and improvement in pronunciation of difficult Bangla words while reading their regular text books. Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance 205. All the sample intervention and comparison schools reported having an SMC. WFP extended technical support to SMCs in the form of training on aspects such as health and nutrition, managing biscuit distribution and safe storage practices. SMCs in all the sampled intervention and comparison schools have received training from either GoB or WFP. A review of the monitoring report (October 2019-March 2020) indicates that 320 PTA and SMCs have been supported as a result of USDA assistance. WFP also provides financial assistance to SMCs to ensure that SMC meetings are conducted regularly. The findings suggests that that on an average, 10 meetings in a year are conducted in both intervention schools and comparison schools. 206. There has been a decrease in the proportion of intervention schools (70%) having seven or more male SMC members from the baseline (83%). This indicates an improved female to male ratio as compared to the baseline in the composition of SMC in intervention schools. Similarly, comparison schools also exhibited an increase in female to male ratio (the proportion of schools that have 7 or more male SMC members being 70% in the baseline to 65% in the end-term). While there is an increase in the female to male ratio, the discussions with the female members of the SMCs reflect that despite such an increase, their participation in SMC meetings remain limited. Some of the female respondents during the FGDs with SMC members indicated that their reluctance to actively participate was associated sometimes with the presence of influential and elderly male members in the meetings. 207. In the sample intervention schools, 74 percent (111 parents) of the parents reported being aware of the existence of SMCs in schools and 83 percent of those aware, reported being aware of the responsibilities of SMCs. In comparison schools, 89 percent of the parents reported awareness on existence of SMC and 88 percent amongst them reported awareness about the responsibilities of SMCs. Amongst parents who were aware about the role of SMCs in both intervention and comparison schools, majority of them perceived improving quality of education and improving school infrastructure as their primary responsibilities. 208. Findings from the primary survey suggest that 93 percent of the sample intervention schools (16 in Ukhiya and 12 in Kutubdia) and 95 percent of the comparison schools reported having a PTA. WFP in collaboration with RtR provides support to PTAs by facilitating four PTA meetings at the school on a yearly basis. Survey findings suggest that 50 percent (8 schools) of the sample intervention schools in Ukhiya and 66 percent (8 schools) in Kutubdia having PTAs have been conducting 3-4 meetings per year with USDA assistance. 209. RtR also provides training to PTA members on quality of education. The findings of the end-term evaluation suggests that during the period of September 2019-February 2020, only 3 of the “They (students) can write a summary of any book. They also try to write stories by themselves. And all of these things…they learn from story books in the library corner.” -Parent in Central Lemshikhali GPS, Kutubdia End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 29 sampled schools in Ukhiya and Kutubdia confirmed receiving training support. The reason for this may be that that the reference period used for this evaluation was shorter (due to COVID-19) as compared to the MTE and the same period also included examinations and commencement of the new academic year, implying that the trainings reported were effectively conducted within a timeframe of 3-4 months. Furthermore, the coverage of training interventions for PTAs in the last year of the programme were affected due to disruptions in the training schedule due to the challenges of lockdown and closure of schools during COVID-19. Evidently, while the USDA assistance has supported regular conduct of PTA meetings, the coverage of training interventions for PTAs in the intervention schools has not been adequate due to implementation challenges posed in the last year of the programme. Number of students enrolled in school receiving USDA assistance 210. A disaggregated enrolment in sample intervention and comparison schools is presented in Table 4 below. The analysis of the data on enrolment in intervention schools in comparison with the baseline indicates a 37 percent increase in average enrolment per sample school during the current grant period. In comparison schools, a reduction of 2.5 percent in enrolment was found as compared to baseline. Enrolment in intervention schools between mid-term and end-term shows an increase of 14 percent. In the case of comparison schools, while there was an increase in enrolment by 10 percent (baseline vs. mid-term), there was a decrease in enrolment by 13 percent from mid-term to end-term. During discussions with SMC members in Ramu, they mentioned that provision of free school bags, uniforms as well as food (“especially dry food or biscuits”) would encourage more parents to enrol their children in school. 211. The analysis of enrolment data at a disaggregated level indicates higher enrolment for girls (196) as compared to boys (173). Discussions with parents revealed that they placed more importance on a girl’s education as compared to a boy’s. This was because they believed that her being educated in the present would ensure that her children also get education in the future. It is interesting to note that the primary motivation to educate a girl child was not to enhance prospects for employability or encourage confidence and independence in the child. Rather, it was to put the onus on her to ensure education of the next generation (fulfilling the gender role of becoming a mother). Therefore, while the higher enrolment amongst girls is a positive development, the reasons behind this progress may be concerning, specifically from a gender and equity lens, as also mentioned above. Table 3: Enrolment in sample intervention and comparison schools Upazila Average enrolment per school (End-term) Boys Girls Students Ukhiya 204 222 426 Kutubdia 111 142 253 Ramu 146 190 336 Number of students regularly (80 percent) attending USDA supported classrooms/schools 212. The end-term findings suggest that 89 percent of students in sampled intervention schools regularly (had a minimum of 80 percent attendance) attend schools as compared to 83 percent in the baseline. In case of comparison schools, 96 percent of students regularly attend schools as compared to 92 percent during baseline. Reference to secondary literature suggests that the higher rates of attendance in Ramu as compared to Ukhiya and Kutubdia may be due to a host of reasons. According to a Rapid Education and Risk Analysis Report96, a higher proportion of teachers in Ukhiya (81%) as compared to Ramu (45%) identified “students’ access to school and attendance” as a major challenge. Transportation challenges including availability, cost, and time 96 https://www.eccnetwork.net/sites/default/files/media/file/Rapid-Education-and-Risk-Analysis-Cox039s-Bazar￾Final-Report.pdf End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 30 were identified as barriers to accessing school in Kutubdia due to its remote location and in Ukhiya due to the traffic jams attributed to the Rohingya influx. Additionally, in Ukhiya, students mentioned that they (especially students from ethnic minority groups) feared conflict between the Rohingya and Bangladeshis and this often kept them from coming to school. Keeping these contextual differences in mind, the increase in attendance of students, in spite of the challenges faced by individuals in Ukhiya and Kutubdia, is an achievement for the programme. Analysis at a disaggregated level suggests a relatively higher increase in regular attendance of girls (by 11.5 percentage points) as compared to baseline in case of intervention schools. In comparison schools, there has been an increase in the regular attendance of girls (by 1.4 percentage points). Therefore, while at an overall level, there is a higher rate of attendance amongst students in comparison schools, it is encouraging to note that the percentage increase of attendance amongst girls is higher in intervention schools. Discussions with parents aligned with these findings wherein it was found that parents encouraged their children, especially girls to regularly attend school. This positive change is an unintended impact of the programme and can be further leveraged by mainstreaming gender in future programmes to improve gender outcomes in intervention areas. 213. In Kutubdia, 92 percent of the students regularly attend schools as compared to 79 percent during baseline. This difference was found to be statistically significant97. However, no significant difference in case of Ukhiya was observed. At the project level, 17,894 boys (against the target of 17,457) regularly attended schools. In case of girls, 23,134 (against the target of 19,689) regularly attend school. Evidently, the project has not only achieved its targets of promoting regular attendance but also exceeded it in case of both boys and girls. During discussions with SMC members, provision of biscuits, free bags, story books and stationery emerged as major enablers of regular attendance in the intervention schools. This was validated through discussions with parents who mentioned that more and more children were now motivated to attend school and did not need to be reminded, as a result of the interventions promoting quality education. 214. During discussions with the SMC members from intervention schools, they mentioned that the need for them to conduct home-visits (to encourage students to come to school) has reduced as the children themselves feel motivated to attend school regularly. They attributed this to the interventions conducted as part of the school feeding programme. This was validated during discussions with mothers who mentioned that previously the children didn’t like attending school but now they don’t need to be reminded to go to school every day. 215. During discussions with parents, SMC members and teachers, the provision of biscuits, bags, tiffin boxes and water bottles by WFP in intervention schools emerged as major drivers for increased student enrolment and attendance. Survey findings suggest that all the teachers in Ukhiya perceive that there will a drop in attendance if biscuit distribution is discontinued. In case of Kutubdia, 92 percent of teachers perceive the same (Figure 1). The quantitative data along with the attribution made to the programme activities by stakeholders point towards the effectiveness of the programme in promoting and improving student enrolment and attendance. 97 Significant difference (@5% level of significance) between baseline and end term values for student attendance in Kutubdia schools End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 31 Number of students receiving deworming medication(s) 216. All the sample intervention and comparison schools, reported administering deworming tablets in the month of October 2019. The deworming campaigns in April 2020 and October 2020 were cancelled by the government due to the COVID-19 pandemic. The data collected from the school records indicate that all the students in the sampled intervention schools have received deworming medication. Average number of school days missed by each student due to illness (for each school and in aggregate) 217. There is no change in the average number of school days (2 days) missed by a student due to illness from the baseline in intervention and comparison schools. Amongst absentees, no change in the proportion of students (parents on behalf of students in the case of ETE) citing illness as the reason for absenteeism was observed in the intervention and comparison schools in the end-term as compared to baseline. Promote Improved Nutrition and Improved Health 218. WFP has promoted nutrition in schools by supporting establishment of vegetable gardens in schools, facilitating training of Little Agriculturalists, and sensitising students and teachers about improved nutrition through Healthy Meal Preparation Days. 219. Analysis of primary data suggests that 67 percent of the schools in Ukhiya and 58 percent in Kutubdia have vegetable gardens. In Ramu, only 10 percent of the sampled schools reported having vegetable gardens. Such a difference highlights the role of the programme acting as a catalyst in supporting GoB to establish vegetable gardens. There were no new vegetable gardens established98 during the period April to September 2020, as schools were closed. Discussions with UEOs in both Kutubdia and Ukhiya also indicated towards collective inability to maintain the school gardens due to closure of schools. 220. The analysis further indicates that 61 percent schools of the sampled schools in Ukhiya and 67 percent schools in Kutubdia have Little Agriculturists. While the findings from the mid-term evaluation indicated all Little Agriculturists in sampled schools had received training from WFP, the findings from the end-term evaluation suggests that 2 out of 3 Little Agriculturists in Ukhiya and 1 out of 2 in Kutubdia received training from WFP. Discussion with WFP representatives indicated that this was due to the postponement of various trainings on account of COVID-19 and the subsequent closure of schools. Number of individuals trained in child health and nutrition as a result of USDA assistance 98 Semi Annual Report April to September 2020 0% 61% 28% 8% 11% 67% 17% 8% 0% 20% 40% 60% 80% 100% No consequence, attendance will remain the same Attendance will drop by 25% Attendance will drop between 25% and 50% Attendance will go down by more than 50% Ukhiya Kutubdia Figure 1: Perception of teachers on impact on students’ attendance if provision of biscuits stopped End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 32 221. The findings of the survey highlight that 7 head teachers and 39 teachers were trained in child health and nutrition as a result of USDA assistance. Similarly, 33 parents reported receiving training/sensitisation on child health and nutrition from WFP. Therefore, 79 individuals (including head teachers, teachers and parents) from the sampled intervention schools have been trained in child health and nutrition in the last year of the programme as a result of USDA assistance. Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance 222. The findings of the end-term evaluation suggests that all the head teachers and teachers trained through USDA assistance, self-reported demonstration of child health and nutrition practices. 223. The status of demonstration of child health and nutrition practices by parents were assessed using two parameters: handwashing at critical times99 and dietary diversity. Out of the parents who were trained by WFP in child health and nutrition, 19 parents (58 percent of the parents trained) demonstrated handwashing practices at critical times100 (Refer table 5 below). Therefore, out of the 79 individuals trained in the sampled intervention schools on child health and nutrition practices, 65 individuals demonstrate handwashing practices at critical times. Table 4: Parents trained by WFP who are aware and demonstrate handwashing practices at critical times Ukhiya Kutubdia Total Treatment Awareness of handwashing by parents (during critical times) Less than 4 times reported 10 5 15 4 to 6 times reported 13 5 18 Total 23 10 33 Demonstration of handwashing by parents (who practice handwashing on 3 or more out of 6 critical times ) Less than 4 times reported 9 5 14 4 to 6 times reported 14 5 19 Total 23 10 33 224. The end-term values with regard to the mean dietary diversity score (DDS)101, for children in sample intervention schools was found to be 4.85 (compared to 4.2 in baseline), based on data reported by parents102. No significant difference was observed. Given that students could not be interviewed during the end-term, sex disaggregated data cannot be presented. Similarly, with regard to the mean dietary diversity score (DDS)103, for children in sample comparison schools (reported by parents), the end-term value is 5.04 (compared to 4.2 in baseline). It is interesting to note that there was no significant difference104 between the scores of boys and girls in intervention and comparison schools during the MTE. 99 The numbers reported here are for those who practice handwashing on 4 or more out of 6 critical times 100 The numbers reported here are for those who practice handwashing on 4 or more out of 6 critical times 101 Feed the Future Guidance (2014) suggests grouping food items into 10 food groups. However, based on discussions with WFP, classification of food was done across 7 categories. Therefore, the scale of food options is that of 7 in this case. 102 The source of data on this indicator for baseline and mid-term were children and parents. However, in the end-term, this data was only collected by parents. (Refer section 1.4) 103 Feed the Future Guidance (2014) suggests grouping food items into 10 food groups. However, based on discussions with WFP, classification of food was done across 7 categories. Therefore, the scale of food options is that of 7 in this case. 104 Non-significant difference at 95% confidence interval (2-tailed) between female and male values of DDS End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 33 225. The mean DDS during the end-term evaluation witnessed a decrease from the mid-term in both sample intervention (4.96 to 4.85) and comparison (5.49 to 5.04) schools. This decrease may be attributed to the ongoing COVID-19 pandemic wherein affordability and availability of diverse nutrition-rich food has been a major challenge, as reported by parents. However, it is interesting to note that the reduction in the mean dietary diversity score is lesser in the schools in Ukhiya and Kutubdia as compared to Ramu. This could be attributed to WFP’s continued messaging on promotion of health, hygiene and nutrition practices during the pandemic through partnerships with organizations such as Sesame Workshop Bangladesh. Such initiatives towards constant reinforcement and promotion of health, hygiene and nutrition practices may have motivated the parents and students in intervention areas to prioritise a diverse and nutrition rich diet, despite challenges posed by the COVID-19 pandemic. 226. Findings at a disaggregated level suggest that despite multiple external challenges with regard to increase in food prices due to the Rohingya refugee influx and unavailability of diverse food items in the context of COVID-19, there was a significant105 increase in the dietary diversity score in Ukhiya from mid-term (3.4) to end-term (4.9). However, this was not the case in Kutubdia. One of the reasons for this maybe the additional disadvantage of remoteness of location and inadequate local production of diverse food items, owing to the salinity of water in the region. 227. The reasons for higher DDS from the baseline in the intervention area can be understood based on the discussions with parents, wherein it was mentioned that children talk about the nutritional qualities of food and demand food based on its nutritional qualities. Parents also mentioned that learnings from demonstrations during events such as the Healthy Meal Preparation Day, has encouraged them to cook nutritious meals for their children at home. Number of educational facilities (i.e. water systems and latrines) rehabilitated/constructed as a result of USDA assistance 228. All the sample schools have received bookshelves for classroom libraries, almirahs, school bags, water bottles and tiffin boxes through USDA assistance. 229. 8 schools (7 in Ukhiya and 1 in Kutubdia) from the sample intervention schools reported receiving support from WFP for rehabilitation/construction of toilets and storage facilities. 11 schools (5 in Ukhiya and 6 in Kutubdia) from the sample intervention schools reported receiving support from WFP for rehabilitation/construction of library, playground and classroom. 230. 4 intervention schools from the sample intervention schools reported receiving support from WFP for drinking water facilities and 8 schools reported receiving support from WFP for water supply in toilets. None of the sampled intervention schools reported receiving support from WFP for water supply in the school garden. Number of schools using an improved water source Number of schools with improved sanitation facilities 231. Based on the assessment of requirements articulated by schools, WFP provided equipment (pumps and motors) to improve the water systems in schools. 29 percent (5 schools) of the sample intervention schools reported receiving support through USDA assistance for rehabilitation or construction of any water system. Availability of improved water sources106 was reported in all the sample intervention and comparison schools as compared to 80 percent schools (for both intervention and comparison schools) in the baseline. Tube well was reported to be the primary source of drinking water for both intervention (90%) and comparison (95%) schools. 232. Number of schools with improved sanitation facilities have been determined on the following three parameters: functionality of toilets, whether it is locked or not, and availability of water. 97 percent of the intervention schools (29 schools) reported having functional toilets as compared to 105 Significant difference at 95% confidence interval (2-tailed) between mid-term and end-term values of DDS in sampled schools of Ukhiya 106 Improved water sources as per WHO: https://www.who.int/water_sanitation_health/monitoring/water.pdf End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 34 11 schools during baseline. Availability of separate functional toilets for boys and girls was observed in 86 percent of the sample intervention schools as compared to 50 percent reported during the MTE. Of the total number of functional toilets in intervention schools, 7 percent (7 toilets) were locked. This is a stark improvement from the mid-term, wherein 28 percent of the functional toilets remained locked. Furthermore, all toilets were reported to have water facilities. In the comparison schools, 9 percent (7 toilets) of the functional toilets were observed to be locked. Here too, a reduction in the number of locked toilets can be seen (27% during the MTE). Availability of water facility in all the functional toilets in the comparison schools was reported. Availability of separate functional toilets for boys and girls was observed in 85 percent of the sample comparison schools (compared to 75% during the MTE). Percent of students who can identify at least three key health and hygiene practices 233. Data for this indicator was not collected as students were not interviewed for the end-term evaluation. The mid-term values is being considered as end-term values. As per the MTE, 39 percent of the students in both the sample intervention schools and comparison schools identified three or more health and hygiene practices. A comparison with baseline values indicated higher levels of awareness among students about health and hygiene practices in the sample intervention schools (baseline-25%) as compared to comparison schools (baseline- 35%), however the difference was not statistically significant. Analysis at a disaggregated level indicated a significant107 difference in awareness among students about health and hygiene practices in Ukhiya (46%) as compared to Kutubdia (29%). Similarly, a statistically significant difference from baseline to midline in the awareness about health and hygiene practices among boys (40% from 16%) and girls (39% from 13%) was observed. 234. Further, two out of three students in the sample intervention and comparison schools identified handwashing as a key health and hygiene practice during the MTE. During discussions with parents and SMC members (during both MTE and end-term) increased adoption of handwashing practices (before meals and after use of toilet) by their children was indicated. Given the extensive messaging and awareness drives conducted by GoB and other development partners and through popular media during COVID-19, the evaluation team believes that there may have been a further increase in the adoption of health and hygiene practices. Number of 'Little Doctor' students supported by WFP 235. The Little Doctors programme has been implemented by GoB across all intervention and comparison schools. WFP contributed to this GoB programme through (a) training/orientation of Little Doctors on health, hygiene and sanitation practices and (b) aprons for easy recognition of little doctors. These aprons were worn by the students during health check-ups and deworming days. 236. Across all the sample schools, a total of 430 Little Doctors (256 in Ukhiya, 174 in Kutubdia) and 99 Little Doctors in Ramu were reported as identified. A comparison with the MTE findings shows that while in intervention schools, the number of little doctors identified remained the same, in the comparison schools of Ramu, there was a reduction by 48 percent (189 little doctors identified during the MTE). On an average, there were 14 Little Doctors per intervention school and 5 Little Doctors per comparison school. Little Doctors in all intervention schools except one had received training from WFP but Little Doctors from only 9 comparison schools (45 percent) received training from the government or any other source. 237. During discussions, parents in intervention schools cited instances where, inspired by Little Doctors in school, students started playing the “Little Doctor” at home. One parent mentioned how his daughter pretended to check her friends and cousins’ fever and taught them steps to maintain cleanliness. Examples of such role plays indicate how the Little Doctor programme has 107 Significant difference at 95% confidence interval (2-tailed) between Ukhiya and Kutubdia’s mid-term values for awareness among students about health and hygiene practices End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 35 positively influenced the students and has been effective in promoting improved nutrition and health practices. Number of individuals trained in safe food preparation, commodity management and storage as a result of USDA assistance 238. WFP has provided trainings to head teachers, teachers and storekeepers on safe storage practices in all intervention schools. 38 individuals (20 storekeepers, 6 head teachers and 12 teachers) from the sample intervention schools reported receiving training on safe storage practices through USDA assistance. In concurrence with the MTE findings, a higher proportion of male teachers and storekeepers (55%) as compared to female teachers and storekeepers reported to have received training on safe storage practices. Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance 239. 66 percent of the trained head teachers and teachers demonstrated the use of safe storage practices. In the case of storekeepers, observations to determine demonstration of safe storage practices could not be conducted during the end-term108 and therefore midterm values were used. Percent of storekeepers who can identify at least three safe storage practices 240. 17 out of the 20 (85%) trained storekeepers in Ukhiya and Kutubdia were able to identify at least three safe storage practices in the end term as compared to 100 percent in the mid-term and 97 percent in the baseline. This decrease in percentage from mid-term to end-term may be attributed to the closure of schools from March 2020 due to which no refresher trainings on safe food handling and storage were conducted for school teachers and government officials. Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance Number of school-age children receiving daily school meals (packet of fortified biscuits as snacks) as a result of USDA assistance 241. As per the semi-annual report of WFP (April 2020 to September 2020), 4,438,900 biscuits have been distributed across 146 schools in the intervention area. Primary study reveals that on average, 4453 students in the sample schools of Ukhiya and 2386 students in those of Kutubdia have received biscuits during the period September 2019 to February 2020. Average biscuit distribution days in a month have been 21 days during these months in the sample intervention schools. Number of teachers, parents and school management committee members attended the community mobilization workshops 242. Given that planned activities like community mobilization workshops for parents had to be postponed due to closure of schools since mid-March 2020, the data presented below captures the number of teachers, parents and SMC members who attended these workshops pre-COVID-19. 243. 94 percent of the parents in sample intervention schools reported being aware of the Read-Play festival. 49 percent of those (aware), reported attending the festival. In most of the sample intervention schools, this Read-Play festival was merged with the Annual Day of the school where along with cultural events, some poetry/recitation competitions were held. The higher awareness about the festival amongst parents is possibly because of the festival being clubbed with the Annual day which is well-known to them. 244. Discussions with parents and SMC members revealed that the Read Play festival was an event that students looked forward to and thoroughly enjoyed. SMC members mentioned that receiving 108 Methodological limitations (Refer section 1.4) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 36 prizes for fluent reading was one of the main reasons that the students issued books from the classroom library corners. According to parents, this festival was a major motivator for students to improve their reading skills. Participation in this event also boosted their confidence in reading and speaking Bangla. 245. WFP through its implementing partners supported the conduction of Grade I reception of students or “nobinboron”. During this event, the newly enrolled children along with their parents are given flowers and tiaras (ribbons) as a welcoming gesture. Cultural programmes like singing, dancing or recitation are also conducted. SMC members too, participate in this event. They talk about the importance of education and request parents to send their children to school every day. None of the comparison schools reported conducting a similar event. 57 percent of parents from the intervention schools reported being aware about Grade I reception day. 63 percent of those (aware) reported attending the same. Foundational Results 246. WFP provided technical assistance to MoPME in developing the “National School Meal Policy 2019”. The support included (i) coordinating with various ministries and development partners at various stages of development of NSMP; (ii) facilitating inter-ministerial joint exposure visits both in￾country and abroad for sensitisation of representatives of different ministries on the relevance of the school feeding programme; (iii) advocating with various ministries for undertaking interventions to support school feeding and finalising NSMP; (iv) providing technical inputs for setting up minimum nutritional requirement109 for school meals. Additionally, WFP through a partnership with Campaign for Popular Education (CAMPE) facilitated an intervention targeting stakeholders from the government and civil society to enhance their understanding on nutrition￾sensitive interventions. During discussions, representatives from MoPME recognised contribution of technical assistance provided under SFP in preparation of nutrition sensitive NSMP. They further confirmed that the technical assistance enhanced their understanding of implementing nutrition￾sensitive interventions, encouraged them to initiate a discourse on providing school meals and helped them to mainstream activities that can promote nutrition, health and hygiene practices within the existing educational framework. 247. In addition to sensitizing different ministries of GoB on the relevance of the school feeding programme and its activities, WFP also provided technical assistance to the GoB Nutrition Technical Committee which provided the formal Government recommendation to include a minimum nutritional requirement in the formulation of NSMP for the pre-primary and primary school children, based on an average of boys and girls aged 3-12 years. 248. The NSMP was approved at the Cabinet Meeting chaired by the Honourable Prime Minister of Bangladesh in August 2019. Through this policy, the GoB has committed to achieve universal coverage of the school meal programme in all the pre-primary and primary schools across the country (around 15 million children). 109 A minimum 30 % of the daily energy requirements of pre- primary and primary school children should be provided through school meals. A minimum 50 % of the recommended micronutrient requirements should be covered by the school meal for half a day school As part of their efforts towards community engagement and awareness raising, RtR also celebrated the Mother Language Day 2020 in primary schools and madrasas in Ukhiya and Kutubdia. The event was marked by rallies, art competitions, poem/rhyme recitation as well as prize distribution for students. According to the UEO of Kutubdia, initiatives such as these can enhance the reading skills of students and encourage them to read more. 1. As part of their efforts towards community engagement and awareness raising, RtR also celebrated the Mother Language Day 2020 in primary schools and madrasas in Ukhiya and Kutubdia. The event was marked by rallies, art competitions, poem/rhyme recitation as well as prize distribution for students. According to the UEO of Kutubdia, initiatives such as these can enhance the reading skills of students and encourage them to read more. 2. Figure 2: Conceptual Frameworks part of their efforts towards community engagement and awareness raising, RtR also celebrated the Mother Language Day 2020 in primary schools and madrasas in Ukhiya and Kutubdia. The event was marked by rallies, art competitions, poem/rhyme recitation as well as prize distribution for students. According to the UEO of Kutubdia, initiatives such as these can enhance the reading skills of students and encourage them to read more. 3. As part of their efforts towards community engagement and awareness raising, RtR also celebrated the Mother Language Day 2020 in primary schools and madrasas in Ukhiya and Kutubdia. The event was marked by rallies, art competitions, poem/rhyme recitation as well as prize distribution for students. According to the UEO of Kutubdia, initiatives such as these can enhance the reading skills of students and encourage them to read more. 4. Figure 3: Conceptual Frameworks part of their efforts towards community engagement and awareness raising, RtR also celebrated the Mother Language Day 2020 in primary schools and madrasas in Ukhiya and Kutubdia. The event was marked by rallies, art competitions, poem/rhyme recitation as well as prize distribution for students. According to the UEO of Kutubdia, initiatives such as these can enhance the reading skills of students and encourage them to read more. 5. As part of their efforts towards community engagement and awareness raising, RtR also celebrated the Mother Language Day 2020 in primary schools and madrasas in Ukhiya and Kutubdia. The event was marked by rallies, art competitions, poem/rhyme recitation as well as prize distribution for students. According to the UEO of Kutubdia, initiatives such as these can enhance the reading skills of students and encourage them to read more. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 37 249. With an objective of ensuring adequate capacity to design and implement interventions under NSMP, MoPME with technical support from WFP, organized a national workshop on the “Systems Approach for Better Education Results (SABER) School Feeding” at Dhaka. The workshop was presided over by the Secretary, MoPME and witnessed participation of around 100 high level officials from different ministries, government technical agencies, DPE, UN agencies and NGO officials including SUN Business Network. The workshop facilitated mapping of existing capacities of MoPME and other ministries and identification of capacity gaps on aspects including (i) creation of an enabling policy and regulatory framework;) ii) preparation of operational plan for implementation, monitoring of programme and ensuring multi-stakeholder collaboration; (iii) sustainable financing; (iv) institutional arrangement, and (v) engagement with the communities and non-state actors in planning, implementation and monitoring, ownership of programme. Specific interventions required to address the capacity gaps were also identified. The workshop also witnessed strong enthusiasm and commitment of GoB and its partners to implement NSMP. 250. WFP also supported MoPME and DPE in operationalizing an online database for school meals in Government assisted 94 Upazilas; and piloting of an e-cash transfer for school meals. Further, it assisted DPE in preparation of guidelines for NGO selection (for implementation of NSMP) and their performance assessment. 251. Additionally, WFP in collaboration with MoPME and Bangladesh Institute of Research and Training on Applied Nutrition (BIRTAN) conducted two batches of the training of trainers (ToTs) for Cooks on Safe Food Preparation Practices and Basic Nutrition at central level. As part of the support, training manuals and handouts were also prepared. Additionally, in partnership with BIRTAN, WFP facilitated training for the cooks at field level. 252. WFP in collaboration with the Department of Public Health Engineering (DPHE) and Directorate of Primary Education (DPE) also contributed to the preparation of a technical specification for Kitchen Construction cum Storage unit to ensure cooking and supply of nutritious meals under NSMP. 253. During COVID-19, WFP worked with NGO partners to realign their activities so as to meet the emerging needs of students and communities. Biscuit distribution to students’ homes is one such example. Towards enabling this, WFP developed the standard operating procedures for the GoB to deliver biscuits to students’ homes. Nearly 3 million students received biscuits at their homes in multiple rounds of distribution. Considering the findings above, it can be inferred that the technical assistance provided by the WFP through USDA assistance, has been effective in contributing towards strengthening the policy framework for ensuring inclusive and equitable quality education and improved nutrition for primary school children (SDG 2 and 4) through a nutrition-sensitive school meal programme. Further, the capacity building support has facilitated MoPME in identification of capacity gaps, preparation and implementation of capacity building to independently undertake implementation of NSMP. It is worth mentioning that the evaluation has not come across any unintended effects of the intervention on human rights and gender equality. 3.3. Efficiency Evaluation Question 6: What was the efficiency of the programme, in terms of transfer cost, cost per beneficiary, logistics, and timeliness of delivery? Evaluation Question 7: How efficient are the operation and approach in terms of capacity building of government towards eventual handover? Biscuit Distribution 254. Through the biscuit distribution activities, WFP was able to cover all the target schools. Demand forecasting for biscuits was done on the basis of the total enrolled children in a school in different grades and grade-wise sum of attendance of children. Schools submitted a Monthly Utilization Report (MUR) of the receipt and utilization of the biscuits for the reporting month. This provided a basis to the implementing partners for forecasting the demand of biscuits for the next month. The End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 38 consolidated demand of all the schools under the two upazilas were submitted to WFP for supply of stocks to the school. 255. All sampled schools in Ukhiya and Kutubdia reported that the biscuit distribution to students (till before closure of schools) was done on a daily basis. During discussions, storekeepers indicated that in case of their absence, the responsibility of biscuit distribution was taken up by the teachers and the headmaster of the school, thereby ensuring regular distribution. The discussions with parents also confirmed that the students received biscuits six days a week. 256. The findings of the ETE indicates that all the sampled schools in Ukhiya reported receiving stock of biscuits on a monthly basis. 10 (out of 12 schools) in Kutubdia reported receiving biscuits on a monthly basis from the implementing partner, during the period September 2019- January 2020. One school reported receiving new stock once in fifteen days. Only one school reported replenishment of stock, once in two months. 257. 10 schools (out of 18 sampled schools) in Ukhiya reported delivery of stock before any request for replenishment. 7 schools reported replenishment of stock within a week of requisition. In case of Kutubdia, one in two schools reported replenishment of stock before any requisition. 6 schools reported replenishment within a week. Only one school in Ukhiya and Kutubdia reported replenishment in fifteen days. 258. With regard to stock out, all except one school each in Ukhiya and Kutubdia confirmed that there were no instances of stock out during the grant period. The store keepers from schools who experienced a waiting period of one week or fifteen days reported maintaining a buffer stock considering the time taken for replenishment. In case of two schools who reported one instance of stock out, the situation was managed by the implementation partners by replenishing the stock from the buffer stock of neighbouring schools. Based on the findings of the primary survey, that reflect (i) only one instance of stock out and; (ii) regular distribution of biscuits to students, it can be inferred that the process of inventory planning, demand forecasting and biscuit distribution is efficient. Monitoring and reporting system 259. The efficiency of the programme can also be determined by (i) the use of an online database/MIS by MoPME for timely reporting of data and its utilisation for planning and timely decision making; (ii) timely reporting of progress on programme implementation by the implementing partners and utilisation of data by WFP. WFP supported DPE in the development and implementation of an online database for reporting and monitoring activities such as commodity management, enrolment, attendance and biscuit distribution. MoPME manages the SFP at the central level and is supported by a PMU which facilitates and establishes systems and processes for monitoring and regular feedback. The data entry in the MIS is done at the Upazila Education Office. 260. Discussions with the representative MoPME and education officers at the district and upazila level indicates that there is regular reporting of data from upazilas. Data is collected by the implementation partner staff at the school level (Monthly Utilisation Reports). This data is consolidated at the upazila level and shared with WFP for analysis and preparation of quarterly reports. The quarterly reports are shared with GoB. These reports are the discussed during review meetings for feedback, resource planning and decision making. 261. With regard to reporting of progress on interventions by the implementing partners to WFP, the discussions with the implementing partner and representatives of WFP indicates that the reporting is done on regular and timely basis. The data is also utilised by WFP for review of implementation and disbursement of funds to partners. No delay in disbursement of funds was reported by the implementing partners. It also emerged during the discussions that the implementing partners’ appraise the Upazila Education office regularly on the progress of implementation of SFP. However, there is irregularity in conduct of joint monitoring visits (officials of District Education Office, Upazila Education Office, Implementation partner and WFP). Agreement of all the stakeholders on a common day for visit owing to their commitments emerged as the major challenge in scheduling joint monitoring visits. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 39 262. The data is also utilised by WFP for review of implementation and disbursement of funds to partners. No delay in disbursement of funds was reported by the implementing partners. 263. On aspects related to literacy, discussions with representatives, RtR highlights adoption of global practices for regular monitoring of classroom library interventions (particularly book transactions in classroom libraries and tracking reading abilities of students). 264. Reference to the CSP Activity Plan & Reporting Tool110 highlights that all the capacity building activities (including those related to operationalization of MIS at national and Upazila level) with the relevant officials at GoB were conducted as per the plan. It can therefore be inferred, that regular reporting and monitoring of data has ensured timely completion of activities thereby reflecting efficiency of processes. Efficiency of implementing partners 265. The implementation partners’ outreach and connectedness with the community and proactiveness plays an important role in ensuring the efficiency of processes. During the discussions with Upazila Education Officers (UEOs), in Ukhiya and Kutubdia, the support received from the Muslim Aid and YPSA in the first two years and from RIC in the final year of implementation was well acknowledged. It also emerged during discussions that implementing partners were perceived to be proactive, supportive and well connected with the community. Discussions with the UEO in Ukhiya revealed that the department was always consulted and informed about the activities undertaken by the implementing partner. The UEO also admitted that both Muslim Aid and RIC shared a good rapport with the school administration. The UEO further mentioned that, this level of transparency and trust is key to ensure successful and well￾accepted implementation of the interventions. 266. As a response to closure of schools during COVID-19, WFP supported MoPME in distributing biscuits to children at home. The guidelines for home distribution of biscuits at home prescribed delivery of at least 25 packets of biscuits per child in a month. Towards ensuring that each child receives biscuits at home, the implementing partner and other NGOs collected enrolment lists from the schools and identified the location/house of each child. Further, the biscuit distribution at home required implementing partners and NGOs to adhere to all safety guidelines prescribed during COVID-19. The remote monitoring111 of 524 households undertaken by WFP in areas implemented by GoB, and therefore not funded by USDA, in June 2020, highlights that 46 percent of the households received biscuits regularly. Further, physical handover of biscuits in 65 percent of the households was done in the presence of teachers and other school staff members and in adherence with the safety protocols prescribed during COVID-19. Evidently, the timely response to the programme needs, measures taken to ensure smooth implementation during and pre COVID-19 and proactive engagement with the government, school administration and the community reflects the efficiency in processes followed by the implementing partners. 3.4. Impact Evaluation Question 8: How did the project contribute to observed impacts? Have there been any unintended outcomes, either positive or negative? What internal and external factors affected the project’s ability to deliver impact? Have the outcomes been specifically affected by COVID-19? What are the spill-over effects of the project? Are local communities (PTA, farmers groups, etc.) fully involved in and contributing toward school feeding? 267. As mentioned in the section on effectiveness earlier, the COVID-19 crisis significantly influenced the implementation of planned activities of the SFP. Further, information on certain indicators has 110 Activity Plan and Report 2019 and 2020 111 School Feeding Programme In Poverty Prone Areas – home delivery of biscuits, Summary Monitoring Report – June 2020, World Food Programme End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 40 not been captured as no discussions were held with the direct beneficiaries (i.e. students)112. Due to this, it is advised that the impact/change reported in this section be understood keeping the limitations of the COVID-19 pandemic in mind. 268. The evaluation, does however highlight an increase in enrolment and attendance, understanding among the community on the benefits of education and improvement in fluency and comprehension among students. Further, findings also suggest that there has been an increase in the overall knowledge about good nutrition and health as well as hygiene and sanitation practices. These positive changes together constitute the impact of the program. Improved Literacy of School-Age Children 269. Findings in the effectiveness section highlight improvement in enrolment as well as attendance in the intervention schools as compared to baseline, enabling the project to exceed its targets (for both boys and girls). Discussions with SMC members and parents also indicated that, owing to the provision of biscuits and free bags in schools, students themselves feel motivated to go to school. Parents also mentioned multiple benefits of education and expressed the need to educate their children (especially girls). Further, both SMC members as well as parents in Ukhiya and Kutubdia believe that owing to biscuit distribution, the attention span of students has also improved as they no longer stay distracted and hungry in class. Therefore, the programme has created an impact on the attendance of students, especially for girls. The programmes interventions have also contributed to the overall improved learning abilities amongst students owing to increased attendance and distribution of biscuits and other provisions. The programme exceeding its targets in terms of attendance further establishes its impact. However, the plastic wrappers of biscuits are often burnt in the open, posing a threat to the environment. Creation of plastic waste and non￾environment friendly waste disposal is an unintended negative impact of the SFP 270. In addition to enrolment and attendance, reading skills of students have also improved. This was validated through discussions with parents and SMCs. Through initiatives such as the classroom libraries, Read Play festival, Mother Language day and SRM period, WFP and its implementation partners have been able to encourage a habit of reading among primary school students even at home and despite the closure of schools. Additionally, during discussions with parents, they mentioned that due to the exposure to story books, their children are now able to write stories of their own. Therefore, apart from reading, students have also been encouraged to engage in creative writing, thus contributing to the overall objective of improving literacy amongst school aged children. 271. There has also been a significant change with regard to prioritisation of education amongst parents and SMC members. Discussions reveal that one of the most discussed topics during the SMC and PTA meetings is the quality of education provided by the schools. SMC members also mentioned conducting home visits to follow up on the educational progress of the children in their area. Especially in the context of COVID-19, the pro-activeness and support to implementing partners demonstrated by parents as well as SMC members to ensure continuation of education during the pandemic is evidence that education has emerged as a priority for the community. This increased ownership to ensure quality education showcases the impact as well as sustainability of the programme with regard to its literacy goals. 272. Considering that there is a growing realization on the importance of education and increasing demand for quality of education, it can be inferred that the project has had an impact with regard to improving literacy of school age children. Increased Use of Health and Dietary Practices 273. As highlighted in the effectiveness section, there is an increase in the adoption of health and hygiene practices in the sample intervention schools. There has been a noteworthy improvement 112 Methodological limitations (refer Section 1.4) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 41 in the number of functional and usable toilets, number of separate toilets for boys and girls, demonstration of handwashing practices amongst parents and so on. 274. In addition to this, WFP’s support to the Little Doctor programme in intervention schools has also promoted the adoption of good health and hygiene practices. Apart from some instances mentioned by parents where students are now playing the Little Doctor game at home, parents have also mentioned an increased aspiration among students to become doctors in the future. This has resulted in an unintended positive impact where more and more students feel motivated to study hard in school in order to achieve their aspirations, thus contributing to the literacy goals of the project as well. As was the case in the MTE, discussions during data collection for the end-term evaluation also indicated that the aforementioned aspiration of students to become doctors can be attributed to the Little Doctor Apron/Uniform provided by WFP. 275. With regard to nutrition, there is an increase in the DDS in sample intervention schools, from the baseline to the end-term. Increased awareness about nutrition through initiatives under SFP, resulting in increased demand from students for nutritious food and efforts from parents to fulfil the demand is one of the important factors for the increase in DDS. It is worth noting that the lesser decrease in DDS in Ukhiya (from mid-term to end-term), despite the challenges caused due to COVID-19 is evidence for the impact created by the project over the grant period wherein the community was able to prioritise a diverse nutrition-rich diet, despite a rise in prices due to COVID-19 and the Rohingya refugee crisis. 276. Therefore, regardless of the challenges posed by the COVID-19 pandemic, the project has been successful in improving awareness about nutrition, health and hygiene. 277. Considering an increased awareness amongst and adoption of health and dietary practices by the parents and their children, it can be inferred that the project has made an impact towards increasing the use of health and dietary practices. 3.5. Sustainability Evaluation Question 9: Is the programme sustainable in the following areas: strategy for sustainability; sound policy alignment; stable funding and budgeting; quality programme design; institutional arrangements; local production and sourcing; partnership and coordination; community participation and ownership? Evaluation Question 10: What progress has the government made toward developing and implementing a nationally owned school feeding programme? Evaluation Question 11: Are local communities fully involved in and contributing toward school feeding and education activities? 278. The approval of the National School Meal Policy by GoB, allocation of USD $76 million and implementation of the school feeding programme in 94 upazilas, of which 16 upazilas provide school meals, provides evidence of the government’s commitment and willingness to take ownership of the programme. The GoB has also made a commitment to achieve universal coverage of school meals programme in all the pre-primary and primary schools in the country (around 15 million children). The nutrition-sensitive nature of the approved National School Meal Policy (NSMP) 2019 lays testimony to GoB’s perceived importance of nutrition and the commitment to mainstream it into government programmes even post withdrawal of technical assistance by WFP. The "Primary School Meal Project (PSMP)" envisions to cover all government primary schoolchildren in all 513 upazilas and municipalities. Going ahead, a MoU will be signed between WFP and the appropriate government authority to provide technical assistance for the effective implementation of the PSMP project. 279. The GOB’s yearly commitment of 4 million USD to WFP is itself an indication that the GoB recognises and values the project. The anticipated approval of the PSMP (late 2021) further validates this commitment. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 42 280. Further, WFP is working with the National Curriculum and Textbook Board (NCTB), the national body that develops all student and teacher curricula material, to ensure that health and nutrition information is mainstreamed into grade-level science curricula in pre-primary grades through grade 5. This is further evidence for GoB recognizing the need for prioritizing and mainstreaming nutrition, ensuring sustainability of the SFP interventions and its associated learnings in the national level programmes. 281. WFP’s support to build capacities of the GoB to independently take over the implementation of the project has been well recognised during discussions with GoB officials. WFP held trainings of trainers for Cooks on Safe Food Preparation Practices and Basic Nutrition in collaboration with BIRTAN and MoPME and also supported the development of training manuals and handouts. In addition to this, WFP also organized a SABER workshop involving various senior officials of the government. The findings of the SABER workshop indicate that WFP has been able to comprehensively address the training needs of the GoB, in order to ensure independent implementation of the PSMP by GoB in the future. 282. WFP’s support to MoPME and DPE in operationalizing an online database for school meals in 94 upazilas has also enabled the GoB officials to independently plan their requisitioning, forecasting, budgeting and expenditure through this data base. The adoption and subsequent utilization of the online database for planning by GoB showcases the sustainability of this mechanism, aimed at an evidence based planning and an efficient takeover of the programme at the national level. 283. The facilitation of inter-ministerial exposure visits by WFP for sensitising ministerial representatives on the relevance of the SFP has established interests of various ministries in mainstreaming nutrition into their activities. This has enabled inter-ministerial coordination to achieve the objectives of NSMP, post withdrawal of support from WFP, thus making a strong case for sustainability. 284. Discussions with representatives of MoPME and DPE indicated a need for further support from WFP to ensure independent procurement of biscuits and other essentials for implementation of the PSMP by the WFP PMU. Additionally, during discussion with a representative of BIRTAN, a need for evidence generation and research initiatives was articulated, especially on nutrition retention, cooking systems and post-harvest management. Support on these aspects would further strengthen GoB’s capacity to successfully carry out the implementation of PSMP. 285. The COVID-19 pandemic adversely affected the capacity building efforts made by WFP. However, strengthened capacities of the GoB over three years ensured that this setback did not significantly affect the achievement of results. In fact, a reach of 3 million children through door to door biscuit distribution achieved by the GoB independently, even in the context of COVID-19, demonstrates GoB’s capacity and commitment to sustain and adopt the SFP, post withdrawal of WFP support. 286. WFP has worked in collaboration with the Cabinet Division, Economic Relations Division (ERD), Ministry of Primary and Mass Education (MoPME) to ensure the SFP’s inclusion in government plans; as a result of which the school feeding programme has been mentioned in the 8th 5 Year Plan of the Government of Bangladesh. This is an important development to set the base for the approval of the new Development Project Proforma (DPP) for 2021-2026. WFP has been providing continuous support to GoB for the preparation and finalization and approval of the DPP- the primary school meal project (PSMP). However, a potential threat to the sustainability of the school meal programme in the next phase is a COVID-19 pandemic induced delay in the approval of the new DPP (depending upon the clearance from planning commission). 287. The inadequate number of teachers in schools continues to remain a challenge in ensuring the programme’s achievement of its literacy goals and its sustainability. WFP, along with its literacy partner RtR, has tried to accommodate all teachers in all trainings and ensure that appropriate instruction is provided in classrooms through ongoing coaching and mentoring support to teachers. However, despite all efforts to build the capacities of teachers, without addressing the institutional gap of shortage of teachers at the overall country level, the sustainability of literacy initiatives of the SFP to ensure quality of education cannot be guaranteed. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 43 288. The findings of the end-term evaluation suggest that the capacities of School Management Committees have been built towards ensuring community’s involvement in school feeding and education activities. SMC members in intervention schools clearly stated that they engage in expressing their concerns to and liaising with the upazila education officer, WFP as well as RtR and extending support for the storage and distribution of biscuits provided by WFP. SMC members also mentioned that whenever a member receives training from WFP, he/she ensures that the knowledge is also passed on to other members, thus ensuring peer learning. Such demonstration of ownership by SMC members, which has also percolated within the community is expected to ensure sustainability of the interventions and its results. 289. Interestingly, during the COVID-19 crisis, community engagement in the SFP activities witnessed a significant boost. Discussions with SMC members and parents highlighted various instances where they took initiatives to ensure smooth door-to-door distribution of biscuits and continuation of education for all students, while the schools remained closed. SMC members also cited instances where they conducted home visits to follow up on students’ education at home and promote the importance of continuing their child’s education amongst parents. During discussions, parents also reported assisting the school teachers in biscuit distribution and taking measures to ensure continuation of their child’s education during the pandemic. These findings indicate towards a prioritization of quality nutrition and education amongst community members. Increased ownership of SFP activities amongst community members, as highlighted above, showcases the readiness of the community to carry out SFP activities with support from GoB. However, to sustain this momentum, there is a need for a community engagement strategy within PSMP to create and strengthen engagement and ownership for smooth implementation of the planned activities. 3.6. Impact of COVID-19 on the programme 290. The COVID-19 pandemic posed numerous challenges that affected the implementation of the SFP. The end-term evaluation examines WFP’s adaptation of its implementation strategy to deliver the SFP in the context of COVID-19, assess the changes made based on emerging needs and the results achieved through them. On examining the status of implementation of recommendations made during the mid-term evaluation, WFP reported that some of the recommended action points could not be undertaken as the mid-term evaluation report was finalised in December 2019 and the restrictions due to COVID-19 were imposed from February 2020. 291. Findings from the monitoring reports and discussions with WFP and RIC representatives at the national and Cox’s Bazar level suggest that the programme activities were re-aligned to ensure that the needs of the students and communities were met during COVID-19 school closures. The programme also attempted to address the changing needs of beneficiaries that emerged as a result of the pandemic, while carrying out its regular activities. Wherever feasible, a shift to remote modalities and direct support to students’ homes to deliver the programme activities was made. WFP along with GoB and implementing partners, collectively worked towards minimizing disruption to students’ learning and nutrition. Some of the new mechanisms created and adaptations made to address the challenges associated with COVID-19 have been mentioned below. 292. WFP, with support from MoPME and DPE, developed the standard operating procedures and guidelines for the distribution of biscuits to students’ homes (home delivery of biscuits-HDB) to address school feeding needs in light of school closures related to COVID-19. Prior to distribution, WFP held online meetings with partner staff to orient them on the new distribution guidelines and provided necessary resource materials to them. Through this, nearly 2.8 million students received biscuits at their homes through multiple rounds of distribution. This ensured that all students continued receiving important micronutrients during the crisis. WFP conducted remote monitoring during and after distribution to ensure effectiveness and continuity of the intervention. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 44 293. To further the literacy activities, WFP participated in two webinars on COVID-19 and education. The first, with UNICEF, the World Bank and Teach for Bangladesh, focused the overall impact on learning. The second, with the MoPME Senior Secretary, Room to Read, and other international experts, focused on social emotional learning and literacy. Additionally, Room to Read re-designed existing materials to create “book cards”, full colour, double-sided cards with stories and questions for students to ensure continuity of efforts to improve reading fluency and comprehension. 35,377 students (Grade 1 to 5) received story cards so that they could continue reading and working towards improving their literacy skills while schools remained closed. Room to Read also distributed 6,850 Grade 1 and 7,477 Grade 2 student workbooks at home to provide students with additional materials to practice their literacy skills. Furthermore, RtR worked with NCTB to mainstream these materials in the school curriculum. 294. Other efforts to ensure continuity of students’ education included leveraging online platforms. To address student learning needs, GoB began virtual classes through a national TV channel for primary school students and Room to Read leveraged social media sites such as Facebook to coordinate with teacher groups to ensure the delivery of education services. Teachers uploaded class wise pre-recorded videos on the Facebook page along with instructions for the parents. SMC members mentioned that in cases where students did not have an access to the internet or a smartphone, they often went over to their neighbour’s house to study with the students there. In addition to this, parents mentioned that they also sought help from private tutors as well as TV and radio channels to help in their child’s education. By disseminating messages through interactive voice recordings (IVR) and training parents on positive reading habits through text messages, at-home academic lessons for students continued at home during the pandemic. 295. Despite all of these efforts, discussions with government officials and RtR revealed constraints with regard to access to online platforms and digital devices especially in Kutubdia, owing to its remoteness and availability of infrastructure. 296. Given the emerging health issues, WFP partnered with the Food Security Sector’s Livelihood Working Group as well as other partners to secure reusable cloth masks for all primary school children in Ukhiya and Kutubdia. Partner (RIC) staff also disseminated messages related to COVID￾19 health measures to parents and children during biscuit distribution to spread awareness and ensure prevention of spread. In addition to sharing awareness messages verbally, the field staff also provided printed leaflets outlining health and safety measures. 297. To further promote hygiene practices and enhance students’ knowledge of health and nutrition, with the aim of improving the Essential Learning Package, WFP partnered with Sesame Workshop Bangladesh to develop materials and conduct virtual ToTs. 298. These interventions as a response to the pandemic and the resultant re-alignment of activities as well as the implementation strategy ensured that the programme remained responsive in the changing context. 299. While WFP continues to work with local, district and national level officials to ensure schools are safe for students’ return upon school reopening, there is a need to for resilient programming to ensure that the impact of the programme is sustained. 4. Conclusions 300. The SFP interventions and strategy are aligned with the priorities of GoB as highlighted in NEP 2010, the National Nutrition Policy (NNP) 2015, the NSSS 2015 as well as NPAN-2 as they promote and support the various objectives on nutrition, health, hygiene and education mentioned as part of these government policies. 301. Further, since the SFP activities address the felt need for interventions to increase children’s interest in education related activities and to improve their performance, aims to address challenges related to awareness around attendance, enrolment and literacy and provide platforms to encourage the same, while also contributing towards strengthening GoB’s existing initiatives, End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 45 the SFP adequately targets the right people with the right type of assistance and was therefore, found to be relevant. 302. Through SFP, WFP has made consistent efforts in sensitising representatives of different ministries of GoB on the relevance of the school feeding programme and its activities, and building consensus at the GoB level on a minimum nutritional requirement for school meals. These efforts contributed to the formulation and approval of the nutrition-sensitive National School Meal Policy 2019. The assistance provided by WFP, in terms of building capacities at MoPME, DPE and strengthening inter-ministerial coordination (MoPME with other ministries), is well recognised by the MoPME and Ministry of Agriculture. The technical assistance provided by WFP enhanced the government’s understanding of implementing nutrition-sensitive interventions, encouraged them to initiate a discourse on providing school meals and helped them to identify opportunities to mainstream complimentary activities within the existing educational framework. 303. At the programme level, the higher end-term values as compared to baseline values indicates that the programme has been able to achieve the intended results (for certain indicators). However, the pandemic caused postponement of certain training activities, due to which the targets have not been able to be met. Comparison of end-term values with mid-term and baseline values for intervention and comparison schools is provided in Annex T. 304. The evaluation findings highlight an increase in understanding among the community on the benefits of education, increase in the skills and knowledge of the teachers and administrators, an increase in the demonstration of new teaching techniques and tools by the teachers and improvement in fluency and comprehension among students, an increase in the overall knowledge about good nutrition, health, hygiene and sanitation as well as safe food preparation and storage practices along with a rise in aspirational levels of students as a result of their identification and recognition as book captains and little doctors. This has further motivated students to demand quality education to fulfil their aspirations and promoted positive nutrition, health and hygiene practices. This, along with the ownership and commitment shown by the community and students to continue their education despite the closure of schools (on account of COVID-19) serves as evidence that the programme has been effective and has been able to achieve its strategic outcome on improving literacy of school age children and increased use of health and dietary practices by the end of the programme. 305. The evaluation brought to light a lack of gender disaggregated data in the monitoring of activities and the findings also point towards a need to mainstream gender within the programme to increase the participation of women in SMCs as well as to involve mothers towards enabling quality education and enhanced nutrition for both boys and girls. The findings indicate a need for gender sensitization amongst mothers who view education only as a means to make girls better mothers in the future which shows a limited understanding of the benefits of education. 306. There has been an increase in the fluency and comprehension abilities of students in intervention schools (based on the findings from MTE). The findings suggest that the program processes have been effective in (a) ensuring that all the students in the class get to read story books as per their abilities (b) motivating and influencing students to adopt and enhance reading habits. The parents have proudly expressed an intent to support their children in pursuing quality education. The enhanced understanding of the importance of education has translated in increased attendance and enrolment of students in intervention schools as compared to baseline. 307. There has also been an increase in the demonstration of new teaching techniques and tools by teachers and head teachers. While WFP may not have met the targets set out for training of teachers, the increased demonstration of teaching techniques is testament to the effectiveness of the capacity building activities conducted by WFP. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 46 308. The process of demand forecasting, supply and distribution of biscuits on a regular basis was found to be efficient. Additionally, the process of reporting progress on various activities in ensuring timely submission of monitoring reports was also found to be efficient. 309. Biscuits are distributed regularly to both boys and girls and there is no gender-based differentiation or discrimination. Despite an increase in demand for and awareness about quality education, the distribution of biscuits continues to be a major influencing factor for parents to send their children to school and for students to attend the school. 310. Joint Monitoring Visits (by WFP staff, GoB officials at district and upazila level and implementing partners) have not been conducted regularly which affects timely review of programme activities and timely course correction. 311. The participation of SMC members in SFP interventions and supporting schools in improving infrastructure, promoting community engagement, encouraging enrolment and regular attendance of students has been effective. The SMC’s active involvement in the implementation of activities during the pandemic was also key to ensure smooth functioning of the interventions and that children’s access to adequate nutrition and education is not disturbed. However, the gender balance in SMC as well the participation of women continues to be skewed in favour of men. 312. The increased participation of the community in school activities such as healthy meal preparation day, assistance for the door-to-door distribution of biscuits and existence of parent teachers’ association provides evidence of increasing community engagement in supporting school￾level implementation of SFP activities. This is an enabler for the sustainability of SFP’s interventions. 313. Realising the importance of SFP, there have been various initiatives taken by the GoB officials at national and district level, teachers and the community to independently take forward interventions. The pending approval of PSMP and significant budget allocation by GoB demonstrates its commitment and willingness to take the ownership of the programme and can be considered as an achievement of an important milestone towards ensuring sustainability of the SFP. 314. Regardless of the challenges posed by the COVID-19 pandemic, the project has been successful in improving awareness about nutrition, health and hygiene. Considering an increased awareness amongst and adoption of health and dietary practices by the parents and their children, it can be inferred that the project has made an impact towards increasing the use of improved health and dietary practices. 315. GoB’s efforts towards mainstreaming new teaching techniques and nutrition knowledge within the existing government teacher training modules and school curriculum have further enabled sustainability of achievement of learning and nutrition outcomes. 316. Significant levels of engagement and strong rapport with the community as well as with the government officials established by the implementing partners has helped in timely implementation of the planned activities under the programme. The adaptations made in the implementation strategy to ensure the continuity of activities due to COVID-19 was also successful due to the involvement of all the stakeholders and the collective ownership showed by each one of them. 317. The technical assistance provided by WFP has supported the GoB in designing the policy framework for ensuring inclusive and equitable quality education and improved nutrition for primary school children. Further, the capacity building support has enabled MoPME to identify capacity gaps, and prepare and implement a capacity building plan to independently undertake implementation of the NSMP. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 47 5. Recommendations Sl. No. Recommendations Rationale Proposed actions Type Timeframe Policy Advocacy (National Level) 1. Provide continued technical assistance to MoPME and DPE in the operationalization and coordination of activities under the NSMP. The NSMP has been approved and MoPME would require technical support in implementation, based on learnings from SFP. WFP can support MoPME in establishing a National School Meal Authority (NSMA) that can coordinate with multiple stakeholders and provide them technical assistance in implementation of NSMP. Further, WFP can support in developing systems for operationalization of the NSMP, including development of standard operating procedures for implementation. Additionally, WFP can provide capacity building support to GoB (national and local level) on the following:  Conducting local and transparent procurement  Programme planning, implementation and monitoring  Promoting food safety and hygiene practices, warehouse management, supply chain management, advanced computer literacy and online database management  Establishing a monitoring framework and reporting system  Design and implementation of a community mobilization strategy  Design and implement a Social Behaviour Change Communication strategy  Documenting processes and learnings Operational Long term 2. Provide technical assistance to MoPME in establishing a Research and Development (R&D) Centre that can draw from and contribute to the global experiences on school feeding/school meal programming (based on learnings from Bangladesh). The R&D Centre would also conduct research on applied nutrition in the local context. The evidence and knowledge generated will be utilized by MoPME and National School Meal Authority (NSMA) would require an effective research wing to build knowledge, generate evidence and provide technical assistance to other relevant ministries as advised in the approved “National School Meal Policy 2019”. WFP can support MoPME, MoA in creating a blue print for establishing the R&D Centre. The technical assistance would include:  Preparing mandate/scope of work for the Centre. This would include identifying areas for research on aspects such as applied nutrition, nutrition retention (post-harvest crop management), developing indigenous and nutritious varieties in hilly areas,  Preparing plan for operationalization of the Centre  Preparing estimates on requirement of human and financial resources and infrastructure.  Conducting research as needed Strategic Short to medium term End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 48 Sl. No. Recommendations Rationale Proposed actions Type Timeframe GoB officials at national and local levels for informed decision making. 3. Increased engagement and advocacy for mainstreaming of new teaching techniques and methods with the existing curriculum prescribed by GoB and the adoption of these techniques at scale. The parents’ perception of teachers’ new teaching techniques is positive. Students (during MTE) and parents (both during MTE and ETE) have indicated that learning through the use of participatory techniques and colourful printed visual aids is faster and more interesting than learning from normal textbooks. However, the shortage of teachers in some schools and the absence of mainstreaming these methods and modules into the existing GoB curriculum acts as a barrier for sustained success. WFP should consider dissemination of evidence on the benefits of the adoption of new teaching techniques generated through this programme at the national level (MoPME, DPE), thereby making the case for consideration by GoB to scale up this initiative. Strategic Medium to long term 4. Explore alternate packaging of biscuits. In the absence of a proper waste disposal mechanism for plastic wrappers, schools follow the practice of burning the wrappers in the open air instead of disposing of them responsibly. Given the adoption of the programme at the national level, WFP should explore alternate packaging of biscuits and minimise the generation of plastic waste. Waste to Art events or competitions may also be organized in schools to serve the dual purpose of raising awareness about fortified biscuits as well as ensuring waste recycling. Operational Short term Actionable (Specific to SFP implementation in Cox’s Bazar) 5. Conduct urban school meal pilots. SFP has demonstrated its relevance and impact in creating an enabling environment for enhancing literacy of students and knowledge of health and hygiene practices among parents and children in rural areas. There is a need for similar interventions in urban areas. WFP should provide technical assistance to MoPME and other relevant ministries to replicate school meal programmes in schools in urban areas. Based on the learning and evidence generated herein, WFP can undertake advocacy for scaling up the programme in schools in urban areas across the country. Strategic and Operational Short to medium term End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 49 Sl. No. Recommendations Rationale Proposed actions Type Timeframe 6. Enhanced focus on improving communities’ access to learning opportunities during school closures. While, there is an increase in awareness on nutrition among students and parents, dietary diversity in intervention schools as compared to comparison schools is still low. Further, the closure of schools (COVID-19), has not only increased the disparity in access to education (between those who can and cannot access learning through online mediums) but has also constrained interventions under the programme to improve literacy levels of students. The need is more pressing for the schools in more poverty prone areas or remote locations in Kutubdia where access to computers or internet is limited.  WFP with the support from RtR should focus on creating digital versions of library books and provide both traditional and e-book formats, as well as interactive formats with options to read text aloud to the schools through online portals. Access to these for download by the teachers, students and parents, for classroom instruction or for reading to children at home can be made available. WFP however, must also ensure access to devices as well as internet in remote locations so that these services, when ready, can be equitably accessed. Operational Short term to medium term 7. Develop new menus for nutritious meals in schools WFP has engaged with BIRTAN and provided training to cooks on aspects such as nutrition, balanced diets and the nutritional value of locally available food. Under the NSMP, students would be provided with cooked meals in schools at least thrice a week.  WFP, with the support of implementing partners and trained cooks, should develop multiple nutritious menus. The menu should also include nutritional information and serving sizes for each meal to promote awareness. Operational Short to medium term 8. Strengthen community engagement to promote increased participation in supporting the implementation of activities under the programme. As compared to the baseline, there is an increase in the number of mothers who have reported their participation in the programme activities. The programme also witnessed active engagement of community members in supporting door-to￾door biscuit distribution during COVID-19.  Highlighting this as evidence, WFP should engage in advocacy by articulating the need and benefits of community participation in the SFP and support MoPME in defining contours for community engagement.  WFP should also consider training community members on procurement of raw materials, preparation of nutritious meals, storage and distribution. The capacity building efforts would enable enhanced community participation in the implementation of the NSMP (at school level) as envisaged in the NSMP. Strategic Medium term End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 50 Sl. No. Recommendations Rationale Proposed actions Type Timeframe 9. Strengthen the Healthy Meal Preparation Day initiative and provide more opportunities for Little Agriculturists to demonstrate their skills. Awareness about Healthy Meal Preparation Day is low as it is often clubbed with regular school picnics. Additionally, Little Agriculturalists (in schools with no vegetable gardens) lack a platform to deliver messages and build awareness about nutrition to their peers. This challenge is further exacerbated due to the non-maintenance of vegetable gardens owing to the closure of schools.  The implementing partners should encourage schools to conduct Healthy Meal Preparation Day separately. Additionally, like Little Doctors and Little Agriculturists should be provided with ample opportunities to demonstrate their acquired skills and deliver messages about nutrition in various events organised by the school. This needs to be coupled with increased efforts to re-instate the vegetable gardens that may have become non-functional during the pandemic. Operational Short term 10. Revitalize Joint Monitoring Visit mechanisms. Joint Monitoring Visits (by WFP staff, GoB officials at district and upazila level and implementing partners) are not conducted regularly.  WFP should re-emphasise the need of conducting joint monitoring visits regularly and provide guidance to the implementing partners so that they can engage with the GoB officials and ensure regularity of these visits Operational Short term 11. Promote enhanced participation of women in SMC activities and incorporate gender aspects in future programming. While there is an increase in the female to male composition ratio of SMCs, participation of women in SMC activities is limited. Further, while there is an increased understanding amongst parents about the value of education, especially for girls, there is a need to generate awareness on gender aspects so that they can move beyond set gendered roles and envision a different future for their daughters (not limited to their role as a mother).  WFP, along with GoB, needs to conduct more gender sensitization workshops with SMCs to ensure equal representation and active participation of women.  WFP should also consider conducting trainings and sensitization workshops with parents to change social norms around gender roles and enhance the understanding of women and girls in society, especially with regard to education and decision-making. This would ensure increased impact on all the programme outcomes for future programmes. An overall mainstreaming of gender within the programme should also be considered while designing future programmes. To ensure mainstreaming of gender in the implementation of activities, inclusion of gender specific indicators for monitoring as well as sensitization activities conducted with the programme staff as well as the beneficiaries should be considered. Strategic Medium term End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 51 Annex A: Map of the Intervention Area End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 52 Indicates (SFP) Intervention Areas within Cox’s Bazar End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 53 Annex B: Project-Level Results End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 54 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 55 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 56 Annex C: Planned Outcomes of WFP Bangladesh McGovern Dole -FY17 Award Table 5: Planned outcomes and annual targets Activity Indicator Target for FY 2017 Target for FY 2018 Target for FY 2019 Target for FY 2020 Target for FY 2021 Build Capacity Number of child healthand nutrition policies, regulations, or administrative procedures in each of the following stages of development as aresult of USDA assistance: 0 1 1 1 0 Number of child healthand nutrition policies, regulations, or administrative procedures in each of the following stages of development as aresult of USDA Assistance:- Stage 1:Analyzed 0 0 0 0 0 Number of child health and nutrition policies, regulations, or administrative proceduresin each of the following stages of development as aresult of USDA Assistance: - Stage 2: Drafted and presented forpublic/stakeholder consultation 0 1 0 0 0 Number of child health and nutrition policies, regulations, or administrative proceduresin each of the following stages of development as a result of USDA 0 0 0 0 0 Assistance: - Stage 3: Presented for legislation/decree - Number of child healthand nutrition policies, regulations, or administrative procedures in each of the following stages of development as a result of USDA Assistance: - Stage 4: Passed/Approved 0 0 1 0 0 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 57 Activity Indicator Target for FY 2017 Target for FY 2018 Target for FY 2019 Target for FY 2020 Target for FY 2021 Number of child healthand nutrition policies, regulations, or administrative procedures in each of the following stages of development as aresult of USDA Assistance:- Stage 5:Passed for which implementation has begun 0 0 0 1 0 Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance: 0 1 1 1 0 Drafted and presented for public/stakeholder consultation Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance: Stage 3: Presented for legislation/decree 0 0 0 0 0 Number of educational policies, regulations and/or administrative procedures in each of thefollowing stages of development as a result ofUSDA assistance: Stage 4: Passed/Approved 0 0 1 0 0 Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance: Stage 5: Passed for which implementation has begun 0 0 0 1 0 Number of Parent-TeacherAssociations (PTAs) or similar “school” governance structures supported as a result of USDA assistance 0 48 146 146 0 Number of public-private partnerships formed as a result of USDA assistance 0 1 0 0 0 Value of new public and private sector investments leveraged as a result of USDA assistance 0 69 23,100,000 23,100,000 0 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 58 Activity Indicator Target for FY 2017 Target for FY 2018 Target for FY 2019 Target for FY 2020 Target for FY 2021 Value of new public and private sector investments leveraged as a result of USDA assistance (HostGovernment): 0 69 23,100,000 23,100,000 0 Value of new public and private sector investments leveraged as a result of USDA assistance (other public): 0 0 0 0 0 Value of new public and private sector investments leveraged as a result of USDA assistance (private): 0 0 0 0 0 Distribute Food Number of daily school meals (Packets of Fortified Biscuits as snacks) provided to school-age children as a result of USDA assistance 0 9,323,176 6,999,982 9,333,310 2,333,328 Number of individuals benefiting directly from USDA-funded interventions 0 48,711 48,643 48,643 48,643 Number of individuals benefiting directly from USDA-funded interventions (continuing): 0 42,444 42,999 42,299 42,299 Number of individuals benefiting directly from USDA-funded interventions (female): 0 26,021 25,211 25,211 25,211 Number of individuals benefiting directly from USDA-funded interventions (male): 0 22,690 23,432 23,432 23,432 Number of individuals benefiting directly from USDA-funded interventions (new): 0 6,267 6,344 6,344 6,344 Number of individuals benefiting indirectly from USDA-funded interventions 0 190,756 167,158 167,158 167,158 Number of individuals receiving take-home rations as a result of USDA assistance (continuing) 0 0 0 0 45,062 Number of individuals receiving take-home rations as a result of USDA assistance (female): 0 0 0 23,883 23,883 Number of individuals receiving take-home rations as a result of USDA assistance (male): 0 0 0 21,179 21,719 Number of individuals receiving take-home rations as a result of USDA assistance (new) 0 0 0 45,062 0 Number of individuals receiving take-home rations as a result of USDA assistance: 0 0 0 45,062 45,062 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 59 Activity Indicator Target for FY 2017 Target for FY 2018 Target for FY 2019 Target for FY 2020 Target for FY 2021 Number of school-age children receiving daily school meals (Packet of Fortified Biscuits as snacks) as a result of USDA assistance 0 47,689 46,433 46,433 46,433 Number of school-age children receiving daily school meals (Packets of Fortified Biscuits as snacks) as a result of USDA assistance (continuing): 0 41,422 40,089 40,089 40,089 Number of school-age children receiving daily school meals (Packets of Fortified Biscuits as snacks) as a result of USDA assistance (female): 0 24,701 24,611 24,611 24,611 Number of school-age children receiving daily school meals (Packets of Fortified Biscuits as snacks) as a result of USDA assistance (male): 0 21,988 21,822 21,822 21,822 Number of school-age children receiving daily school meals (Packets of Fortified Biscuits as snacks) as a result of USDA assistance (new): 0 6,267 6,344 6,344 6,344 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (continuing): 0 41,422 40,089 40,089 40,089 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (female): 0 25,701 24,611 24,611 24,611 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (male): 0 21,988 21,822 21,822 21,822 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance new): 0 6,267 6,344 6,344 6,344 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance: 0 47,689 46,433 46,433 46,433 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 60 Activity Indicator Target for FY 2017 Target for FY 2018 Target for FY 2019 Target for FY 2020 Target for FY 2021 Quantity of take-home rations provided (biscuits in metric tons) as a result of USDA assistance: 0 0 0 338 253 Improve Student Enrollment and Attendance Number of students enrolled in school receiving USDA assistance 0 47,689 46,433 46,433 46,433 Number of students enrolled in school receiving USDA assistance (female): 0 25,701 24,611 24,611 24,611 Number of students enrolled in school receiving USDA assistance (male): 0 21,988 21,822 21,822 21,822 Number of students regularly (80%) attending USDA supported classrooms/schools 0 38,151 37,146 37,146 37,146 Number of students regularly (80%) attending USDA supported classrooms/schools (female) 0 20,561 19,689 19,689 19,689 Number of students regularly (80%) attending USDA supported classrooms/schools (Male): 0 17,590 17,457 17,457 17,547 Promote Improved Health Number of educational facilities (i.e. water systems and latrines) rehabilitated/ constructed as a result of USDA assistance 0 12 35 0 0 Number of individuals trained in child health and nutrition as a result of USDA assistance (female): 0 232 292 292 292 Number of individuals trained in child health and nutrition as a result of USDA assistance (male): 0 348 438 438 438 Number of individuals trained in child health and nutrition as a result of USDA assistance: 0 580 730 730 730 Number of individuals who demonstrate use of new child health and nutrition practices as a result of USD assistance: 0 28,600 28,600 28,600 28,600 Number of schools using an improved water source 0 48 146 146 146 Number of schools with improved sanitation facilities 0 48 146 146 146 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 61 Activity Indicator Target for FY 2017 Target for FY 2018 Target for FY 2019 Target for FY 2020 Target for FY 2021 Number of students receiving deworming medication(s) 0 47,689 46,433 46,433 46,433 Number of students receiving sanitation kits 0 0 0 0 45,062 Promote Improved Nutrition Number of individuals trained in child health and nutrition as a result of USDA assistance 0 580 730 730 730 Support Improved Literacy Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance 0 92 92 116 116 Number of school administrators and officials trained or certified as a result of USDA assistance 0 116 116 146 146 Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance 0 464 584 620 620 Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance 0 580 730 730 730 Number of textbooks and other teaching and learning materials provided as a result of USDA assistance 0 146,728 90,125 90,024 45,062 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text 0 25 35 35 35 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (female): 0 25 35 35 35 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 62 Activity Indicator Target for FY 2017 Target for FY 2018 Target for FY 2019 Target for FY 2020 Target for FY 2021 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (male): 0 25 35 35 35 Support Improved Safe Food Preparation and Storage Number of individuals trained in safe food preparation, commodity management and storage as a result of USDA assistance 0 322 438 438 0 Number of individuals trained in safe food preparation, commodity management and storage as a result of USDA assistance (female): 0 49 66 66 0 Number of individuals trained in safe food preparation, commodity management and storage as a result of USDA assistance (male): 0 273 372 372 0 Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance 0 290 438 438 438 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 63 Annex D: Activity wise Graduation Timeline Table 6: Activity wise Graduation Timelines Activity Key Milestones Timeline Build Capacity  School Feeding National Policy will be finalized by first quarter 2018  Follow up of SABER 1-day annual workshop to track progress against the five policy goals will be done by last quarter each year  Draft School Feeding Policy Implementation Strategy will be finalized by early 2020  Learning sharing workshop based on pilot school feeding modalities will be carried out by last quarter 2019  Technical support for strengthening the GoB system to establish school feeding supply chain and procurement and quality control (2018-2020)  Support to asset management online reporting and tracking (2018- 2020) January 2018- September 2020 Distribute Food  Agreement signed, EP and PMP approved, Call forward initiated  Wheat will arrive in Chittagong Port by April 2018, biscuit factory (supplier) receives wheat for production, production ensues, biscuits will be delivered to programme locations (10-week long process)  Biscuits will be served in school by10: 30 am to grade 1-5 July 2018-September 2020 Improve Student Enrolment and Attendance  Programme launch at each school (146 total)  Distribution of student stationaries and school bag for all grades (pencils, crayon, Eraser, Sharpener & kit.) approximately 48,000 students  Library opening ceremony at each school (146)  Quarterly programme review meeting  Annual progress sharing meeting  Student Recognition Day January 2018- September 2020 Promote Improved Health  Reconstruction of latrines and water systems in 35 schools begins in a year, late 2018.  Identification and orientation to “Little Doctors” will be done by first quarter 2018. Approximately 15 little doctors per school (2,190 total). Approximately 146 orientations will be given in total, one per school. January 2018- September 2020 Promote Improved Nutrition  Establish 60 garden demonstration plot at school January 2018- September 2020 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 64 Activity Key Milestones Timeline  Train approximately 5 teachers at each school (730 total) in healthy meal preparation Support Improved Literacy  Community mobilization orientation once per year at each community  Publication of children's magazine (15 copies per school/once a year)  Library bookshelf distribution (once) for 146 government primary school classes and 55 madrasa classes.  Grade 1-2 teacher training in Bangla reading instruction, approximately 292 teachers  Primary Training Institution and Upazila Resource Center instructor ToT  Head teacher training on school performance, quality improvement, governance, transparency management, monitoring, and evaluation, approx. 146, one per school  Basic training for the programme on instruction and programme implementation January 2018- September 2020 Support Safe Food Preparation and Storage  Initial training at field level (18 total, 25 people each) in year 1  Central level training for factory officials (25-35 people; 1 training); May 2018  Yearly refresher training by NGOs at school level approximately 438, three per school January 2018- September 2020 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 65 Annex E: McGovern Dole Target Beneficiaries and Funding for WFP School Feeding Programme A. McGovern Dole Target Beneficiaries Project Population Participant Approximate number Upazilas 2 Schools 146 Students 47,689 Parents 95,738 School Administrators 146 Teachers 730 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 66 Annex F: Role of Partners Table 7: Role of partners Partners Role of Partners MoPME and DPE  Management of programme with adequate quality control measures by establishing supply chain, food storage facilities etc.  Establishing proper monitoring and reporting systems  Designing and implementing pilots with various combinations of ingredients for the future  Engaging and learning from community awareness activities to make school feeding nutrition sensitive  Independent procurement of biscuits  Establishing a National School Feeding Policy and Strategy  Generating evidence on best practices during implementation of National School Feeding Policy and Strategy MoPME, DPE  Provision of fortified biscuits during 2018, 2019 and 2020 school years in grades 1-5  Delivery and distribution of food using WFP’s extensive pipeline and supply chain management system  Monitoring distribution of food and it’s safe storing, tracking and reporting  Providing guidance to school administrators, monitor performance and providing feedback for further improvement RtR, YPSA, MA and RIC Room to Read  Procure bookshelves for libraries for 709 government primary school classes  Initiate Book Checkout Register, Supplementary Reading Material register, visitor Registrar to the schools  Distribute books to library (both RtR and purchased) and Book leveling sticker  Conduct Library Rating System. RtR will introduce the rating tools but rating conducted by YPSA/MA Literacy Facilitators  Lead teacher training on Bangla  Lead training on Library Management for teachers, primary training institute and Upazila Resource Center  Conduct baseline and midline assessment on reading schools  Lead ToT on introduction at Primary Training Institute and Upazila Resource Center  Lead Quarterly Programme review meeting with implementing partners (3 meetings); for key people from WFP to also attend at district level  Lead Annual Progress Sharing Meeting with WFP and MA/YPSA for key people from WFP to also attend at district level  Lead Progress Sharing Meeting with Government Officials twice a year in two Upazilas  Lead Technical Review Meetings with Literacy Programme Officer Technical and Literacy Facilitators (literacy team) in two Upazilas YPSA, MA and RIC  Support implementation and distribution of fortified biscuits to programme schools.  Liaising with WFP on distribution of FAO designed health and hygiene posters (5 per school)  Liaise with Room to Read’s Literacy Project Officer on the implementation and facilitation of trainings, distribution of materials  Identify any potential issues with programme implementation and communicate such issues with WFP  Participate, facilitate and encourage active engagement in programme launch, distribution of programme materials (books, pens, backpacks, library shelves) at the school level  Support student tracking on library usage for grades 1 and 2.  Creating literate environment in the school in grade 1 and 2 (budget classroom wise). This involves o classroom coloring and decoration o setting display board (for display of students' work) o providing wall clock o displaying materials (grade specific posters on rhymes and poems etc.) on as needed basis.  Initiate wall magazine development – once a year, school based; once a year, Upazila level  Initiate wall magazine competition  Initiate Upazila based recognition event for school teachers, SMCs  Organize Community Mobilizer orientations  Provide WFP with monitoring data such as amount of food distributed, average student attendance, and number of latrines and water systems rehabilitated YPSA, MA and RIC  WFP will guide and monitor YPSA and MA in hygiene reconstruction and information communication messaging. The community will be engaged through monthly SMC meetings.  YPSA and MA will support rehabilitation of water systems and latrines. This will involve identifying what materials are needed for rehabilitation, procuring the materials and rehabilitating the water systems or toilets. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 67 Partners Role of Partners  WFP will train YPSA and MA on the ‘Little Doctors’ activity which includes lessons on how to measure weight, height, vision test and assist in government distribution of deworming.  WFP will monitor this activity through field monitors and maintain consistent communication between government, schools, MA and YPSA.  YPSA and MA will be trained to provide on the spot guidance, as needed, to teachers and students during routine monitoring. YPSA. MA and RIC  WFP will monitor the project and train YPSA and MA on basic agriculture skills.  YPSA and MA will implement this activity at the school level, training teachers and overseeing project performance.  YPSA and MA will be trained to provide on the spot guidance, as needed, to teachers and students during routine monitoring.  YPSA and MA will re-enforce “Little Agriculturalist” and the setting up of gardens RtR, MA, YPSA  RtR will develop the training package, provide ToT and co-facilitate the event with other government trainers at the Upazila level  RtR and WFP will liaise with government to obtain training approvals  MA and YPSA will facilitate school level trainings MoPME, YPSA, MA, DPEO  WFP will organise trainings to YPSA and MA and government officials (DPEO) on food storage practices.  YPSA and MA will train school teachers, head administrator and SMC members at the local level.  MoPME officials will be trained in monitoring the biscuit manufacturers in food safety standards for safe preparation of biscuits and appropriate biscuit storage practices.  Government officials will be trained in monitoring the biscuit manufacturers. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 68 Annex G: Stakeholder Analysis Table 8: Stakeholder analysis Stakeholder Interest in the [Intervention/Project/Operation] Involvement in Evaluation and likely use Who (specifically for the Evaluation) Internal (WFP) Stakeholders WFP CO Bangladesh  Responsible for overall management and provide support relating to logistics, procurement and finance.  Responsible for overall coordination, liaison with stakeholders, implementation oversight and capacity building.  Responsible for the coordination of the activity’s implementation.  Supervising the proper management of warehouses, establishing the supply chain for biscuits and providing capacity building support to the Government.  Overseeing the overall financial management.  Overseeing collection of data and maintaining data base; data analysis and reporting, capacity support in institutionalizing monitoring and reporting system in MoPME, DPE etc.  Central level coordination, planning, designing, guiding and training for implementation, progress tracking, demonstration of results including reporting  Providing field support, liaison with sub-offices and providing M&E support  The key informant, primary stakeholders, and users of this evaluation.  Initial briefing and overview of WFP work in Bangladesh, programme documents, help evaluation team better understand the context of implementation and strategy for future;  Support the evaluation team through an introduction to key stakeholders;  Review Inception Report and Draft End – Term Evaluation Report  Involved in using evaluation findings of USDA Mc Govern Dole FY17-20 grant  As evidence to inform operational and strategic decision￾making  To understand the impact of interventions and explore the strength of the exit strategy with a focus on achieving programme sustainability  CD  DCD (Programme)  DCD (Operations Support)  Head of Programme  Activity Manager  Evaluation Manager  Programme Policy Officer￾School Feeding (Implementati on)  Senior Programme Officer (SF)  Senior Programme Assistant  Senior Programme Officer (Resource Management) WFP Field Offices Cox’s Bazaar  Coordination between Cox’s Bazar and Dhaka Offices on school feeding programme; management of overall project timeline; ensuring timely submission of good quality deliverables; communication of programme process to relevant stakeholders.  Providing overall programme support for Ukhiya and Kutubdia sub-districts, ensuring communication flow to Senior Programme Officer; maintaining timelines, reviewing deliverables for submission to donor  Providing support for pipeline management and assisting the implementing agencies in proper tracking supply and utilization of commodities  Overseeing the overall financial management of project, providing support at field level for biscuits delivery, warehouse management  Coordinating, supervising and guiding implementation, monitoring and reporting at the field level  Sharing feedback to improve programme performance  The key informant, primary stakeholders.  As users of the evaluation findings to understand the extent to which needs of the targeted population have been met.  Programme Policy Officer￾SF  Programme Associate  Logistics Assistant End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 69 Stakeholder Interest in the [Intervention/Project/Operation] Involvement in Evaluation and likely use Who (specifically for the Evaluation) RBB Responsible for both oversight of COs and technical guidance and support. The RB management has an interest in an independent/impartial account of the operational performance. The Regional Evaluation Officer supports CO/RB  Key informant and primary stakeholder – involved in planning for next USDA Mc Govern Dole  Interested in the independent account of USDA MGD performance  Applying learning from evaluation to other country offices. No interviews will be conducted WFP HQ Policy and Programme WFP HQ technical units are responsible for issuing and overseeing the rollout of normative guidance on corporate programme themes, activities and modalities, as well as of overarching corporate policies and strategies. They also have an interest in the lessons that emerge from evaluation, as many may have relevance beyond the geographical area of focus. No interviews will be conducted WFP Executive Board (EB) The WFP governing body has an interest in being informed about the effectiveness of WFP operations. The WFP governing body has an interest in being informed about the effectiveness of WFP operations. This evaluation will not be presented to the EB but its findings may feed into annual syntheses and into corporate learning processes. Office of Evaluation (OEV) Provides indirect independent oversight DE Help Desk No interviews will be conducted External stakeholders SFP beneficiarie s They are the ultimate recipients of WFP support and therefore, have a stake in WFP determining whether its assistance is appropriate and effective. As such, the level of participation in the review of women, men, boys, and girls from different groups will be determined and their respective perspectives will be sought. Key informants and primary stakeholder – providing perspective on results, outcomes and emerging impact of WFP’s intervention Will be affected by the decision to continue or to stop the intervention A sample of schools to be selected and within each school, a sample of the following respondents will be interviewed:  School-going children in primary schools (equal number of boys and girls)  Teachers, Storekeeper, Headmaster  Parents , SMC members  Efforts would be made to ensure interviews with equal numbers of boys and girls  Individual (structured interviews) Government of Bangladesh End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 70 Stakeholder Interest in the [Intervention/Project/Operation] Involvement in Evaluation and likely use Who (specifically for the Evaluation) Ministry of Primary and Mass Education (MoPME) Directorate of Primary Education (DPE)Ministry of Health  Responsible for providing guidance on School Feeding priorities and approaches.  Receiving capacity support for School Feeding programme design and implementation at the national and sub national level  Management of programme with adequate quality control measures by establishing supply chain, food storage facilities etc.  Establishing proper monitoring and reporting systems  Engaging and learning from community awareness activities to make school feeding nutrition sensitive  Independent procurement of biscuits Key informant and primary stakeholder on government policy, priorities, views on support by WFP and on expanding school feeding, on GoB’s commitment to providing/mobilize resources, issues, and opportunities in handing over of the programme, capacities, and convergence to be explored. Will have perspective on sustaining the FY17-20 programme. Relevant officials District & Upazila Level education authorities Responsible for overseeing education sector performance, including implementation of National School Meal Policy, and liaising with other government departments at a decentralized level Primary stakeholder and key informant – on the implementation of the school feeding components. Will have perspective on challenges and achievements.  District Primary Education Officer  Upazila Education Officers (Ukhiya and Kutubdia)  Upazila Resource Centre Instructors Capacity Support Unit (in DPE) (WFP)  Leading the capacity support unit-CSU’s team on development of a national school feeding programme, supporting the Government’s scaling up school feeding and in the formulation of national school meal policy and implementation strategy  Administrating and programme implementation support. Support and providing on the job training to the relevant GoB officials  Supporting MoPME for the operationalization of National School Meal Policy and action plan  Providing liaison and programme support May be useful in mapping the assessing the existing capacities of MoPME and DPE.  iProgramme Policy Officer (SFCS)  Programme Assistants  Consultant  Washington DRO UN country team UNICEF UNICEF is also contributing to the development of Bangladesh in the sectors of health, nutrition, education and WASH. No interviews will be conducted DONOR agency USDA Funder of WFP school feeding programme Primary stakeholder and informant - on the relationship with WFP and the priorities moving forward Based on the advice of WFP, a Skype call can be organized if required. NGOs Room to Read  Procure bookshelves for libraries for 709 government primary school classes  Initiate Book Checkout Register, Supplementary Reading Material register, visitor Registrar to the schools  Distribute books to library (both RtR and purchased) and Book levelling sticker  Conduct Library Rating System. RtR will introduce the rating tools but rating conducted by YPSA/MA Literacy Facilitators  Lead teacher training on Bangla The results of the evaluation might affect future implementation modalities, strategic orientations, and partnerships.  Programme Operations Director  Literacy Director  Research Monitoring & Evaluation  Literacy Facilitators  Project Manager End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 71 Stakeholder Interest in the [Intervention/Project/Operation] Involvement in Evaluation and likely use Who (specifically for the Evaluation)  Lead training on Library Management for teachers, primary training institute and Upazila Resource Center  Conduct baseline and midline assessment on reading schools  Lead ToT on introduction at Primary Training Institute and Upazila Resource Center  Lead Quarterly Programme review meeting with implementing partners (3 meetings); for key people from WFP to also attend at district level  Lead Annual Progress Sharing Meeting with WFP and MA/YPSA for key people from WFP to also attend at district level  Lead Progress Sharing Meeting with Government Officials twice a year in two Upazilas  Lead Technical Review Meetings with Literacy Programme Officer Technical and Literacy Facilitators (literacy team) in two Upazilas Other Implementing partners such as Resource Integration Centre  Support implementation and distribution of fortified biscuits to programme schools.  Liaising with WFP on distribution of FAO designed health and hygiene posters (5 per school)  Liaise with Room to Read’s Literacy Project Officer on the implementation and facilitation of trainings, distribution of materials  Identify any potential issues with programme implementation and communicate such issues with WFP  Participate, facilitate and encourage active engagement in programme launch, distribution of programme materials (books, pens, backpacks, library shelves) at the school level  Support student tracking on library usage for Grades I and II.  Creating literate environment in the school in grade 1 and 2 (budget classroom wise). This involves  Classroom colouring and decoration  setting display board (for display of students' work)  Displaying materials (grade specific posters on rhymes and poems etc.) on as needed basis.  Initiate wall magazine development – once a year, school based; once a year, Upazila level  Initiate wall magazine competition  Initiate Upazila based recognition event for school teachers, SMCs  Organize Community Mobilizer orientations  Provide WFP with monitoring data such as amount of food distributed, average student attendance, and number of latrines and water systems rehabilitated Their respective perspectives will be sought as the engagement of these actors influences the effectiveness of the programme as well as its sustainability.  Upazila Coordinator  Field Monitors End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 72 Annex H: Summary of the methodology presenting the method of data collection, the target groups/sources of information, the nature of the interviews, and the indicative outputs Table 9: Summary of the Evaluation Methodology Methods of data collection Target Group/ Source of Information Primary (Virtual) Survey Secondary Review Quantitative (Structured Interviews)  Parent Questionnaire  School Questionnaire  School Head Questionnaire  School teachers Questionnaire  School Storekeeper Questionnaire  Project design document, results framework, annual action plans, and project activities and monitoring reports (quarterly, monthly, yearly) NPAN-2  MIS and monitoring data Report on National Education and School Feeding Policy Stakeholders Engagement Conference Qualitative (Focus Group Discussions and KIIs)  FGD with parents  FGD with SMC members Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE. Indicative outputs of data collection Establish achievement of targets at end-term for various indicators (excluding some due to COVID-19 crisis)  Compare end-term values of indicators such as enrolment and attendance (from direct and indirect influence) with baseline values; consider mid-term values for those indicators for which data cannot be collected  Identify key enablers and barriers with special emphasis on the COVID-19 context  Status of the performance and achievement of outcomes of the project and COVID-19’s effect on the same  Perceptions and overall experiences of the beneficiaries with the project Wherever appropriate sex disaggregated analysis has been provided  Establish relevance of the project and understand how the project activities were realigned to support GoB during COVID-19.  Establish alignment of the programme with government initiatives and WFP’s country Analyse the resources utilised  Evaluate the project performance  Identify key design elements that contributed to direct and indirect change  Explore pathways created for sustainability; scope of replicability; key considerations for future project design, especially from a global pandemic perspective End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 73 Annex I: Conceptual Framework for the End-Term Evaluation 1. The key envisaged impact of the MGD SFP is to improve the literacy of school going children and enhance their learning outcomes so as to advance their future health, productivity, and capabilities for overall well￾being. The distribution of high energy biscuits have several foreseen outcomes which include alleviation of short term hunger which is likely to improve the overall attentiveness of children during classes and school days; improved overall nutrition status of children through provision of calorific and fortified biscuits; and incentivise parents to send their children to school given the reduction in cost of schooling which is likely to translate into improved demand for education. 2. The MGD SFP also includes supporting interventions on the supply side that aim at building school teachers’ and school administrators’ skills and capacities and improve their attendance and performance in schools. Combined with better access to school supplies and materials, enhanced teaching capabilities create an enabling environment that is likely to augment regular attendance in school and improve learning outcomes for students. 3. Activities such as training on food preparation and storage practices and better access to water and sanitation services in school, promote safe health and dietary practices. Establishment of school gardens demonstrates to children and families how to diversify and improve their diet. 4. Through capacity development of the School Management Committee, the programme also aims to engage community and strengthen their local capacities to demand improved education. 5. In this light, the evaluation team has used an evaluation framework that includes the key thematic and functional areas that the interventions work upon. These can be viewed as four domains:  Individual (school going children),  Institutional (school teachers, school administrators, and store keepers),  Social (parents and community), and  External environment (policy, government support, civil society). 6. A schematic of the evaluation framework is presented below. 7. The external policy influence acts as the larger enabling environment which requires supportive Government policies. The cross cutting factors act as triggers or barriers influencing each of the actors and Figure 9: Conceptual Framework End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 74 structures in the three inner domains. For instance, the capacities of the teachers and the infrastructural support determine the quality of literacy instruction which has a key influence on the learning levels of the students. Similarly, the social and cultural norms often determine whether a girl child will go to school or stay at home and help with household chores. Likewise, boys may be pulled out of school for working in the fields especially during sowing and harvesting season or working as daily wage labour for contributing to household income. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 75 Annex J: Sampling Protocol 1 The sample was spread across two sub-districts of Cox’s Bazar. Fifty Government Public Schools (GPS) have been sampled. Out of these, 30 were intervention schools and the balance were comparison schools. The schools have been selected randomly in the same proportion as the number of intervention schools in the districts, thus ensuring the statistical representation of the sample. 2 While sampling, the diversity of the geographical locations of the schools (i.e., schools that are easily accessible by road and those that are located in remote/ difficult-to-reach areas) was ensured. 3 The sample size has been calculated at the programme level using the ‘differences method’ formula with a finite population (confidence interval of 1.96 and estimated difference set at 5 percent) as per Cochran (1977)[1] using the following formula: (𝑧 2 )𝑝𝑞 𝑑 2 p = estimated proportion of the population, set at 50% for maximum variance; q = 1 – p; z = standard score corresponding to the confidence interval (95%) set at 1.96; d = estimated difference, set at 5% for this initiative Accordingly, the sample size for treatment schools will be 390 students and that for comparison schools will be 260. In light of the fact that students were not sampled for the end-term evaluation113, parents from intervention (150) and comparison schools (100) were sampled. [1] Cochran, W. G. (1977) Sampling Techniques. 3rd Edition, John Wiley & Sons, New York. 113 Methodological Limitations (refer Section 1.4) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 76 Annex K: List of Sample Schools Upazila- Ukhiya S. No. Union School 1 Rajapalong Ukhiya Model Government Primary School 2 Rajapalong Moddo Rajapalong Government Primary School 3 Rajapalong Gohnarpara Shapi Government Primary School 4 Rajapalong Khoirati Government Primary School 5 Rajapalong Dailpara Government Primary School 6 Rajapalong Aminpara Ebtedayi Madrasha 7 Holodipalong D. Holdiapalong Government Primary School 8 Holodipalong Rumka Government Primary School 9 Holodipalong Nolbunia Government Primary School 10 Holodipalong Shaleh Bulbul Government Primary School 11 Ratnapalong Ruholladeba Government Primary School 12 Ratnapalong Telipara Government Primary School 13 Ratnapalong Valukia Ebtedayee Madrasha 14 Jaliapalong Lomboripara Government Primary School 15 Jaliapalong Motherbunia Government Primary School 16 Jaliapalong Dailpara Government Primary School 17 Palongkhali Balukhali Government Primary School 18 Palongkhali Anjumanara Government Primary School Upazila- Kutubdia S.No Union Name of School 1 Uttar Dhurong Char Dhurong Government Primary School 2 Uttar Dhurong Teliakata Government Primary School 3 Uttar Dhurong Baingakata Government Primary School 4 Dakhin Dhurong Jalilia Government Primary School 5 Dakhin Dhurong Elahia Government Primary School 6 Lemsikhali South Lemsikhali Government Primary School 7 Lemsikhali Central Lemsikhali Government Primary School 8 Koierbil Koierbil G.M Government Primary School 9 Borghop Kutubdia Model Government Primary School 10 Borghop Middle Aliakbor Dail Government Primary School 11 Borghop North Borghop Government Primary School 12 Aliakbor Dail East Aliakbor Dail Government Primary School End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 77 Upazila- Ramu S.No School Code School Name 1 91412040906 Kalaroada Government Primary School 2 91412040402 Ghonarpada Government Primary School 3 91412040411 Nonachodi Government Primary School 4 91412040501 Ramu Kendriya Government Primary School 5 91412040506 Ramu Khijari Burmese Government Primary School 6 91412040510 Paschim Merongloya Government Primary School 7 91412040101 Garjoniya Government Primary School 8 91412040105 Fakrikata Government Primary School 9 91412040203 Jhumchodi Government Primary School 10 91412040701 Rajarkul Government Primary School 11 91412040802 Panerchodha Government Primary School 12 91412040903 Dariyar Dighi Government Primary School 13 91412040303 Hasnakata Government Primary School 14 91412040503 Ramu Government Primary School 15 99412049002 Kajerbil Reg. Pvt. Primary School 16 99412049006 Ultakhali Reg. Pvt. Primary School 17 99412049010 Bado Jangchhadi Reg. Pvt. Primary School 18 99412049015 Farida Rashid Adarsh Reg. Pvt. Primary School 19 99412049203 Banolata Community Primary School 20 99412049022 Nasrat Aziya Reg. Pvt. Primary School End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 78 Annex L: Evaluation Matrix Table 10: Evaluation matrix Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability RELEVANCE Did the project reach the intended beneficiaries with the right mix of assistance?  Responsiveness of the programme in the context of Ukhiya and Kutubdia on the following aspects: o Prevalent issues and concerns of affected population in light of education, nutrition, social aspects (for boys and girls) o External administrative, social and political environment o Infrastructure availability (water supply, toilets) and access for students (for boys and girls)  WFP annual report  Baseline and Mid-term report  NPAN 2  WFP Bangladesh Country Strategic Plan 2017-2020  Data from Key Informant Interviews  Document review  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE, parents and school staff  FGD with Parents and SMC members  Narrative/ thematic analysis of secondary data  Analysis of Qualitative data (KIIs and FGDs)  Comparing views of GoB, WFP, implementing partners, parents and SMC members Strong Is the project aligned with national government’s policies and strategies including education, school feeding, safety net and national five years plan? o Consideration of school feeding programme priorities and interventions within GoB’s policy and planning frameworks such as, National School Meal policy, NPAN 2, Vision 2021, 7th five year plan and the national COVID response strategy o Number of interventions and activities that fulfil the institutional and capacity building needs, supports provision of education, school feeding, safety net services as identified by National  NPAN 2  National School Meal policy  Vision 2021  GoB COVID-19 response strategy  7th five year plan  National Nutrition Policy  National Social Security Strategy  Data from Key Informant Interviews  Document review  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE  Narrative/ thematic analysis of secondary data  Analysis of Qualitative data (KIIs)  Comparing the views of GoB and WFP. Strong End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 79 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability Nutrition Policy, National Social Security Strategy. o Does the Capacity Building component address the capacity needs of the functionaries at the MoPME to implement the School Feeding Programme? o Activities strengthening GoB’s ongoing initiatives o Considering the COVID-19 context, did the project adapt /realign its activities to support GoB in responding to the challenges? Is the project perceived as relevant in addressing the needs of target population? Is the project perceived to be relevant during a pandemic such as COVID-19?  Stakeholder perceptions regarding the degree to which needs of different groups were addressed appropriately  Modifications made in the implementation strategy to make the activities more responsive during the pandemic  Stakeholder perceptions of relevance of the project, in the COVID-19 crisis  Data from Key Informant Interviews  WFP monitoring reports  Document review  IDI/FGD with parents and teachers  IDI with WFP and implementing partners  Narrative/ thematic analysis of secondary data  Analysis of Qualitative data (KIIs and FGDs)  Comparing views of GoB, WFP, implementing partners, parents and teachers Strong Does the project complement other donor-funded and government initiatives?  Extent to which the school feeding programme activities complement GoB’s ongoing initiatives.  Extent to which project initiatives complement initiatives undertaken by other development partners.  Extent to which project initiatives complement initiatives undertaken by other development partners as a response to COVID-19  Action plans and assessment reports on activities undertaken by other development partners  Data from Key Informant Interviews  Document review  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE and  Narrative/ thematic analysis of secondary data  Analysis of Qualitative data (KIIs)  Comparing the views of GoB, WFP, implementing partners Fair End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 80 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability other development partners EFFECTIVENESS To what extent the outputs led to the realization of the expected results and outcomes – were the set targets achieved?  Results of activities planned and undertaken (as per action plan)  Results of engagement with GoB and other development partners  Extent to which the capacities of representatives, GoB have been developed on: o Preparation of operational guidelines for the national school feeding programme o Preparation of monitoring and reporting framework for scaling up national school feeding programme o Use of online database for reporting on school feeding programme for programme strengthening o Operationalizing multi-sectoral committee on school feeding o Integration of complementary components such as WASH, Health, Nutrition and Dietary Practices and Literacy in programme planning (at the govt. level) o Developing community participation strategy and plan for implementation  USDA approved work plan  Monthly/quarterly reports from implementing partners  School attendance and enrolment records  Biscuit distribution and receipt records  Programme Implementation Plan (PIP and Action plan)  Baseline and midterm data  Programme monitoring reports  SMC minutes of meetings  Data from quantitative survey with school staff, teachers and parents  Data from focused group discussions with mothers, and SMC members  Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE, implementing partners  Document review  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE, implementing partners  Focus group discussions with mothers and SMC members  Outcome reports of other similar programmes  Quantitative survey with the store keeper, teachers, headmaster and parents  Narrative/ thematic analysis of secondary data  Qualitative and quantitative analysis of primary data (KIIs)  Methodological triangulation (Quantitative and Qualitative)  Comparing the data on attendance from schools records with information received from parents  Comparing the response from WFP and GoB  Comparing the information on availability of infrastructure with response from parents  Comparison of end term values with baseline values  Analysis of school attendance, biscuit Strong End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 81 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability  SABER-SF Country Report 2020 distribution and receipt records What was the output and were all the project activities carried out as planned?  Comparison of data on performance indicators (standard and custom) with the baseline and targets (disaggregated by sex).  Key enablers and barriers (both internal and external) towards achievement of results  Stakeholders’ awareness and perceptions about programme interventions o Encouragement of reading habits in school and at home o Perception of parents on the read-play festival o Degree to which parents perceive that such events have encouraged students to read books (apart from textbooks) o Degree to which parents perceive that the use of new teaching techniques has aided their child’s understanding of school curriculum o Percent of parents in target communities who can name at least three benefits of primary education o Perception of parents on adequacy of teaching materials and methods o Degree to which parents are satisfied with the teaching methods in school o Perceptions regarding Read Play Festival, book corners, vegetable gardens (Essential Learning Package) and its  USDA approved work plan  Monthly/quarterly reports from implementing partners  School attendance and enrolment records  Biscuit distribution and receipt records  Programme Implementation Plan (PIP and Action plan)  Baseline data and midterm data  Programme monitoring reports  SMC minutes of meetings  Data from quantitative survey with, school staff, teachers and parents  Data from focused group discussions with mothers, and SMC members  Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE, implementing partners  Document review  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE, implementing partners  Focus group discussions with mothers and SMC members  Outcome reports of other similar programmes  Quantitative survey with the store keeper, teachers, headmaster and parents  Narrative/ thematic analysis of secondary data  Qualitative and quantitative analysis of primary data (KIIs)  Methodological triangulation (Quantitative and Qualitative)  Comparing the data on attendance from schools records with information received from parents  Participation of women in highlighting and resolving issues during the SMC meetings.  Comparing the response from school and WFP regarding community participation with response from parents Strong End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 82 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability effect on children and community; demand for other such provisions o Degree to which programme activities have influenced parents to demand better quality of education services  Degree to which parents perceive that training on health, hygiene and nutrition has influenced adoption of better health, hygiene and nutrition practices  Degree to which GoB perceives the interventions from WFP have been useful in building their capacities to implement the programme independently  Degree to which SMC members perceive increase in involvement and participation of women in decision-making  Comparing the response from school and WFP regarding use of new teaching techniques with response from parents  Comparison of midterm/end-term values with baseline values  Analysis of school attendance, biscuit distribution and receipt records In what ways was the project affected as a result of COVID-19?  Changes in activities planned and undertaken (as per action plan), if any  Changes in timelines for engagement with GoB and other development partners  Effect of COVID-19 on the achievement of intended outputs and outcomes  Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE, implementing partners  WFP monitoring reports  Document review  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE, implementing partners  Narrative/ thematic analysis of secondary data  Comparison of end￾term values with midterm and baseline values  Analysis of school attendance, biscuit distribution and receipt records EFFICIENCY What was the efficiency of the  Timely and complete disbursement of funds (year wise) as per the programme implementation plan  Biscuit distribution and receipt records  Document review  Workshop/ Key informant interviews  Narrative/ thematic analysis of secondary data Strong End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 83 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability programme, in terms of transfer cost, cost per beneficiary, logistics, and timeliness of delivery?  Timely and complete achievement of the programme outputs (year wise)  Timely and complete utilization of resources by the intervention areas (geographic and thematic)  Timely submission and approvals of monthly/quarterly/yearly action plans  Inventory planning for biscuits and demand forecasting  Timely supply (from WFP and implementing partners) and receipt (by implementing partners and schools)  Use of systems and processes for monitoring interventions and feedback  Degree to which implementing partners perceive the biscuit distribution process as efficient (timely availability and distribution)  What effect did the COVID-19 crisis have on utilisation of resources (financial as well as human capital)? What was the effect of reallocation (if any) on the project’s implementation and results?  Programme monitoring reports(including data from MIS)  Data from quantitative survey with school staff, teachers and parents  Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE, implementing partners  Data from focused group discussions with mothers and SMC members  Financial Disbursement and Utilization Report (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE  Quantitative survey with the store keeper, teachers, headmaster and parents  Focus group discussions with mothers and SMC members  Qualitative and quantitative analysis of primary data (KIIs), survey and FGDs  Methodological triangulation (Quantitative and Qualitative analysis)  Analysis of biscuit distribution, receipt records End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 84 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability How efficient the operation and approach are in terms of capacity building of government towards eventual handover?  Timely completion of planned capacity building activities  Use of online MIS for (resource) planning, implementation and decision-making  Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE.  Document review  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE  Narrative/ thematic analysis of secondary data  Analysis of Qualitative data (KIIs)  Comparing the views of GoB and WFP. Strong SUSTAINABILITY What remains to be done in order to achieve a full handover and nationally-owned school feeding programme? Is the programme sustainable in the following areas: strategy for sustainability; sound policy alignment; stable funding and budgeting; quality programme design; institutional arrangements; local production and sourcing; partnership and coordination; community participation and ownership?  Ownership and commitment of government and school officials to take forward or adopt best practices or innovations from the initiative  Institutionalisation of delivery model and approaches within existing administrative structures at national and subnational levels  Capacity of the line departments and functionaries to ensure that the programme functions independently  Institutionalization of coordination mechanisms for the independent implementation of the school feeding programme at national and sub-national levels  Institutionalization and use of online database for real-time tracking at national and sub￾national levels  Availability of a post-project sustainability plan (exit strategy)  Key enablers and barriers (both internal and external) towards sustainability of the programme  NPAN 2  School Feeding Policy and other policy documents  Programme documents and monitoring reports  Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE  Data from focused group discussions with mothers and SMC members  KII with representatives of MoPME and WFP  Desk Review of government document, policies and action plan  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE  Focus group discussions with mothers and SMC members  Narrative/ thematic analysis of secondary data  Analysis of data through KIIs and FGDs  Comparing views of GoB and WFP Strong End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 85 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability  Effect of COVID-19 on exit strategy  In the context of a pandemic, how does WFP plan to ensure sustainability of the project What progress has the government made toward developing and implementing a National School Meals Programme?  Progress after approval of National School Meals Policy a) Preparation of action plan for implementation of National School Meals Policy b) Inclusion of programme components in govt. priorities/policies (NPAN2)  Budgetary provisions/commitments by GoB to implement the programme independently (inclusion in revenue budget) to ensure a) Provision of hot meals b) Supporting infrastructure requirements at school level (kitchen, storeroom/WASH infrastructure)  What changes (if any) have been made to the timelines and modalities in the process of handing over to GoB, after the advent of the pandemic?  NPAN 2  School Feeding Policy and other policy documents  Programme documents and monitoring reports  Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE  KII with representatives of MoPME and WFP  Desk Review of government document, policies and action plan  Workshop/ Key informant interviews (KII) with WFP programme staff, representatives MoPME, DPE  Narrative/ thematic analysis of secondary data  Comparing views of GoB and WFP Strong Are local communities fully involved in and contributing toward school feeding and education activities?  Preparation and operationalization of community engagement strategy  Community involvement in monitoring and existence/use of feedback mechanism  Community participation in school management and biscuit distribution  NPAN 2  School Feeding Policy and other policy documents  Programme documents and monitoring reports  Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE  Data from focused group discussions with mothers and SMC members  KII with representatives of MoPME and WFP  Desk Review of government document, policies and action plan  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners,  Narrative/ thematic analysis of secondary data  Analysis of data through KIIs and FGDs  Comparing views of GoB and WFP  Comparing views of WFP and schools with SMC and parents Strong End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 86 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability representatives MoPME, DPE  Focus group discussions with mothers and SMC members IMPACT How did the project contribute to observed impacts? Have there been any unintended outcomes, either positive or negative? What internal and external factors affected the project’s ability to deliver impact? Have the outcomes been specifically affected by COVID￾19? What are spillover effects of the project? Are local communities (PTA, farmers groups, etc.) fully involved in and contributing  Number of students regularly (80%) attending USDA supported classrooms/schools  Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance  Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance  Number of students enrolled in school receiving USDA assistance  Number of public-private partnerships formed as a result of USDA assistance  Value of new public and private sector investments leveraged as a result of USDA assistance  Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance  Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance  Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance  School attendance and enrolment records  Biscuit distribution and receipt records  Programme Implementation Plan (PIP and Action plan)  Baseline and midterm evaluation report  Programme monitoring reports  SMC minutes of meetings  Data from quantitative survey with school staff, teachers and parents  Data from focused group discussions with mothers, and SMC members  Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE, implementing partners  Document review  Storeroom observation  Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE, implementing partners  Focus group discussions with mothers and SMC members  Outcome reports of other similar programmes  Quantitative survey with the store keeper, teachers, headmaster and parents  Narrative/ thematic analysis of secondary data  Qualitative and quantitative analysis of primary data (KIIs)  Methodological triangulation (Quantitative and Qualitative)  Comparing the data on attendance from schools records with information received from parents  Comparing the data received from school on storage management practices with observation  Comparing the response from WFP and GoB  Comparing the information on Strong End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 87 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability toward school feeding?  Number of schools using an improved water source  Number of schools with improved sanitation facilities  Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text  Degree to which community and school administration perceives the impact of programme activities in improving attendance, reading abilities, increased adoption of better health, hygiene and nutrition practices  Support from GoB to ensure infrastructural availability (for creating an enabling environment) to achieve impact  Extent of community engagement by SMC/school administration for supporting SFP activities and achievement of desired results  Extent of community involvement in supporting SFP activities and achievement of desired results  Unintended impact on any difference of impact on girls and boys availability of infrastructure with response from parents  Comparison of end term values with midterm with baseline values  Analysis of school attendance, biscuit distribution and receipt records  Analysis of SMC minutes of meeting Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 88 Annex M: Data Collection Tools The table below maps the indicators with the type of tools and the questions. The quantitative and qualitative tools have also been presented in this annex. It is important to highlight that on account of schools being closed and students not considered as a respondent category, it will be difficult to collect data on some of the indicators. For such indicators, it is proposed that midterm values are considered for the end term evaluation. Since interactions with children will not be carried out during data collection, tools such as the student questionnaire, student FGD guide, EGRA tool and Classroom Observation will not be administered. *Indicators for which values collected during the mid-term will be used for the end-term evaluation Table 11: Mapping of the indicators with the type of tools and the questions Standard Indicator Number Performance Indicator Source Type of Questionnaire Question No. 1 Number of students regularly (80%) attending USDA supported classrooms/schools Primary survey School Questionnaire Questions 86, 87 2 Number of textbooks and other teaching and learning materials provided as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Teacher Questionnaire Questions 7,8 Question 14 3 Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Headmaster Questionnaire Question 85 Questions 4,5 4 Number of school administrators and officials trained or certified as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Headmaster Questionnaire Questions 85 Questions 2,3 5 Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Primary survey School Questionnaire Teacher Questionnaire Headmaster Questionnaire Question 85 Questions 11, 12, 13 Questions 4,5 6 Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Teacher Questionnaire Headmaster Questionnaire Question 85 Questions 9, 10 Questions 2,3 7 Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Questions 14, 20, 21,55 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 89 Standard Indicator Number Performance Indicator Source Type of Questionnaire Question No. 8 Number of students enrolled in school receiving USDA assistance WFP monitoring reports Primary survey School Questionnaire Questions 80, 81 9 Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance Primary survey School Questionnaire Questions 57,58,71,72 10 Number of public-private partnerships formed as a result of USDA assistance Secondary; WFP monitoring reports 11 Value of new public and private sector investments leveraged as a result of USDA assistance Secondary; WFP monitoring reports 12 Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance: Secondary; WFP monitoring reports Stage 1: Analysed Stage 2: Drafted and presented for public/stakeholder consultation Stage 3: Presented for legislation/decree Stage 4: Passed/Approved Stage 5: Passed for which implementation has begun 15 Number of daily school meals (breakfast, snack, lunch)114 provided to school-age children as a result of USDA assistance Primary survey School Questionnaire Storekeeper Questionnaire Questions 53,79 Question 37 16 Number of school-age children receiving daily school meals (breakfast, snack, lunch)115 as a result of USDA assistance Primary survey School Questionnaire Storekeeper Questionnaire Questions 53,83 Question 37 17 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance Secondary; WFP monitoring reports 18 Number of individuals trained in child health and nutrition as a result of USDA assistance Primary Survey WFP; monitoring reports School Questionnaire Parents Questionnaire Questions 34,39,40,41,54, 85 Questions40,41 114 School Feeding Programme in Bangladesh provides biscuits. Hence, data will be obtained is for number of biscuits provided to school age children as a result of USDA. 115 Data will be collected for number of children receiving biscuits as a result of USDA assistance. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 90 Standard Indicator Number Performance Indicator Source Type of Questionnaire Question No. 19 *Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance Primary survey Mid Term value Parent Questionnaire Questions 33,34,35,37 20 Number of individuals trained in safe food preparation and storage as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Store Keeper Questionnaire Question 85 Questions 9,10 21 Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Store Keeper Questionnaire Question 85 Questions 7,8,11 22 Number of schools using an improved water source WFP monitoring reports Primary survey School Questionnaire Question 10 23 Number of schools with improved sanitation facilities WFP monitoring reports Primary survey School Questionnaire Question 17 24 Number of students receiving deworming medication(s) WFP monitoring reports Primary survey School Questionnaire Questions 54 26 *Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text Secondary; RtR monitoring reports Midterm Value 27 Number of individuals benefiting directly from USDA￾funded interventions WFP monitoring reports 28 Number of individuals benefiting indirectly from USDA-funded interventions WFP monitoring reports Custom 1 Percent of teachers in target schools who attend and teach school at least 90 percent of scheduled school days per school year The data for this indicator could not be collected in midline evaluation due to government’s circular that restricts access of teacher’s attendance data outside the education department. For the end term evaluation as Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 91 Standard Indicator Number Performance Indicator Source Type of Questionnaire Question No. well no data can be reported. 2 Number of classroom libraries (book shelves with books) established as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Question 6,7,8 3 *Percent of students in classrooms identified as attentive by their teachers Midterm value 4 *Average number of school days missed by each student due to illness (for each school and in aggregate) Midterm value 5 Percent of parents in target communities who can name at least three benefits of primary education Primary survey Parent Questionnaire Question 12 6 *Percent of students who can identify at least three key health and hygiene practices Midterm value 7 Percent of storekeepers who can identify at least three safe storage practices Primary survey Storekeeper Questionnaire Questions 11, 29-36 9 Number of 'Little Doctor' students supported by WFP WFP monitoring reports Primary survey School Questionnaire Question 38 11 Number of meetings/workshops/training sessions held for institutional capacity to implement SF as a result of USDA assistance Secondary; WFP monitoring reports 12 Number of pilot initiatives supported to design SF modalities as a result of USDA assistance Secondary; WFP monitoring reports 13 Number of government staff trained as a result of USDA assistance Secondary; WFP monitoring reports 14 Number of technical training for system development for service/food procurement, quality control, supply chain, and strengthened online database system and gender mainstreaming in programme as a result of USDA assistance Secondary; WFP monitoring reports 15 Number of schools supported by the Government with school feeding as a result of USDA assistance Secondary; WFP monitoring reports Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 92 Standard Indicator Number Performance Indicator Source Type of Questionnaire Question No. 16 Number of social mobilization/community meetings as a result of USDA assistance Secondary; WFP monitoring reports 17 Number of community mobilization workshops organized as a result of USDA assistance Secondary; WFP monitoring reports 18 Number of teachers, parents and school management committee members attended the community mobilization workshops Secondary; WFP monitoring reports Primary Survey Parents Questionnaire Teachers Questionnaire Question 39 Questions 22 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 93 1. SCHOOL QUESTIONNAIRE INTERVIEW INFORMATION A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. Name of the school A4. School code A5. Number of grades in the school Multiple responses possible Pre-school 1 Class 1 2 Class 2 3 Class 3 4 Class 4 5 Class 5 6 A6. Number of children in the school (Please note from school records) Boys Girls Pre-school Primary A7. Number of teachers Male Female Pre-school Primary A8. Date of visit (dd/mm/yy) A9. Time of survey (in am/pm) Morning 1 Afternoon 2 A10. Name of the supervisor A11. Name of the enumerator A12. Designation of the person being interviewed (Multiple responses possible) School principal/head master 1 School vice-principal 2 School teacher 3 A13. Name of the head master / teacher A14. Phone number of the headmaster A15. Area type Urban 1 Rural 2 Peri-urban 3 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 94 SCHOOL FACILITIES S.No. Question Options Codes Skip For each facility that is present, OBSERVE the facility and code accordingly. Ask For Both Programme And Comparison Schools 1. Number of classrooms in the school 2. Does the school have a separate classroom for each grade? (considering 2 shifts separately) Yes 1 Go to 4 No 2 3. If no, then how many such grades are sharing their classroom with other grades? Morning ___No. Afternoon ___No. 4. Does the school have a Library? Yes 1 No 2 Go to 6 5. If yes, how many supplementary books does the school have in the library? (No. of books from the library records) 6. Does the school have Classroom Libraries? Yes 1 No 2 7. Has the school received books from any organization? Yes 1 No 2 Go to 9 8. If yes, what is the name of the organization? RtR/WFP/MA/RIC /YPSA116 1 Government 2 Institutes/College s 3 Donated by individuals/any other NGOs 4 Don’t Know 5 Others (specify) 9. Does the school have a source of drinking water for students near or at school? Inside school premises 1 Outside school premises 2 No drinking water facility 3 Go to 13 10. What is the main source of drinking water? Tap water 1 Bottled water 2 RO plant within school 3 Borehole 4 Protected dug well 5 Unprotected well 6 Protected spring 7 Unprotected spring 8 116 RtR: Room to Read, WFP: World Food Programme, MA: Muslim Aid, YPSA: Young Power in Social Action, Resource Integration Centre Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 95 SCHOOL FACILITIES Rainwater collection 9 Rivers or ponds 10 Vendor-provided water 11 Tanker truck water 12 Other (Specify) 11. Has the water been treated for drinking by boiling etc.? Yes 1 No 2 Don’t know 3 Not required 4 12. . Is this drinking water available during all times of the day as well? (Ask as well as observe by going to the drinking water facility) Only available during lunch hour 1 Available during all times of the day 2 13. If No, how do the majority of students get drinking water during school hours? Buy Bottled water from shops 1 Children carry water from home 2 Get water from neighbours 3 Other (Specify) 14. Has any water supply systems been rehabilitated / constructed with WFP/support? Yes 1 No 2 15. Does the school have a common Hand Washing facility? Yes 1 No 2 16. Do the students wash their hands using the Hand Washing facility at school? (as per your observation) Yes 1 No 2 17. Does the school have a functioning toilet for the students? Yes 1 No 2 18. Are there separate toilets for boys and girls? Yes 1 No 2 19. How many toilets are there? A. Total toilets / B. Functional toilets 20. Are there toilets which have been rehabilitated / constructed with WFP support? Yes 1 No 2 21. . Has the school been provided with needed pumps and motors to improve water systems through WFP support? Yes 1 No 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 96 22. For each functioning toilet, ask the following set of questions (WASH) Record through Observation Functioning Toilet Type of toilet 1. Flush or pour/flush facilities connected to a: (Piped sewer, septic, pit latrine) 2. pour/flush toilets without a sewer connection 3. Pit latrines with a slab 4. Pit latrines without slab/open pit 5. Ventilated improved pit latrines 6. Composting toilets 7. Bucket latrines 8. Hanging toilets/latrines Is the toilet for boys or girls? Boys...1 Girls…2 Mixed…3 Is the toilet locked? Yes…1 No…2 Does the toilet have water inside for use? Yes, piped water connection…1 Yes, water kept in a container…2 No water inside the toilet…3 Does the toilet have a mug? Yes…1 No…2 Is the toilet maintained? Yes: 1 No: 2 Do the toilets have hand washing facilities within the school? Yes: 1 No: 2 Does the hand washing facility have continuous water supply? Yes: 1 No: 2 Does the hand washing facility have soap? Yes: 1 No: 2 How often is it a challenge to have sufficient water for the hand washing facility? Never=1 Rarely=2 Sometimes =3 Always=4 1 2 3 4 5 23. Do the toilets have hand washing facilities within the school? Yes 1 No 2 Go to 27 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 97 24. Does the handwashing facility have continuous water supply? Yes 1 No 2 25. Does the hand washing facility have soap? Yes 1 No 2 26. How often is it a challenge to have sufficient water for the hand washing facility? Never 1 Rarely 2 Sometimes 3 Always 4 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 98 SCHOOL GARDEN and LITTLE DOCTORS S.No. Question Options Code s Skip Only Ask For Programme Schools. Not to Be Asked in Comparison Schools 27. Does the school have a vegetable garden? Ask the school head to show you the school garden. Only after observing the school garden, mark the correct answer. Yes 1 No 2 Go to 41 28. Has there been any training on establishing and maintenance of vegetable garden? Yes 1 No 2 Go to 30 29. If yes, who has provided the training? WFP/MA/YPSA/RIC 1 Government 2 Any other, please specify 30. Is there any vegetable/fruit sown or growing in the garden Yes 1 No 2 Go to 32 31. Write the names of the vegetables/fruits that you see have planted in the school garden. 1 2 3 4 5 6 7 8 9 1 0 32. If no, What are the reasons? Soil is not fertile 1 Don’t have anyone to take care of the garden 2 Have not received any training 3 There was no protection from cattle 4 The training content was not enough to learn gardening 5 Trainer was not good 6 Any other, please specify 33. What is the source of water for irrigating the school garden? Water supply not available 1 River/stream/canal 2 Pond within the school 3 Pond outside of the school premises 4 Rainwater 5 Piped water from the district administration 6 Others (specify) 34. Never 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 99 SCHOOL GARDEN and LITTLE DOCTORS How often is it a challenge to have sufficient water for the garden? Rarely 2 Sometimes 3 Always 4 35. Have any “Little Agriculturists” been identified for vegetable gardens? Yes 1 No 2 Go to 41 36. If yes, how many? (Specify number). Also capture the same for previous years Boys (in 2020) Boys (in 2019) Boys (in 2018) Girls (in 2020) Girls (in 2019) Girls (in 2018) 37. Has there been any training provided to the little agriculturists? Yes 1 No 2 38. Who has provided training to Little Agriculturists? WFP/MA/YPSA/RIC 1 Government 2 Any other, please specify 39. On what all aspects have they been provided training? Cultivation practices, setting up and maintenance of school garden 1 Nutritional value of fruits and vegetables 2 Healthy meal preparation 3 Any other, please specify 40. What are the activities that Little Agriculturists perform? (Multiple responses possible) Setting up and maintenance of school garden 1 Disseminating information of nutritional value of fruits and vegetables 2 Supporting healthy meal preparation 3 Any other, please specify 41. Have any “Little Doctors” been identified in the school? Yes 1 No 2 Go to 48 42. If yes, how many? (Specify number). Also capture the same for previous years Boys (in 2020) Boys (in 2019) Boys (in 2018) Girls (in 2020) Girls (in 2019) Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 100 SCHOOL GARDEN and LITTLE DOCTORS Girls (in 2018) 43. Has there been any training provided to Little Doctors? Yes 1 No 2 Go to 46 44. Who has provided training to Little Doctors? WFP/MA/YPSA/RIC 1 Government 2 Any other, please specify 45. On what all aspects have they been provided training? (Multiple response possible) Health and hygiene practices 1 Delivery and administration of deworming tablets 2 Conducting health check ups 3 Any other, please specify 46. What are the activities that Little Doctors perform? (Multiple response possible) Support in delivery and administration of deworming tablets 1 Support in Health Check-ups 2 Any other, please specify 47. What is the frequency of support? Once a month 1 Once in 3 months 2 Once in 6 months 3 Once in a year 4 Any other, please specify SCHOOL FEEDING PROGRAMME S.No. Question Options Code s Skip Only Ask For Programme Schools. Not To Be Asked To Comparison Schools. 48. How many days was the school open in the January 2020? (before COVID-19 ) ___Days 49. In the month of January 2020, how many days were school biscuits provided to the students? ___Days 50. What are the main challenges your school has faced in regularly providing fortified biscuits? (pre COVID-19) Multiple responses possible. No biscuits available due to delay in delivery of stock 1 No adequate supply of biscuits 2 Students do not like the biscuits 3 Frequent absenteeism of storekeeper 4 Others (specify) 51. Have you faced any challenges in regular supply of biscuits to No biscuits available due to delay in delivery of stock 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 101 school during COVID-19 period? Is yes, what were the main challenges Multiple responses possible. No adequate supply of biscuits 2 52. Does your school have a separate storage room/facility to store the fortified biscuits? Yes, within the school 1 Yes, outside the school premises 2 No 3 53. If no, then where are the biscuits stored? Multiple response possible In a class room 1 In teacher’s room 2 Open space 3 In the house of the school/ head/teacher 4 Neighbour’s house 5 Others (Specify) 6 54. Does the school have a Pest Management Plan for their food storage facilities? Yes 1 No 2 55. Does the storage room have wooden pallets above the ground for storage of commodities? (through observation) Yes 1 No 2 Could not observe 3 56. Has the storage room/s been rehabilitated / constructed with WFP/USDA support? Yes 1 No 2 57. Since when is the school receiving support from the school feeding programme supported by WFP? Ask for the year in which the WFP support started. Accordingly calculate the number of years till the time of survey (that is 2019). a. Mention the year and month when it started: ____ Year ___ Month b. Duration: ___ Years ___ Months Put “99” for years and months, if the school have not received any support yet, but will receive 58. What type of support is the school receiving/has received from WFP/RtR/ MA/YPSA/ RIC in the last 1 year? (Ask the Headmaster about all the options mentioned here, and at the end also check if there is any additional support) 59. Type of support Yes=1 No=2 Source 1: Government 2: WFP/RtR/MA/YPSA. RIC 3: Others 1. Fortified biscuits for the students 1 2 2. Textbooks 1 2 3. School gardening support 1 2 4. Infrastructure support (toilet/store room etc.) 1 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 102 5. Infrastructure support (library/playground/classroom etc.) 1 2 6. Teaching materials / guidelines 1 2 7. Water supply for drinking 1 2 8. Water supply for toilets 1 2 9. Water supply for school garden 1 2 10. Training for teachers / administrators / support staff 1 2 11. Receiving deworming medication(s) 1 2 12. Training for students / parents / others 1 2 13. Any other, specify 60. Other than WFP/YPSA/RtR/MA/RIC is this school currently benefitting from any other project with a partner, for example an NGO, charity, private company or the government? Yes 1 No 2 61. If yes, then ask: What type of project or projects is the school participating in? Multiple responses possible Water, hygiene and sanitation (WASH)) 1 Education / enrolment (general) 2 Education / enrolment (girls) 3 Nutrition 4 Anti-malaria or anti￾Dengue (mosquito nets etc.) 5 Others (specify) School Management Committee Ask For Both Programme And Comparison Schools 62. Does the school have a School Management Committee (SMC)? Yes 1 No 2 63. How many members are there in the SMC? Male Female Total 64. Has the SMC received any form of training/workshop? Yes 1 No 2 Go to 67 65. If yes, who has provided the training/conducted the workshop? WFP/YPSA/MA/RtR/ RIC 1 Government 2 Others, please specify 3 66. If provided by WFP/YPSA/MA/RtR/RIC what were the aspects? Health, Hygiene and Nutrition awareness 1 School infrastructure 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 103 School Feeding Programme 3 Quality of education 4 Environment Education 5 Gender 6 School Management 7 Any others, please specify 67. Does the SMC have plans for regular meetings? Yes 1 No 2 Go to 69 68. What is the frequency of planned meeting per academic year? No. of planned meeting s 69. How many times did the committee actually meet in the last one year? 70. How many members attended the last meeting? 71. How many female members attended the last meeting? 72. Do you maintain minutes of the meeting? Please validate it by checking the minutes of the meeting from the record book and collect copy of the same Yes 1 No 2 73. What are the topics discussed during SMC meetings? Health, Hygiene and Nutrition awareness 1 School infrastructure 2 School Feeding Programme 3 Quality of education 4 Environment Education 5 Others (Specify) 74. In general, how would you describe the functionality of the SMC? Only one response Non-functional / Provides no support to the school. 1 Somewhat functional /Provides some support to the school 2 Highly functional / Provides strong support to the school 3 Only Ask For Programme Schools. Not To Be Asked For Comparison Schools 75. What is the kind of support that the SMC provides to the school feeding programme? Multiple responses possible Management of biscuits stock 1 Management of Parent Teacher Association/ (PTA) 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 104 Coordination with RtR to execute the activities streamlined under the programme 3 Grievance redressal of provision of fortified biscuits related issues 4 Disseminate materials to visualize good hygiene practises 5 Improvement in School infrastructure 6 Community Engagement￾facilitating participation of community in school activities 7 No support 8 Others (specify) 76. Does the school have Parent Teacher Association (PTA)? Yes 1 No 2 77. How many members are there? _____Male(number) _____Female(number) 78. Have the PTA members received any training from WFP/RtR/MA/YPSA/RIC Yes 1 No 2 79. If yes, on what all aspects have PTA members received training Health, Hygiene and Nutrition awareness 1 School Feeding Programme 2 Quality of education 3 Environment Education 4 Gender 5 School Management 6 Community Engagement￾facilitating participation of community in school activities 7 Any others, please specify 80. What is the frequency of planned meeting per academic year? 81. How many times did the PTA actually meet in the last one year? 82. What are the topics discussed during PTA meetings? Health, Hygiene and Nutrition awareness 1 School infrastructure 2 School Feeding Programme 3 Quality of education 4 Environment Education 5 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 105 FOOD UTILISATION DATA ONLY FOR PROGRAMME SCHOOLS. NOT TO BE FILLED FOR COMPARISON SCHOOLS S.No. Question Options Codes Skip 83. Does the school have information for biscuit utilization at school level? Yes 1 No 2 If coded 2 in Q.78, then skip to Q80 Enter “888” if data is not available S.No Months Number of students receiving daily school biscuits Number of days in the month when the school biscuits were provided Total Male Female # 84. 84.1 84.2 84.3 84.4 1 September 2019 2 October 2019 3 November 2019 4 December 2019 5 January 2020 6 February 2020/ while the schools were closed (during last 6 months) STUDENT ENROLMENT AND ATTENDANCE Students Current Enrolment and Attendance Data To Be Asked To Both Programme And Comparison Schools S.No. Question Options Codes Skip 85. Does the school have records of Student Enrolment for past academic years? Yes complete records 1 Yes partial records 2 No 3 If coded 3 in Q.80, then skip to the next section 86. ENROLMENT FOR 2019 AND 2020 ACADEMIC SCHOOL YEARS Complete for Pre-school and Primary only (Grades 1-5) Enter "999" if the class does not exist Enter "888" if data is not available Academic year Enrolment at start of academic school year Total number of students present (headcount) Total number of students absent Transfers or deceased Left without reason (dropout) Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls 2019 a. b. c. d. e. f. g. h. Pre￾school 1 2 Grievance redressal of provision of fortified biscuits related issues 6 Others (Specify) ¨ Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 106 STUDENT ENROLMENT AND ATTENDANCE Students Current Enrolment and Attendance Data To Be Asked To Both Programme And Comparison Schools 3 4 5 2020 Pre￾school 1 2 3 4 5 Students Attendance Data in the last 1 year TO BE ASKED TO BOTH PROGRAMME AND COMPARISON SCHOOLS S.No. Question Options Code s Skip 87. Does the school have records of Student Attendance for last academic years? Yes complete records 1 Yes partial records 2 No 3 If coded 3 in Q.82 then skip to the next section If coded 1 or 2 in Q.82, then Complete this section for Pre-school and Primary (grades 1-5) only Enter “999” if the class does not exist Enter “888” if data is not available First, enter the total number of school days for each month (not including holidays or school closures). Then, from the attendance records (2019) write the attendance for students selected for the survey. Enter the number of days the students attended school each month. September 2019 October 2019 November 2019 1 2 3 88. Number of schools days Grade Student code Sex (male=1, female=2) September 2019 October 2019 November 2019 a. b c d d e 1(In Grade 2 in 2020, Grade 1 in 2019) 2 (In Grade 3 in 2020, Grade 2 in 2019) 3(In Grade 4 in 2020, Grade 3 in 2019) 4 (In Grade 5 in 2020, Grade 4 in 2019 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 107 TEACHERS TRAINING 89. Teachers Training Data in the last 1 year a. b. c. d. e. f. g h. i. j. k. l. m. Teachers training data (Have the teachers received training from WFP? If the answer is a no, enumerator to ask reason for the same and record the response separately. (Use of training received) Source Teacher ID Teacher’s Name (start with Head teacher’s information) ) Sex 1. Male 2. Femal e Position in the school 1. Head teacher/ principal 2. Deputy head teacher 3. Teacher Full time/ Part time 1. Full￾time 2. Part￾time Which grade does the teacher teach? Storage practices Teaching / Learning techniques (at least 2 days or 16 hours) Hygiene and WASH Child health and nutrition If trained, whether the teacher is using storage practices? If trained, does s/he demonstrate use of new and quality teaching techniques or improved literacy instruction identified by his / her supervisor/ head teacher? If trained, does he/she demonstrate use of child health and nutrition practices? Code 1: Government 2: WFP 3:Others 1 2 3 4 5 6 7 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 108 2. TEACHER QUESTIONNAIRE Interview Information A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. Name of the school A4. School code A5. Date of visit (dd/mm/yy) A6. Time of survey (in am/pm) Morning 1 Afternoon 2 A7. Name of the supervisor A8. Name of the enumerator A9. Has the teacher given her/his consent to participate in the interview? Yes 1 No 2 If the teacher says no, then thank them and proceed to the next teacher on your list. S.No. Question Options Codes Skip 1. What is your name? 2. Sex (Observe only) Male 1 Female 2 3. How many years have you been teaching? ________Number of years 4. For how many years have you been teaching in this school? ________Number of years 5. What position do you hold in this school? Head teacher/principal 1 Assistant/ Deputy head teacher 2 Senior Teacher 3 Teacher (permanent/regular) 4 Teacher (paid contract) 5 Teacher (volunteer) 6 Others 6. What is your highest educational qualification? MSc or higher 1 Bachelor 2 Diploma 3 Higher/Advanced diploma 4 Technical/vocational 5 Secondary (G9-G11) 6 Middle (G6-G8) 7 Primary (G1-G5) 8 Untrained 9 Other 1 0 7. Which grades do you teach? Multiple responses possible Class/grade 1 1 Class/grade 2 2 Class/grade 3 3 Class/grade 4 4 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 109 Class/grade 5 5 8. What subject do you teach at school? Multiple responses possible/ Literature / Language (Bengali) 1 Literature / Language (English / other foreign language) 2 Science 3 Mathematics 4 Social science 5 History 6 Religion 7 Arts / Painting 8 Life skills 9 Others (specify) 9. Have you received any training from RtR/WFP? Yes 1 No 2 10. On what all aspects have you received training and a manual? Teaching instruction training Library management Leadership and school management Traini ng 1 Yes 2 No Manu al 1 Yes 2 No Traini ng 1 Yes 2 No Manu al 1 Yes 2 No Traini ng 1 Yes 2 No Manu al 1 Yes 2 No 11. Do you find these trainings and manuals useful? Traini ng 1 Yes 2 No Manu al 1 Yes 2 No Traini ng 1 Yes 2 No Manu al 1 Yes 2 No Traini ng 1 Yes 2 No Manu al 1 Yes 2 No 12. Do you apply the learnings (received from the trainings and manuals)? Yes 1 No 2 13. How often do you apply these learnings? All the time (for all students and all topics) 1 Only for certain topics 2 Only for some students 3 14. What proportion of students in your classroom has the required textbook/s for the class? None 1 Less than 25% 2 25-50% 3 50-75% 4 75-99% 5 100% 6 Don’t Know 7 15. What proportion of students in your class has a desk and chair? None 1 Less than 25% 2 25-50% 3 50-75% 4 75-99% 5 100% 6 Don’t Know 7 16. Only to be asked for intervention schools. Yes 1 No 2 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 110 Do you think that since the school feeding programme has started, children are more attentive in class? Don’t Know 3 17. Do you think that the provision of biscuits increases the number of children coming to school? Yes 1 No 2 Don’t Know 3 18. For intervention schools, ask According to you, if the provision of biscuits stopped from today, what would be the consequence of students’ attendance? For comparison schools where provision of biscuits is not going on ask, According to you, if the biscuits are provided from today, what would be the result on student attendance? No consequence, attendance will remain the same 1 Attendance will drop by 25% 2 Attendance will drop between 25% and 50% 3 Attendance will go up by more than 50% 4 Don’t Know 5 19. For intervention schools, ask According to you, if the provision of biscuits stopped from today, what would be the consequence on student enrolment? For comparison schools where provision of biscuits is not going on, ask According to you, if the biscuits are provided from today, what would be the consequence on student enrolment? No consequence, enrolment will remain the same 1 Enrolment will drop by 25% 2 Enrolment will drop between 25% and 50% 3 Enrolment go up by more than 25% 4 Don’t Know 5 20. Only to be asked to intervention schools What type of trainings have you received from WFP/RtR on the following Type of training No. of trainin gs Durat ion Utilizat ion (Y/N) Teaching / Learning techniques (at least 2 days or 16 hours) Storage practices Hygiene and WASH Child health and nutrition 21. Only to be asked to intervention schools Are you aware of any of these community mobilisation initiatives? 1. Read Play Festival Yes 1 No 2 2. Grade-I Reception Day Yes 1 No 2 3. Healthy Meal Preparation Day Yes 1 No 2 4. Other, please specify 22. Have you attended any of the community mobilisation initiatives? End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 111 1. Read Play Festival Yes 1 No 2 2. Grade-I Reception Day Yes 1 No 2 3. Healthy Meal Preparation Day Yes 1 No 2 4. Other, please specify End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 112 3. PARENT QUESTIONNAIRE Interview Information A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. Name of the school in which the child is studying A4. School code A5. Student name for which the parent is being interviewed. A6. Respondents name A7. Household address A8. Household size A9. Date of visit (dd/mm/yy) A10. Time of survey (in am/pm) Morning 1 Afternoon 2 A11. Name of the supervisor A12. Name of the enumerator A13. Has the parent given consent for the interview? Yes 1 No 2 If the parent says no, then thank them and proceed to the next parent on your list. Socio-Demographics Characteristics S. No. Question Options Codes Skip 1. Relationship of the respondent with the child Father 1 Mother 2 Both 3 Others (Specify) 2. What is the total number of school aged children in the household? (age 5-18 years) _______Boys _______Girls 3. Number of children in primary school in the household _______Boys _______Girls End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 113 Socio-Demographics Characteristics S. No. Question Options Codes Skip 4. Which of the following items do you have in your household? ITEMS Yes No Radio 1 2 Electricity 1 2 Refrigerator 1 2 Bicycle 1 2 Toilet 1 2 Mobile Phone 1 2 Television 1 2 Motorbike 1 2 Car 1 2 Tractor 1 2 5. What is the main source of drinking water? Tubewell 1 Piped water into the house 2 Piped water to yard/plot of the house 3 Surface water (river, lake, stream, canal, etc.) 4 Protected well 5 Unprotected well 6 Protected spring 7 Unprotected spring 8 Bore well 9 Rainwater 10 Bottled water/water vendor 11 Tanker 12 Others (Specify) 13 6. What is your relationship to the head of the household? See code 7. What is the highest level of education of the household head? See code 8. What is the main occupation of the head of the household? See code End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 114 Socio-Demographics Characteristics S. No. Question Options Codes Skip 9. What is his/her average monthly income (either in cash or kind or both)? _______________Amount in taka 10. How many members in the household earn an income (including farming)? S.No. Member Relationship to the child Sex Education (Code) Main Occupation (Code) Monthly income 1 2 3 4 5 6 7 QUESTIONS CODES 9, 13 (3): Relationship: What is your relationship to the household head? 1= head, 2 = spouse, 3 = child,, 4 = sibling,, 5 = son/daughter –in-law, 6 = other (specify) 10, 13 (5): Education - What is the highest level of education? 0= no schooling, 1-12=last Grade passed, 13= higher diploma; 14= technical/ vocational diploma, 15= Bachelor or equivalent, 16= Master or equivalent, 17=Pre-primary/Just enrolled, 18= Don’t know, 19=Other (and specify) 11, 13 (6): Main Occupation What is your / their main occupation? 0 = Unemployed, 1=Farmer, 2=Casual, 3=Fishing, 4=Petty trade business, 5=Official employee (public /private service), 6=Livestock: Poultry rearing; 7=Cottage industry /handicraft 8=Domestic maid, 9=Rickshaw/van push cart, 10=Transport worker (e.g. bus/truck), 11=Housewife, 12=Student, 19 = Too old or too young to work, Other (and specify) Questions related to attitude/perspectives about the school/education S. No. Question Options Codes Skip 11. Do you think education is important and beneficial? Yes 1 No 2 12. If Yes, what according to you are the benefits of primary education? Improves literacy rate 1 Improves future opportunities of work for children 2 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 115 Questions related to attitude/perspectives about the school/education S. No. Question Options Codes Skip Multiple responses possible Helps child’s skill development 3 Helps girls to remain in school and delay early marriage 4 Helps children from different social and ethnic groups to bond 5 Helps children learn more about the world 6 Helps break the cycle of poverty 7 Others 13. Has your child been given textbooks from school? Yes 1 No 2 Go to 15 14. If yes, who provided these textbooks? WFP/RtR/MA/YPSA 1 Government 2 Other, please specify 15. If no, how do you manage? We buy 1 We borrow from others 2 Take used books from senior students 3 Other, please specify 16. At home, do you or someone in your family help (name of the child) in studies or completing the school work? Yes 1 No 2 17. . At home, do you or someone in your family read stories to (name of the child Yes 1 No 2 18. Only to be asked for intervention schools According to you what are the benefits of the school feeding programme? Multiple responses possible My child is getting nutritious food for lunch 1 My child stays in school all day/improves attendance 2 The child does not stay hungry in school 3 The child can pay more attention in class 4 Less expense on food 5 Others End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 116 Questions related to attitude/perspectives about the school/education S. No. Question Options Codes Skip 19. How are you managing your child’s education while the schools are closed (due to COVID 19)? They are taught by family member/ relatives 1 Taught by private tutor/local hujur 2 Use of online platforms 3 Through radio/television broadcast 4 Not being able to manage 5 20. Is there a Parent Teacher Association (PTA) in your school? Yes 1 No 2 Can’t say 3 21. If yes, then are you a part of the Parent Teacher Association (PTA)? Yes 1 No 2 22. How many meetings have been conducted in the last one year? Number of meetings………… 23. How many meetings have you attended in the last one year? Number of meetings………… 24. Are you aware of the existence of the School Management Committee (SMC)? Yes 1 No 2 25. If yes, what is your perception of the SMC’s involvement in the school activities? High 1 Medium 2 Low 3 26. Are you aware about the responsibilities of SMC? Yes 1 No 2 Go to 28 27. If yes, what all activities do they perform? (Do not prompt) Improving quality of education 1 Improving school infrastructure 2 Improving management of school 3 Improving school feeding programme 4 Others, please specify 28. What is the perception on the role of SMC in…? Improving quality of education Improving school infrastructure End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 117 Questions related to attitude/perspectives about the school/education S. No. Question Options Codes Skip Use code: 1 = Useful, 2 = Satisfactory 3 = Not useful Improving management of school Improving school feeding programme Others 29. . Only to be asked for intervention schools What are the existing avenues through which you can get information or make complaints on the school feeding programme? Meetings with SMC 1 Meetings with School Administrators 2 Suggestion Box 3 Informal communication (verbal) with teachers / SMC members 4 My child (student) 5 None 6 Others Questions related to health and hygiene S. No. Question Options Codes Skip 30. Do you have a toilet in your house? Yes inside the dwelling 1 Yes, inside the plot of the house 2 No 3 31. Do you have access to water inside the toilet? Yes, piped water 1 Yes, a container for water 2 No 3 32. Do the household members use the toilet? Yes, all members 1 Yes, but only some members 2 No 3 33. Do you have a hand washing facility inside your household/in the yard/plot? Yes, tubewell 1 Yes, water in a bucket or a container 2 Yes, tippy tap 3 No 4 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 118 Questions related to health and hygiene S. No. Question Options Codes Skip 34. Do you use soap for hand washing in your household? Yes 1 No 2 35. Can you tell me when someone should wash their hands? Multiple responses possible Before eating a meal 1 After eating a meal 2 Before feeding a child 3 Before preparing/handling food 4 After using toilet 5 After handling farm work/animals 6 36. What are the times when you wash hands? (Ask with soap- yes or no for each coded option) Multiple responses possible Options Code s With soap yes-1 no-2 Before eating a meal 1 After eating a meal 2 Before feeding a child 3 Before preparing/handling food 4 After using toilet 5 After handling farm work/animals 6 Others 37. Where do you get your information related to health, hygiene and nutrition? Local health clinic/hospital 1 School Health and hygiene Brochures 2 School teachers 3 Community meetings 4 School PTA meetings 5 NGO/GoB Community health workers 6 Poster and Pamphlet 7 Notice board/ wall magazine / Wall paintings/hording board 8 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 119 Questions related to health and hygiene S. No. Question Options Codes Skip Radio / Television 9 Video/Documentary Street Drama Show 10 Newspaper / Magazine 11 Others Questions related to the dietary diversity of the child 38. 1=Yes 2=No 9 = Don’t know Food Items (In the last 24 hours (during the day and night), did (CHILD NAME) eat any of these food items? Ask about every single items and record the answer. If any items are consumed less than one tea spoon, record response “2. NO”; Only count them “1.YES” if consumed ≥1 teaspoon.) 1 Cereals, grains, roots and tubers (Food made from grains such as bread, rice, noodles, biscuits, or any other foods made from millet, sorghum, maize, rice, wheat, white potatoes, white yams, manioc, cassava, [other local root crops] or any other foods made from roots or tubers) 2 Legumes/nuts (any foods made from beans, peas, or lentils, or foods made from nuts or seeds such as pumpkin flower seed) 3 Milk and dairy products like cheese, yogurt 4 Meat, fish and eggs 5 Vegetables and leaves (any dark green leafy vegetables such as [local dark green leafy vegetables] 6 Fruits (ripe mangoes, ripe papayas or [other local vitamin A-rich fruits] guava 7 Foods made with red palm oil, red palm nut, or red palm nut pulp sauce (Vitamin A rich oil), etc. Questions related to community mobilisation initiatives (only for intervention school) S. No. Question Options Codes 39. Are you aware of any community mobilisation initiatives/the following events? Read Play Festival Yes 1 No 2 Grade-I Reception Day Yes 1 No 2 Other, please specify Yes 1 No 2 40. Have you attended any of the community mobilisation initiatives/events? Read Play Festival Yes 1 No 2 Grade-I Reception Day Yes 1 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 120 Questions related to the dietary diversity of the child No 2 Other Yes 1 No 2 41. Have you received any training/workshop on health, hygiene and nutrition? Yes 1 No 2 42. If yes, who provided the training/conducted the workshop? WFP/RtR/YPSA/MA/RIC 1 Government officials 2 CHWs 3 Any other, please specify End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 121 4. SCHOOL STOREKEEPER QUESTIONNAIRE Interview Information A1. Name of the district A2. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 A3. Name of the village A4. School Name A5. School Code A6. Date of visit (dd/mm/yy) A7. Time of survey (in am/pm) Morning 1 Afternoon 2 A8. Name of the supervisor A9. Name of the enumerator A10. Has the storekeeper given her/his consent to participate in the interview? Yes 1 No 2 If the storekeeper says no, then thank them and proceed to the next storekeeper on your list. S.No. Question Options Codes Skip 1. What is your name? 2. Gender (Observe only) Male 1 Female 2 3. Since when have you been appointed as the storekeeper? ________Number of Years and _______ number of months 4. As a store keeper, how much time do you spend on this role in a day? ________Number of hours 5. Do you have a record book where you keep a stock of the food? (Request if you could see the record book.) Yes 1 No 2 If response is 2, Skip to Q7 6. This week 1 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 122 S.No. Question Options Codes Skip When was the last time that the record book was updated? Verify from the record book Last week 2 Last month 3 Two months back 4 More than 2 months back 5 Last year 6 No record found in the book 7 7. Does the school have a pest/insects management plan? Yes 1 No 2 8. Does the school carry out pest/insects control measures? Yes 1 9. Are you trained in safe food preparation and storage practices? Yes 1 If response is 2, Skip to Q11 No 2 10. If yes, who has provided training? WFP/MA/YPSA 1 Government 2 Any other, please specify 11. What according to you constitute safe storage practices? (Do not prompt) Keeping the store room properly ventilated? 1Yes 2 No Pest Control 1 Yes 2 No Keeping the room dry and free from moisture 1 Yes 2 No Stacking the stocks at an elevated level (using pallets) 1 Yes 2 No Others, please specify S.No. Question Options Codes Skip End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 123 12. How many such trainings did you receive in last one year? Write 99 if does not remember 13. Have you received a book about Warehouse management within the last one year? Yes 1 No 2 Don’t remember 3 14. Are you paid a salary for being a storekeeper here? Yes 1 No 2 15. Does your school have a dedicated store room for biscuits? Yes, within the school 1 Yes, outside the school 2 No 3 If response is 3, Skip to Q17 16. If the store room is outside the school, then how far is the distance to the store room? _____in km 17. If No, where are the biscuits stored? In a classroom 1 In teacher’s room 2 Open space 3 In the house of head master/teacher 4 Neighbour’s home 5 Others 18. Can the food storage room be locked? Yes 1 No 2 19. When you are on leave who manages biscuit distribution? Head Master 1 Teacher 2 School Support Staff(non-teaching) 3 No one manages, room is locked 4 20. When do you receive new stock of biscuits Every week 1 Every 15 days 2 Every month 3 Once in two month 4 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 124 Any other please specify 21. How much time does it take to receive the biscuits, once the requisition is made Stock is delivered before any requisition 1 One week 2 15 days 3 One month 4 More than a month 5 22. Has there been an instance during the last one year when the stock delivered was less than quantity demanded Yes 1 No 2 Go to 24 23. If yes, how many times this has happened in the last one year Enumerator to record reasons Once 1 2 times 2 More than 2 times 3 S.No. Question Options Codes Skip 24. Has there been an instance of stock out during the period September 2019-January 2020 Yes 1 No 2 Go to 27 25. If yes, how many times has this happened? Enumerator to record reasons Once 1 2 times 2 More than 2 times 3 26. What was the common reason for stock out Delayed supply from MA/YPSA/WFP despite timely requisition 1 Delay in requisition 2 Supply of stock not fit for consumption 3 27. Do you maintain buffer stock Yes 1 No 2 28. If yes for how many days One week 1 15 days 2 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 125 One month 3 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 126 29. Biscuit Receipt Record (Demand and Supply) Month Quantity of biscuits required Quantity of biscuits received Date of request Actual delivery date Quantity of biscuits distributed to students* February 2020 January 2020 December 2019 November 2019 October 2019 September 2019 *Please verify from the consumption record book and click a picture of the page End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 127 5. HEADMASTER QUESTIONNAIRE Interview Information A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. School Name A4. School code A5. Sex Male 1 Female 2 A6. Date of visit (dd/mm/yy) A7. Time of survey (in am/pm) Morning 1 Afternoon 2 A8. Name of the supervisor A9. Name of the enumerator 1. What does the school authority or administrator do to improve teaching/classroom environment? (Multiple Response) Nothing 1 Regular monitoring of teaching techniques of teachers 2 Provide feedback to teachers for improvement of the quality and techniques 3 Conduct regular training for teachers 4 Other (Specify) 2. Have the teachers in your school received training from WFP/RtR in the last one year (for Intervention Schools Only) Yes No 3. If yes, on what all aspects have the teachers received training(last one year) (Multiple Response) Teaching / Learning techniques (at least 2 days or 16 hours) 1 Storage practices 2 Hygiene and WASH 3 Child health and nutrition 4 4. How do you support adoption of new techniques by the teachers? (Multiple Response) Do nothing 1 Supplementary reading materials are made available to teachers 2 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 128 Encourage teachers to use audio-visual aid during class 3 Encourage teachers to prepare handmade posters / locally made visual aids 4 Encourage teachers to adopt participatory teaching techniques (e.g. Role play / Story telling /Group Discussion / Problem solving) 5 Reward/recognition for teachers 6 Other (Specify) 5. How do you monitor adoption of new techniques by teachers (Multiple Response) Do nothing 1 Classroom observation 2 Feedback from students 3 Feedback from teachers 4 Feedback from parents 5 6. What proportion of students in your school has the required textbook/s for the class? None 1 Less than 25% 2 25-50% 3 50-75% 4 75-99% 5 100% 6 Don’t Know 7 7. What proportion of students in your school has a desk and chair? None 1 Less than 25% 2 25-50% 3 50-75% 4 75-99% 5 100% 6 Don’t Know 7 8. Only to be asked for intervention schools. Do you think that since the school feeding programme has started, children are more attentive in class? Yes 1 No 2 Don’t Know 3 9. Only to be asked for intervention schools. Do you think that the provision of biscuits increases the number of children coming to school? Yes 1 No 2 Don’t Know 3 10. For intervention schools, ask According to you, if the provision of biscuits stopped from today, what would be the consequence of students’ attendance? For comparison schools where provision of biscuits is not going on ask, No consequence, attendance will remain the same 1 Attendance will drop by 25% 2 Attendance will drop between 25% and 50% 3 Attendance will go up by more than 50% 4 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 129 According to you, if the biscuits are provided from today, what would be the result on student attendance? 11. For intervention schools, ask According to you, if the provision of biscuits stopped from today, what would be the consequence on student enrolment? For comparison schools where provision of biscuits is not going on, ask According to you, if the biscuits are provided from today, what would be the consequence on student enrolment? No consequence, enrolment will remain the same 1 Enrolment will drop by 25% 2 Enrolment will drop between 25% and 50% 3 Enrolment go up by more than 25% 4 12. Are there measures in place/functional for improvement and maintenance of school infrastructure? (Multiple Response) No 1 Cleanliness of the school surrounding 2 Regular cleaning of classrooms 3 Regular cleaning of teacher’s rooms 4 Proper maintenance of school latrines 5 Proper maintenance of classroom and furniture 6 Availability of clean water for teachers and students 7 Other (specify) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 130 6. FOCUS GROUP DISCUSSIONS: PARENTS Name of the School: Name of the Village: Name of the Upazila: No. of Children in Primary School S. No Name of the Parent Age Gender Boy/s Girl/s Rapport Building 1. Do all children in your village go to school? Are there any households in your village where children do not go to school? If so, then why? 2. Why do you think it is important for children to go to school? (Probe for: different reasons like understanding the importance of education, or they want to keep them at school as they have to go to work, or they receive nutritious meals in the school, etc.) 3. Do you think the school is adequately addressing your expectations? If not, what more is required? 4. At home, who decides if a child should go to school or not? How many years of school would you like to see your own children complete and why? (Probe for differences between boys and girls) 5. How frequently have you attended parent-teacher meetings? Have there been any benefits? 6. Are you satisfied with the teaching quality in school? What more do you think they be done to benefit the children? 7. What according to you have been the benefits of the following activities? Please mention anything that you think could have been improved?  Biscuit distribution  Vegetable Gardens  Healthy Meal Preparation Day  Little Doctors  Deworming Programme  Book Captains  Library Corners  Remedial Classes in Bangla  Read Play Festival End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 131 8. Where do you get your information regarding these activities? How often?(Probe: any community mobilisation activities like Read Play Festival, Healthy Meal Preparation Day, Grade 1 Reception Day) 9. Do you or anyone in the household help your child read story-books and complete homework?  Does your child ask for this help? 10. Has the school conducted any training or activities for parents or the community? If yes, what are these and how do you think these trainings/activities have benefited the community. Please describe. (Probe: Read Play Festival, awareness on adequate nutrition and hygiene practices, healthy meal preparation day etc.) 11. How do you/would you like to contribute to these activities? 12. In order to do these, what additional support would you require? (Probe: training, workshops) 13. Do you think the trainings/activities initiated by the school has increased awareness on children’s health, nutrition, education and WASH related issues? How? Please give examples. 14. How have these trainings/activities helped you? Can you illustrate any daily practice you may have changed as a result of these trainings? 15. Has this change in practice caused a change in your environment? If yes, how? 16. Where apart from schools do you/your child get information regarding adequate nutrition and healthy eating? (Probe: children’s school, family, teachers, television, internet, IEC material, community health workers, community programmes etc.) 17. How have you been managing to continue your child’s education since the closure of schools due to COVID-19? End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 132 7. FOCUS GROUP DISCUSSIONS: SCHOOL MANAGEMENT COMMITTEE (SMC) Name of the School/SMC: Name of the Village: Name of the Upazila: No. of members: S. No Name Age Gender Position A. APPOINTMENT, ROLES & RESPONSIBILITIES 1. Details on formation, composition and frequency of meetings of SMCs to be collected by enumerator from school administration and copy MoMs need to be collected. 2. Have there been any instances where planned meetings have not been conducted? Kindly provide reasons. 3. How do you think the role of SMC has evolved in the last 3 years? 4. How do you think the SMC trainings have benefited the functioning of SMCs or the schools?  Would you like to receive training on other aspects? Like what? 5. What have been the various challenges faced in the school? (Probe: lack of infrastructure, facilities, high drop outs, absenteeism etc.) How has the SMC contributed in addressing these? Please explain. 6. Has there been a change in the school’s environment through the SMC’s efforts? What has changes and how? 7. What support has the SMC provided in implementing SFP activities? 8. Has the SMC’s roles and responsibilities been modified in any way since the inception of the programme? How? 9. Do you face any challenges in the programme’s implementation? What were these?  What steps were been taken to address these? 10. How do you think the programme has impacted the children? (Probe: improved health and nutrition, improved attendance, improved reading abilities, improved awareness with regard to health etc.) Please give examples. What has helped bring about this change? 11. Have you observed any change in the parent’s/community’s attitude in perceiving the importance of education and nutrition of their children? How? Please describe. What has helped bring about this change? 12. Do the parents/community support/participate in the programme in any way? If yes, how and on which occasions? 13. What measures have been taken to ensure the continuity in children’s education during the closure of schools? End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 133 8. Checklist for Capacity Building Activities (GoB) 1. What has been the progress on the ‘National School Meal Policy 2019’ since its approval? Kindly elaborate on the following￾a) Has an implementation plan for the policy been prepared? b) What challenges are foreseen in the adoption and implementation of the same? c) What support would MoPME, MoHFW and other ministries require from WFP to address some of these challenges? 2. What is the nature of responsiveness and involvement demonstrated by inter-agency committee members at the national and sub-national level? 3. What kind of dissemination of the NSMP has been undertaken so far? Specify platforms. 4. Has the dissemination and wider sharing of NSMP led to any feedback/actions? Please elaborate. 5. How do you think the technical workshops and support received by WFP has capacitated GoB to take up SFP independently? a) What more support is required for the government to be able to independently undertake such targeting for scale up? 6. Has the monitoring and reporting framework for the scale up areas been created? If yes, has it been agreed upon the various ministries and departments? (Request documents for the same) 7. Online Database and Monitoring a) How many districts are currently using the online MIS? b) How has the online monitoring helped improve the implementation of SFP? c) Are any challenges being faced in use of online MIS? d) What more is required to ensure its institutionalization? 8. What kind of support has WFP provided MoPME in ensuring selection of NGOs/implementing partners independently? a) Has DPE begun identifying implementing partners on their own? b) If yes, what kind of orientation activities have been undertaken and by whom? 9. Have any meetings with the Ministry of Finance been held for sustainable financing of NSMP? What is the progress on its inclusion in the revenue budget? 10. What other initiatives/ activities have been undertaken to strengthen and enhance engagement of local organizations and community groups? (Probe: training manuals for community mobilization/advocacy materials/IEC and media coverage) 11. What has been the response of the communities as a result of the same? (examples of engagement and involvement) 12. What have been some of the key learnings from the technical assistance provided by WFP to GoB/MoPME? a) Have these learnings been documented? b) How are these learnings being incorporated to inform future activities? 13. What changes (if any) have been made to the timelines and modalities for building capacity of GoB officials through the school feeding programme? 14. What remains to be done to initiate the roll out of the implementation strategy of the National School meals Programme? End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 134 9. Checklist for Capacity Building Activities (WFP) 1. What has been the progress on the ‘National School Meal Policy 2019’ since its preparation? 2. Do you think that the CB activities have led to increased ownership or a perception change of certain ministries? Kindly give examples 3. How have the findings/learnings of the SABER country report 2020 being incorporated in the operationalization of NSMP? 4. What has been the response of the Ministry of Finance to ensure sustainable financing of NSMP? What is the progress on its inclusion in the revenue budget? 5. What kind of challenges remain with regards to scale up and expansion of SFP? 6. What more support is required for the government to be able to independently implement the programme? 7. Has the monitoring and reporting framework for the scale up areas been created? If yes, has it been agreed upon the various ministries and departments? (Request documents for the same) 8. What kind of support has WFP provided MoPME in ensuring selection of NGOs/implementing partners independently? Has MoPME begun identifying implementing partners on their own? 9. If yes, what kind of orientation activities have been undertaken and by whom? 10. How is government integrating complementary components such as WASH, health, nutrition, dietary practices and literacy of school children in the hand-over areas? 11. How is the government working towards improving learning environment for children? (including feeding and nutrition support services for pre-school children and partnership with ECD network) Please provide details. 12. What initiatives/activities have been undertaken to strengthen and enhance engagement of local organizations and community groups? (Probe: Circular of functioning of PTA, training manuals for community mobilization/advocacy materials/IEC and media coverage) 13. What have been some of the key learnings from the technical assistance provided by WFP to GoB/MoPME? a) Have these learnings been documented? b) How are these learnings being incorporated to inform future activities? 14. What are the aspects that WFP will continue to provide support on and aspects it will stop supporting as a result of adequate capacities built? 15. Is there an exit strategy for the CB activities for GoB? If yes, has it been implemented? 16. What changes (if any) have been made to the timelines and modalities for building capacity of GoB officials through the school feeding programme? 17. How were the planned activities affected during COVID-19? What is the resultant change in the implementation strategy? 18. What remains to be done in order to achieve a full handover of the school feeding programme? 19. How has this strategy been affected by COVID 19? In the context of a pandemic, how does WFP plan to ensure sustainability of the project? End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 135 Annex N: Evaluation Mission Schedule Table 12: Evaluation mission schedule Dates Activity Stakeholders Locations/ Sites Inception Phase- Scoping Discussion 16th -25th April 2020  Meeting with WFP – CO team-Evaluation Manager, School Feeding Team , Government Capacity Strengthening Officer, Head of M&E unit  Meeting with representatives of Implementation Partners  Discussion with RBB representatives NRMC, WFP CO, Implementation Partners New Delhi (Online ) Inception Phase- Inception Report Preparation and Finalisation 26th April –7th May 2019 Literature Review Preparation of quantitative and qualitative tools Finalization of evaluation approach and methodology Preparation and submission of Inception Report to WFP CO NRMC New Delhi 5th October 2020 Preparation of revised inception report ( with revisions in the methodology considering COVID-19) NRMC New Delhi 6th October – 4th November 2020 Review of Inception report and tools by WFP, DEQAS and USDA Incorporating comments in the report Finalization of the Inception report Identification of field enumerators and supervisors for data collection Translation of tool and coding into CAPI NRMC, WFP CO, DEQAS, USDA Dhaka, New Delhi Training 16th December – 17th December 2020 Training of field enumerators and field practice (Virtual). NRMC, DMA Cox’s Bazar Delhi End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 136 Dates Activity Stakeholders Locations/ Sites 19th December 2020 Field Practice in three schools in Cox Bazar and debriefing with the data collection teams Cox’s Bazar Data Collection 20th December 2020- 7th January 2021 Primary Qualitative and Quantitative Survey in treatment schools (Virtual) Debriefing of team at the end of each field day Quality Check of all filled questionnaires and data collected with the supervisors NRMC Delhi, Ukhiya, Kutubdia and Ramu KIIs with representatives GoB (MoPME, DPE) NRMC Dhaka ,Delhi Data Analysis, Reporting and Dissemination Phase 7th January 2020 – 5th March 2021 Quantitative data cleaning, data analysis, documentation of field notes and findings, development of analysis plan and report structure Preparation of draft evaluation report NRMC New Delhi 5th March 2021 Submission of Draft End- Term Evaluation Report to WFP NRMC New Delhi 5th March 2021 – 2nd April 2021 Review of Draft End- Term Evaluation Report by WFP, DEQAS and USDA Incorporating comments in the report Preparation of Final End- Term Evaluation Report NRMC, WFP CO, DEQAS, USDA Dhaka, New Delhi 2nd April 2021 Submission of the Final End- Term Evaluation Report NRMC New Delhi End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 137 Annex O: Documents Gathered WFP CO has shared a host of documents for the evaluation. The table below describes the nature of documents reviewed. Table 13: Documents Gathered and Reviewed Document Type Comment / Titles & dates of documents received Received - Y/N (N/A) Link to Evaluation matrix Project related documents Y Appraisal mission report N/A Project document (including Logical Framework in Annex F) Country Strategic Plan, Bangladesh (2017-20), Project Agreement between the FAS and the WFP including the results framework Y Relevance Standard Project Reports Y Budget Revisions Financial Report Y Note for the record (NFR) from Programme Review Committee meeting (for original operation and budget revisions if any) N/A Approved Excel budget (for original intervention and budget revisions if any) Programme budget summary and narrative Y Relevance, Efficiency Intervention/Project Plan (breakdown of beneficiary figures and food requirements by region/activity/month and partners) Project Agreement between the FAS and the WFP Y Relevance, Effectiveness Other Country Office Strategic Documents (if applicable) N/A Country Strategy Document (if any) Country Strategic Plan, Bangladesh (2017-20), Y Relevance, Sustainability Other Assessment Reports [if applicable] SABER Country Report 2020 Y Effectiveness, Impact and Sustainability Baseline Evaluation Report Y Comprehensive Food Security and Vulnerability Assessments N/A Crop and Food Security Assessments (FAO/WFP) N/A Emergency Food Security Assessments N/A End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 138 Document Type Comment / Titles & dates of documents received Received - Y/N (N/A) Link to Evaluation matrix Food Security Monitoring System Bulletins N/A Market Assessments and Bulletins N/A Joint Assessment Missions (UNHCR/WFP) N/A Inter-Agency Assessments N/A Rapid needs assessments N/A Cash and voucher feasibility studies N/A Other Monitoring & Reporting (if applicable) MGD FY17 semi-annual report Y Efficiency, Effectiveness M&E Plan Performance Monitoring Plan and New Monitoring Frameworks Y Relevance, Effectiveness, Impact M&E Policy USDA Monitoring and Evaluation Policy Y Relevance M&E Indicators and Definitions USDA and McGovern-Dole Indicators and Definitions Y Effectiveness, Impact Country Situation Report (SITREP) N Country Executive Brief WFP Bangladesh Country Brief Y Relevance Food Distribution and Post-distribution Monitoring Reports N Effectiveness, Impact Monthly Monitoring Reports Semi-annual project reports till March 2020 Y Effectiveness, Impact, Efficiency Beneficiary Verification Reports N/A Donor specific reports Bangladesh FY 17 Semi-annual Report Y Effectiveness, Impact Output monitoring reports (if applicable) Quarterly and Monthly Monitoring Reports of RtR and MA Y Effectiveness, Efficiency Actual and Planned beneficiaries by activity and district/ location by year Actual and Planned beneficiaries by activity by year; location wise monitoring data is not available, USDA semi-annual reports Y Effectiveness, Impact, Efficiency Male vs. Female beneficiaries by activity and district/ location by year Would be required to evaluate the gender component of the programme N Beneficiaries by age group N/A Actual and Planned tonnage distributed by activity by year FY17 Semi-annual report Y Effectiveness, Impact, Efficiency Commodity type by activity N/A End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 139 Document Type Comment / Titles & dates of documents received Received - Y/N (N/A) Link to Evaluation matrix Actual and Planned cash/voucher requirements (US$) by activity by year N/A Operational documents (if applicable) N/A Organogram for main office and sub-offices BDCO McGovern Dole Annual Work Plan FY2019 Y Effectiveness, Efficiency, Sustainability Activity Guidelines N Mission Reports N Pipeline overview for the period covered by the evaluation N Logistics capacity assessment N/A Partners (if applicable) Terms of Reference Y Annual reports from cooperating partners From RtR, MA, YPSA, RIC (Project completion report, if any) Y Effectiveness, Efficiency, Impact, Sustainability List of partners (Government, NGOs, UN agencies) by location/ activity/ role/ tonnage handled List of partners has been provided in ToR. Relevant documents have been shared for their roles. Y Effectiveness, Sustainability Field level agreements (FLAs), Memorandum of Understanding (MOUs) Roles and Responsibilities of various partners Y Effectiveness, Efficiency, Sustainability Cluster/ Coordination meetings (if applicable) N/A Logistics/Food Security/nutrition cluster documents N/A NFRs of coordination meetings N Sustainability Other Evaluations/ Reviews Y Relevance, Effectiveness, Impact, Sustainability Evaluations/ reviews of past or on-going operation USDA McGovern-Dole International Food for Education and Child Nutrition Programme’s Support (2014-2016) to WFP Bangladesh Country Programme: Baseline and End Term Report FY17 Baseline report Y Relevance, Effectiveness, Impact, Sustainability Resource mobilisation (if applicable) Y Efficiency, Effectiveness Resource Situation N Efficiency, Effectiveness Contribution statistics by month N/A End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 140 Document Type Comment / Titles & dates of documents received Received - Y/N (N/A) Link to Evaluation matrix Resource mobilization strategy N Efficiency, Effectiveness NFRs Donor meetings N Maps (if applicable) Y Relevance Operational Map BDCO McGovern Dole Annual Work Plan FY2019 Y Relevance Logistics Map N Food/Cash/voucher Distribution Location Map N/A Food Security Map N/A Other documents collected by the team (including external ones) (if applicable) NPAN 2, End line evaluation of Reading Outcomes in Government Primary Schools (USAID’s Reading Enhancement for Advancing Development (READ) Activity Y Templates Y Work plans Y Effectiveness, Efficiency Norms and standards Y Specify End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 141 Annex P: Ethical Considerations 1. NRMC understands the United Nations Evaluation Group (UNEG) norms, standards and ethical guidelines and aspires to strictly adhere to the same. 2. NRMC adheres to ethical practice and code of conduct during all its evaluations following its own ethical guidelines and that of the clients. Particularly, in relation to evaluations that cater to sensitive population groups such as women and girls, social and religious minorities, disaster affected populations, migratory populations, elderly, disabled and pregnant women, we ensure extra care in the process. 3. The team for this assignment was guided by the UNEG ethical guidance principles throughout, which ensured that no violations, like collecting data without consent, collecting data not pertaining to this assignment, accessing areas within the institution premises for which approval has not been taken etc., were committed during the data collection. 4. For the interviews, a verbal consent from the respondents was taken. Prior to the consent, the respondent was informed about their voluntary participation and confidentiality of information being collected. They were given assurance that the data will be kept confidential and no personal identifiers will be mentioned in the report. 5. The evaluation team as well as the data collection team adhered to the following three categories of ethical norms:  Informed consent: All interviews were conducted with prior verbal consent of the respondents. The participants were clearly briefed on the intent of the discussions and the use of the findings from the interviews towards research purposes. The respondents were informed of their right to decline their participation in the interview or not disclose any information that they did not want to reveal.  Right to safeguard integrity: No information obtained from the responses was made public at any stage of the surveys and thereafter. The database did not have the name of the respondent or family members. All such information was encrypted. A complete privacy of the respondent was maintained.  Protection from physical, mental and emotional harm: During the survey, enumerators did not ask any questions or pose any cross-questions that were personal or sensitive, or that would have physically, mentally or emotionally harmed the respondent.  Access to information regarding research: Data collectors provided all information related to the survey and its objectives. If the respondent wanted to seek more information, they were asked to contact the senior team members for answers.  Protection of privacy and well-being: Personal information of the respondent was not shared with anyone and was kept confidential. It was used for evaluation purposes only. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 142 Annex Q: NRMC’s Internal Protocols 1. NRMC’s Internal Quality Assurance procedures adhered to the following three categories of norms as follows: 2. Our internal quality control measures were followed at each stage of our evaluations starting from design of tools, to hiring and training of field teams; from data collection in the field to monitoring of the process; from data analysis till the reporting stage. Some of our key measures undertaken to ensure data quality is as follows: a. A three-stage internal review process of key deliverables: Our evaluation teams was structured in a manner that all deliverables including data collection tools, inception report, data analysis plans, and reports are reviewed by the project coordinator, the team leader and an external technical backstopping expert assigned internally for the project. b. Our evaluation teams travelled extensively to the field work areas to ensure that the data quality is excellent. We engaged our core evaluation team members to travel for field visits especially for qualitative interviews with sensitive populations. We devised efficient and effective methods to deploy coordinators and supervisors to monitor the data collection process continuously. c. Assuring data quality during analysis: Using interactive checking, validation of sample data, and data cleaning by our data analysts, NRMC ensured the quality of data from qualitative or quantitative methods. We ensured that transcripts of the qualitative interviews along with field notes are made available to the evaluation team for better analysis. d. Strong support teams: Training for field teams was conducted by core team members (from the evaluation team). Our professional editor ensured that all reports are edited before sharing with the client. Our internal data processing team ensured that all errors are resolved quickly and thoroughly. The staff on payroll or contract always demonstrates honesty, integrity, and professionalism at all times. The staff is aware of applicable statutes, regulations, practices, and ethical standards governing data collection and reporting. The team reports information accurately and without bias. The team is accountable, and holds others accountable, for ethical use of data. Integrity The team promotes data quality by adhering to best practices and operating standards. The team provides all relevant data, definitions, and documentation to promote comprehensive understanding and accurate analysis when releasing information. Data Quality The team treats data systems as valuable organizational assets and hence data backup is a mandatory affair. The team safeguards sensitive data to guarantee privacy and confidentiality as our servers are accessible to limited staff only. Data Security End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 143 Annex R: Team Composition and Specific Tasks Table 14: Team Composition and Responsibilities Core Team Members Primary Role Specific tasks within the Evaluation Deliverables Dates Rahul Agrawal Team Leader and Evaluation Manager  Overall project backstopping and coordination of assignment including interaction with WFP CO  Technical inputs on research design, development of research tools and conceptual framework, sampling design analysis of data, and quality assurance of the inception, draft, and final report  Technical inputs on detailing results framework  Oversee quality of deliverables to WFP CO, DEQAS and USDA  Designing evaluation frameworks, key indicators and analytical frameworks  Technical inputs on evaluation methods and tools  Technical backstopping for research design implementation  Field work quality assurance  Training of data collection teams  Tabulation plan, validation of tables and indicators  Data analysis  Report Writing and presentation Inception report, Draft report and Final report As per the timelines in Annexe N. Prof. Hafizur Rehman Advisor- Primary Education  Technical inputs and advisory on the development of the evaluation methodology  Technical inputs on quantitative and qualitative tools  Technical backstopping  Technical inputs on detailing results framework  Designing of EGRA tool and classroom observations Inception report, Draft report and Final report As per the timelines in Annexe N. Jayesh Bhatia Quality Assurance Expert  Defining quality assurance protocols for data collection, analysis and deliverables and ensuring its implementation Inception report, Draft report and Final report As per the timelines in Annexe N. Mrinalini Gender Expert  Development of conceptual framework and evaluation design with a specific focus on gender  Development of evaluation indicators and tools  Development of tools with inclusion of equity and gender components  Contextual analysis of the data with a gender lens  Data collection and monitoring of field work  Tabulation plan, validation of tables and indicators  Development of analytical framework and data analysis  Report Writing Inception report, Draft report and Final report As per the timelines in Annexe N. Aditi Chordia Research Manager  Development of evaluation indicators and tools  Training of data collection teams  Data collection and monitoring of field work(including quality assurance)  Tabulation plan, validation of tables and indicators  Development of analytical framework and data analysis  Report Writing Inception report, Draft report and Final report As per the timelines in Annexe N. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 144 Annex S: The Training Schedule of the Data Collection Team Table 15: Training Schedule S. No. Time Topic Responsibility Day 1 1. 9:30 am–10:00 am Team Introductions NRMC and DMA 2. 10:00 am- 10:30 am Training on WHO norms and guidelines for COVID-19 NRMC 3. 10:30 am-11:30 am Training of data collection teams on context of the programme, about the programme and the various components of the evaluation. Overview of the tools. NRMC 4. 11:30 am-12:00 pm Ethical guidelines to be followed during the evaluation survey NRMC 5. 12:00 pm-1:30 pm Training on School Questionnaire NRMC and DMA 6. 1:30 pm-2:15 pm Lunch break 7. 2:15 pm–4:00 pm Training of school questionnaire contd. NRMC and DMA 8. 4:00 pm-4:15 pm Break 9. 4:00 pm-5:30 pm Questionnaire for Parents, Headmaster NRMC and DMA 10. 5:30pm-6:00 pm Formation of groups (4 groups) basis on the exercise conducted to assess their capabilities NRMC/DMA Day 2 1. 9:00 am-9:30 am Recap of last day NRMC and DMA 2. 9:30 am-11:00 am Questionnaires Teacher, Storekeeper NRMC and DMA 3. 11:00 am-11:15pm Break NRMC and DMA 4. 11:15 am-1:15pm Mock calls of all questionnaires using CAPI NRMC and DMA 5. 1:15 pm-2:00 pm Lunch Break 6. 2:00 pm-4:00 pm Training on FGDs NRMC and DMA 7. 4:00 pm-4:15 pm Break 8. 4:15 pm-5:30 pm Mock calls and Revision for the day NRMC and DMA Day 3 1. 7:30 am-12:30 pm Field Practice NRMC and DMA 2. 12:30 pm-1:00 pm Lunch 3. 1:00 pm-3:00 pm Field practice NRMC and DMA 4. 3:00 pm-4:00 pm Field enumerators share experiences, lessons and queries NRMC and DMA 5. 5:00 pm-6:00 pm De-briefing of the team and query resolution NRMC and DMA 6. 6:00 pm –6:30 pm Finalize field movement plan NRMC and DMA End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 145 Annex T: End-Term Values of Key Indicators Table 1 Note: In the Status column, the colour codes indicate whether the set target for the indicator was achieved (green) or not achieved (red). For indicators where the MTE values were considered due to methodological limitations (refer section 2.3), the cells have been coded white. For one indicator related to teacher’s attendance that the ETE has not reported on (indicator no 1.1), the colour grey has been used. Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t SO1 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of the grade-level text Students reading and understanding ability were assessed using the EGRA tool. The tool was administered with 420 students in intervention and 280 students in comparison schools in Grade 3. Students were categorised into emergent, beginner and reader categories with 35% 6.9% 8.4% 5.5% 28% 25% 32% 28% 25% 32% 117 All midterm values are on the basis of WFP monitoring report and extrapolated on the basis of findings from the sample survey of the mid-term evaluation End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 146 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t comprehension ability based on a sub-test score, as recommended by EGRA guidelines. Students who can read more than 45 words and correctly answer 4-5 questions are indicated here. 1.1 Number of school administrators and officials in target schools who demonstrate the use of new techniques or tools as a result of USDA assistance No. of trained headmasters who reported using and demonstrating the use of new learning techniques and tools 116 0 0 0 78 68 10 88 (extrap olated) 14 4 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 147 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 1.1 Number of teachers/educat ors/teaching assistants in target schools who demonstrate the use of new and quality teaching techniques or tools as a result of USDA assistance Teachers, educators, teaching assistants who have successfully completed a pre- or in-services training programme to teach in schools or equivalent non￾school based settings over the past one year were then assessed on whether they are using the learned techniques. Use of techniques was assessed by asking their supervisors whether the teacher is using the techniques he/she learned in the classroom. 584 117 40 77 338 175 163 496 (extrap olated) 39 63 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 148 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 1.1 Percent of teachers in target schools who attend and teach school at least 90 percent of scheduled school days per school year118 Regular teacher attendance was defined as attending more than or equal to 90 percent of school days; teacher attendance was assessed using monthly school records of all the teacher from sample schools and the number of working days per month during the period July 2017- September 2017 75% 82.3% 90.9% 76.1% -- -- -- -- -- -- 118 The data for this indicator could not collected in midterm evaluation due to government’s circular which restricts access to teacher’s attendance data for anyone apart from education department. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 149 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 1.2 Percent of students in classrooms identified as attentive by their teachers Attentiveness of students is based on the perception of teachers on the attentiveness of sampled students from each school 70% 70.3% 68.8% 71.7% 75.1% 71.4% 78.9% 75.1% 71.4% 78.9% 1.3 Number of students enrolled in school receiving USDA assistance Student enrolment is based on monthly school records (pre￾primary and primary) of the sample schools for the academic year commencing January 2018 47,586 268 (average per school) 119 125 (averag e per school) 142 (avera ge per school) 322 (averag e per school) 47586 (from monitor ing report) 153 (avera ge per school) 169 (avera ge per school) 368 (avera ge per school) 45062 (from monito ring report) 173 (avera ge per school) 20897 (from monito ring report) 196 (avera ge per school) 24165 (from monito ring report) 119 Comparable values were not available as the presented number was derived from the sample of the baseline (data for all 146 schools was not available). End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 150 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 1.3 Percent of students regularly (at least 80 percent of the school days) attending school Student attendance was measured using the attendance record of fifteen randomly selected students from every sample school for the last academic semester (September 2017- February 2018). 82% 83.3% 84.4% 82.4% 85.2 % 82% 88.3% 88.7% 82.4% 94% 1.3 Average number of school days missed by each student due to illness Data was collected by asking students the no. of school days missed in the last 1 week and out of the missed school days, how many were missed due to illness. 5 2 2 2 2 2 2 2 2 2 1.3 Percent of parents in target communities who can name at least three Data was collected through interviews with parents and asking them about 70% 67.8% 61.3% 72.4% 73.2% 80% 66.4% 71% 80% 69% End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 151 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t benefits of primary education the benefits of primary education. 2.1 Percent of students who can identify at least three key health and hygiene practices Students who could identify three or more key health and hygiene practices during the primary survey 65% 25.1% 27.6% 22.7% 39.2% 39.6% 38.9% 39.2% 39.6% 38.9% 2.2 Number of individuals trained in safe food preparation and storage as a result of USDA assistance The data represents teachers, headmasters and storekeepers who reported having received training on storage practices 322 0 0 0 94 66 28 94 66 28 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 152 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 2.2 Number of individuals who demonstrate the use of new safe food preparation and storage practices as a result of USDA assistance This data was collected through observation in the storage facilities during the primary survey. 290 35 22 13 94 66 28 94 66 28 2.2 Percent of storekeepers who can identify at least three safe storage practices This data was collected through the interviews with the storekeepers 50% 96.7% NA NA 100% 100% 100% 85% 92% 75% 2.4 Number of schools using an improved water source Schools that had any of the following sources of water: Tap water, RO plant within the school, Borehole, protected 146 26 NA NA 96 NA NA 134 NA NA End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 153 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t dug well or Protected spring Data collected from WFP Monitoring report 2.4 Number of schools with improved sanitation facilities Schools that have any of the following sanitation facilities: Flush or pour/flush facilities connected to a piped sewer/septic/pit latrine, flush or pour/flush toilets without a sewer connection, Pit latrine with a slab, Ventilated improved pit latrines or Composting toilets Data collected from WFP Monitoring report 146 11 NA NA 99 NA NA 134120 NA NA 120 WFP monitoring report Oct 19-Mar 20 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 154 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 2.5 Number of individuals trained in child health and nutrition as a result of USDA assistance121 Teachers and headmasters who received training on health, hygiene and nutrition 730 0 0 0 25 (as per survey) 1480 (from monitoring report) 17 8 79 (46 head teacher and teacher and 33 parent s) 22 57 SO2 Number of individuals who demonstrate the use of new child health and nutrition practices as a result of USDA assistance122 Data for handwashing was collected from parents. The values represent those parents who were trained by WFP and among them who reported washing hands for 4 or more out of 6 critical times Adding to this, the number of head 28,600 114 NA NA 204 (extrapolat ed) 42 (25 head teacher s and teacher s + 17 parents ) 19 23 65 (46 head teacher s and teacher s + 19 parent s) 22 43 121 Individuals consist of headmasters, teachers and parents for this indicator. While none of the individuals have been trained on child health and nutrition practices under this programme yet, the values given in the table represents the number of headmasters and teachers who have received training on health, hygiene, and nutrition from the GoB or any other source. 122 Child health and nutrition practices were assessed using 3 parameters; Handwashing, Dietary Diversity and Deworming (data for deworming has been reported separately) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 155 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t teachers and teachers who demonstrated use of child health and nutrition practices. Dietary diversity of pupils was measured through interviewing parents using 24 hours recall method. Mean Dietary Diversity Score is presented here. 4.2 4.2 4.3 4.97 4.96 4.99 4.85 4.88 4.82 2.5 Number of students receiving deworming medication(s) This data was collected from the headmasters 47,486 0 0 0 47,000 22,266 24,734 44,491 123 - - 123 Reported from WFP Monitoring Report (October 2019 to March 2020) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 156 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 1.3.3 Number of educational facilities (i.e. water systems and latrines) rehabilitated/co nstructed as a result of USDA assistance This data was collected from the headmasters through school questionnaires 35 N.A. N.A. N.A. 18124 NA N.A. 32125 NA NA SO1 Number of individuals benefitting directly from USDA-funded interventions WFP Monitoring reports 43,438 N.A. N.A. N.A. 51,232 126 N.A. N.A. 45,062 127 20,897 24,165 SO1 Number of individuals benefitting indirectly from WFP Monitoring Reports 175,044 N.A. N.A. N.A. 175,044 128 N.A. N.A. 18024 8 N.A. N.A. 124 Schools that reported to receive support from WFP for rehabilitation/construction of toilets, storage facilities, water systems, library, playground, classroom etc. 125 Reported from WFP Monitoring Report (October 2019 to March 2020) 126 Individuals included in this indicator are students, headteacher, Grade I and Grade II Bangla teachers, Govt. officials, SRM teachers and SMC members. 128 Individuals reported for this indicator includes parents of the students receiving school meals End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 157 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t USDA-funded interventions 2.4 Number of “Little Doctor” students supported by WFP This was collected through school questionnaire 2,025 N.A. N.A. N.A. 2,093 N.A. N.A. 2044 N.A. N.A. 1.4.4 Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance This is based on an interview with headmasters on the existence of SMCs and PTAs in the school 334 59129 NA NA 189 NA NA 189 NA NA 129 All these schools had school governance structures prior to commencement of activities under the current USDA grant End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 158 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 1.4.4 Number of public-private partnerships formed as a result of USDA assistance WFP monitoring reports 1 N.A. N.A. N.A. N.A. N.A. N.A. 1 N.A. N.A. 1.4.4 Value of new public and private sector investments leveraged as a result of USDA assistance WFP Monitoring Reports $227.1 N.A. N.A. N.A. USD 19 million dollars N.A. N.A. USD 5.2 million dollars 130, 1.3. million dollars 131 N.A. N.A. 1.4.2 The number of educational policies, regulations and/or administrative procedures in each of the WFP Monitoring Reports 2 N.A. N.A. N.A. 2 N.A. N.A. 2 N.A. N.A. 130 value of GoB in school feeding 131 funding support from Japan and Danish Government End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 159 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t following stages of development as a result of USDA assistance:  Stage 1: Analyse d  Stage 2: Drafted and present ed for public/ stakeho lder consult ation  Stage 3: Present ed for legislati on/decr ee  Stage 4: Passed/ End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 160 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t Approv ed Stage 5: Passed for which implementation has begun 1.4.4, 1.4.1 Number of meetings/works hops/training sessions held for institutional capacity to implement SF as a result of USDA assistance WFP Monitoring Reports 142 N.A. N.A. N.A. 6 N.A. N.A. 139 N.A. N.A. 1.4.4 Number of community mobilization workshops organized as a result of USDA assistance WFP Monitoring Reports 38 N.A. N.A. N.A. N.A. N.A. N.A. N.A.132 N.A. N.A. 132 Planned activities like community mobilization workshop for parents and health and hygiene sessions for students had to be postponed as schools are closed since mid-March 2020. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 161 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 1.4.1, 1.4.2, 1.4.4 Number of pilot initiatives supported to design SF modalities as a result of USDA assistance WFP Monitoring Reports 4 N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. 1.4.1 Number of government staff trained as a result of USDA assistance WFP Monitoring Reports 20,400 N.A. N.A. N.A. 837 N.A. N.A. 5,227 2.2, 1.4.1, Number of technical training for system development for service/food procurement, quality control, supply chain, and strengthened WFP Monitoring Reports 284 N.A. N.A. N.A. 30133 N.A. N.A. 133 From WFP monitoring reports End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 162 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t online database system and gender mainstreaming in the programme as a result of USDA assistance SO1, SO1 Number of schools supported by the Government with school feeding as a result of USDA assistance WFP Monitoring Reports 13,482 N.A. N.A. N.A. N.A. N.A. 1.4.4 Number of social mobilization/co mmunity meetings as a result of USDA assistance WFP Monitoring Reports 84 N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 163 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value 117 Midline Value by sex End Value End line Value by sex Status Project Male Femal e Project Male Femal e Projec t Male Femal e Project Projec t Projec t Projec t Projec t Projec t 1.4.4 Number of teachers, parents and school management committee members attended the community mobilization workshops WFP Monitoring Reports 1,168 N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. 1.2.1.1 & 1.3.1.1 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance WFP Monitoring Reports 41,334 N.A. N.A. N.A. 47,586 22,588 24,998 45,062 20,897 24,165 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 164 Table 2 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison SO1 Percent of students who, by the end of two grades of primary schooling, demonstr ate that they can read and understa nd the meaning of the grade￾level text Students reading and understanding ability were assessed using the EGRA tool. The tool was administered with 420 students in intervention and 280 students in comparison schools in Grade 3. Students were categorised into emergent, beginner and reader categories with comprehension ability based on a sub-test score, as recommended by EGRA guidelines. Students who can read more than 45 words and correctly answer 4-5 questions are indicated here. 6.90 % 13.6 0% 8.4 0% 11.70 % 5.50 % 15.40 % 28% 17% 25% 12% 32% 23% 28% 17% 25% 12% 32% 23% End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 165 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison 1.1 Number of textbooks and other teaching and learning materials provided as a result of USDA assistanc e Number of textbooks and other teaching materials provided 0 0 N.A N.A N.A N.A 82,8 07 0 NA NA NA NA 2,60,72 8 0 NA NA NA NA 1.1 Number of school administr ators and officials trained or certified as a result of USDA assistanc e No. of headmasters who have successfully completed a pre￾or in-services training programme to teach in schools or equivalent non-school based settings over the past year. 0 0 0 0 0 0 88 0 68 0 20 0 97 (extrap olated) 20 (based on survey) 15 (base d on surve y) 5 (based on survey) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 166 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison 1.1 Number of school administr ators and officials in target schools who demonstr ate the use of new technique s or tools as a result of USDA assistanc e No. of trained headmasters who reported using and demonstrating the use of new learning techniques and tools 0 0 0 0 0 0 78 0 68 0 10 0 88 (extrap olated) 18 (based on survey) 14 (base d on surve y) 4 (based on survey) 1.1 Number of teachers/ educators /teaching assistants trained or certified as a result of USDA assistanc e Teachers, educators, teaching assistants who have successfully completed a pre￾or in-services training programme to teach in schools or equivalent non-school based settings over the past year. 0 0 0 0 0 0 404 0 209 0 195 0 550 (extrap olated) 113 (based on survey) 43 (as per surve y) 70 (as per survey) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 167 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison 1.1 Number of teachers/ educators /teaching assistants in target schools who demonstr ate the use of new and quality teaching technique s or tools as a result of USDA assistanc e Teachers, educators, teaching assistants who have successfully completed a pre￾or in-services training programme to teach in schools or equivalent non-school based settings over the past one year were then assessed on whether they are using the learned techniques. Use of techniques was assessed by asking their supervisors whether the teacher is using the techniques he/she learned in the classroom. 117 110 40 37 77 73 338 0 175 0 163 0 496 (extrap olated) 102 (based on survey) 39 (estim ated based on surve y findin gs) 63 (estima ted based on survey finding s) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 168 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison 1.1 Percent of teachers in target schools who attend and teach school at least 90 percent of schedule d school days per school year Regular teacher attendance was defined as attending more than or equal to 90 percent of school days; teacher attendance was assessed using monthly school records of all the teacher from sample schools and the number of working days per month during the period July 2017- September 2017 82.3 0% 86.8 0% 90. 90 % 92.50 % 76.1 0% 83.10 % -- -- -- -- -- -- -- -- -- -- -- -- 1.1 Number of classroo m libraries (book shelves with books) establish ed as a result of USDA assistanc e No. of schools that have classroom libraries within their premises 0 0 NA NA NA NA 30 0 NA NA NA NA 30 0 NA 0 NA 0 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 169 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison 1.2 Number of daily school meals (breakfas t, snack, lunch) provided to school￾age children as a result of USDA assistanc e This data was collected from the school records NA NA NA NA NA NA 24,8 8,91 4 0 NA 0 NA 0 44,38,9 00 0 NA 0 NA 0 1.2 Number of school￾age children receiving daily school meals (breakfas t, snack, lunch) as a result of USDA assistanc e This data was collected from the school records NA NA NA NA NA NA 46,7 91 0 22,211 0 24,58 0 0 45,062 0 21,39 1 0 23,671 0 1.2 Percent of students in classroo ms Attentiveness of students is based on the perception of teachers on the attentiveness of 70.3 0% 89.2 0% 68. 80 % 89.60 % 71.7 0% 88.90 % 75.1 0% 71.20 % 71.40 % 71.30 % 78.90 % 71% 75.10 % 71.20 % 71.40 % 71.30 % 78.90 % 71% End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 170 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison identified as attentive by their teachers sampled students from each school 1.3 Number of students enrolled in school receiving USDA assistanc e Student enrolment is based on school records (pre￾primary and primary) of the sample schools for the most recent academic year 268 364 125 163 142 200 322 401 153 182 169 219 368 355 173 157 196 197 (Ave rage per sam ple scho ol) (Ave rage per sam ple scho ol) (Av era ge per sam ple sch ool) (Aver age per sampl e school ) (Ave rage per sam ple scho ol) (Aver age per sampl e school ) (Ave rage per sam ple scho ol) (Aver age per sampl e school ) (Avera ge per sample school) (Avera ge per sample school) (Aver age per sampl e school ) (Aver age per sampl e schoo l) (Avera ge per sample school) (Aver age per sampl e school ) (Aver age per sampl e school ) (Avera ge per sample school) (Avera ge per sample school) (Averag e per sample school) Percent of students regularly (at least 80 percent of the school days) attending school Student attendance was measured using the attendance record of fifteen randomly selected students from every sample school for the last academic semester (September 2017-February 2018). 83.3 0% 91.8 0% 84. 40 % 89.80 % 82.4 0% 93.80 % 85.2 0% 93.60 % 82% 92.70 % 88.30 % 94.60 % 88.70 % 96.00 % 82.4% 97.40 % 93.90 % 95.20% End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 171 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison 1.3 Average number of school days missed by each student due to illness Data was collected by asking students the no. of school days missed in the last 1 week and out of the missed school days, how many were missed due to illness. 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1.3 Percent of parents in target communi ties who can name at least three benefits of primary education Data was collected through interviews with parents and asking them about the benefits of primary education. 67.8 0% 66.0 0% 61. 30 % 63.20 % 72.4 0% 67.70 % 73.2 0% 70% 80% 85.70 % 66.40 % 69% 71% 77% 80% 65% 69% 80% 2.1 Percent of students who can identify at least three key health and hygiene practices Students who could identify three or more key health and hygiene practices during the primary survey 25.1 0% 34.6 0% 27. 60 % 31.70 % 22.7 0% 37.30 % 39.2 0% 38.80 % 39.60 % 37.20 % 38.90 % 40.50 % 39.20 % 38.80 % 39.60 % 37.20 % 38.90 % 40.50% End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 172 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison 2.2 Number of individua ls trained in safe food preparati on and storage as a result of USDA assistanc e The data represents teachers and headmasters who reported having received training on storage practices 0 0 0 0 0 0 94 0 66 0 28 0 94 0 66 0 28 0 2.2 Number of individua ls who demonstr ate the use of new safe food preparati on and storage practices as a result of USDA assistanc e This data was collected through observation in the storage facilities during the primary survey and the numbers of teachers and headmasters who reported to demonstrate the use of new safe food preparation and storage practices and represents the numbers out of the individuals who are trained 35 NA 22 NA 13 NA 94 0 66 0 28 0 94 0 66 0 28 0 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 173 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison 2.2 Percent of storekeep ers who can identify at least three safe storage practices This data was collected through interviews with storekeepers 96.7 0% NA NA NA NA NA 100 % 0 100% 0 100% 0 85% 0 92.0% 0 75.0% 0 2.4 Number of schools using an improved water source Schools that had any of the following sources of water: 26 14 NA NA NA NA 30 20 NA NA NA NA 30 20 NA NA NA NA Tap water, RO plant within the school, Borehole, protected dug well or Protected spring 2.4 Number of schools with improved sanitation facilities Schools that have any of the following sanitation facilities: Flush or pour/flush facilities connected to a piped sewer/septic/pit latrine, flush or pour/flush toilets without a sewer connection, Pit latrine with a slab, Ventilated improved pit 11 17 NA NA NA NA 29 20 NA NA NA NA 29 20 NA NA NA NA End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 174 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison latrines or Composting toilets 2.5 Number of individua ls trained in child health and nutrition as a result of USDA assistanc e134 Teachers and headmasters who received training on health, hygiene and nutrition 0 0 0 0 0 0 25 (19 teac hers and 6 head teac hers) (as per surv ey) 0 17 0 8 0 79 (7 head teacher s and 39 teacher s and 33 parents ) (as per survey) 0 22 (16 teache rs and 4 head teache rs + 2 father s) (as per surve y) 0 57 (23 teacher s and 3 head teacher s + 31 mother s) (as per survey) 0 2.5 Number of individua ls who demonstr ate the use of new child health and nutrition practices as a result of USDA Data for handwashing was collected from parents. The values represent those who are trained by WFP and among them who reported washing hands for 4 or more out of 6 critical times. 114 43 NA NA NA NA 42(2 5 head 0 19 0 23 0 65 (46 head teacher s and teacher s +19 parents 0 20 head teache rs and teache rs + 2 father s 0 26 head teacher s and teacher + 17 mother s 0 134 Individuals consist of headmasters, teachers and parents for this indicator. While none of the individuals have been trained on child health and nutrition practices under this programme yet, the values given in the table represents the number of headmasters and teachers who have received training on health, hygiene, and nutrition from the GoB or any other source. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 175 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison assistanc e135 Adding to this, the number of head teachers and teachers who demonstrated use of child health and nutrition practices. teac hers and teac hers + 17 pare nts) Dietary diversity of pupils was measured through interviewing parents using 24 hours recall method. Mean Dietary Diversity Score is presented here. 4.2 4.2 4.2 4.2 4.3 4.1 4.97 5.49 4.96 5.5 4.99 5.48 4.85 5.04 4.88 5.11 4.82 5 2.5 Number of students receiving dewormi ng medicatio n(s) This data was collected from the headmasters 0 0 0 0 0 0 47,0 00 29,64 4 22,266 14,257 24,73 4 15,38 7 44,491 (WFP monito ring report Oct 19 to March 20) NA NA 135 Child health and nutrition practices were assessed using 3 parameters; Handwashing, Dietary Diversity and Deworming (data for deworming has been reported separately) End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 176 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex End value Value by sex Proj ect Com pari son Male Female Proj ect Comp ariso n Male Female Projec t Comp ariso n Male Female Pro ject Comp ariso n Proj ect Comp ariso n Projec t Compa rison Proje ct Comp ariso n Proje ct Compa rison Projec t Compar ison Foundat ional results Number of Parent￾Teacher Associati ons (PTAs) or similar “school” governan ce structure s supporte d as a result of USDA assistanc e This is based on an interview with headmasters on the existence of SMCs and PTAs in the school 59136 40 N.A. N.A. N.A. N.A. 189 N.A. N.A. N.A. N.A. N.A. 320 (WFP monito ring report Oct 19 to Mar 20) N.A. N.A. N.A. N.A. N.A. 136 All these schools had school governance structures prior to commencement of activities under the current USDA grant End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 177 Annex U: Terms of Reference for End-term Evaluation The Terms of Reference (TOR) presented here are for the Fiscal Year (FY) 2017 – FY2020 McGovern￾Dole International Food for Education and Child Nutrition Grant (McGovern-Dole) funded through the U.S. Department of Agriculture (USDA). This programme aims to reach 47,689 primary school aged children per year with fortified biscuits in two sub-districts of Cox’s Bazar, Bangladesh. The sub￾districts are Ukhiya and Kutubdia. The programme will target 146 government and non-government organization supported schools. The biscuits will be produced by a local biscuit factory using the wheat donated under this agreement. The biscuits will be fortified with vitamins and minerals to reduce micronutrient deficiencies in students. Complimentary activities aimed at improving literacy by increasing classroom instruction, awareness of health and hygiene practices, and community engagement in education complement the provision of biscuits to students. The three-year programme was developed with support from the Ministry of Primary and Mass Education (MoPME), particularly with the soon to be approval of the School Feeding Programme in Poverty Prone Areas (SFPPA). WFP and MoPME have a close working relationship which entails four WFP staff seconded to MoPME to assist with capacity strengthening for managing the implementation of the School Feeding Programme. WFP and MoPME hold formal bi-monthly meetings with the SFPPA project director and four to five key government staff on the programme. The McGovern-Dole project will engage the sub-recipient Room to Read (RtR) to implement targeted education activities specifically designed to achieve McGovern-Dole’s Strategic Objectives (SO). RtR will implement their innovative and nationally aligned early grade literacy activities and provide overall technical assistance to two local NGOs. Muslim Aid (MA) and Young Power in Social Action (YPSA) are two additional sub-recipients who will be implementing literacy, school meals, nutrition and capacity strengthening activities at the local and district level. MA will primarily work in Ukhiya whereas YPSA will work in Kutubdia. Both organizations were selected to work on the McGovern￾Dole programme because they were already partners with WFP on school meals programmes prior to the McGovern-Dole award. The purpose of this TOR is two-fold. Firstly, it provides key information to the evaluation team and helps guide them throughout the evaluation process; and secondly, it provides key information to stakeholders about the proposed evaluations. The evaluation process will be managed by an Evaluation Manager (WFP - EM) appointed by the WFP Bangladesh Office. The EM will be the main focal point for day to day contact with the hired evaluation firm during the evaluation period. The WFP – EM will be supported by the M&E Unit not associated with the implementation of the programme in the WFP Bangladesh country office. A competitive bidding process will result in an outside firm to be contracted to carry out the actual evaluation. Appropriate safeguards to ensure the impartiality and independence of the evaluation are outlined within this TOR. The evaluation process entails three deliverables and will be commissioned by the Bangladesh Country Office. They will be undertaken in a single assignment (contract). The specific deliverables (timeframes mentioned are subject to change) include: End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 178 Evaluation Date Baseline evaluation report January – June 2018 Midterm evaluation report June – December 2019 End-line evaluation report June – December 2020 The first deliverable will provide information about the pre-project situation, establish a baseline and review project targets. The second deliverable will provide an evidence-based assessment of the project outcome, independent evaluation of the operation and associated interventions about halfway through the project so that WFP-Bangladesh can adjust the project design/implementation for the remaining project period. The final deliverable will be a final evaluation of the three-year programme to inform any future project design, outcomes and lessons learned. This FY2017 programme is located in a different district than the previous three-year funding support from USDA. As such, programme design, targeting and interventions are unique to this new cycle of funding. Noting these differences, the evaluation firm may wish to review previous programme evaluations and findings to support the evaluation design. Bangladesh’s School Meals Context and Subject Bangladesh is one of the most densely populated and disaster-prone countries in the world. Its population is estimated at over 160 million and it is classified as a least-developed, low-income, food￾deficit country. It falls in the low human development category, ranking 142 out of 185 countries on the Human Development Index. Despite significant gains in terms of macro-economic growth and human development over the past decade, Bangladesh continues to experience high levels of extreme poverty, and high rates of food insecurity and under-nutrition. Forty-one percent of children under the age of five are stunted, 16% are wasted, and 36% are underweight; levels that are above public emergency thresholds. Bangladesh is also highly vulnerable to natural disasters, such as flooding and cyclones, which exacerbates food insecurity status of millions of people. Bangladesh also faces the human development challenge of illiteracy. The national literacy rate is 50.5% (11-45 years) and among 11-14-year old, 19.5% are non-literate and 10.4% are semi-literate. In recent years, Bangladesh has made significant progress in its efforts to address illiteracy, especially with regard to increasing access to education and gender equity at the primary level, and is on track to reach the net enrolment target of Millennium Development Goal 2, universal primary education, by 2015. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 179 Targeted Beneficiaries and Regions: In line with USDA and GoB priorities, the most vulnerable and food insecure areas of Bangladesh will be targeted by this McGovern-Dole programme. The programme will operate in two sub-districts of Cox’s Bazar, Ukhiya and Kutubdia. Cox’s Bazar has a poverty rate of 32% and an extreme poverty rate of 17% and is highly food insecure with over 34% of the population below the food consumption poverty line. The region is also experiencing a Level 3 emergency with the influx of over 600,000 ethnic Rohingya refugees from Myanmar. The education system in Bangladesh consists of approximately 150,000 institutions, 40 million students, and a million teachers. In addition, there are thousands of non-formal primary education centres and quomi (indigenous) faith-based madrasas not included in education statistics. There are approximately 18 million students and 80,000 institutions in primary education. Participation in education has expanded remarkably over the past two decades. The Government of Bangladesh has implemented several mechanisms for incentivizing school attendance, including a national school feeding programme for 2.5 million children with the support of WFP; the distribution of financial stipends to 13 million poor children and their families (including disabled children) under a cash for education programme; and the provision of free textbooks. As a result, at the primary level, there is over 97% enrolment while gender parity in primary enrolment has also been achieved. The expansion in education (SDG4) has been widely celebrated but the quality of education remains a serious concern as are high drop-out rates, which increasingly affect young girls and boys dropping out of school to pursue economic activities. Significant education The graduation of USDA funded programmes to GoB District Upazila No. schools No. of children Graduation Year/month Kurigram Kurigram Sadar 177 45,846 12-Jun Ulipur 408 69,978 12-Jun Chilmari 146 24,943 14-Dec Rajarhat 36 1,074 14-Dec Nageshwari 315 59,909 14-Dec Bhurungamari 153 34,819 12-Jun Fulbari 199 29,034 14-Dec Rowmari 142 36,175 14-Dec Rajibpur 69 16,494 14-Dec Barisal Bakergonj 348 86964 12-Feb Gaibandha Gobindhagonj 273 70002 14-Dec Saghata 182 45693 14-Dec Gaibandha Sadar 224 51,694 17-Dec Fulchari 116 26,992 17 Dec Sundarganj 226 69,267 17-Dec Total 3,050 668,885 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 180 challenges remain however relating to attendance, completion, and a prevailing low national literacy rate (51.3%). These challenges are amplified in Cox’s Bazar. WFP has shifted from direct delivery of biscuits to supporting the government in managing the national school feeding programme. In 2007, WFP assisted directly two million children. This number reduced to one million in 2014 and was further reduced to 500,000 in 2016 as a result of the government gradually taking over from WFP. The graduation of McGovern-Dole supported schools to the Government of Bangladesh began in 2012. Since this time, 520,931 students in 2,448 schools have been handed over to the Government of Bangladesh. For specific district and school information, please see the chart above. Since the start of the SFPPA, WFP has been providing technical assistance to the Government to manage school feeding activities through an embedded Capacity Support Unit (CSU) in the Department of Primary Education. Programme Interventions The project will use McGovern-Dole commodities and cash funding to contribute directly towards both of the McGovern-Dole programme’s highest-level Strategic Objectives, McGovern-Dole SO1: Improved Literacy of School-Aged Children and McGovern-Dole SO2: Increased Use of Health and Dietary Practices. WFP has developed a comprehensive programme to provide fortified biscuits at the early hour of school and complimentary educational interventions. The following activities will be undertaken and contribute toward the achievement of SO1 and SO2:  Build Capacity: WFP will provide technical assistance to MoPME to finalize the SFPPA and strategy and support scaling up school feeding. WFP will develop an action plan for policy implementation and will conduct an assessment for the implementation strategy.  Distribute Food: WFP in collaboration with the MoPME will provide fortified biscuits to school children in the Cox’s Bazar District of Bangladesh. WFP will arrange for the fortified biscuits to be produced by local biscuit factories, using the wheat donated under this agreement.  Improve Student Enrolment/Attendance: Through sub recipient, Room to Read, WFP will organize awareness campaigns on the importance of education at both the community and central level.  Promote Improved Health: In cooperation with the NGOs Muslim Aid UK and Young Power in Social Action, WFP will improve water systems and latrine facilities in selected participating primary schools and provide the schools with the needed pumps and motors. The NGOs will organize parent gathering sessions at the school level to discuss topics related to health, hygiene and nutrition awareness; support deworming distribution and develop and disseminate information education materials to help visualize good hygiene practices.  Promote Improved Nutrition: WFP will establish school gardens, school garden clubs, and provide training to SMC’s, teachers, and community members on the establishment and maintenance of school gardens.  Support Improved Literacy: Through the sub-recipient Room to Read, instruction materials will be developed and distributed to teachers and school administrators in grades 1 and 2 in all participating schools. School reading corners in all of the project schools will be End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 181 established in grades 1-5. A variety of additional education related activities will also be implemented.  Support Improved Safe Food Prep and Storage: WFP will train biscuit manufacturers in food safety standards for safe preparation of biscuits and appropriate storage practices. MoPME officials will be trained in monitoring the biscuit manufacturers. WFP will organize training on food storage practices for MoPME, school teachers, and administrators at the local level. Project Population Participant Approximate number Number of Upazila/sub-district 2 Number of Schools 146 Student 47,689 Parents 95,378 School Administrators 146 Teacher 730 District & Upazila/ sub￾district level education officials 10 (Cox’s Bazar district & two Upazila education officials) Evaluation Criteria and Questions The evaluations proposed will use the standard evaluation criteria of Relevance, Effectiveness, Efficiency, Sustainability, and Impact as well as Adequacy, Transparency, and Timeliness. Gender Equality and the Empowerment of Women (GEEW) should be mainstreamed throughout. The FY17 mid-term and end-line evaluation will address the proposed key evaluation questions (see Annex III: Evaluation Criteria and Questions FY17 baseline, mid-term and end-line evaluations). These are only the key indicative questions and sub-questions, in order to provide the background to the evaluation team. The evaluation team is therefore required to further elaborate the questions and sub questions under each criterion during the inception phase of each evaluation. Collectively, the questions aim at highlighting the existing circumstances, performance of school meal activities during the project period and key lessons learnt, which could inform future strategic and operational decisions. Evaluability is the extent to which an activity or a programme can be evaluated in a reliable and credible fashion. A preliminary evaluability assessment will be done by the Country Office at the initial stage of the project cycle, which will be deepened by the evaluation team in each inception package relating to deliverables. The evaluation team shall notably critically assess data availability and take evaluability limitations into consideration in its choice of evaluation methods. In doing so, the team will also critically review the evaluability of the gender aspects of the operation, identify related challenges and mitigation measures and determine whether additional indicators are required to include gender empowerment and gender equality dimensions. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 182 The evaluations will take a programme theory approach based on the results framework. It will draw on the existing body of documented data as far as possible and complement and triangulate this with information to be collected in the field. Stakeholders and Users Stakeholders A number of stakeholders both inside and outside of WFP have interests in the results of the evaluation and some of these will be asked to play a role in the evaluation process. Those include WFP Country Office, Regional Bureau, and WFP Washington Office, as well as key headquarters Divisions (Programme Policy Division, the Performance Management, and Monitoring Division and the Office of Evaluation among others). Externally, USDA including the Food Assistance Division in Washington, DC and the regional Agricultural Attaché, and other key project partners, including Government of Bangladesh, specifically the Ministry of Primary and Mass Education (MoPME), Directorate of Primary Education (DPE), Local Government Division (LGD), Department of Public Health and Engineering (DPHE), Ministry of Health and Family Welfare. In addition, NGOs and international organizations Room to Read, Muslim Aid-UK, YPSA, BRAC, GAIN (Global Alliance for Improved Nutrition), UNICEF and FAO also have a stake in the evaluation. An Evaluation Reference Group (ERG) will be formed with representation from WFP CO, RBB, OEV HQ, and externally USDA, MoPME/DPE, Room to Read, Muslim Aid-UK, YPSA and UNICEF. The ERG members will review and comment on the draft evaluation products and act as key informants in order to safeguard against bias and influence. Users The primary users of this evaluation will be:  WFP-Bangladesh and its government partner to adjust joint activities to implement a school feeding programme and to inform future project design and implementation  USDA to inform changes in project strategy, results framework, and critical assumptions  NGOs to inform current activities or future project design  The GoB is expected to take over the management and monitoring of the school feeding programme over time, therefore, information on whether the programme is yielding the desired results is of primary importance  The WFP Regional Bureau to provide strategic guidance, programme support, oversight, and extract lessons for sharing across the region  WFP HQ for wider organizational learning and accountability  Office of Evaluation to feed into evaluation syntheses  Other COs may also benefit from the findings, which can contribute to corporate learning on the implementation of capacity development interventions Documents for review could include previous evaluations of the school feeding programme, noting that such evaluations were in a different location and different combination of interventions. These documents may include the McGovern-Dole FY14 baseline study and mid-term evaluation of the School Meals Programme, the Country Programme 2013-2017 mid-term evaluation, and the recently End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 183 commissioned outcome survey for the USDA funded McGovern-Dole programme in Gaibandha. Additional documents may include monitoring data currently available with the CO and partners. These will be made available separately to the Evaluation Team. End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 184 End-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 185 Annex V: Terms of Reference for Evaluation Reference Group Context: The McGovern-Dole school feeding programme in Cox’s Bazar began in January 2018 and will continue through September 2020. The main objectives of the programme are improved literacy and nutrition outcomes for children in Government public schools and madrasahs (NGO schools are not part of the literacy component of the programme). More broadly, the programme seeks to understand the impact of school feeding, literacy interventions, and health and nutrition messaging on students’ experiences in school. The purpose of the evaluation is to understand the extent to which the current objectives are being met, to refine our programming for the remaining funding cycle, and to hear the perspectives of school teachers, children, parents and other relevant community members (school management committee). Purpose: - Evaluation Committee (EC): The purpose of this committee is to ensure a credible, transparent, impartial and quality evaluation in accordance with WFP Evaluation Policy 2016-2021. It will achieve this by supporting the evaluation manager in making decisions, reviewing draft deliverables (TOR, inception report and evaluation report) and submitting them for approval by the CD who will be the chair of the committee. - Evaluation Reference Group (ERG): it is a group of key internal and external evaluation stakeholders who review and comments on the draft Terms of Reference (TOR), Inception and Evaluation reports. The ERG members act as advisors during the evaluation process, and they are not taking key decisions about the evaluation. The composition of the EC and ERG: Group members EC ERG Officer in charge DCD (Chair of the EC) Piet Vochten Evaluation Manager (EC Secretariat) Antonio Battista Head of Programme Rezaul Karim Programme officer MGD Katelyn Gless Regional Evaluation Officer (REO) Yumiko Kanemitsu Country Office M&E Officer Ezaz Nabi School feeding Policy Officer Farzana Akter School feeding CS Officer Abdullah Patwary McGovern Dole Advisor Niamh O’Grady Regional School Feeding Officer Nadya Frank Representative from Directorate of Primary Education (DPE) - SFPPPA Mr Ruhul Amin Khan Representative from local District (District Primary Education Officer) Md. Shafiul Alam Representative from RtR Zakir Hossain Tasks of EC and ERG (relevant for MTE, to be updated for Endline): Evaluation phases Tasks of the Evaluation Committee Tasks of the Evaluation Reference Group Preparation Phase  Select and establish ERG membership.  Reviews the revised draft ToR prepared by the EM on the basis of: o The outsourced Quality Support service feedback, ERG comments, the EM responses documented in the comments matrix  Approves the final TOR.  Approves the final evaluation team and budget.  Review and comment on the draft.  Ensure that the ToR will lead to a credible. and useful evaluation and provide additional information to inform the finalization of the TOR.  Where appropriate, provide input on the evaluation questions.  Identify source documents useful to the evaluation team.  Attend ERG meeting/conference call etc. Inception Phase  Briefs the evaluation team about the evaluation.  Informs evaluation design.  Supports identifying field visit sites based on selection criteria, defined by the evaluation team in the IR though the EC should not influence actual selection.  Reviews the revised draft IR based on: o The outsourced Quality Support service and EM feedback, ERG comments, the Evaluation team responses in the comments matrix  Approves the final IR.  Meet with evaluation team to discuss how the evaluation team can design a realistic/practical, relevant and useful evaluation.  Identify and facilitate dialogues with key stakeholders for interviews, as required.  Identify and access documents and data  Help identify appropriate field sites according to selection criteria set up by the evaluation team in the inception report.  Review and comment on the draft Inception Report. Data Collection Phase  Act as key informants: responds to interview questions.  Facilitates access to sources of contextual information and data, and to stakeholders.  Attends the end of field work debriefing(s) meeting.  Supports the team in clarifying emerging issues and identifying how to fill any data gaps.  Act as a key informant: respond to interview questions.  Provide information sources and facilitate access to data.  Attend the evaluation team’s end of field work debriefing. Data Analysis and Reporting Phase  Review the revised draft ER based on: o The outsourced Quality Support service and EM feedback, ERG comments, the Evaluation team responses in the comments matrix  Approves the final ER.  Review and comment on the draft evaluation, focusing on accuracy, quality and comprehensiveness of findings, and of links to conclusions and recommendations. The latter should be relevant, targeted, realistic and actionable.  The ERG, being advisory, it must respect the decision of the independent evaluators about whether feedback is incorporated, as long as the process is transparent, including rationale for not incorporating feedback. Evaluation phases Tasks of the Evaluation Committee Tasks of the Evaluation Reference Group Disseminatio n and Follow-up Phase - Leads the preparation to the management response to the evaluation. - Decides whether management agrees, partially agrees or does not agree with the recommendations. - Clears the management response. - Disseminates the Management Response to key stakeholders. - Disseminate final report internally and externally, as relevant. - Share findings within units, organizations, networks and at events. - Provide input to management response and its implementation (as appropriate). Procedures of engagement:  The CD will appoint members of the EC - The EM will notify the members of the date, time, location and agenda of meetings at least one week before the meeting and share any background materials for preparation.  EC meetings will be held face-to face (or Skype is face-to-face not possible).  The Chair of the Evaluation Committee will appoint the members of the ERG.  The EM will notify the ERG members the time, location and agenda of calls and share any relevant background materials.  ERG meetings will be via electronic conference call/Skype – seldom they will be face to face when deemed appropriate.  The ERG will meet at least once at every evaluation phase.  ERG members, representing their organizations, will be interviewed by the evaluation team as part of inception and data collection.  ERG members will provide feedback electronically to the EM on the draft ToR, Inception Report and Evaluation Report. Comments will be recorded in a comments matrix to help ensure a transparent and credible process. Annex W: Bibliography 1. Ahmed, A. U. (2004). Impact of feeding children in school: Evidence from Bangladesh. Washington, DC: International Food Policy Research Institute. 2. Ahmed, Akhter. (2017). Strategic Review of Food Security and Nutrition in Bangladesh. 10.13140/RG.2.2.24673.71529. 3. Ahsan, K. Z., Streatfield, P. K., Ijdi, R. E., Escudero, G. M., Khan, A. W., & Reza, M. M. (2015). Fifteen years of sector-wide approach (SWAp) in Bangladesh health sector: an assessment of progress. Health policy and planning, 31(5), 612-623. 4. Alston, M., & Akhter, B. (2016). Gender and food security in Bangladesh: the impact of climate change. Gender, Place & Culture, 23(10), 1450-1464. 5. Bangladesh Education Statistics 2018; BANBEIS 6. FAO, WFP and IFAD. 2012. The State of Food Insecurity in the World 2012. Economic growth is necessary but not sufficient to accelerate reduction of hunger and malnutrition. Rome, FAO. 7. Hasib, Md & Hassan, Md & Hasan, Mehedi & Khan, md. shafiqul islam. (2020). Effect of nutritional status on Rohingya under-five children in Bangladesh. International Journal of Public Health Science (IJPHS). 09. 101-106. 10.11591/ijphs.v9i4.20546. 8. Health SDG Profile: Bangladesh (2019) 9. Helen Keller International and James P. Grant School of Public Health). (2015). State of Food Security and Nutrition in Bangladesh: 2014. BRAC University: Dhaka 10. Howlader, et al. (2012). Investing in Nutrition Now: A Smart Start for Our Children, Our Future. Estimates of Benefits and Costs of a Comprehensive Program for Nutrition in Bangladesh, 2011– 2021. PROFILES and Nutrition Costing Technical Report. Washington, DC: Food and Nutrition Technical Assistance III Project (FANTA), FHI 360 11. Human Development Report 2016-Human Development for Everyone: Briefing note for countries on the 2016 Human Development Report 12. icddr,b, UNICEF, Bangladesh, Global Alliance for Improved Nutrition (GAIN), Institute of Public Health and Nutrition (IPHN) (2011-12). National Micronutrients Survey. Dhaka, Bangladesh 13. International Labour Organization, ILOSTAT database (2017) 14. Islam, M. S., & Hasan, M. N. (2017). Bangladesh. In The World Guide to Sustainable Enterprise (pp. 36-41). Routledge. 15. Ministry of Health and Family Welfare, GOB, 2017. Second National Plan of Action for Nutrition (NPAN-2) 16. Ministry of Health and Family Welfare, Government of Bangladesh, 2017. Second National Plan of Action for Nutrition (2016-2025) 17. Nutrition country profile accessed from www.globalnutritionreport.org Source: UNICEF/WHO/World Bank Group: Joint child malnutrition estimates. (2018) 18. Rahman, M. M. (2017). Biannually school-based deworming by Mebendazole 500mg has reduced the worm load drastically in Bangladesh. EC Bacteriology and Virology Research, 2, 113-114. 19. Terms of Reference (TOR) of Fiscal Year (FY) 2017 – FY2020 McGovern-Dole International Food for Education and Child Nutrition Grant (McGovern-Dole) 20. The ‘Global Water Supply and Sanitation Assessment’ by World Health Organization (WHO), United Nations Children Fund (UNICEF), Water Supply and Sanitation Collaborative Council (WSSCC) 21. The Global Gender Gap Report benchmarks countries on their progress towards gender parity across four thematic dimensions: Economic Participation and Opportunity, Educational Attainment, Health and Survival, and Political Empowerment. 22. UN Renewed Efforts against Child Hunger (REACH), Undernutrition in Bangladesh A Common Narrative. 2014 23. UNEG Indicators on Gender Equality and Human Rights in UNEG Quality Checklist for Evaluation 24. United Nations Development Programme, Human development indices and indicators: 2018 Statistical update. 25. United Nations Development Programme. (2020). Human development indices and indicators: 2019 Statistical update. 26. World Health Organization. (2017). New perspectives on global health spending for universal health coverage (No. WHO/HIS/HGF/HF Working Paper/17.10). World Health Organization. 27. Retrieved from: a. https://www.globalhungerindex.org/bangladesh.html b. http://www.fao.org/3/CA3252EN/ca3252en.pdf c. https://washwatch.org/en/countries/bangladesh/summary/statistics/ d. https://reliefweb.int/sites/reliefweb.int/files/resources/operation_presence_map_wash_ sector_august_2018_ta_0.pdf e. http://www.dghs.gov.bd/images/docs/Publicaations/HealthBulletin2017Final13_01_201 8.pdf f. http://app.dghs.gov.bd/localhealthBulletin2016/publish/publish.php?org=10000924&ye ar=2016&lvl=1 g. https://www.wateraid.org/bd/media/wateraid-launches-the-wash4urbanpoor-project￾aiming-to-serve-450000-urban-poor-with-water h. https://info.undp.org/docs/pdc/Documents/BGD/UNGA_Booklet_2017.pdf i. http://documents.worldbank.org/curated/en/857071529206219039/pdf/BANGLADESH -QLEAP-PAD-05252018.pdf j. https://knoema.com/atlas/Bangladesh/topics/Health/Nutrition/Prevalence-of￾undernourishment k. https://data.worldbank.org/indicator/SH.XPD.CHEX.GD.ZS?locations=8S l. https://data.worldbank.org/indicator/SH.XPD.CHEX.GD.ZS?end=2017&name_desc=false& start=2000&view=chart m. http://hdr.undp.org/en/indicators/57506 n. http://app.dghs.gov.bd/localhealthBulletin2016/publish/publish.php?org=10000924&ye ar=2016&lvl=1 o. https://www.ceicdata.com/en/bangladesh/education-statistics/bd-government￾expenditure-on-education-total--of-government-expenditure p. http://www.xinhuanet.com/english/2019-02/27/c_137855105.htm q. https://thefinancialexpress.com.bd/views/improving-quality-of-education-main￾challenge-1580309749 r. https://www.humanitarianresponse.info/en/operations/bangladesh/document/j-msna￾refugee-and-host-communities-preliminary-findings-october-2020 s. https://data.unicef.org/topic/water-and-sanitation/wash-in-schools/ t. https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/do cuments/files/cxb_wash_sector_operation_presence_map_january_2019_0.pdf u. https://data2.unhcr.org/en/documents/download/72273 v. https://www.worldbank.org/en/news/feature/2019/04/29/social-safety-nets-in￾bangladesh-help-reduce-poverty-and-improve-human-capital w. https://www.thedailystar.net/country/cabinet-okays-national-school-meal-policy-2019- 1.5-crore-children-1787272 x. https://www.fas.usda.gov/programs/food-progress y. https://www.feedthefuture.gov/country/bangladesh/ z. https://www.usaid.gov/bangladesh/agriculture-and-food-security aa. https://www.foreignassistance.gov/explore/country/Bangladesh bb. https://www.worldbank.org/en/news/press-release/2020/09/25/world-bank-helps￾bangladesh-ensure-safe-water-and-sanitation-in-rural-areas cc. https://www.lightcastlebd.com/insights/2020/12/how-dependent-is-bangladesh-on￾development aid#:~:text=In%20the%20beginning%20of%20FY,from%20the%20Covid%2D19%20fall out Annex X: List of Acronyms ASPR Annual Sector Performance Report (prepared by Directorate of Primary Education) AUEOs Assistant Upazila Education Officers BANBEIS Bangladesh Bureau of Educational Information and Statistics BRAC Bangladesh Rural Advancement Committee COVID-19 Coronavirus Disease CPD Continuing Professional Development CWW Children Without Worms DAE Directorate of Agriculture Extension DEQAS Decentralized Evaluation Quality Assurance System DGHS Directorate General of Health Services DHS Demography and Health Survey DID Difference-in-Difference DMA Data Management Aid DPE Directorate of Primary Education DPEd Diploma in Primary Education DPEOs District Primary Education Officers DPHE Department of Public Health and Engineering EGRA Early Grade Reading Assessment EiE Education in Emergency EQAS Evaluation Quality Assurance System ERG Evaluation Reference Group FAS Foreign Agricultural Service FAO Food and Agriculture Organisation FGD Focus Group Discussion FY Five Year FYP Five Year Plan GAIN Global Alliance for Improved Nutrition GDI Gender Development Index GDP Gross Domestic Product GEEW Gender Equality and Empowerment of Women GHI Global Hunger Index GNI Gross National Index GoB Government of Bangladesh GPS Government Primary School HDI Human Development Index HDR Human Development Report HNP Health, Nutrition and Population IFAD International Fund for Agricultural Development ILO International Labour Organisation IQ Intelligent Quotient IR Inception Report KIIs Key Informant Interviews LGD Local Government Division M&E Monitoring and Evaluation MA Muslim Aid MGD Mc Govern-Dole MIS Management Information System MoA Ministry of Agriculture MoHFW Ministry of Health and Family Welfare MoPME Ministry of Primary and Mass Education MPI Multidimensional Poverty Index NCTB National Curriculum and Textbooks Board NER Net Enrolment Rate NGO Non-Government Organization NPAN2 National Plan of Action for Nutrition - 2 NRMC NR Management Consultants India Pvt Ltd NSFPPA National School Feeding Programme in Poverty Prone Areas NSFPS National School Feeding Policy and Strategy NSSS National Social Security Strategy OEV Office of Evaluation, WFP PEDP Primary Education Development Programme PESP Primary Education Stipend Project PTA Parent Teacher Association PTI Primary Teachers Training Institute QLEAP Quality Learning for All Programme RBB Bangkok Regional Bureau RIC Resource Integration Centre RtR Room to Read SABER Systems Approach for Better Education Results SDG Sustainable Development Goals SFP School Feeding Programme SFPPA School Feeding Programme in Poverty Prone Areas SMC School Management Committee SOs Strategic Objectives STH Soil-transmitted helminth SWAp Sector-wide approach ToR Terms of Reference ToT Training of Trainer TSER Transforming Secondary Education for Results UEOs Upazila Education Officers UN United Nations UNEG United Nations Evaluation Group UNICEF United Nations International Children's Emergency Fund URC Upazila Resource Center USAID United States Agency for International Development USDA US Department of Agriculture VAM Vulnerability Assessment and Mapping WASH Water, Sanitation and Hygiene WFO HQ WFP Head Quarter WFP World Food Programme WFP-CO WFP Country Office WHO World Health Organization YPSA Young Power in Social Action [Dhaka, March 2021, Final Report, End-Term Evaluation Report]