Foreign Agricultural Service, United States Department of Agriculture Bangladesh McGovern-Dole Project MIDTERM EVALUATION February 2020 DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared independently NRMC. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 in Bangladesh Agreement Number: FFE-388-2017/019-00 Funding Year: Fiscal Year 2017 Project Duration: 2017-2020 Implemented by: World Food Programme Evaluation Authored by: NRMC Rahul Agrawal Ankita Singh Mrinalini Mazumdar DECENTRALIZED EVALUATION FOR EVIDENCE-BASED DECISION MAKING Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 in Bangladesh Final Report February 2020 World Food Programme Evaluation Manager: Antonio Battista Prepared by Team Leader: Rahul Agrawal Gender Specialist: Ankita Singh Research Manager: Mrinalini Mazumdar Decentralized Evaluation Acknowledgements The NRMC evaluation team wishes to acknowledge the guidance, support, and cooperation received from all the participants in the evaluation. NRMC takes this opportunity to extend sincere thanks to the distinguished Government officials from Ministry of Primary and Mass Education, Directorate of Primary Education at National and District and Upazila level, Bangladesh National Nutrition Council (BNNC), Ministry of Health and Family Welfare and Department of Agricultural Extension, Ministry of Agriculture for their time and precious inputs. The NRMC Evaluation Team expresses its gratitude to Mr. Rezaul Karim, Mr. Ezaz Nabi, Mr. Antonio Battista, Ms. Katelyn Runyan-Gless, Ms. Farzana Akter, Mr. Md. Abdullah-Al-Mamun Patwary, Ms. Sneha Lata and Mr. George Suman Karmaker for their valuable suggestions and guidance for the evaluation. We would also like to thank the staff of Room to Read, Muslim Aid, and Young Power in Social Action who took time out to meet us and provide their views on the school feeding programme and facilitate the logistics during the survey. We are thankful to the team from Data Management Aid for their partnership with the NRMC team during the evaluation, particularly on data collection. Last but not the least, the evaluation team wishes to acknowledge the cooperation received from all informants, including school children, school heads, teachers, parents, cooks, storekeepers, and SMC members, during the primary survey. Disclaimer The opinions expressed in this report are those of the Evaluation Team, and do not necessarily reflect those of the World Food Programme. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. Table of Contents LIST OF TABLES................................................................................................................................... II LIST OF FIGURES ................................................................................................................................. II EXECUTIVE SUMMARY......................................................................................................................... I 1. INTRODUCTION........................................................................................................................... 1 1.1. EVALUATION OVERVIEW ............................................................................................................................................. 1 1.2. OVERVIEW OF THE EVALUATION SUBJECT ............................................................................................................... 1 1.3. CONTEXT ........................................................................................................................................................................ 3 1.3.1 Food Security and Nutrition ................................................................................................................................... 3 1.3.2 Health................................................................................................................................................................................ 4 1.3.3 Education......................................................................................................................................................................... 4 1.3.4 Water, Sanitation and Hygiene ............................................................................................................................. 5 1.3.5 School Feeding Needs................................................................................................................................................. 5 1.3.6 Gender Analysis............................................................................................................................................................. 6 1.3.7 Government Policies and Programmes.............................................................................................................. 6 1.3.8 Development Assistance in Bangladesh ............................................................................................................ 7 1.4. EVALUATION METHODOLOGY AND LIMITATIONS ................................................................................................... 9 1.4.2 Gender dimensions of the mid-term evaluation.......................................................................................... 12 1.4.3 Data Analysis.............................................................................................................................................................. 12 1.4.4 Limitations and Risks.............................................................................................................................................. 13 1.4.5 Ensuring Quality and Adherence to Ethical Standards............................................................................... 14 2. EVALUATION FINDINGS .............................................................................................................15 2.1. RELEVANCE................................................................................................................................................................. 15 2.2. EFFECTIVENESS..........................................................................................................................................................17 2.3. EFFICIENCY ................................................................................................................................................................. 31 2.4. IMPACT.........................................................................................................................................................................33 2.5. SUSTAINABILITY.........................................................................................................................................................35 2.6. OBSERVATIONS OF THE PARTICIPANTS IN COX’S BAZAR AND DHAKA ON THE FINDINGS OF MTE DURING THE VALIDATION WORKSHOP .................................................................................................................................................36 3. CONCLUSIONS .............................................................................................................................37 4. RECOMMENDATIONS .................................................................................................................39 ANNEX A: MAP OF THE INTERVENTION AREA................................................................................44 ANNEX B: PROJECT-LEVEL RESULTS ................................................................................................46 ANNEX C: PLANNED OUTCOMES OF WFP BANGLADESH MCGOVERN DOLE -FY17 AWARD ........49 ANNEX D: ACTIVITY WISE GRADUATION TIMELINE.......................................................................54 ANNEX E: MCGOVERN DOLE TARGET BENEFICIARIES AND FUNDING FOR WFP SCHOOL FEEDING PROGRAMME......................................................................................................................56 ANNEX F: ROLE OF PARTNERS ..........................................................................................................59 ANNEX G: STAKEHOLDER ANALYSIS ................................................................................................62 ANNEX H: SUMMARY OF THE METHODOLOGY PRESENTING THE METHOD OF DATA COLLECTION, THE TARGET GROUPS/SOURCES OF INFORMATION, THE NATURE OF THE INTERVIEWS, AND THE INDICATIVE OUTPUTS...............................................................................69 ANNEX I: DATA ANALYSIS METHODS ...............................................................................................70 ANNEX J: SAMPLE SIZE COVERED BY RESPONDENT CATEGORY....................................................72 ANNEX K: LIST OF SAMPLE SCHOOLS...............................................................................................73 ANNEX L: EVALUATION MATRIX.......................................................................................................75 ANNEX M: DATA COLLECTION TOOLS ..............................................................................................84 ANNEX N: EVALUATION MISSION SCHEDULE ................................................................................154 ANNEX O: DOCUMENTS GATHERED ...............................................................................................156 ANNEX P: NRMC’S INTERNAL PROTOCOLS ....................................................................................160 ANNEX Q: TEAM COMPOSITION AND SPECIFIC TASKS ................................................................. 161 ANNEX R: THE TRAINING SCHEDULE OF THE DATA COLLECTION TEAM ...................................162 ANNEX S: MIDLINE VALUES OF KEY INDICATORS .........................................................................164 ANNEX T: TERMS OF REFERENCE FOR MID-TERM EVALUATION ................................................185 ANNEX U: BIBLIOGRAPHY ...............................................................................................................190 ANNEX V: LIST OF ACRONYMS ........................................................................................................192 List of Tables Table 1: Commodity Direct Distribution: Bangladesh FY 2017 Award ............................................................... 2 Table 2: Enrolment in sample intervention and comparison schools................................................................23 Table 3: Students' attendance in sample intervention and comparison schools...........................................23 Table 4: Parents trained by WFP who are aware and demonstrate handwashing practices at critical times ..............................................................................................................................................................................................25 Table 5: Planned outcomes and annual targets...........................................................................................................49 Table 6: Activity wise Graduation Timelines ................................................................................................................54 Table 7: Role of partners .......................................................................................................................................................59 Table 8: Stakeholder analysis..............................................................................................................................................62 Table 9: Summary of the Evaluation Methodology ....................................................................................................69 Table 10: Sample Size .............................................................................................................................................................72 Table 11: Evaluation matrix.................................................................................................................................................75 Table 12: Mapping of the indicators with the type of tools and the questions...............................................84 Table 13: Evaluation mission schedule........................................................................................................................154 Table 14: Documents Gathered and Reviewed .........................................................................................................156 Table 15: Team Composition and Responsibilities ................................................................................................. 161 Table 16: Training Schedule .............................................................................................................................................162 List of Figures Figure 1 Percent of students who can both read more than 45 words per minute and comprehend more than seven questions ..................................................................................................................................................21 Figure 2 Number of students who gave more than 7 correct answers (comprehension) and number of students who read more than 45 correct words in a minute.................................................................................21 Figure 3: Perception of teachers (of intervention schools) on impact on students’ attendance if provision of biscuits stopped ..............................................................................................................................................24 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017- 2020 i Executive Summary 1 Commissioned by the WFP Country Office Bangladesh (WFP-CO), the mid-term evaluation (MTE) of the USDA McGovern-Dole (FY17) funded school feeding programme considers the evaluation questions specifically for the mid-term of the FY17 award. 2 The MTE serves several critical purposes intended for both accountability and learning. The objective of the evaluation is to highlight the existing circumstances and performance of school feeding activities during the project period, compare the mid-term values with the baseline values for all performance indicators as outlined in the Terms of Reference (ToR), assess the progress of implementation of programme activities post the baseline study and underline the key lessons learnt that can inform future strategic and operational decisions. 3 While the prevalence of undernourishment has declined in Bangladesh, there still are 26 million food-insecure people in the country. Cox’s Bazar alone has 695,000 people that are severely food-insecure with over 34 percent of the population living under the food consumption poverty line. It has also been performing poorly on indicators such as Net Enrolment Rate (NER) and dropout rates 4 The USDA McGovern-Dole funded school feeding programme (SFP) is designed to provide school feeding assistance in the form of micronutrient-fortified biscuits to pre-primary and primary school children in Ukhiya and Kutubdia upazilas (sub-districts) in Cox’s Bazar district of Bangladesh. The program aims to reach 47,689 school children across 146 government primary schools and Ebtadayee Madrasas in three years. 5 Over three years, WFP with on-field assistance from implementing partners like Room to Read (RtR), Muslim Aid (MA) and Young Power in Social Action (YPSA) is carrying out project activities aiming to (i) increase student enrolment and student-teacher attendance through the provision of school meals and recognition of high-performing teachers, (ii) improve literacy of school-aged children and quality of education through early grade reading and other learning interventions, (iii) improve health and dietary practices of students by improving water systems and latrine facilities and through water, sanitation and hygiene education initiatives, (iv) increase government ownership through capacity building activities, and (v) increase awareness of the importance of education among parents and community members through ongoing advocacy campaigns. 6 The findings of the MTE of the SFP will primarily be used by (i) WFP-Bangladesh and its implementing partners to adjust joint activities to implement the SFP and to inform future project design and implementation, (ii) USDA to inform changes in project strategy, results framework, and critical assumptions, (iii) Government of Bangladesh (GoB) (including local government officials like District Education Officer (DEO), Upazila Education Officer (UEO), head teachers and teachers as they are expected to eventually take over the management and monitoring of the SFP and (iv) other WFP offices like the Regional Bureau for wider organizational learning and accountability. 7 A validation workshop was conducted in Cox’s Bazar and Dhaka to share and validate the evaluation findings with all stakeholders including beneficiaries and GoB officials. The aim of the workshop was not only to validate the evaluation findings but also to explore potential use of the findings and prioritize actions based on feasibility and priorities of the GoB. Methodology 8 The MTE adopted a mixed method approach to assess and report on the performance and results of the FY17 project (in comparison with the baseline), draw lessons and identify corrective actions through the lens of relevance, effectiveness, efficiency, impact and sustainability. The performance of intervention schools was examined vis-à-vis comparison schools in Ramu upazila in Cox’s Bazar district. The rationale for choosing this upazila was that it has never received any form of school feeding support, contrary to other upazilas. 9 Though the SFP does not have a specific focus on gender related issues an attempt has been made to integrate Gender Equality and Empowerment of Women (GEEW) throughout the evaluation. The sampling of students ensured equal representation of boys and girls, and the school-level data was disaggregated by sex. 10 Following the simple random sampling approach adopted during baseline, 50 Government Public Schools (GPS) were identified as sample (30 intervention and 20 comparison). The schools were selected randomly in the same proportion as the number of programme schools in Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017- 2020 ii the districts, thus ensuring the statistical representation of the sample. The sample size was calculated at the programme level using the ‘differences method’ formula with a finite population1. 390 students and 260 students from grade I-V were sampled from intervention and comparison schools respectively. One parent from each grade of the selected students was covered. From each school, one head teacher, one teacher, and one storekeeper were interviewed. The sample size to administer Early Grade Reading Assessment (EGRA) tool was 14 students from Grade III per school. Overall, 30 focus group discussions were held with students, School Management Committee (SMC) members and parents (10 for each category). 20 percent of the schools from the sample were randomly identified for these Focus Group Discussions (FGDs). 11 The implementation of the programme activities commenced from October 2018. Due to the limited duration of implementation prior to commencement of data collection (July 2019) for the MTE, the evaluation may not highlight/report significant change/impact of the programme’s activities. Key Findings Relevance 12 The SFP has been found to be well aligned with the national government’s priorities, policies and strategies. This includes the National Education Policy (2010), the National Nutrition Policy 2015, the National Social Security Strategy (2015) as well as the Second National Plan of Action for Nutrition. 13 In the context of need for quality education in Cox’s Bazar, SFP’s implementation strategies such as creating awareness on the importance of education (through events such as the Read Play festival), providing training to teachers on new techniques and establishment of class room libraries targets the right people and was therefore found to be relevant. Provisioning of fortified biscuits to students and implementing complimentary education interventions as part of SFP, extends the development partners’ (such as UNICEF) support to GoB from early child development centres to primary schooling and complements other donor-funded initiatives. Effectiveness 14 WFP’s efforts to sensitize representatives of different ministries of GoB on the relevance of SFP and its activities, and building consensus at the GoB level on minimum nutritional requirements for school meals contributed to the formulation and approval of the National School Meal Policy 2019 and GoB’s commitment to initiate school meal programme in 16 Upazilas, reaching 410,238 children in 2,166 primary schools. The assistance provided by WFP in terms of building capacities at the Ministry of Primary Mass Education (MoPME), Directorate of Primary Education (DPE) level and strengthening inter-ministerial coordination is well recognised by the MoPME, Ministry of Agriculture (MoA) and the Bangladesh National Nutrition Council (BNNC). 15 At the programme level, an increase in the midline values as compared to baseline values (for 30 out of 35 main and custom indicators) along with the qualitative findings indicates that the programme is on track to carry out all planned activities and achieve the intended results. Efficiency 16 All intervention schools reported regular receipt of stock and distribution of biscuits. The Management Information System and processes, managed by the MoPME and a Project Management Unit (PMU), wherein WFP provided technical support to DPE for real-time monitoring of the National SFP, were observed to be efficient. The timely response to programme needs, measures taken to ensure smooth implementation and proactive engagement with GoB as well as the community reflects efficiency of implementing partners. The intensity of monitoring, frequency of engagement and feedback by the implementing partners was observed to be efficient for classroom libraries. However, this monitoring mechanism was found to involve too many steps which often resulted in a lack of coordination between RtR and the implementing partners. Additionally, there is irregularity in conducting district joint monitoring visits owing to the varied commitments of the district level officials involved. Therefore, while the processes of the programme are being implemented 1 Confidence interval of 1.96 and estimated difference set at 5%, as per Cochran (1977), Cochran, W. G. (1977) Sampling Techniques. 3rd Edition, John Wiley & Sons, New York Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017- 2020 iii efficiently, there is a need to further improve coordination among stakeholders and improve the monitoring system to enhance the programme’s efficiency. Impact 17 Despite the limited period of implementation (9 months) prior to the commencement of data collection (July 2019) for the MTE, the evaluation findings highlight an increase in understanding among the community on the benefits of education, increase in the skills and knowledge of the teachers and administrators and improvement in fluency and comprehension among students. The findings also suggest that there has been an increase in the overall knowledge about good nutrition, health, hygiene and sanitation practices as well as safe food preparation and storage practices. These positive changes together constitute the intermediate impact of the program. 18 MTE findings suggest that the programme is also on track to achieve its strategic outcomes on improved literacy of school age children and increased use of health and dietary practices. The percentage of students who have completed Grade 2 and can read and understand the meaning of the grade-level text has increased from 6.9 percent at baseline to 28 percent at MTE. Discussions with parents and students indicated that initiatives such as Book Captains and Little Doctors have also led to a recognition of students’ capabilities by their peer groups and has created aspirations among them to achieve similar recognition by demonstrating reading skills and healthy practices. There is also an increase in (a) demonstration of new teaching techniques and tools by the teachers (b) attentiveness and attendance of the students (c) discourse on improving quality of education. With regard to health and dietary practices, there is an increase in the dietary diversity score in intervention schools from baseline (4.15) to midterm (5.49) due to an increased demand from students for nutritious food as a result of awareness generation initiatives of the SFP. 19 However, an unintended negative impact was found as a result of the programme wherein, the absence of a proper waste disposal mechanism for the plastic wrappers of biscuits, resulted in schools burning the wrappers in open instead of disposing them responsibly. This poses a major environmental concern and requires immediate correction. 20 Considering that there is a growing realization on the importance of education, increasing demand for quality of education and adoption of health and hygiene practices, and project indicators are improving, the project is on track to achieve its strategic outcome on improving literacy of school age children. Sustainability 21 The approval of the National School Meal Policy by the GoB and allocation of USD $19.5 million for the implementation of the programme in 16 upazilas (stage I) provides evidence on the government’s commitment to take ownership of the programme. The government has expressed its willingness to independently undertake procurement by 2021. This will act as a major enabler for the sustainability of the SFP. Mainstreaming the new teaching methods and tools would further ensure sustainability of the intervention. 22 Mothers narrating instances of participation in school activities such as healthy meal preparation day and existence of parent teachers’ association provides evidence of increasing community engagement in supporting school in implementation of SFP activities. This has the potential to act as an enabler for sustainability. 23 Realising the importance of SFP, there is intent from the GoB officials at national and district level, teachers and the community, to independently take forward interventions. The actual implementation however, is dependent on continued financial assistance from the GoB or WFP. The approval of NSMP and allocation of budget by GoB demonstrates its commitment and willingness to take the ownership of the programme and can be considered as an achievement of important milestone towards ensuring sustainability of SFP. 24 During the discussions at the validation workshop in Cox’s Bazar, the representatives from the community clearly articulated their interest in contributing towards school feeding and education activities. Towards this, an effective community engagement strategy and capacitated SMC can act as drivers to ensure sustainability of SFP. The DEO and UEO also expressed their interest in independently implementing the interventions undertaken through SFP, if provided with financial assistance. Despite the expressed intent of the community and the other local stakeholders in Cox’s Bazar, the actual implementation of the interventions is still dependent on Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017- 2020 iv continued financial assistance from the GoB or WFP, posing a hindrance to the programme’s sustainability. Recommendations Policy Advocacy (National level) 25 Since the School Meal Policy has been approved, WFP should now provide technical assistance to MoPME in developing guidelines and a plan for its implementation, based on learnings from the SFP. WFP can also provide capacity building support on following: (a) Improved documentation of process and learning and (b) Design and implementation of a monitoring and evaluation (M&E) framework for National School Meal Policy (NSMP). 26 A positive change has been observed since the baseline with an increase in the number of mothers who have reported their participation in the programme activities. This offers a chance to further strengthen community engagement. WFP should leverage this opportunity to engage in advocacy by articulating the need and benefits of community participation in SFP and support MoPME in defining contours for community engagement. 27 The shortage in number of teachers and the absence of mainstreaming of new and innovative teaching techniques and methods in the existing GoB curriculum acts as an inertia for the teachers to fully adopt such teaching methods. WFP should consider advocacy through dissemination of evidence on benefits of adoption of new teaching techniques generated through this programme at the national level. They should also ensure mainstreaming of these methods within the existing government prescribed curriculum. 28 The practice of burning the plastic biscuit wrappers, in the absence of a proper waste disposal mechanism needs urgent correction. WFP and GoB should explore alternate packaging of biscuits and minimize the generation of plastic waste. Waste to Art events/competitions may also be organized in schools to serve the dual purpose of raising awareness about fortified biscuits as well as ensuring waste recycling. Actionable (Specific to SFP implementation in Cox’s Bazar) 29 There is a need for WFP to revitalize the joint monitoring visit mechanism at the district level as envisaged in the programme. 30 Given the lack of availability of spaces in schools for establishing vegetable gardens, WFP should liaise with the Department of Agriculture to create awareness and build capacities of the school administration to explore and ensure utilization of alternate spaces for vegetable gardens such as the terrace. 31 To address Kutubdia’s challenge of access to safe drinking water (caused due to the ingress of sea water contaminating the aquifers) WFP needs to engage in advocacy actions with different stakeholders to improve access to safe water in these targeted schools. 32 While there is an increase in the female to male ratio in the composition of SMCs, participation of women in SMC activities has been found to be very limited. It is suggested that WFP undertake gender sensitization workshops with SMCs to ensure equal representation and active participation of women. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 1 1. Introduction 1.1. Evaluation Overview 1. This document is the mid-term evaluation (MTE) report of the School Feeding Programme (SFP), being implemented during the period January 2018-December 2020 in Cox’s Bazar district, supported by McGovern-Dole International Food for Education and Child Nutrition Grant through the US Department of Agriculture (USDA). The MTE is commissioned by WFP Country Office, Bangladesh (WFP-CO). The MTE considers the evaluation questions specifically for the mid-term of FY17 award. This evaluation was conducted during the period April 2019 to November 2019, wherein the data collection was done during the period, 15th July 2019 to 1st August 2019. 2. The MTE serves several critical purposes intended for accountability. The aim of the evaluation is to highlight the existing circumstances and performance of school feeding activities during the project period, compare the mid-term values with the baseline values for all performance indicators as outlined in the Terms of Reference (ToR), assess the progress of implementation of programme activities post the baseline study and underline the key lessons learnt that can inform future strategic and operational decisions. The evaluation will be used to track the progress of the programme in achievement of indicators for McGovern Dole’s two strategic objectives as well as the highest-level results that feed into the strategic objectives. Furthermore, the MTE provides key insights regarding the progress of the project, draws lessons from the interventions and helps understand the factors affecting the performance of the project. The evaluation on the basis of relevance, effectiveness, efficiency, sustainability, and impact aims to provide corrective measures and adjustments required for the remaining period of the programme, to achieve the intended results. 1.2. Overview of the Evaluation Subject 3. The subject of the evaluation is the WFP implementing USDA-funded SFP in Bangladesh from 2017-2020. Over the course of three years, WFP is carrying out project activities aimed at (i) increasing student enrolment and student-teacher attendance through the provision of school meals and recognition of high-performing teachers, (ii) improving literacy among school-aged children and the quality of education through early grade reading and other learning interventions, (iii) improving the health and dietary practices of students by improving water systems and latrine facilities and through water, sanitation and hygiene education initiatives, (iv) increasing government ownership through capacity building activities, and (v) increasing awareness regarding the importance of education among parents and community members through ongoing advocacy campaigns. 4. WFP is providing micronutrient-fortified biscuits to around 47,689 primary-school-aged children per year in two upazilas in Cox’s Bazar district - Ukhiya and Kutubdia. The programme is targeting 146 government- schools and 11 madrasahs in Ukhiya. The programme intervention map is presented in Annex A. The status of distribution during the period October 2018- March 2019 is as given below: Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 2 Table 1: Commodity Direct Distribution: Bangladesh FY 2017 Award Period (October 1, 2018 - March 31, 2019) Commodity Direct Distribution Report (Oct 2018 - March 2019 ) Commodity Quantity (MT) Distribution Region Beneficiaries Comment Total Received Expected Distributions Actual Distributions Fortified biscuit 118.78 133.44 118.78 Ukhiya 31,380 The distribution was started from end of Nov' 18 in Kutubdia and from Jan' 18 in Fortified biscuit 67.889 73.224 67.889 Kutubdia 15,411 Ukhiya. Total 186.669 Source: WFP Semi Annual Report 5. An end line evaluation of the 2015-17 SFP highlighted the need to strengthen the functioning and role of School Management Committees (SMCs) (especially women members) in the SFP, improve teaching and learning environment in schools, improve training (and re-training) of teachers on new teaching techniques and methods, and strengthen coordination and partnerships with the Ministry of Primary and Mass Education (MoPME), Government of Bangladesh (GoB). The poorly accomplished outputs on training in safe food preparation and storage practices was also elaborated upon. Teacher’s shortage and inadequate school infrastructure were identified as external factors affecting the literacy outcome. The SFP-FY17 award incorporated all the recommendations (in the programme design) from the previous evaluation except addressing the shortage of teachers. This required direct intervention from GoB. 6. The scope of the evaluation is to address the proposed key evaluation questions including activities and processes related to the programme’s formulation, implementation, resourcing, monitoring, and evaluation. 7. In accordance with the advice from the government to the district level officials, that restricts the access to teacher’s attendance to district and upazila officers, evaluation of indicators based on teacher’s attendance has been excluded from the scope of evaluation. 8. Objectives, Outcomes and Activities: The strategic objectives and foundational results and expected outcomes have been presented in the results framework in Annex B. Key activities under each of the strategic objectives of the programme and indicators are presented in Annex C. 9. Planned Outputs and Beneficiaries: The USDA McGovern-Dole FY-17 project is expected to benefit 47,689 schoolchildren of Grades I-V in 146 schools by providing approximately 9.3 million micro-nutrient fortified biscuits per year and supporting complimentary education interventions in two upazilas (Ukhiya and Kutubdia) of Cox’s Bazar. The project is also expected to aid the formulation and operationalization of the first National School Meal Policy (NSMP) and help mainstream Government of Bangladesh’s National School Feeding in Poverty Prone Areas Programme (NSFPPA) into the Primary Education Development Programme (PEDP IV). Details of the planned outcomes have been provided in Annex D. The overall targeted beneficiaries and regions in line with the GoB and USDA priorities are presented in Annex E. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 3 10. Partners: WFP has established strategic partnerships and leveraged support from government institutions, other UN agencies and NGOs towards achieving the McGovern-Dole goals. The list of partners along with their responsibilities is presented in Annex F. 11. Resource Requirements and Funding of USDA McGovern-Dole SFP: WFP has allocated $17,119,720.00 for donations of commodities, transportation, and financial assistance through the USDA McGovern-Dole SFP. WFP is providing financial support to its implementing partners for covering the direct programme costs and administration expenses. The detailed programme costs, line item costs and activity-wise project operating budget is provided in Annex E. 12. Gender Dimensions of intervention: SFP by design, does not have a specific focus on gender related issues. However, WFP aims to encourage the equal participation of women and girls in all project activities. Distribution of biscuits is done in a gender equitable manner, wherein all boys and girls in the target schools are provided biscuits. 1.3. Context 13. With a population of over 160 million, Bangladesh has increased its HDI value by 57.1% during the period 1990 to 2017, putting itself in the medium human development category2. Gross National Income (GNI) per capita has also increased by 178.6 percent during the same period. However, despite significant gains in macro-economic growth and human development over the past decade, Bangladesh’s Human Development Index (HDI) of 0.608 is below the average of 0.645 for countries in the medium human development group. The Multidimensional Poverty Index (MPI)3 which includes education, health, and standard of living also highlights widespread income poverty in Bangladesh. 40.7 percent of the country’s population (64 million people) is multi￾dimensionally poor. The contributions of deprivation from education, health, and standard of living to overall poverty are 28.4, 26.1, and 45.5 percentage points respectively. 1.3.1 Food Security and Nutrition 14. Bangladesh has made impressive gains with regard to improving the state of food security over the past few decades, wherein the prevalence of undernourishment has declined from 32 percent in 1990 to 15.2 percent in 20164. However, there are still 26 million people who are food-insecure. 15. The Food Security and Nutrition Surveillance Project reports shows that while food insecurity has declined for all the quintiles, it has declined relatively slowly for the poorer ones. In 2010, the proportions of food-insecure households were found to be 68 percent for the bottom two quintiles and 20 percent for the top two quintiles. By 2014, these proportions had come down to 43 percent among the bottom two quintiles and 6 percent among the top two5. These figures suggest that the rate of progress was much slower for the poorer groups6. Moreover, existing household-level food insecurity is exacerbated by poor access to food and limited dietary diversity, in turn worsening undernutrition. 2 United Nations Development Programme. (2018). Human development indices and indicators: 2018 Statistical update. 3 Human Development Report 2016-Human Development for Everyone: Briefing note for countries on the 2016 Human Development Report Note: The HDR 2018 Statistical Update does not contain the MPI due to methodology revisions and will be available in due course. If made available, the same shall be updated in the evaluation report. 4 https://knoema.com/atlas/Bangladesh/topics/Health/Nutrition/Prevalence-of-undernourishment 5 Helen Keller International and James P. Grant School of Public Health). (2015). State of Food Security and Nutrition in Bangladesh: 2014. BRAC University: Dhaka 6 Ahmed, Akhter. (2017). Strategic Review of Food Security and Nutrition in Bangladesh. 10.13140/RG.2.2.24673.71529. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 4 16. The concerns about food security and nutrition are emerging as a result of climate change. About half of the population already suffers from iron and zinc deficiency and there exists, a genuine concern that rising carbon dioxide in the atmosphere will exacerbate the problem of micronutrient deficiencies. Increased salinity due to the inflow of sea water into groundwater in the coastal areas, especially in districts like Cox’s Bazar, poses the threat of altering the micronutrient content of staple foods, including rice. 17. The Joint Response Plan (JRP) for Rohingya Humanitarian Crisis 20197 by the Food and Agriculture Organization (FAO) suggests that 695,000 people are severely food-insecure in Cox’s Bazar. The JRP also reports that Cox’s Bazar has borne significant negative impacts on the environment leading to a reduction in agricultural production. A study conducted in Bangladesh by FAO on gender and food security8 highlighted the gender differences in access to food and related resources. 1.3.2 Health 18. With a current expenditure of 3 percent of its Gross Domestic Product (GDP) on health9, Bangladesh spends well below the already-low South East Asia regional average of 4.6 percent10. 19. While the Human Development Report (HDR) 2018, indicates improvement in life expectancy at birth from 58 to 73 years during 1990-2018, the under-five mortality rate per 1,000 live births is 34.2.11 In case of SFP intervention upazilas (in Cox’s Bazar), the under-five mortality rate in 2015 was 47.4 and 5.6 for Kutubdia and Ukhiya respectively12 . 20. All 64 districts in Bangladesh are endemic with parasitic diseases, with 78 million (55%) infected with roundworm13. The most vulnerable to such infections are the 5-14 years age group as roundworm causes impaired growth and cognition development among children. To address this, GoB’s Ministry of Health and Family Welfare (MoHFW) initiated a deworming programme in 2010. It also initiated the “Little Doctor” initiative, with an aim to increase drug coverage (for deworming) as well as knowledge on important health matters among students, especially on personal hygiene. Deworming is now conducted for children aged 5-12 years in all primary level schools in the country. 1.3.3 Education 21. With over 2 percent of GDP and 18.8 percent of total government expenditure being spent on education14, there has been a significant improvement in the status of children’s education. The country’s net enrolment rate (NER) at the primary school level has increased from 80 to 95 percent during 2000-2017. In 2017, female NER was 93 percent and male NER 89 percent in primary schools. Furthermore, as per the World Health Organization, the Gender Parity Index (GPI)15 for gross enrolment ratio in primary education for Bangladesh was 66.2 in 2015. 7 http://www.fao.org/3/CA3252EN/ca3252en.pdf 8 Alston, M., & Akhter, B. (2016). Gender and food security in Bangladesh: the impact of climate change. Gender, Place & Culture, 23(10), 1450-1464. 9 2018 Health SDG Profile: Bangladesh 10 World Health Organization. (2017). New perspectives on global health spending for universal health coverage (No. WHO/HIS/HGF/HFWorkingPaper/17.10). World Health Organization. 11 United Nations Development Programme, Human development indices and indicators: 2018 Statistical update. 12 http://app.dghs.gov.bd/localhealthBulletin2016/publish/publish.php?org=10000924&year=2016&lvl=1 13 Rahman, M. M. (2017). Biannually school-based deworming by Mebendazole 500mg has reduced the worm load drastically in Bangladesh. EC Bacteriology and Virology Research, 2, 113-114. 14 Bangladesh Education Statistics 2018; BANBEIS 15 The GPI is the ratio of girls to boys enrolled at primary level in public and private schools. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 5 22. The quality of education, however, has not been able to match the level of enrolment rate. According to the World Development Report 2018, 35 percent of grade III students scored too low to even be tested on reading comprehension in Bangla, and only 25 percent of grade V students in the country passed the minimum threshold in mathematics. National learning assessments by the GoB also show poor literacy and numeracy skills among students wherein only 25 to 44 percent of the students in Grades V-VIII have mastery over Bangla, English and Mathematics. 23. It is also interesting to note that when the HDI for Bangladesh is discounted for the inequalities, especially in education, the loss is of 37.3 percent, indicating a high variation in the mean years of schooling within the country. Furthermore, the GDI indicates that the mean years of schooling for girls is lower (5.2) as compared to boys (6.7). 24. It is important to highlight that among all districts, Cox’s Bazar’s performance on indicators such as primary education NER and dropout rates is lowest in the country. The NER in Cox’s Bazar is 72.6 percent for boys and 69 percent for girls, compared to the national average of 98 percent. Further, the net attendance ratio for primary school age children is 65.6 percent16 as compared to national average of 73.2 percent. Similarly, while the national average dropout rate for boys and girls is 14 percent and 11 percent respectively, it is 39.6 percent (for boys) and 23 percent (for girls) in Cox’s Bazar. 1.3.4 Water, Sanitation and Hygiene 25. According to the Joint Monitoring Programme (JMP) Report 2017, the proportion of population with access to safely managed water is 56 percent while the proportion of population with access to ‘at least basic’ sanitation is 48 percent. 26. With regard to the WASH situation in schools, the WASH Watch report revealed that as of 2016, 73 percent of primary schools had access to ‘basic water service’ whereas 17 percent did not have access to a water service at all. Also, while 57 percent primary schools had access to a ‘basic’ sanitation service, only 39 percent of primary schools had access to basic hygiene services (facility with water and soap). 27. Furthermore, as per the WASH Sector Cox’s Bazar Situation Report17 (August 2018) by the Inter Sector Coordination Group, 1.05 million people in Cox’s Bazar are still in need of immediate WASH support. 1.3.5 School Feeding Needs 28. Malnutrition estimates from the Global Nutrition Report (2018) show that wasting in Bangladesh is at 14 percent among children under five years and has not shown much progress against the global nutrition targets of 201818. Similarly, about one out of three children (32.6%) under five years are underweight. In case of Cox’s Bazar, 40.5 percent of children under five are under weight, while 49.5 percent19 children are stunted. 29. The WFP-CO’s Vulnerability Assessment and Mapping (VAM) Unit assessed priority districts and sub-districts by using a weighted index combining food insecurity, nutrition, poverty, and education indicators. The report pointed to a high need of intervention in Cox’s Bazar district. The 16 Multiple Indicator Cluster Surveys (MICS) 2012-13. 17https://reliefweb.int/sites/reliefweb.int/files/resources/operation_presence_map_wash_sector_august_2018_ta_0.pdf 18 2018 Nutrition country profile accessed from www.globalnutritionreport.org Source: UNICEF/WHO/World Bank Group: Joint child malnutrition estimates. 19 Multiple Indicator Cluster Surveys (MICS) 2012-13. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 6 district has a poverty rate of 22 percent and extreme poverty rate of 11.3 percent. Further, it is highly food insecure with over 34 percent of the population living below the food consumption poverty line20. The region is also experiencing a Level 2 emergency with the influx of over 900,000 Rohingya refugees from Myanmar which is further exacerbating food insecurity. 1.3.6 Gender Analysis 30. Although the constitution of Bangladesh guarantees equal rights, women and girls in Bangladesh still lag behind men in many aspects of life. Discrimination against women and male dominance is a prevalent feature of society, especially in rural areas. 31. The World Economic Forum’s Global Gender Gap Reports (2017 and 2018)21 revealed that Bangladesh has progressed significantly from its position as 72nd in 2016, 47th (in a tally of 144 countries) in 2017, and 48th amongst 149 countries in 2018, leading South Asia in terms of gender equality. The country was ranked 135 in labour force participation, 105 in wage equality, 114 in earned income, and 135 in legislators, professionals/technical workers, thereby concealing the continued struggle with violence against women, wage disparity and the participation of women in workforce. 32. While poverty rates in Bangladesh are decreasing, vulnerability to poverty continues to have concrete gender dimensions. Significant disparities in employment and wage rates persist (female GNI 2,041; male GNI 5,285)22 which, combined with considerable gaps in asset ownership, seriously limit women’s economic opportunities. ILO reports that participation of women in the formal labour market (43.1%) lags far behind that of men (81%)23, and the rates of business ownership by women are even lower. With a large population mostly engaged in informal labour, women are faced with lower income, greater insecurity, and unfavourable working conditions. 33. Gender-based capability poverty continues to be a key issue in the health and education sectors despite significant improvements in recent years, and is reflected in poor nutrition, maternal mortality and child mortality indicators. For every 100,00024 live births, 176 women die from pregnancy-related causes. While the adolescent fertility rate has shown improvement since 2006, it still remains an area of concern. 1.3.7 Government Policies and Programmes 34. The Government of Bangladesh envisions transforming Bangladesh into a middle-income country by 2021 and a prosperous country by 2041. The government has translated this vision into an actionable agenda by formulating the Perspective Plan (2010-2021) and two Five Year Plans (6th and 7th FYPs)25 by integrating Sustainable Development Goals (SDGs) into it. Given the comprehensiveness and cross-cutting nature of SDGs, policy coherence is very critical. 35. The government is implementing the National Education Policy since 2010. During the course of implementation, it extended the compulsory pre-primary and primary education to Grade VIII. This, however, had to be rolled back due to the lack of infrastructure. Efforts to increase the 20 Terms of Reference (TOR) of Fiscal Year (FY) 2017 – FY2020 McGovern-Dole International Food for Education and Child Nutrition Grant (McGovern-Dole) 21 The Global Gender Gap Report benchmarks countries on their progress towards gender parity across four thematic dimensions: Economic Participation and Opportunity, Educational Attainment, Health and Survival, and Political Empowerment. 22 United Nations Development Programme. (2018). Human development indices and indicators: 2018 Statistical update. 23 International Labour Organization, ILOSTAT database (2017) 24 Islam, M. S., & Hasan, M. N. (2017). Bangladesh. In The World Guide to Sustainable Enterprise (pp. 36-41). Routledge. 25 https://info.undp.org/docs/pdc/Documents/BGD/UNGA_Booklet_2017.pdf Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 7 educational participation (enrolment, attendance, persistence etc.) are being made through the Primary Education Stipend Project (PESP). 36. The recent Fourth Primary Education Development Programme (PEDP4), GoB’s flagship programme in the education sector for the duration 2019 to 2023, aims at providing children of pre-primary to grade V quality education with key emphasis on efficiency, inclusivity and equity. The programme aims at deploying quality teachers who would be provided with continuous professional development.26 37. The MoHFW embarked on a sector-wide approach (SWAp) modality for the health, nutrition and population (HNP) sectors in 1998. Assessments indicate that the MOHFW has made substantial progress in health outcomes and health systems strengthening27. 38. However, undernutrition costs Bangladesh more than 7,000 crore BDT (US$ 1 billion) in lost productivity every year28, and even more in healthcare costs29. The Second National Plan of Action for Nutrition (NPAN-2), an integrated and multi-sectoral framework for improving the nutritional status in the country, acknowledges that ensuring effective investments in nutrition is estimated to lead to economic gains in Bangladesh, through an estimated increased productivity, exceeding 70,000 crore BDT by 202130 and possibly even more31. 39. GoB has also formulated a comprehensive National Social Security Strategy (NSSS) to integrate all safety net programmes. The coverage of social safety net recipients is 27.80 percent and the allocation has been increased from 1.9 percent of GDP in FY09 to 2.3 percent in FY17 (Household Income and Expenditure Survey, 2016). 1.3.8 Development Assistance in Bangladesh 40. Bangladesh receives development assistance from various sources. The World Bank is implementing a “Transforming Secondary Education for Results (TSER)” project to support the government’s Secondary Education Development Programme, which is expected to benefit 13 million students from Grades VI-XII. It is expected to enhance the quality of teaching and learning as well as improve access and retention of students, especially girls and children from poor households. 41. Further, aiming to reach over 18 million children studying in pre-primary level to grade V, World Bank is implementing the Quality Learning for All Programme (QLEAP) across the country. The programme will help improve the quality of education and ensure equitable access to primary education. The programme aims to bring about one million out-of-school children to learning centres that would follow national primary education curriculum, and thus help them integrate with the formal education system. The project will also build close to 95,000 classrooms, teachers’ rooms, and multipurpose rooms to ensure international standard students-to-classroom ratio. It will also build 80,000 WASH blocks and 15,000 safe water sources with special emphasis on 26 http://documents.worldbank.org/curated/en/857071529206219039/pdf/BANGLADESH-QLEAP-PAD-05252018.pdf 27 Ahsan, K. Z., Streatfield, P. K., Ijdi, R. E., Escudero, G. M., Khan, A. W., & Reza, M. M. (2015). Fifteen years of sector-wide approach (SWAp) in Bangladesh health sector: an assessment of progress. Health policy and planning, 31(5), 612-623. 28 FAO, WFP and IFAD. 2012. The State of Food Insecurity in the World 2012. Economic growth is necessary but not sufficient to accelerate reduction of hunger and malnutrition. Rome, FAO. 29 UN Renewed Efforts Against Child Hunger (REACH), Undernutrition in Bangladesh A Common Narrative. 2014 30 Howlader, et al. (2012). Investing in Nutrition Now: A Smart Start for Our Children, Our Future. Estimates of Benefits and Costs of a Comprehensive Program for Nutrition in Bangladesh, 2011– 2021. PROFILES and Nutrition Costing Technical Report. Washington, DC: Food and Nutrition Technical Assistance III Project (FANTA), FHI 360 31 Ministry of Health and Family Welfare, Government of Bangladesh, 2017. Second National Plan of Action for Nutrition (2016-2025) Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 8 facilities for girl students and female teachers. Moreover, hygiene promotion will be integrated in the revised curricula and textbooks. 42. Bangladesh also receives development assistance from the US Government and its agencies. USDA is providing assistance through Winrock International as part of the Food for Progress programme. This programme aims to improve agricultural productivity and expand the trade of agricultural products in Bangladesh.32 43. Additionally, the Feed the Future programme (part of the US Government-funded Global Hunger and Food Security Initiative) helps more than 700,000 farmers to increase their production of food, commercial crops, fish, and livestock, gain access to markets, and improve the nutrition of their families. 33 44. USAID is also providing support to GoB towards enhancing agricultural productivity, diversifying production to improve nutrition, building capacities of farmers to upgrade to new technologies, improving market systems and contributing to women’s empowerment in agriculture. 34 45. Further, a relevant US Government Foreign Assistance supported programme is the MaMoni Maternal and Newborn Care Strengthening activity, implemented by Save the Children for the period of 2018 – 2032. This programme aims to contribute towards providing quality maternal and newborn health care and nutrition services through technical capacity building of human resources, improvement of health information systems, development of appropriate guidelines and protocols, assistance in policy and strategy development, and strengthening of governance at national and sub-national levels. 35 46. Towards education, USAID is assisting the GoB in improving reading skills amongst primary school students through teacher trainings, provision of supplementary reading materials and creation of community reading camps. Additionally, in collaboration with BRAC, it is also helping in increasing access to schooling among out-of-school children in urban slums. 36 47. In the WASH sector, Water Aid has also launched the WASH4UrbanPoor project to impact the lives of 450,000 staying in slums and low-income urban communities through sustainable WASH services in slums, schools, healthcare centres and public places.37 48. The World Bank is implementing the Multipurpose Disaster Shelter Project in Bangladesh with the objective of reducing the vulnerability of the coastal population across selected districts of Bangladesh to natural disasters. These shelters will also serve as schools and allow for continued use and upkeep of the infrastructure in non-disaster periods. 49. In addition, NGOs are a major performer working on diverse areas like healthcare, education and safe drinking water. For instance, BRAC in Bangladesh has been providing education in a cost￾effective way which has boosted both school attendance and retention. BRAC schools have initiated two measures; separate toilets and two free sets of school uniform for girls. 50. UNICEF is also contributing to the development of Bangladesh in the sector of health, nutrition, education and WASH. 51. The United Kingdom's Department for International Development has provided more than £80 million to UNOPS for addressing challenges related to the humanitarian context of Cox’s Bazar 32 https://www.fas.usda.gov/programs/food-progress 33 https://www.feedthefuture.gov/country/bangladesh/ 34 https://www.usaid.gov/bangladesh/agriculture-and-food-security 35 https://www.foreignassistance.gov/explore/country/Bangladesh 36 https://www.usaid.gov/bangladesh/education 37 https://www.wateraid.org/bd/media/wateraid-launches-the-wash4urbanpoor-project-aiming-to-serve-450000-urban￾poor-with-water Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 9 district. UNOPS is working closely with 15 implementing partners to extend support through improving access to food, education, health, sanitation and jobs. These partners include Action Against Hunger, BRAC, Christian Aid, Handicap International, HelpAge International, International Organization for Migration, Oxfam in Bangladesh, Save the Children International, Solidarités International, UNICEF, United Nations High Commissioner for Refugees, UNFPA, UN Women, World Food Programme and World Health Organization. 1.4. Evaluation Methodology and Limitations 52. The MTE adopts mixed methods approach (quantitative and qualitative) to present the evaluation findings. The difference in the status of indicators from baseline to mid-term is examined through the performance of comparison schools vis-a-vis intervention schools supported by FY17 project (provision of fortified biscuits and trainings). 53. The methodology entailed a desk review of project documents and other supporting documents along with primary data collection which included both quantitative and qualitative methods (key informant interviews and focus group discussions). Data from the primary survey has been triangulated to assess the reliability and validity of the data by comparison of perspectives of different stakeholders on the same issue. In order to mainstream GEEW throughout the analysis, all school-level data includes analyses disaggregated by sex, and the sampling of students ensured equal representation of boys and girls. 54. A summary of the methodology presenting the method of data collection, the target groups/sources of information, the nature of the interviews, and the indicative outputs is presented in Annex H. 55. Evaluation Criteria and Questions: The evaluation through the lens of relevance, efficiency, impact, effectiveness and sustainability (a) assesses and reports on the performance and results of the FY17 project (with reference to the baseline values); (b) draws lessons for learning – identifying reasons why certain results occurred or not; (c) identifies corrective actions and adjustments for the remaining period of the project to ensure the achievement of targets and sustainability of results. The evaluation matrix (Annex L) highlights the key information areas mapped with the mid-term evaluation questions, the approach for data collection and data sources. Wherever appropriate, gender dimensions have been factored into the sub￾questions/key information areas for each evaluation question. 56. Through the use of both quantitative and qualitative tools, the evaluation has determined the current status of achievement against the indicators and reasons that are leading to the progress or affecting the performance of the project. Further, it highlights the perceptions of the programme beneficiaries (such as parents, students and SMC members) and their overall experience with regards to the services being provided as part of the project. This will help understand the enablers and barriers in the achievement of desired outcomes and identify what more needs to be done. The MTE also assesses the extent to which the recommendations made in baseline study have been implemented. 57. The quantitative questionnaires presented in Annex M covers key programme outcomes: literacy, attendance, and attentiveness. Other outcomes include enrolment, disease-related absenteeism of students, dietary diversity score, and status of availability of WASH facilities in schools are also included. 58. The qualitative tools are also presented in Annex C. The qualitative tools focussed on understanding beneficiaries’ experience with the project activities as well as getting their views Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 10 on what is missing from a school feeding programme and what WFP can do better. The structured tools were administered with the following respondents: a. School-going children: Children from Grades I-V were selected through random sampling and interviewed. The questions mainly catered to their receipt of biscuits, attendance, illness￾related absenteeism, and their home and class learning environment. Further, it also attempted to understand their participation in the various activities being implemented under the programme. For children in Grade III, the Early Grade Reading Assessment (EGRA) tool was administered to assess their learning levels. Both quantitative and qualitative tools were administered with students. b. Parents: Parents of the sampled school going children were interviewed to collect data on receipt of biscuits, their awareness about project activities and contribution to the project, the dietary diversity of children, educational environment at home, awareness on child health and nutrition practices, adoption of these practices, and validate the information collected from their wards. Both quantitative and qualitative tools were administered with parents. c. School-based respondents: Questionnaire for the school provided information on aspects (either through observations or through discussions with the headmaster) related to enrolment data, attendance data (of students), student attentiveness, infrastructure, maintenance, biscuit utilisation, and specific questions on the provision of biscuits. Additionally, questions related to classroom libraries, school gardens, little doctors and WASH were also included. Further, a questionnaire was developed for the headmaster to capture the roles and responsibilities played by him/her as school administrator. The other respondents from school included school teachers and storekeepers. The mapping of the questionnaire with the indicators is provided in the form of a table in Annex M along with the tools. Further, classroom observations helped understand if the teachers demonstrate the use of new techniques with students for easy comprehension and use improved teaching materials for better learning outcomes. Discussions were also undertaken with SMC members to understand their role in the project as well as their involvement in school related decisions. d. KIIs were also held with government officials at the national and district level, focusing on their current engagement with the implementation of the FY17 project and SFP, the contribution of WFP in augmenting their capacities to implement SFP and the support required to ensure implementation of activities independently in a sustained manner. The discussion guide for the capacity building component with the representatives, GoB is provided in Annex M. A communication was sent to the relevant government departments highlighting the evaluation plan and schedule of the meetings. e. Representatives, WFP CO: In order to further understand the challenges faced during one year of implementation, success stories and learnings, in depth discussions with representatives of WFP CO were conducted. The discussion guide for the capacity building component with the representatives, WFP is provided in Annex M f. KIIs with representatives from RtR, MA, and YPSA were conducted to better understand their role and engagement in the programme. 59. Site Mapping: The MTE covers the diversity of geographical locations of the schools (i.e., both schools that are easily accessible by roads and those that are located in difficult terrain). The advantage of this approach is that the intensity of the evaluation vis-à-vis community response in diverse geographies increases, adding value to the data. This ensures upholding both WFP’s mandate and the government’s concerns. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 11 60. The sample for MTE spreads across two sub-districts (Ukhiya and Kutubdia) of Cox’s Bazar. Following the approach adopted during baseline, 50 schools were sampled. Of these, 30 were intervention schools (in Ukhiya and Kutubdia) and the remaining were comparison schools in Ramu (same district, different sub-district/upazila). The sample of the schools spread across the district was in the same proportion as the number of programme schools present in the district, thus ensuring statistical representation of the sample. The schools were selected using simple random sampling, in line with the baseline methodology. 61. The sample size was calculated at the project level, using the ‘differences method’ formula with a finite population (confidence interval of 1.96, estimated difference set at 5%). Accordingly, the sample size for intervention schools for quantitative data collection was 390 students (13 students per school). By allocating schools as per their probability proportionate to size (i.e. the total number of GPS in the two upazilas), 12 schools in Kutubdia and 18 schools in Ukhiya were selected randomly. Two students in each grade were randomly selected from Grade I-V. For Grade III, the sample size was 5 students per school (refer Annex J for detailed sampling). Deviation from the above was made in the case of administering the EGRA38 tool and for estimating the attendance. The sample size to administer EGRA tool was 14 students of Grade III per school; 9 more students were randomly selected in addition to the selected 5 students (for quantitative data collection) in Grade III. Similarly, 260 students from 20 comparative schools (13 per school) in Ramu were selected. Overall, 650 students from 50 schools were covered. The list of sample schools is presented in Annex K. 62. In each sampled school, equal number of boys and girls were selected randomly, ensuring gender representation at the time of the survey. The school administrator was requested to inform the students and their parents in advance about the survey. This approach increased the probability of parents’ availability for the interviews, who otherwise might have been engaged in their daily work and chores. 63. For focus group discussions (FGD), 20 percent of the schools from the sample were randomly identified. For each category, namely students, SMC members and parents, 6 FGDs from the intervention schools and 4 FGDs from the comparative schools were conducted (per respondent type, through random selection). Overall, 30 FGDs (10 for each category) were conducted. The FGDs were conducted in the school premises to ensure availability of all respondents (for which, prior intimation was ensured). 64. The data collection team (through Data Management Aid), consisted of locals from Chittagong/Cox’s Bazar/Dhaka. NRMC provided the data collection team a detailed explanation of the tools (three-day class room training), followed by a field practice for one day. During training, specific sessions were conducted on ethical considerations during data collection, data integrity and interacting with children. 1.4.1 Validation Workshop 38 EGRA test is a reading-comprehension test. As outlined in the methodology section, the reading test was administered by counting the number of words read correctly in a minute. For testing comprehension, the students were allowed time to complete reading the entire text and then answer the questions. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 12 65. With an objective of validating findings of MTE and aid cross learning among stakeholders (MoPME and DPE officials at national and district level, implementation partners, SMC members, teachers and parents) a validation workshop was conducted in Cox’s Bazar39 and Dhaka. Further, workshop aimed at triggering discussions around responsibilities that each category of stakeholders can commit to ensure achievement of targets of SFP during the remaining period of the project and, foster collective action. 66. The workshop in Cox’s Bazar was attended by education officials at the district and upazila level, WFP representatives, SMC members, teachers and parents. Post the presentation on findings of MTE, four groups were created for discussion. Three groups were homogenous representing (i) parents; (ii) teachers and headmasters and (iii) SMC members. The fourth group represented officials at district and upazila level and implementing partners. Each group discussed on (i) What’s working well with SFP; (ii) What needs to improve; (iii) How can each group (as representative of a category of stakeholders) support in achievement of targets under SFP and , (iv) What support would a group require from other groups in order to enable them fulfil their commitments. 67. In Dhaka, workshop was attended by officials from MoPME, DPE, Information Management Division (IMD), Department of Agricultural Extension (DAE) and Bangladesh Institute of Research and Training on Applied Nutrition (BIRTAN). In addition to the discussion on findings of MTE, the participants specifically discussed recommendations in terms of (i) importance to SFP; (ii) feasibility of implementation. 1.4.2 Gender dimensions of the mid-term evaluation 68. For the evaluation, gender analysis was considered as a prerequisite to ensure that the design, planning, implementation, monitoring and evaluation of the programme take into account the existing gender disparities. The evaluation has incorporated inputs from the Technical Note on Integrating Gender in WFP Evaluation, 2019 and Gender Equality and Empowerment of Women (GEEW) framework to adequately focus on the gender dimensions within the programme. The design of the evaluation (methodology and the analytical framework) has taken into consideration the United Nations Evaluation Group (UNEG) indicators, specifically relating to gender equality and human rights. Towards this, equal representation of boys and girls was ensured in the discussions with students. Additionally, separate FGDs with mothers and fathers was conducted. Primary data on progress against outputs and outcomes has been disaggregated by gender. 69. The evaluation is guided by WFP’s latest Gender Policy (2015-20). Aligning with these principles, we understand that development programmes affect women, men, boys, and girls differently. Hence, the evaluation emphasises on testing the programme’s coherence with the national policy on gender, and operates inclusive strategies and implementation based on a sound gender analysis. The evaluation focusses on data disaggregated by sex and age, providing empirical insights that can be used and built upon by the users. 1.4.3 Data Analysis 70. Considering that the Difference-in-Difference analysis was not possible during the baseline, an activity evaluation was proposed in the inception report to collect comparative monitoring data 39 Workshop in Cox’s Bazar was conducted on 11th November 2019. In Dhaka, it was conducted on 13th November 2019. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 13 from comparison schools. Accordingly, for school feeding indicators, a direct comparison between the intervention and comparison schools has been undertaken. On literacy indicators, comparison between the intervention and comparison schools was not possible during the baseline, given that the Reading Enhancement for Advancing Development (READ) programmes40 had ended just prior to the commencement of the baseline survey. The MTE therefore, followed the hypothesis formulated during baseline that while the performance against the literacy indicators will improve in the intervention schools, with the cessation of the literacy intervention in Ramu, higher levels of literacy indicators are unlikely to be maintained. Considering this, for literacy indicators, a comparison between the baseline values and mid-term values of intervention schools was carried out to measure change. 71. Qualitative data on the other hand, was analysed by categorizing the findings as per the functional areas under the project level results framework provided in the ToR. These findings were then mapped against the evaluation questions and the OECD-DAC criteria to help identify the strength of the evidence to answer each question. Triangulation was undertaken within methods, where appropriate, by comparison of perspectives of different stakeholders on the same issue. For example, if the students mentioned that they were absent for five days in the last month, this was cross-checked with the attendance data available in the school and through their parents. Complementary observations (through KIIs and FGDs) have been triangulated with the findings of quantitative survey for final analysis and preparation of the evaluation report. 1.4.4 Limitations and Risks 72. The implementation of the programme activities commenced from October 2018. Due to limited duration of implementation prior to commencement of data collection (July 2019) for MTE, the evaluation may not highlight/report significant change/impact of the programme’s activities. 73. The evaluation measures the attentiveness of children based on information from their school teachers, which may be biased based on the teacher’s perception of the child. Attentiveness of a student is a function of different variables such as short-term hunger, poor teaching methods, and lack of interest in the subject matter being taught in class, and incidents at home, among others. 74. Following the advice from WFP during baseline, questions on oral fluency and comprehension have been asked through the EGRA tool during the mid-term. This approach was adopted considering that in case the student was unable to clear the comprehension test, it would be prudent to examine whether s/he could read sentences. 75. Due to the unavailability of the teacher’s attendance data, in accordance with the circular from GoB, the evaluation does not report any findings on the related indicator (custom indicator 1.1.1). This was discussed and agreed upon with WFP. 76. In order to ensure availability of respondents, students’ attendance records and biscuit distribution records, the visit plan was shared with schools prior to the commencement of survey. There is a possibility that with prior information, schools were better prepared and the observations that the evaluation team made may not reflect the reality. Triangulation, where appropriate, by comparison of perspectives of different stakeholders both quantitatively and through FGDs helped in addressing this limitation. 40 READ was a four and a half-year project (2013-2018), supported by the US Agency for International Development, to raise reading skill in the early grades of school, implemented by Save the Children in Cox’s Bazar and other districts. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 14 77. Though the SFP does not have a specific focus on gender related issues, an attempt has been made to integrate Gender Equality and Empowerment of Women (GEEW) throughout the evaluation. The sampling of students ensured equal representation of boys and girls, and the school-level data was disaggregated by sex. The study also examined the status of participation of women during SMC meetings. Data obtained for the gender-related questions has been analysed comprehensively to report on the gender dimensions of the evaluation. Data on attainment of outputs and outcomes has also been disaggregated by gender. 1.4.5 Ensuring Quality and Adherence to Ethical Standards 78. WFP’s is closely aligned with UNEG norms, standards and good practices of the international evaluation community to ensure that the evaluation process and products conform to the same. These standards and guidelines guided the evaluation team at NRMC. The quality assurance expert at NRMC ensured that the prescribed quality protocols were integrated with the data collection process to obtain reliable data and ensure the quality of deliverables. Some of the key steps to ensure quality at various stages of the survey have been mentioned in Annex P. 79. The evaluation team made sure that no data was collected without consent of the participants or approval of the concerned authorities. For this, a verbal consent was sought from all respondents. Further, the participants were informed about a) their choice to decline their participation in the interview at any time and b) about their rights to confidentiality, anonymity, prevention from any harm and access to information about the evaluation. Considering that the MTE engaged children (primary school students), special care was sought in ensuring ethical behaviour and understanding in data collection. The data collection team was trained on child-sensitive and child-friendly techniques so that they are able to engage with children effectively. Students were spoken to in a manner that was sensitive and did not cause intimidation, for which the enumerators were specially trained. It was ensured that the girl students were interviewed by women enumerators to make them comfortable. During the survey, it was ensured that the evaluation team did not ask any questions or pose any cross-questions that were personal or sensitive, or that might physically, mentally or emotionally harm the respondent (especially children). Moreover, ice-breaking games were played with children and informal/casual discussions were held with parents, teachers, SMC members and headmasters prior to administering any tool, to build rapport and ensure their comfort and ease. Considering that the evaluation team was interacting with people from the host communities (given the huge influx of Rohingya in Cox’s Bazar), all matters related to access, availability and affordability of goods and services were discussed keeping the sensitivity of the context in mind. 80. Some other measures taken to ensure quality data collection were as follows: a. The data collection team comprised of personnel well-experienced in collecting and collating both quantitative and qualitative information. Separate teams were made for each tool/questionnaire to ensure that the tasks are clearly assigned and the teams are well-versed with their respective tools. KIIs with the government officials, WFP field office staff and partners were conducted by the NRMC core team (details provided in Annex Q) to ensure the quality of discussions with these stakeholders. b. As part of quality control and to ensure timeliness of collection of data, NRMC developed a detailed field movement plan in advance of the survey and strictly followed it. There was no delay in data collection even during monsoons. c. A robust monitoring process was followed for quality assurance during data collection. At least one member of the core evaluation team of NRMC was present in the field during the Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 15 entire period of data collection, accompanying the enumerators for both qualitative and quantitative interviews. d. Regular updates regarding the progress of the evaluation were provided to the Evaluation Manager, WFP – CO by the evaluation team. The team members followed quality norms prescribed by WFP. 2. Evaluation Findings 81. This section presents the field findings and assesses the relevance, effectiveness, efficiency, impact, and sustainability of the programme with gender equity and inclusion mainstreamed throughout. The findings have been reported based on the data collected from primary surveys, interviews and FGDs as well as secondary review. For the primary data collection, quantitative and qualitative data from the stakeholders was triangulated before converging into the overall findings. Wherever appropriate, gender dimensions were factored into the sub-questions, judgement criteria, and indicators for each evaluation question. Snapshot of various extracurricular activities undertaken as part of SFP Book Captains: Students selected to issue books to their peers from the Classroom Libraries provided by Room to Read. They receive a badge, cap and t-shirt in order to distinguish them from other students. Little Doctors: Students selected to assist in de-worming drives and disseminate health and hygiene related messages to their peers in assemblies, classrooms etc. They receive a white apron in order to distinguish them from other students. Vegetable Garden: A garden established in the school premises where vegetables and fruits are grown. The purpose is to teach students about the importance of nutrients and promote nutritious dietary practices through practical demonstration. Little Agriculturalist: Students selected to be in-charge of the vegetable gardens and disseminate nutrition and safe storage related messages to their peers. Read Play Festival: A community event to encourage game-based learning. Students play word making, reading and spelling games and so on. Healthy Meal Preparation Day: A community event to demonstrate nutritious and healthy cooking practices. Mothers of the school students cook for this event using vegetables from the school vegetable garden. Discussions around nutritious properties of food are also held. Grade 1 Reception Day: An event hosted by schools to welcome Grade 1 students and felicitate their parents for enrolling their child in the school. The aim is to encourage enrolment through recognition. 2.1. Relevance Evaluation Question 1: Is the project implementation strategy designed to reach the right people with the right type of assistance? Evaluation Question 2: Is the project aligned with the national government’s policies and strategies including education, school feeding, safety net and national five year plans? Evaluation Question 3: Does the project complement other donor-funded and government initiatives? Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 16 82. Considering the poor performance of Cox’s Bazar on indicators of NER and dropout rates41, fortified biscuits are one of the means to improve attendance and attentiveness of students. This was validated by the views of parents during FGDs, where it was clearly articulated that distribution of biscuits act as one of the triggers for students to attend school regularly. It can therefore, be inferred that the biscuit distribution intervention is relevant in terms of generating interest among students to attend schools regularly. 83. Despite the improvement in the enrolment rates at the national level, quality of education still remains a challenge, as also indicated in the World Development Report 201842. The key influencing factors affecting the poor results in Bangla (including reading and comprehension) and Mathematics are (i) lack of access to early childhood development programmes, (ii) low quality of teaching practices, (iii) challenges related to poor school management, and (iv) low levels of overall spending on public education. 84. In this context, SFP’s implementation strategy that comprises of (i) provision of trainings to teachers and headmasters on new teaching techniques, library management and effective school management and (ii) establishment of reading corners to improve Bangla reading comprehension, find its relevance. Community participation events such as “Read-Play festival” and “Grade I Reception day” celebrated as part of SFP also contribute to the relevance of the programme since they aim to create awareness around attendance, enrolment and literacy and provide platforms to encourage the same. 85. Discussions with parents revealed that there is a felt need for interventions to increase children’s interest in education related activities and to improve their performance. Through interventions such as the Read Play festival as well as Book Corners, parents believe that the programme is addressing the need for improving the quality of education. Such interventions were appreciated as they were not only contributing towards the enhancement of children’s reading fluency and comprehension skills but also increasing their interest in education. 86. SFP’s technical support to the Little Doctor initiative contributes towards strengthening GoB’s initiative by augmenting the capacities of Little Doctors to distribute deworming tablets, conduct height, weight and eye check-ups and disseminate health and hygiene related messages to peer students. 87. The support towards rehabilitation and construction of water systems and toilets in schools addresses the challenge of availability of functional toilets in schools, as highlighted in the baseline study that indicated towards poor maintenance and lack of water facilities in toilets. Sensitisation of teachers, parents and students on health and hygiene practices planned under SFP further creates awareness and influences the target population to adopt good health and hygiene practices. Hence, the implementation strategy adopted by the programme, adequately targets the right people with the right type of assistance and was therefore, found to be relevant. 88. The National Education Policy (NEP), 2010 articulates GoB’s approach to support ‘education for all’, the eradication of illiteracy and improvements in the quality of education. SFP’s activities such as (i) creating awareness on the importance of education, (ii) developing new teaching techniques and tools and providing training to teachers on the same, (iii) establishment of class room libraries to promote a reading habit among students and; (iv) providing support to GoB in strengthening 41 Details presented in the section on context 42 Details mentioned in the section on context Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 17 school governance structures are aligned with Bangladesh’s NEP (2010) and supports MoPME in achieving its objectives. 89. GoB’s flagship programme in the education sector (PEDP4) aims at providing pre-primary to grade V students quality education, with key emphasis on efficiency, inclusivity and equity from 2019- 2023. Towards this, the support provided through SFP in terms of providing training to teachers on new teaching techniques is aligned with PEDP4. 90. The NPAN-2 has highlighted the need for and focus towards (i) social protection programmes that include school meals and school feeding, (ii) strengthening multi-sectoral programmes towards ensuring nutrition and increasing joint efforts and coordination among ministries with regard to social safety nets, education, water, sanitation and hygiene, (iii) strengthening/integrating nutrition education in regular curricula of primary schools and scaling up school health, school feeding and school gardening programmes. Therefore SFP’s activities on (i) distribution of fortified biscuits; (ii) supporting vegetable gardens; and (iii) creating awareness on health, hygiene and nutrition, aligns with NPAN-2. Furthermore, the SFP also supports MoPME in reporting its contribution in the joint efforts of various ministries for implementation of NPAN-2. 91. Additionally, the NSSS (2015) acknowledged the need to scale-up and increase outreach, access and coverage of existing safety net programmes. The implementation strategy of adopting a saturated approach in the two upazilas ensures improved coverage of SFP. 92. In light of the findings above, it can be inferred that the programme is well aligned with the national government’s priorities, policies and strategies. 93. Besides WFP, other development partners such as UNICEF, UNHCR, USAID, BRAC and Save the Children are also implementing programmes in Cox’s Bazar. These partners are primarily working for Rohingya refugees on aspects such as food distribution, health, water and sanitation, among others. For the host community, UNICEF is supporting the creation of child friendly schools and early childhood development centres (for preschool children aged 3-6 years). Provision of nutrition support services is one of the important support measures to early childhood development centres. SFP, through its provision of fortified biscuits to students and complimentary education interventions, extends the development partners’ support to GoB from early child development centres to primary schooling. The SFP activities were therefore, found to be complementing other donor-funded and government initiatives. 2.2. Effectiveness Evaluation Question 4: What is the output and progress of project implementation – is the project on track to carry out all the activities as planned? Evaluation Question 5: To what degree have the interventions resulted in the expected results and outcomes – is the project on track to reach set targets? Support Improved Literacy and Improve Student Enrolment and Attendance Number of textbooks and other teaching and learning materials provided as a result of USDA assistance 94. The survey findings indicate that all the sampled intervention schools have received books from WFP as a result of USDA assistance. The periodic monitoring report43 highlights that 82,807 textbooks and learning materials have been provided to all intervention schools. Additionally, 43 October 18- March 2019; Source: WFP Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 18 WFP is also providing training to teachers and headmasters with an objective of improving literacy outcomes in students. All the sample intervention schools (except one in Ukhiya)44 during the quantitative survey, reported receiving training on improving literacy in the past year. All the head teachers and 83 percent of the teachers in the sample school confirmed receipt of such trainings. Number of classroom libraries (book shelves with books) established as a result of USDA assistance 95. Classroom libraries for each grade were observed in all the sampled intervention schools. In case of shared classrooms, separate libraries for each grade were observed. These classroom libraries have books that are colour-coded based on the difficulty levels of reading. Discussions with the headmasters, teachers and students provided an affirmation on support received by the intervention schools on aspects related to (a) building capacities of book captains and teachers on library management, (b) provision of supplementary books and uniforms for book captains and (c) facilitating regular monitoring of students’ reading and comprehension abilities. WFP has also initiated a separate Supplementary Reading Material (SRM) period for children, so that they can read storybooks in class. This period is conducted once a week wherein teachers and sometimes Literacy Facilitators (LFs) interact with students, and help them to read books available in the library. 96. The analysis of findings at a disaggregated level (on the basis of gender) suggest that 85 percent boys and 91 percent girls in sampled schools of Ukhiya, and 79 percent boys and 84 percent girls in Kutubdia read books from the classroom library. However, in the case of Ramu, analysis highlights that only 49 percent boys and 47 percent girls in the sampled comparison schools read extracurricular books from the library45. Discussions with students and parents confirmed issuing of supplementary reading materials by students from the classroom libraries on a regular basis (weekly) in the intervention schools. During discussions, parents in Ukhiya were able to better highlight the functioning of classroom libraries and the roles and responsibilities of book captains as compared to Kutubdia. In Ramu, there was no mention of book captains and classroom libraries during discussions. This was corroborated from the findings of the quantitative survey where all the students indicated that their school(in Ramu) does not have a separate class room library. The parents and students however, mentioned having school libraries. 97. Further, discussions with students in Ramu revealed that they have read all the supplementary books provided under the READ programme. It also emerged from discussions with teachers and headmasters that there have not been any concerted efforts towards ensuring availability of new books post the READ programme (since 2018), resulting in a decreasing engagement of students with the existing books. Number of school administrators and officials trained or certified as a result of USDA assistance 44 The questionnaire (quantitative survey) did not capture reasons for not receiving training on improving literacy. The same shall be added during the end-term survey 45 Supported under READ Programme; Donor: USAID, IREX, Porticus Foundation, SC UK, and Losan; Duration: September 2013 - March 2018 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 19 Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance 98. The periodic monitoring reports highlight that a total of 484 teachers/educators/teaching assistants (146 head teachers and 338 teachers) across all intervention schools have been trained as a result of USDA assistance. The reported number is lower than the SFP target of training 730 teachers and educators by Year 2. The survey findings indicate that 18 head teachers (75%) and 85 teachers (73%) in the sampled intervention schools have been trained on new teaching techniques as a result of USDA assistance. At a disaggregated level, 9 head teachers and 39 teachers in Ukhiya and 9 head teachers and 46 teachers in Kutubdia were trained by WFP. Number of school administrators and officials in target schools who demonstrate the use of new techniques or tools as a result of USDA assistance Number of teachers/educators/teaching assistants in target schools who demonstrate the use of new and quality teaching techniques or tools as a result of USDA assistance 99. 89 percent of the head teachers and 98 percent of the teachers trained through USDA assistance in the sampled intervention schools reported the use of new teaching techniques or tools. The classroom visits validate these findings, wherein the use of participatory techniques (storytelling and role play) by trained teachers were observed across all the sampled intervention schools. It is worth mentioning that the use of participatory techniques by teachers was observed in comparison schools as well. However, only 25 percentteachers in the sampled comparison schools were observed to be using printed visual aids as compared to 73 percent in sampled intervention schools. During FGDs, students expressed that learning through the use of participatory techniques and colourful printed visual aids is faster and more interesting than learning from normal textbooks. This was confirmed by parents during FGDs where they mentioned children demonstrating improved learning through new techniques and discussing the new techniques and tools at home. Evidently, the implementation activities have been able to promote teacher’s knowledge and skills, the demonstration and utility of which has also been validated by the students and parents. Percentage of parents in target communities who can name at least three benefits of primary education 100. While 69 percent of the parents in the comparison schools and 66 percent of the parents in sampled intervention schools could name at least three benefits of primary education, no significant difference46 was observed between the midline and baseline values for the same. The three most common responses from parents (both intervention and comparison schools) on benefits of primary education were that it a) improves future opportunities of work for children and b) helps break the cycle of poverty and c) helps child’s skill development. 101. During the FGDs, parents in the intervention schools clearly articulated the need for quality of education. This can be supported by analysis of data from the quantitative survey that suggests that in 90 percent (45 schools) of the sample schools (83% in Ukhiya, 100% in Kutubdia), quality of education is the most commonly discussed agenda in the parent-teacher meetings. 90 percent of schools in Ramu also reported having discussions on quality of education during parent teacher meetings. It can, therefore, be inferred that parents discuss and demand quality education for their children. 46 Non-significant difference (@5% level of significance) between midterm and baseline values Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 20 Percentage of students in classrooms identified as attentive by their teachers 102. There is a significant difference47 in the proportion of students identified as attentive by their teachers (75%) in the intervention schools as compared to the baseline (70%). For the comparison schools, there is a significant decrease in the proportion of students identified as attentive by their teachers (71%) as compared to the baseline (89%). During discussions, teachers from the comparison schools expressed their realization on school feeding initiative as a key enabler for improving attentiveness amongst students. Another contributing factor to this decrease in the proportion of attentive students can be the decline in the adoption of innovative teaching techniques by teachers from the comparison schools. This can be validated by findings from classroom observations conducted by the evaluation team. The findings of the classroom observation during MTE suggests lower adoption of innovative teaching techniques (participatory techniques, audio visual aids and putting effort to engage the students) in comparison schools as compared to those observed during baseline. This also emerged during the FGDs where students highlighted limited participation in classroom activities. Percentage of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text 103. The findings of the EGRA test indicates, a significant difference48 between mid-term (28%) and baseline values (6.9%) for demonstration of reading comprehension abilities (combination of fluency and comprehension) among students in intervention schools. For comparison schools, the difference from baseline (15.4%) to midline (17%) was not statistically significant. Furthermore, the increase in the number of students (who by the end of two grades of primary schooling, could read and understand the meaning of grade level text) from baseline to midline is higher in intervention schools (118 students from 29 students), than comparison schools (48 students from 42 students). One of the contributing factors to this relatively lesser percentage increase in comparison schools could be the lack of availability of new supplementary reading material leading to low motivation among students to read, post withdrawal of READ programme. In the case of intervention schools, books are issued based on regular assessment of students’ reading abilities. During discussions, the students expressed that being able to read a book of the level that corresponds to their reading ability encourages them to read regularly. 104. Analysis of data on the basis of gender in intervention schools and comparison schools has been presented in Figure 1 below. This data suggests that 32 percent girls and 25 percent boys in sample intervention schools could fluently read and understand grade II level text. Within the sample intervention schools, Ukhiya had a higher proportion of students (30%) demonstrating proficiency in reading comprehension skills as compared to Kutubdia (27%). During FGDs in Ukhiya, the parents proudly cited instances of children reading out stories to them and siblings. …aamar mey badi eshe aamae boi theke golpo shonae, aamar shunte khoob moja lage. Aami to podte pari na, kintu aamar mey khoob bhalo pode, aamar khoob garbo hoye… 47 *Significant difference (@5% level of significance) between baseline and midterm values for student attentiveness in comparison schools 48 Significant difference at 95% confidence interval (2-tailed) between baseline and midterm value of sample intervention schools for demonstration of reading and comprehension abilities Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 21 …my daughter read out stories to me from books when she comes back home, I really enjoy those stories. I can’t read but my daughter reads really well, I’m very proud of her… - mother in Ukhiya 105. Further, they mentioned that the supplementary reading materials with interesting stories and colourful pictures have generated interest among students in reading and helped in improving Bangla reading skills. They also highlighted improvement in “uccharon” or pronunciation of difficult Bangla words. Enthused by the improvement, the parents also talked about their efforts to support children in reading, by reading books/narrating stories themselves or taking help from neighbours, or relatives. Figure 1 Percent of students who can both read more than 45 words per minute and comprehend more than seven questions Figure 2 Number of students who gave more than 7 correct answers (comprehension) and number of students who read more than 45 correct words in a minute 106. The analysis of EGRA test indicate that 37 percent students in intervention schools and 32 percent students in comparison schools were able to read more than 45 words correctly in a minute. Analysis at a disaggregated level based on gender, suggests that a higher proportion of girls were able to read more than 45 words correctly in both intervention and comparison schools (Figure 2). 107. With regard to comprehension, there is a significant difference49 in the number of correct answers given by students in intervention schools from baseline. For schools in Ramu, 17 percent students answered more than 7 correct answers as compared to 28 percent in intervention schools. Analysis at a disaggregated level based on gender in intervention schools suggests that a greater proportion of girls in the area (32%) were able to give more than 7 correct answers as compared to boys (25%). Similar findings were observed for Ramu as 23 percent girls and 12 percent boys could answer more than 7 questions correctly. 49 *Significant difference (@5% level of significance) between number of correct answers given in Ukhiya and Kutubdia. 28% 25% 27% 20% 41% 30% 12% 23% 17% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Male Female Total Male Female Total Male Female Total Ukhiya Kutubdia Ramu Intervention Comparison 33% 41% 37% 26% 38% 32% 25% 32% 28% 12% 23% 17% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Male Female Total Male Female Total Intervention Comparison Read more than 45 correct words Answer more than 7 correct answers Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 22 Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance 108. All the sample intervention and comparison schools reported having an SMC. WFP extends technical support to SMCs in the form of training on aspects such as health and nutrition, managing biscuit distribution and safe storage practices. 94 percent of SMCs in Ukhiya and 83 percent of SMCs in Kutubdia reported receiving training from WFP. WFP also provides financial assistance to SMCs to ensure that SMC meetings are conducted regularly. The findings suggests that on an average, 10 meetings in a year are conducted in both intervention and comparison schools. One in two schools in Ukhiya and one in three schools in Kutubdia (as in the case of Ramu) conduct SMC meetings on a monthly basis. 109. There has been a significant decrease50 in the proportion of intervention schools (56%) having seven or more male SMC members from the baseline (83%). This indicates an improved female to male ratio as compared to the baseline in the composition of SMCs in intervention schools. In contrast, comparison schools exhibit a decrease in the female to male ratio, with an increase in the proportion of schools that have seven or more male SMC members (from 70 percent in the baseline to 75 percent in the midterm). While there is an increase in the female to male ratio, the discussions with the female members of the SMCs reflect that despite an increase, there are limited opportunities for them to contribute in discussions related to identifying ways to improve school infrastructure or management. 110. With regard to the functioning of SMCs, 58 percent of the sample parents in intervention schools and 68 percent in the comparison schools were aware about the activities of the SMC. The discussions with the SMC members indicate that quality of education and school infrastructure are the most commonly discussed topics during SMC meetings. This can also be corroborated with the findings of quantitative survey that suggest, 58 percent parents in the intervention schools expressing that SMCs engage in improving quality of education. A higher proportion of parents (99%) in comparison schools responded that SMCs engage in improving quality of education. A higher proportion in comparison schools can be attributed to support provided to strengthen the working of SMCs under the READ program Two of three intervention schools reported that SMCs also provide support to the school feeding programme in the form of stock management and grievance redressal for the distribution of fortified biscuits. 111. 97 percent of the sample intervention schools (17 in Ukhiya and 12 in Kutubdia) and 100 percent of the comparison schools reported having a PTA. WFP facilitates four parent-teacher meetings at school in a year. 78 percent (14 schools) of the sample intervention schools in Ukhiya and 75 percent (9 schools) in Kutubdia are conducting 3-4 meetings per year. WFP has also provided training to these PTAs on health, hygiene and nutrition awareness, and quality of education. Headmasters from 8 schools (44 %) in Ukhiya and 4 schools (33 %) in Kutubdia confirmed having received training support to PTAs in their schools from WFP. Number of students enrolled in school receiving USDA assistance 112. A disaggregated enrolment in sample intervention and comparison schools is presented in Table 2 below. The analysis of the data on enrolment in intervention schools and comparison with the baseline indicates a 21 percent increase in average enrolment per sample school. Similarly, an increase in average enrolment per sample school by 10 percent as compared to the baseline value 50 *Significant difference (@5% level of significance) between baseline and midterm values of proportion of schools with seven or more male SMC members. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 23 can be observed in comparison area. While there is an increase (from baseline to midterm) in average enrolment in the intervention and control schools, it was not found to be statistically significant. Furthermore, a relatively higher increase in the average enrolment of girls per sample intervention school during mid-term as compared to comparison schools can be observed (17% and 9%). Table 2 Enrolment in sample intervention and comparison schools Average Enrolment per school Baseline Mid-term Boys Girls Total Boys Girls Total Intervention 125 142 268 153 169 322 Comparison 163 200 364 182 219 401 Number of students regularly (80 percent) attending USDA supported classrooms/schools 113. As can be observed from Table 3 below, in the baseline, 324 students (83 %) in sample intervention school regularly attended school (had a minimum of 80 percent attendance). In comparison, the findings for the mid-term suggests that 332 students (85 %) in sample intervention schools regularly attend school. Though there is an increase, this change is not statistically significant. In case of comparison schools, 244 students (94 %) students regularly attend schools as compared to 92 percent (239) students during baseline. This change is also not statistically significant. Analysis at a disaggregated level suggests, a relatively higher increase in regular attendance of girls (88% as compared to baseline value of 82%) in case of intervention schools as compared to comparison schools (from 94% in baseline to 95 % at mid-term). This increase in regular attendance of girls in intervention schools has been found to be statistically significant51 whereas the increase in the regular attendance of girls in comparison schools is non￾significant. In the case of boys, changes in attendance from baseline to midterm in both intervention and comparison schools was not found to be statistically significant. Analysis at disaggregated level (upazila) suggests that, there is an increase in the number of students who regularly attended school (88%) in Kutubdia from baseline (79%) and the difference was found to be statistically significant52. No significant difference in case of Ukhiya was observed. Table 3: Students' attendance in sample intervention and comparison schools Baseline Mid-term Intervention Comparison Intervention Comparison Total Boys Girls Total Boys Girls Total Boys Girls Total Boys Girls 83.3% 84.4% 82.4% 91.8% 89.8% 93.8% 85.2% 82% 88.3% 93.6% 92.7% 94.6% 51 Significant difference (@5% level of significance) between baseline and midterm values for girls’ attendance in intervention schools 52 Significant difference (@5% level of significance) between baseline and midterm values for student attendance in Kutubdia schools Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 24 114. During discussions with parents, SMC members and teachers, the provision of biscuits, bags, lunchboxes and water bottles by WFP in intervention schools emerged as major drivers for increased student enrolment and attendance. Survey findings also suggest that 78 percent teachers in Ukhiya and 71 percent in Kutubdia perceive that attendance will drop by 25 percent if biscuit distribution is discontinued. Further 17 percent teachers in Ukhiya and 29 percent in Kutubdia expressed that attendance will drop by more than 25 percent. (Figure 3) Number of students receiving deworming medication(s) 115. All the sample intervention and comparison schools, reported administering deworming tablets twice between July 2018 and June 2019. 19,370 students (97%) in the sample intervention schools have received deworming medications. Using midline value as reference, it can be inferred that 46,258 students across 146 schools have received deworming medications. Average number of school days missed by each student due to illness (for each school and in aggregate) 116. There is no change in the average number of school days (2 days) missed by a student due to illness from the baseline in intervention and comparison schools. Amongst absentees, no change in the proportion of students citing illness as the reason for absenteeism was observed in the intervention and comparison schools in the midterm as compared to baseline. Promote Improved Nutrition and Improved Health 117. WFP has promoted nutrition in schools by supporting establishment of vegetable gardens in schools, facilitating training of Little Agriculturalists, and sensitising students and teachers about improved nutrition through Healthy Meal Preparation Days). 118. Analysis of primary data suggests that 50 percent of the schools in Ukhiya and 33 percent in Kutubdia have vegetable gardens. Vegetable gardens were not found in any of the sample comparison schools. The analysis further indicates that 55 percent of the sample schools in Ukhiya and 75 percent of sample schools in Kutubdia had Little Agriculturists. None of the sample comparison schools had Little Agriculturists. All little agriculturists in sample schools reported 6% 78% 14% 3% 0% 71% 21% 8% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% No consequence, attendance will remain the same Attendance will drop by 25% Attendance will drop between 25% and 50% Attendance will go down by more than 50% Ukhiya Kutubdia Figure 3: Perception of teachers (of intervention schools) on impact on students’ attendance if provision of biscuits stopped Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 25 receiving training from WFP on aspects related to cultivation practices, setting up and maintenance of vegetable gardens, and nutritional value of fruits and vegetables. During FGDs with students it emerged that awareness about Little Agriculturalists is low. Discussions with those who were aware, highlights that low awareness is on account of limited opportunities /availability of a platform for Little Agriculturists to disseminate messages and utilize their training in influencing other students about nutrition and good dietary practices. The observations from the discussions can be corroborated through quantitative data that highlights awareness about Little Agriculturists limited to 44 percent students in Ukhiya and 32 percent students in Kutubdia. 119. Further, only 44 percent of the students in the sample intervention schools reported that they were aware of a Healthy Meal Preparation Day. The reason for low awareness can be understood from the discussions with students where it was revealed that Healthy Meal Preparation Day was often merged with a “Picnic” Day. While recognition of Healthy Meal Preparation Day as a separate event did not emerge from discussions, the students did recall teachers talking about nutrition practices during the picnic day. Number of individuals trained in child health and nutrition as a result of USDA assistance 120. The monitoring report indicates that 1,480 individuals were trained in child health and nutrition. The findings of the survey highlight that 32 teachers and 14 head teachers across 30 intervention schools have received training on child health and nutrition through multiple sources53. Out of these, 19 teachers (59%) and 6 head teachers (43%) were trained in child health and nutrition specifically through USDA assistance. Considering the survey findings as the basis, it can be estimated that about 126 teachers and headmasters across the target schools may have been trained in child health and nutrition as result of USDA assistance. Similarly, 28 parents reported receiving training/sensitisation on child health and nutrition from WFP. Estimation (through extrapolation to all the target schools) based on the survey findings indicates that 17,740 parents may have received training from WFP on child health and nutrition. Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance 121. All the head teachers and teachers trained through USDA assistance, reported demonstrating the use of child health and nutrition practices. Considering the survey findings as the basis, it can be estimated that 121 head teachers and teachers are demonstrating the use of child health and nutrition practices across 146 schools in the intervention area. 122. The status of demonstration of child health and nutrition practices by parents were assessed using two parameters: handwashing at critical times and dietary diversity. Out of the parents who were trained by WFP in child health and nutrition, 17 parents (61 percent) demonstrated handwashing practices at critical times54 in intervention schools (Refer table4 below). Table 4: Parents trained by WFP who are aware and demonstrate handwashing practices at critical times in intervention schools 53 GoB, WFP, NGOs 54 The numbers reported here are for those who practice handwashing on 3 or more out of 6 critical times Ukhiya Kutubdia Total Intervention Awareness of handwashing by parents (during critical times) Less than 3 times reported 9 1 10 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 26 123. A significant55 difference between the baseline (4.2) and midterm (4.96) values with regard to the mean dietary diversity score (DDS)56, for children in sample intervention schools was observed. Similarly, a significant difference between the baseline (4.15) and midterm (5.49) values with regard to the mean dietary diversity score (DDS) for children in sample comparison schools was observed. One of the factors contributing to a higher DDS from the baseline in the intervention area is increased dialogue around nutrition leading to improved dietary diversity among students and parents. This can also be understood based on the discussions with parents in the intervention areas, wherein it was mentioned that children have started talking about the nutritional qualities of food and demanding food based on its nutritional qualities. A relatively higher DDS in the comparison areas as compared to intervention areas can be attributed to certain exogenous factors (outside of the purview of the programme) at play in Ukhiya and Kutubdia that have limited the adoption of improved dietary practices. These external factors include lack of availability of diverse food items in Kutubdia due to its geographical location and the surge in prices of various food items due to the Rohingya crisis in Ukhiya. If these factors are addressed, the intervention areas would also witness a similar increase in the DDS as the comparison areas. 124. It is interesting to note that no gender discrimination was reported with regard to access to food at home, post consumption of biscuits at school. This can also be validated by dietary diversity scores where no significant difference57 was found between the scores of boys and girls. Exposure to different nutritional messages provided to children in school was highlighted as the primary reason for increased awareness about nutrition among students. They further elucidated that while they have been consuming vegetables and fruits earlier as well, it is only now that they are aware about the nutritional qualities. The translation of increased awareness to improved DDS (in intervention schools as in case of Ramu) is limited due to external factors specific to Ukhiya and Kutubdia. Ukhiya has the challenge of a surge in food prices due to the Rohingya crisis making it difficult for the parents (with limited resources) to fulfil the demand for nutritious food by children. Kutubdia, faces the challenge of inadequate local production of diverse food items, due to the salinity of water thereby, affecting the availability of diverse food items and limiting consumption to fish and rice only. 55 Significant difference at 95% confidence interval (2-tailed) between baseline and midterm values of DDS in sample intervention schools 56 Feed the Future Guidance (2014) suggests grouping food items into 10 food groups. However, based on discussions with WFP, classification of food was done across 7 categories. Therefore, the scale of food options is that of 7 in this case. 57 Non-significant difference at 95% confidence interval (2-tailed) between female and male values of DDS 4 to 6 times reported 18 0 18 Total 27 1 28 Demonstration of handwashing by parents (who practice handwashing on 3 or more out of 6 critical times ) Less than 3 times reported 10 1 11 4 to 6 times reported 17 0 17 Total 27 1 28 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 27 …aamar chele aamake badite school jemon baagan aache shemni baagan korte boleche. Oo bollo school e to aamaye fol bhaag kore khete hoye. Badite gaach thakle puro fol ta aami hi khete parbo… …my son asked me to grow a garden in house like he has in his school. He said that he has to share the fruits grown in school’s garden. But if there’s garden in house he can have the whole fruit by himself… -mother in Ukhiya Number of educational facilities (i.e. water systems and latrines) rehabilitated/constructed as a result of USDA assistance 125. All the sample schools have received book racks for classroom libraries, almirahs, school bags, water bottles and tiffin boxes through USDA assistance. 7 schools (4 in Ukhiya and 3 in Kutubdia) from the sample schools reported receiving support from WFP for rehabilitation/construction of toilets and storage facilities. 6 schools (2 in Ukhiya and 4 in Kutubdia) from the sample reported receiving support from WFP for rehabilitation/construction of library, playground and classroom. 126. 3 intervention schools from the sample reported receiving support from WFP for drinking water facilities. 5 schools reported receiving support from WFP for water supply in toilets and 5 schools reported receiving support from WFP for water supply in the school garden. Number of schools using an improved water source Number of schools with improved sanitation facilities 127. Based on the assessment of requirements articulated by schools, WFP provides equipment (pumps and motors) to improve the water systems in schools. 13 percent (4 schools) of the sample intervention schools reported receiving support through USDA assistance for rehabilitation or construction of any water system. Availability of improved water sources58 was observed in all the sample intervention and comparison schools as compared to 80 percent schools in the baseline. Tube well was observed to be the primary source of drinking water for both intervention (90%) and comparison (92%) schools. However, from the discussions with teachers and students, salinity of water emerged as a major concern in Kutubdia. While it was observed that some students carry water bottles from home, most of the teachers and students drink saline water that poses a major health risk. 128. Number of schools with improved sanitation facilities have been determined on the following three parameters: functionality of toilets, whether it is locked or not, and availability of water. 97 percent of the intervention schools (29)59 reported having functional toilets as compared to 11 schools during baseline. Availability of separate functional toilets for boys and girls was observed in 50 percent of the sample intervention schools. Of the total number of functional toilets in intervention schools, 28 percent (24 toilets) were locked. Students during discussions reported that these toilets are exclusively used by teachers and locked for students. Moreover, 7 percent (6 toilets, 5 in Ukhiya and 1 in Kutubdia) of the functional toilets were found to have no water facility. In the comparison schools, 27 percent (23 toilets) of the functional toilets were observed to be locked. Availability of water facility in all the functional toilets in the comparison schools was 58 Improved water sources as per WHO: https://www.who.int/water_sanitation_health/monitoring/water.pdf 59 The only sample intervention school having no functional toilets was Paschim Ratna GPS in Ukhiya. This was due to a delay in the construction of a new WASH block by BRAC. The children reportedly used the toilets of houses close to the school. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 28 observed. Availability of separate functional toilets for boys and girls was observed in 75 percent of the sample comparison schools. Percent of students who can identify at least three key health and hygiene practices 129. 39 percent of the students in both the sample intervention schools and comparison schools identified three or more health and hygiene practices. A comparison with baseline values indicates higher levels of awareness among students about health and hygiene practices in the sample intervention schools (baseline-25%) as compared to comparison schools (baseline- 35%), however the difference is not statistically significant. Analysis at a disaggregated level, indicates a significant60 difference in awareness among students about health and hygiene practices in Ukhiya was higher (46%) as compared to Kutubdia (29%). Similarly, a statistically significant difference61 from baseline to midline in the awareness about health and hygiene practices among boys (40% from 16%) and girls (39% from 13%) was observed. 130. Further, two out of three students in the sample intervention and comparison schools identified handwashing as a key health and hygiene practice. During discussions, parents (from both intervention and comparison schools) indicated increased adoption of handwashing practices (before meals and after use of toilet) by their children during the past one year. During visit to schools for data collection, students were also found washing hands before meals. Evidently, there is an increase in the adoption of key health and hygiene practices among students. Evidently SFPs implementation has supported promotion of health, hygiene and nutrition knowledge and sanitation condition of the pre-primary and primary schools. Number of 'Little Doctor' students supported by WFP 131. The Little Doctors programme has been implemented by GoB across all intervention and comparison schools. WFP contributes to this GoB programme through providing (a) training/orientation of Little Doctors on health, hygiene and sanitation practices and (b) aprons for easy recognition of little doctors. These aprons are worn by the students during health check￾ups and deworming days. 132. Across all the sample schools, a total of 430 Little Doctors (256 in Ukhiya, 174 in Kutubdia) and 189 Little Doctors in Ramu were reported as identified. On an average, there were 14 Little Doctors per intervention school and 9 Little Doctors per comparison school. Of the Little Doctors in intervention schools, all had received training from WFP and only 45 percent of the Little Doctors in comparison schools had received training from the government or any other source. 133. A significant difference62 in awareness about Little Doctors among students between intervention and comparison schools was observed. The primary data indicates that 88 percent of the students in intervention schools and 30 percent of the students in comparison schools are aware about little doctors. 134. The findings of survey were corroborated during FGDs with students (from intervention schools) wherein they were able to cite instances of Little Doctors disseminating health and hygiene messages (including safe handwashing practices, cutting nails, personal hygiene and cleanliness 60 Significant difference at 95% confidence interval (2-tailed) between Ukhiya and Kutubdia’s midterm values for awareness among students about health and hygiene practices 62 *Significant difference (@5% level of significance) between intervention and comparison values of awareness about Little Doctors. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 29 of uniform) during school activities such as morning assemblies. They also mentioned that Little Doctors wear aprons and assist the teachers during deworming activities. Parents during discussions also mentioned hearing about Little Doctors from their children. No such validation emerged out of the discussions with parents and students in Ramu. It can therefore, be inferred that the visibility of the Little Doctors in intervention schools is higher than in comparison schools. Number of individuals trained in safe food preparation, commodity management and storage as a result of USDA assistance Percent of storekeepers who can identify at least three safe storage practices Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance 135. WFP has provided trainings to headmasters, teachers and storekeepers on safe storage practices in all intervention schools. 95 individuals (27 storekeepers, 27 headmasters and 41 teachers) from the sample intervention schools reported receiving training on safe storage practices. A higher proportion of male teachers and storekeepers (71%) as compared to female teachers and storekeepers have received training on safe storage practices. 97 percent (92) of the trained teachers and storekeepers demonstrated the use of safe storage practices. All the trained storekeepers in Ukhiya and Kutubdia were able to identify at least three safe storage practices. Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance Number of school-age children receiving daily school meals (packet of fortified biscuits as snacks) as a result of USDA assistance 136. As per the semi-annual report of WFP (October 2018 to March 2019), 2,488,914 biscuits have been distributed across 146 schools in the intervention area. Primary study reveals that 6,427 students in the sample schools of Ukhiya and 3,249 students in those of Kutubdia have received biscuits in the past six months (October 2018 to March 2019). Average biscuit distribution days in a month have been 16 days for 6 months in the sample intervention schools. 137. Further, 61 percent of the schools in Ukhiya and 75 percent schools in Kutubdia reported maintaining a buffer stock. Out of those schools that maintain a buffer stock, 73 percent schools in Ukhiya maintain stock for 7 days, and 56 percent schools in Kutubdia keep stock for 7 days. 44 percent schools in Kutubdia also reported keeping stock for 15 days. During discussions, headmasters and store keepers in Kutubdia mentioned that since it is a remote island, keeping stock for longer periods ensures regular and timely distribution of biscuits. Number of teachers, parents and school management committee members attended the community mobilization workshops 138. 93 percent of the parents in sample schools reported being aware of the Read-Play festival. 55 percent of those (aware), reported attending the festival. However, during discussions with headmasters, only 2 or 3 schools reported having this festival. In most of the sample intervention schools, this Read-Play festival was merged with the Annual Day of the school where along with cultural events, some poetry/recitation competitions were held. The higher awareness about the Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 30 festival amongst parents is possibly because of the festival being clubbed with the Annual day which is well-known to them. 139. WFP through its implementing partners facilitate the conduction of Grade I reception of students or “nobinboron”. During this event, the newly enrolled children along with their parents are given flowers and tiaras (ribbons) as a welcoming gesture. Cultural programmes like singing, dancing or recitation are also conducted. SMC members also participate in this event. They talk about the importance of education and request parents to send their children to school every day. None of the comparison schools reported conducting a similar event. 63 percent of parents from the intervention schools reported being aware about Grade I reception day. 65 percent of those (aware) reported attending the same. Importantly, during the FGDs, parents admitted that social recognition through such events has influenced parents to enrol their children in school. Foundational Results 140. WFP provided support to MoPME in the form of facilitating inter-ministerial joint exposure visits for sensitising representatives of different ministries on the relevance of the school feeding programme. Discussions with the Bangladesh National Nutrition Council (BNNC) representatives revealed that post these visits, they advocated with various ministries for undertaking interventions to support school feeding and persuaded MoPME to finalise the National School Meals Policy (NSMP). Additionally, WFP has also facilitated a Campaign for Popular Education (CAMPE) targeting stakeholders from the government and civil society to enhance their understanding on nutrition-sensitive interventions. Discussions with MoPME, Ministry of Agriculture and the BNNC indicate that these efforts have encouraged them to initiate a discourse on provisioning of school meals and explore opportunities to mainstream complimentary activities within the existing educational framework. These consultations facilitated preparation of nutrition sensitive NSMP and GoB’s commitment to initiate school meals programme in 16 upazilas, reaching 410,238 children in 2,166 primary schools. 141. WFP also provided support to MoPME in coordination with various ministries and development partners at various stages for the development of the NSMP. One of the important support measures from WFP recognised by the MoPME is its contribution to the technical committee established to set the minimal nutritional requirement63 for school meals. WFP also provided extensive technical assistance to DPE for NGO partner selection in 11 upazilas (WFP handed over 9 upazilas to the government) as well as NGO partners’ performance assessment for the Government-assisted 93 upazilas. Discussions with the Project Director, DPE highlight that these interventions resulted in a financial commitment (USD $19.5 million) from the Government of Bangladesh towards the implementation of NSMP. 142. As part of the implementation of NSMP, a hot cooked meal for five days a week and high energy fortified biscuits (once a week) will be served to students. The meal will be prepared with eggs, fortified rice, lentils and oil and freshly procured vegetables to ensure variety, taste and minimum nutritional requirements as defined in the policy. Additionally, it is envisaged that the selection of the menu will be done in consultation with parents and the local community. 63 A minimum 30 % of the daily energy requirements of pre- primary and primary school children should be provided through school meals. A minimum 50 % of the recommended micronutrient requirements should be covered by the school meal for half a day school Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 31 143. In order to scale up the implementation of the policy, responsibilities at different levels have been defined. At the national level, the MoPME will be responsible for corrections and modifications in the policy as well providing direction in the implementation of the policy. The DPE, as the implementation agency, is expected to prepare an operational plan and undertake the responsibility for implementation, supervision and monitoring of the programme. At the division, district and upazila levels, the offices of the Divisional Deputy-Director (Primary Education), District Primary Education, and Upazila Education will be involved in implementing the NSMP along with the Upazila Chairman and Upazila Nirbahi Officer. 144. Considering the findings above, it can be inferred that the SFP is on the right track to carry out all the planned activities and reach set targets. Further, it is worth mentioning that the evaluation has not come across any unintended effects of the intervention on human rights and gender equality. 2.3. Efficiency Evaluation Question 6: What was the efficiency of the processes programme, in terms of transfer cost, cost/beneficiary, logistics, and timeliness of delivery? Evaluation Question 7: How efficient the operation and approach are in terms of capacity building of government toward eventual handover? Biscuit Distribution 145. Through the biscuit distribution activities, WFP has been able to cover all the target schools. Demand forecasting for biscuits is based on the total enrolled children in a school in different grades and grade-wise sum of attendance of children in the school. Schools submit a Monthly Utilization Report (MUR) of the receipt and utilization of the biscuits for the reporting month. This provides a basis to the implementing partners for forecasting the demand of biscuits for the next month. Accordingly, the consolidated demand of all the school under the two upazilas are submitted to WFP for supply of stocks to the school. 146. All sample schools in Ukhiya and Kutubdia reported that the distribution of biscuits to students is done on a daily basis. During discussions, storekeepers indicated that in the case of their absence, the responsibility of biscuit distribution is taken up by the teachers and the headmaster of the school in order to ensure regular distribution. The discussions with students and parents also confirmed that all the students receive biscuits 6 days a week. 147. 4 out of 5 sample schools in Ukhiya and Kutubdia confirmed receiving stock of biscuits on a monthly basis. Other schools reported receiving biscuits fortnightly. All the sample schools reported that they receive the stock before any fresh requisition is made. With regard to stock out, all except one school each in Ukhiya and Kutubdia (one instance) confirmed that there are no instances of stock out. For these two schools, stock out was managed by the implementation partners by replenishing the stock from the buffer stock of neighbouring schools. Based on the findings of the primary survey, it can be inferred that the process of biscuit distribution is efficient. Monitoring and reporting system 148. MoPME manages the SFP at the central level and is supported by a PMU which facilitates and establishes systems and processes for the monitoring and regular feedback of interventions. The centralized MIS provides data that is decentralized up to the level of the Upazila Education Office. Implementation partners collect MURs from schools and enter the data into the MIS at the upazila Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 32 level for further consolidation. This reporting system was observed to be well established and operated efficiently. 149. Other programme interventions such as the promotion of vegetable gardens and classroom libraries are primarily monitored by the staff of the implementation partners. The intensity of monitoring and frequency of engagement and feedback was observed to be more for classroom libraries as compared to vegetable gardens. RtR has adopted their global practice of monitoring the classroom library interventions (particularly book transactions in classroom libraries and tracking reading abilities of students) using their own tools and methods. The random check of library register and discussions with students indicate that the system of monitoring of book transactions and tracking reading abilities is well established. 150. Data for monitoring reports is collected by the implementation partner staff at the school level. This data is consolidated at the upazila level by implementation partner and shared with WFP for preparation of quarterly reports. The quarterly reports are shared with GoB and RtR. Discussions with RtR representatives revealed that, this monitoring mechanism involves too many steps and lacks coordination between implementing partners and RtR. 151. Further, discussions with implementing partners’ highlight that while they apprise the Upazila Education office regularly on the progress of implementation of SFP, there is irregularity in conduct of joint monitoring visits (officials of District Education Office, Upazila Education Office, Implementation partner and WFP). Agreement of all the stakeholders on a common day for visit owing to their commitments emerged as the major challenge in scheduling the joint monitoring visit. Therefore, while the processes of the programme are being implemented efficiently, the delay in submission of quarterly reports and timely action by RtR on any observations highlights the need for further improving the efficiency of the monitoring system. 152. Based on discussions with WFP, it was agreed that the efficiency of operations and approach in terms of capacity building of government toward eventual handover would be evaluated as part of end term evaluation. Efficiency of implementing partners 153. The implementation partners’ outreach and connectedness with the community and proactiveness plays an important role in ensuring their efficiency. On this aspect, Muslim Aid’s capacity was observed to be higher as compared to YPSA. This can be validated by an instance observed during the field visit where in order to avoid delays in conducting trainings due to logistic issues, Muslim Aid proactively arranged the training sessions in their own office. During the discussions with Upazila Education officers in Ukhiya and Kutubdia, the support received from the Muslim Aid and YPSA in implementation of SFP was acknowledged. However, it also emerged during discussions that Muslim Aid is perceived to be more proactive in its approach, supportive and better connected with the community as compared to YPSA. Discussions with Upazila Education officer (UEO), Ukhiya revealed that the department is always consulted and informed about the activities that Muslim Aid undertakes. UEO also admitted that Muslim Aid shares good rapport with the schools. UEO further mentioned that, this kind of transparency and trust is always required for such partnerships for better implementation of the interventions. The timely response to the programme needs, measures taken to ensure smooth implementation and proactive engagement with the government and the community reflects the efficiency of implementing partners. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 33 2.4. Impact Evaluation Question 8: What is the intermediate impact of the project? Have there been any unintended outcomes, either positive or negative? What internal and external factors affect the project’s achievement of intended results? 154. Due to limited duration (nine months) of implementation prior to commencement of data collection (July 2019) for MTE, the evaluation does not highlight/report significant change/impact of the programme’s activities. However, the evaluation is still able to highlight an increase in understanding among the community on the benefits of education, increase in the skills and knowledge of the teachers and administrators and improvement in fluency and comprehension among students. Further, findings also suggest that there has been an increase in the overall knowledge about good nutrition, health, hygiene and sanitation as well as safe food preparation and storage practices. These positive changes together constitute the intermediate impact of the program. 155. The plastic wrappers of biscuits are often burnt in the open, posing a threat to the environment. Creation of this plastic waste for which there is no proper mechanism of responsible disposal is an unintended outcome of the SFP. A positive unintended outcome has been the rise in aspirational levels of students as a result of their identification and recognition as Book Captains and Little Doctors. This has motivated and influenced other students to demand quality of education to fulfil their aspirations. 156. There are many internal and external factors that may affect the achievement of intended results by the end of the project. For instance, the acceptance of implementing partners amongst teachers, GOB officials (especially DEOs and UEOs) and the community is an internal factor that can positively affect the progress of the program. Similarly, the approval of the School Meal Policy is an external factor that will mainstream nutrition and hence contribute to the likelihood of the project to reach its intended outcomes. However, a rise in conflict or tensions within the humanitarian context of Ukhiya is likely to adversely affect the outcomes of the SFP. Due to the salinity of water, there is negligible agricultural production in Kutubdia. This can also adversely affect project outcomes as a rise in prices of the food affects the dietary diversity score of the local population. Further, any change in the implementation mechanism (internal factor) of the project is bound to affect the achievement of project outcomes. Improved Literacy of School-Age Children 157. The findings in the effectiveness section highlight improvement in reading and comprehension skills of students in sample intervention schools as compared to the baseline. Discussions with parents and students also indicated, that students have started reading out stories at home to their parents. Enthused by the demonstration of reading abilities by their children, parents have started encouraging and supporting them in enhancing their reading and comprehension skills. During the discussions, parents also mentioned that through PTA meetings and other platforms for engagement with school authorities their participation has also increased. They have started expressing their concerns that may hamper the learning of their children. “…amaar meye bole je oo ooi shada poshak ta shara din porte chaye, kano ki ota to school shesh hole niye naye. Tai aami oke bollam jodi tui eei poshak shob shomay porte chash tahole bhalo kore podha kor aar bado hoye doctor ho. Taholei tui sharakkhun eei shada poshak porte parbi…” Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 34 “…my girl said that she wants to wear the white apron whole day but she had to return it to school after school hours. To which I said if you want to wear that all the time then you should study really well and become a doctor. Only then you’ll be able to wear the white apron all the time…” - one of the Little doctor’s mother in Kutubdia 158. Students during FGDs mentioned that initiatives such as Book Captains and Little Doctors has led to the recognition of students’ capabilities by their peer group and parents. The identification and recognition of students as achievers has majorly influenced other students. They now aspire to achieve similar recognition in the next academic year and have started reading books as they believe that demonstration of reading skills is an important criterion for gaining recognition. Evidently, SFP through its interventions has influenced the students to pursue education, demonstrate their abilities and has nudged their parents to demand for quality education in order to fulfil their aspirations. …baniyeche aar oder kano na, tokhun aamra oder boli ki aamra bhalo kore podte pari tai hoyechi. Jodi tomrao book captain hote chao tahole tomrao bhalo kore poda koro… …Our friends also want to become book captains. So, when they ask us why we were chosen to be book captains and not them, we tell them that we read well and are hence chosen. If you too want to be a book captain then you should read well too… - group of three book captains in Ukhiya 159. Further, as emerged through the findings of the survey, there is an increase in the demonstration of new teaching techniques and tools by the teachers. The students and parents have also acknowledged that the adoption of new techniques and tools have aided the learning process. An increase in attentiveness and attendance of the students was also observed as compared to the baseline. Discussions with SMC also revealed an increasing discourse on improving the quality of education during the meetings efforts towards encouraging parents to ensure regular attendance of students. Evidently, SFP has nudged the school administration to respond to the growing demand from parents on improving the quality of education. 160. Considering that there is a growing realization on the importance of education and increasing demand for quality of education and the momentum that has been created, it can be inferred that the project is on track to achieve its strategic outcome on improving literacy of school age children. Increased Use of Health and Dietary Practices 161. As highlighted in the effectiveness section, there is an increase in DDS in sample intervention schools from baseline to midline. Increased awareness about nutrition through initiatives under SFP, resulting in increased demand from students for nutritious food and efforts from parents to fulfil the demand is one of the important factors for the increase in DDS. Further, increased awareness about health and hygiene practices (handwashing at critical times) leading to increased adoption of these practices was also observed. 162. Considering an increased awareness and adoption of health and dietary practices by the students and parents, it can be inferred that the project is likely to achieve its strategic outcome with regards to increased use of health and dietary practices by the end of the programme. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 35 2.5. Sustainability Evaluation Question 9: Is the programme sustainable in the following areas: strategy for sustainability; sound policy alignment; stable funding and budgeting; quality programme design; institutional arrangements; local production and sourcing; partnership and coordination; community participation and ownership? Evaluation Question 10: What progress has the government made toward developing and implementing a nationally owned school feeding programme? Evaluation Question 11: Are local communities fully involved in and contributing toward school feeding and education activities? 163. The approval of the National School Meal Policy by GoB and allocation of USD $19.5 million (during the period April- June 2019) for the implementation of programme in 16 upazilas (stage I, post January 2020) provides evidence of the government’s commitment and willingness to take ownership of the programme. As per the plan as in April 201964, hot cooked meals will be provided under the NSMP for five days a week and high energy fortified biscuits will be served once a week. Further, efforts will be made to ensure the minimum nutritional requirements as per the recommendation of the technical committee of the MoHFW. In addition to this, fortified rice, lentils and oil will be used and vegetables will be freshly procured. Further, the menu will be selected in consultation with parents and the local community. 164. In order to scale up the implementation of the policy, responsibilities at different levels have also been defined. At the national level, the MoPME will be responsible for corrections and modifications in the policy as well providing direction in the implementation of the policy. The DPE, as the implementation agency, is expected to prepare an operational plan and undertake the responsibility for implementation, supervision and monitoring of the programme. At the division, district and upazila levels, the offices of the Divisional Deputy-Director (Primary Education), District Primary Education, and Upazila Education will be involved in implementing the NSMP along with the Upazila Chairman and Upazila Nirbahi Officer 165. The operational plan was not prepared till the time survey was conducted. During the discussions with representatives, MoPME and DPE it emerged that they will require technical assistance from WFP in formulating the operational guidelines for the National School Meal Policy. These guidelines would also describe the institutional arrangements and strategy for implementation. 166. Discussions with representatives MoPME and DPE indicate that while there is intent of GoB to roll out the NSMP across all upazilas, the subsequent scaling up would depend on the learnings from implementation of Stage I (roll out in 16 upazilas). As a result, during MTE, it is difficult to comment on sustainability of institutional arrangements for implementation of SFP and roll out of SFP across all upazilas. 167. WFP’s support on aspects such as sensitisation of representatives of different ministries on the relevance of school feeding programme and its activities (through inter-ministerial joint visits), facilitating inter-ministerial consultations on minimum nutritional requirement for school meals, coordination with various ministries and development partners at various stages of policy development is well recognised by the MoPME, Ministry of Agriculture and the Bangladesh National Nutrition Council. 64 WFP and MoPME note on the National School Meal Policy 2019 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 36 168. The facilitation of inter-ministerial joint visits by WFP for sensitising ministerial representatives on the relevance of the SFP has established interests of various ministries in mainstreaming nutrition into their activities. During the discussions with Representative Directorate of Agriculture Extension, it emerged that the directorate has prepared a plan for training the farmers (specific focus on women farmers) on good agricultural practices and production of vegetables (considering nutritious requirements) required to support school meals for students in nearby schools. WFP, on the other hand would facilitate technical support to DPE on cooking of nutritious food through the Bangladesh Institute of Research and Training on Applied Nutrition (BIRTAN). Collaboration between the Ministry of Agriculture and Ministry of Primary and Mass Education would set an example for other ministries to mainstream nutrition into their programmes and build a case for coordinated efforts to achieve the objectives of NSMP, post withdrawal of support from WFP. 169. A continued dependence of MoPME on WFP support to facilitate procurement of vendors for implementation of various activities under the programme was observed. During discussions, it emerged that the government prefers procurement by WFP in order to avoid any political or interpersonal influence during the procurement process. The lack of willingness of the government to independently undertake procurement can act as a hindrance in sustaining the SFP post withdrawal of technical assistance by WFP. 170. Discussions with representatives DPE and teachers highlighted shortage of teachers across the project intervention areas. Further, as the new techniques methods and tools introduced through SFP have not yet been mainstreamed with the government teacher training modules and curriculum, the adoption of new teaching methods on a regular basis is considered as an additional burden. Mainstreaming of newer teaching methods and tools, if not addressed, can limit the sustainability of the intervention. The observations from Ramu provide an instance wherein the improvement in reading and comprehension abilities among children during the implementation of READ programme could not be sustained post the closure of the programme. 171. The findings of the evaluation reflect that the capacities of School Management Committees have been built in order to encourage the community’s involvement in school feeding and education activities. SMCs are now increasingly contributing to biscuit distribution and working towards increasing enrolment and attendance of students. However, their involvement needs to be enhanced. Moreover, the participation of mothers in celebration of events in school highlight their contribution in supporting implementation of SFP. An effective community engagement and capacitated SMC can act as drivers to ensure sustainability of SFP. During the discussions at the validation workshop in Cox’s Bazar, the representatives from the community clearly articulated their interest in contributing towards school feeding and education activities. The DEO and UEO also expressed their interest in independently implementing the interventions undertaken through SFP. They will however, require financial assistance from GoB or WFP during implementation. Evidently, despite the intent (of the community and the other local stakeholders in Cox’s Bazar) the actual implementation of the interventions is dependent on continued financial assistance from the GoB or WFP. 2.6. Observations of the participants in Cox’s Bazar and Dhaka on the findings of MTE during the validation workshop 172. All the four groups validated the findings of MTE in terms of what is working well with SFP. This included (i) timely and regular availability of biscuits; (ii) support provided to Little Doctors, Book Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 37 Captains and Little Agriculturists and its effectiveness; (iii) training of teachers and its effectiveness; (iv) availability and improvement of sanitation facilities; (v) conduct of events such as health meal preparation day, quiz competition and community participation in these events. The groups also articulated areas of improvement for consideration in the remaining period on aspects such as (i) introducing innovative teaching methods to enhance learning of weaker students; (ii) introducing variants (taste) of biscuits; (iii) conducting joint monitoring visits regularly; (iv) promoting participation of women in SMCs and (v) supporting mid-day meal for students. 173. The participants in Dhaka also validated the findings of MTE. The discussions on importance of recommendations for SFP and their feasibility in implementation has been highlighted in section on recommendations. 3. Conclusions 174. The SFP interventions and strategy are aligned with the priorities of GoB as highlighted in NEP 2010, the National Nutrition Policy (NNP) 2015, the NSSS 2015 as well as NPAN-2 as they promote and support the various objectives on nutrition, health, hygiene and education mentioned as part of these government policies. 175. Further, since the SFP activities address the felt need for interventions to increase children’s interest in education related activities and to improve their performance, aims to address challenges related to awareness around attendance, enrolment and literacy and provides platforms to encourage the same while also contributing towards strengthening GoB’s existing initiatives, the SFP adequately targets the right people with the right type of assistance and was therefore, found to be relevant. 176. Through SFP, WFP has made consistent efforts in sensitising representatives of different ministries of GoB on the relevance of school feeding programme and its activities, and building consensus at the GoB level on minimum nutritional requirement for school meals. These efforts contributed to the formulation and approval of the National School Meal Policy 2019. The assistance provided by WFP in terms of building capacities at MoPME, DPE level and strengthening inter-ministerial coordination (MoPME with other ministries) is well recognised by the MoPME, Ministry of Agriculture and the Bangladesh National Nutrition Council. 177. At the programme level, the higher midline values as compared to baseline values (for 30 out of 35 indicators) indicates that the programme is on track to carry out all planned activities and achieve the intended results. Comparison of midline values and baseline values for intervention and comparison schools is provided in Annex S. 178. The evaluation findings highlight an increase in understanding among the community on the benefits of education, increase in the skills and knowledge of the teachers and administrators, an increase in the demonstration of new teaching techniques and tools by the teachers and improvement in fluency and comprehension among students, an increase in the overall knowledge about good nutrition, health, hygiene and sanitation as well as safe food preparation and storage practices along with a rise in aspirational levels of students as a result of their identification and recognition as book captains and little doctors. This has further motivated students to demand quality of education to fulfil their aspirations and promoted positive nutrition, health and hygiene practices. This serves as evidence that the programme has been effective and that the project is on track to achieve its strategic outcome on improving literacy of school age children and increased use of health and dietary practices by the end of the programme. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 38 179. There has been a significant increase in the fluency and comprehension abilities of students in intervention schools. Therefore, the program processes has been effective in (a) ensuring that all the students in the class get to read story books as per their abilities (b) motivating and influencing (from their peers) students to adopt reading habits and enhance their reading abilities. The parents have proudly expressed an intent to support their children in pursuing quality education. The perceived importance of education has translated in increased attendance and enrolment of students in intervention schools as compared to baseline. 180. Teachers have confirmed receiving training on new teaching methods from WFP. However, adoption of new teaching techniques is perceived by the teachers as an additional burden due to work overload, thereby limiting its practice. 181. The process of demand forecasting, supply and distribution of biscuits on a regular basis was found to be efficient. Additionally, the process of reporting progress on various activities in ensuring timely submission of monitoring reports was also found to be efficient. 182. Biscuits are distributed regularly to both boys and girls and there is no gender-based discrimination. Despite an increase in demand for and awareness about quality education, the distribution of biscuits continues to be a major influencing factor for parents to send their children to school and for students to attend the school. 183. The participation of SMC members in SFP interventions and supporting schools in improving infrastructure, encouraging enrolment and regular attendance of students has increased. The gender balance in SMC however, continues to be skewed in favour of men. 184. Mothers narrating instances of participation in school activities such as healthy meal preparation day and existence of parent teachers’ association provides evidence of increasing community engagement in supporting school in implementation of SFP activities. This has the potential to act as an enabler for sustainability. 185. Joint Monitoring Visits (by WFP staff, GoB officials at district and upazila level and implementing partners) are not being conducted regularly which has a bearing on timely review and supervision of the programme implementation activities. 186. Realising the importance of SFP, there is intent (of the GoB officials at national and district level, teachers and the community) on independently taking forward interventions, the actual implementation however, is dependent on continued financial assistance from the GoB or WFP. The approval of NSMP and allocation of budget by GoB demonstrates its commitment and willingness to take the ownership of the programme and can be considered as an achievement of important milestone towards ensuring sustainability of SFP. 187. GoB’s efforts towards ensuring adequate availability of teachers and mainstreaming new techniques with the existing government teacher training modules and curriculum would enable sustainability of achievement of learning outcomes, post the withdrawal of support from WFP. 188. Levels of engagement and rapport with the community as well as with the government officials established by the implementing partners has helped in timely implementation of the planned activities under the programme. It is therefore, essential that the current implementation structure is continued till the end of SFP. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 39 4. Recommendations Sl. No. Recommendations Rationale Proposed actions Type Timeframe Priority (based on rating of recommendations on scale of 1-565 by GoB representatives during validation workshop in Dhaka66) Proposed action Policy Advocacy (National Level) 1. Provide technical assistance to MoPME in developing guidelines and plan for implementation of National School Meal Policy School Meal Policy has been approved and MoPME would require technical support in implementation, based on learnings from SFP. WFP can support the MoPME in the preparation and execution of the ‘implementation’ plan. Further, it can provide capacity building support on following: • Improved documentation of process and learning • Designing and implementing M&E framework for NSMP Operational Short term Ranked 5 on importance, 5 on feasibility MoPME would require technical support from WFP in preparation of guidelines and plan for implementation 2. Engaging into advocacy activities for mainstreaming of new techniques and methods with the existing curriculum prescribed by GoB and The student’s perception on experience with the adoption of new techniques by the teachers is positive. Students and parents have indicated that learning through the use of participatory techniques and colourful WFP should consider dissemination of evidence on benefits of adoption of new teaching techniques generated through this programme at the national level (MoPME, DPE), thereby making case for consideration on scaling up of this initiative by GoB. It should Strategic Medium to long term Ranked 5 on importance, 4 on feasibility MoPME officials saw merit in adoption of new techniques to improve the academic performance of students and agreed to work towards mainstreaming 65 1-lowest , 5- highest, 66 Rating recommendations and determining priority based on(i) importance for the programme; (ii) feasibility of implementation Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 40 Sl. No. Recommendations Rationale Proposed actions Type Timeframe Priority (based on rating of recommendations on scale of 1-565 by GoB representatives during validation workshop in Dhaka66) Proposed action adoption of these techniques at scale. printed visual aids is faster and more interesting than learning from normal textbooks. However, shortage in number of teachers in some schools and absence of mainstreaming of these methods and modules in the existing GoB curriculum acts as an inertia for the teachers to fully adopt these methods. Adoption of new teaching techniques is therefore, perceived by the teachers as an additional burden, thereby limiting its practice. also make efforts towards ensuring mainstreaming of these methods with the existing curriculum, prescribed teaching methods. 3. Utilization of alternative spaces for vegetable gardens. There is a lack of availability of space in schools for establishing vegetable gardens. WFP should liaise with the Department of Agriculture for creating awareness and building capacity of the school administration to ensure utilisation of alternate spaces for vegetable gardens such as terrace. Strategic Long term Ranked 5 on importance, 3 on feasibility MoPME would liaise with Ministry of Agriculture for technical assistance in creating roof gardens and creating awareness amongst community on growing nutritious vegetables Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 41 Sl. No. Recommendations Rationale Proposed actions Type Timeframe Priority (based on rating of recommendations on scale of 1-565 by GoB representatives during validation workshop in Dhaka66) Proposed action 4. Strengthening community engagement for their enhanced participation in supporting implementation of activities under the programme. As compared to the baseline, there is an increase in the number of mothers who have reported their participation in the programme activities. Highlighting this as an evidence, WFP should engage in advocacy by articulating the need and benefits of community participation in SFP and support MoPME in defining contours for community engagement. WFP may also consider supporting MoPME in devising a social audit mechanism in schools whereby accountability of all stakeholders is ensured through increased participation of communities. Strategic Medium term Recommendation by the evaluation team, not discussed during validation workshop) 5. Exploring alternate packaging of biscuits. In the absence of a proper waste disposal mechanism for plastic wrapper, schools follow the practice of burning the wrappers in open instead of disposing them responsibly. WFP should explore alternate packaging of biscuits and minimise generation of plastic waste. Waste to Art events/competitions may also be organized in schools to serve the dual purpose of raising awareness about fortified biscuits as well as ensuring waste recycling. Operational Medium term Ranked 5 on importance, 4 on feasibility MoPME would engage with Ministry of Jute and Textile to explore alternate packaging Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 42 Sl. No. Recommendations Rationale Proposed actions Type Timeframe Priority (based on rating of recommendations on scale of 1-565 by GoB representatives during validation workshop in Dhaka66) Proposed action Actionable (Specific to SFP implementation in Cox’s Bazar) 6. Support the school administration in ensuring access to safe water in schools. Schools in Kutubdia face the challenge of access to safe drinking water due to the ingress of sea water contaminating the aquifers. WFP should engage in advocacy actions with Local Government and Engineering Department to ensure availability of safe drinking water in targeted schools. Operational Long term Ranked 5 on importance, 4 on feasibility The group agreed to liaise with LGED to ensure access to safe drinking water. They also agreed to explore with LGED with regard to involvement of community in rainwater harvesting 7. Revitalizing Joint Monitoring Visit mechanisms. Joint Monitoring Visits (by WFP staff, GoB officials at district and upazila level and implementing partners) are not conducted regularly. WFP needs to revitalize the joint monitoring visit mechanism envisaged in the programme and ensure regularity of these visits. Operational Short term Recommended by the evaluation team, validated by the district officials in Cox Bazar, not discussed in Dhaka) The DEO agreed to revitalise joint monitoring visits in consultation with WFP and implementation partners 8. Strengthening Healthy Meal Preparation Day initiative and providing more opportunities to Little Agriculturists for demonstrating their skills. Awareness about Healthy Meal Preparation Day is low as it is often clubbed with regular school picnics. Additionally, Little Agriculturalists (in schools with no vegetable gardens) lack a platform to deliver messages and build The implementing partners should encourage schools to conduct Healthy Meal Preparation Day separately. Additionally, like Little Doctors, Little Agriculturists should also be provided opportunity to demonstrate their acquired skills and deliver messages about Operational Short term Ranked 4 on importance, 5 on feasibility MoPME would coordinate with Ministry of Health and Family Welfare (MoHFW) to (i) leverage on events such as World Food Day and Nutrition Week and (ii)form nutrition committees in every upazila to create Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 43 Sl. No. Recommendations Rationale Proposed actions Type Timeframe Priority (based on rating of recommendations on scale of 1-565 by GoB representatives during validation workshop in Dhaka66) Proposed action awareness about nutrition in their peers. nutrition in various events organised by the schools. awareness and promote positive nutrition behaviour 9. Promoting enhanced participation of women in SMC activities. While there is an increase in female to male ratio in the composition of SMC, participation of women in SMC activities is limited. WFP needs to conduct more gender sensitization workshops with SMCs to ensure equal representation and active participation of women. Strategic Medium term Ranked 5 on importance, 3 on feasibility MoPME would explore creation of mid-day meal committees ensuring high representation and involvement of women 44 Annex A: Map of the Intervention Area 45 Indicates (SFP) Intervention Areas within Cox’s Bazar 46 Annex B: Project-Level Results 47 48 49 Annex C: Planned Outcomes of WFP Bangladesh McGovern Dole -FY17 Award Table 5: Planned outcomes and annual targets USDA Standard Indicator Number Results Framework (indicator no.) Performance Indicator Year 1 Year 2 Year 3 (Only applicable for Capacity Building) Life of project 1 1.3 Number of students regularly (80percent) attending USDA supported classrooms/schools 80 % 82 % 0% 82% 2 1.1.2 Number of textbooks and other teaching and learning materials provided as a result of USDA assistance 1,46,728 1,23,546 0 2,70,274 3 1.1.5 Number of school administrators and officials in target schools who demonstrate the use of new techniques or tools as a result of USDA assistance 92 116 0 116 4 1.1.5 Number of school administrators and officials trained or certified as a result of USDA assistance 116 146 0 146 5 1.1.4 Number of teachers/educators/teaching assistants in target schools who demonstrate the use of new and quality teaching techniques or tools as a result of USDA assistance 464 584 0 584 6 1.1.4 Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance 580 730 0 730 7 1.3.3 Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance 12 35 0 35 8 1.3.4 Number of students enrolled in school receiving USDA assistance 47,689 47,689 0 47,689 50 USDA Standard Indicator Number Results Framework (indicator no.) Performance Indicator Year 1 Year 2 Year 3 (Only applicable for Capacity Building) Life of project 9 1.4.4 Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance 48 128 334 334 10 1.4.4 Number of public-private partnerships formed as a result of USDA assistance 1 0 0 1 11 1.4.4 Value of new public and private sector investments leveraged as a result of USDA assistance $68.65 $78.13 $80.31 $227.1 12 1.4.2 Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance: Stage 1: Analyzed Stage 2: Drafted and presented for public/stakeholder consultation Stage 3: Presented for legislation/decree Stage 4: Passed/Approved Stage 5: Passed for which implementation has begun 1 0 1 2 15 1.2.1.1 Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance 93,23,176 93,23,176 0 1,86,46,352 16 1.2.1.1 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance 47,689 47,689 0 47,689 51 USDA Standard Indicator Number Results Framework (indicator no.) Performance Indicator Year 1 Year 2 Year 3 (Only applicable for Capacity Building) Life of project 17 1.2.1.1 & 1.3.1.1 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance 47,689 47,689 0 47,689 18 2.5 Number of individuals trained in child health and nutrition as a result of USDA assistance 580 730 0 730 19 SO2 Number of individuals who demonstrate the use of new child health and nutrition practices as a result of USDA assistance 28600 (60 percent of the total children) 28,600 0 28,600 20 2.2 Number of individuals trained in safe food preparation and storage as a result of USDA assistance 322 322 0 322 21 SO2 Number of individuals who demonstrate the use of new safe food preparation and storage practices as a result of USDA assistance 290 290 0 290 22 2.4 Number of schools using an improved water source 48 98 0 146 23 2.4 Number of schools with improved sanitation facilities 48 98 0 146 24 2.5 Number of students receiving deworming medication(s) 47,689 47,689 0 47,689 26 SO1 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text 25.00 35.00 - 35% 27 SO1 Number of individuals benefiting directly from USDA￾funded interventions 48,711 48,711 0 48,711 52 USDA Standard Indicator Number Results Framework (indicator no.) Performance Indicator Year 1 Year 2 Year 3 (Only applicable for Capacity Building) Life of project 28 SO1 Number of individuals benefiting indirectly from USDA￾funded interventions 1,90,756 1,90,756 0 1,90,756 Custom 1 1.1.1 Percent of teachers in target schools who attend and teach school at least 90 percent of scheduled school days per school year 65% 75% 0% 75% 2 1.1.2 Number of classroom libraries (book shelves with books) established as a result of USDA assistance 243 487 0 487 3 1.2 Percent of students in classrooms identified as attentive by their teachers 60% 70% 0% 70% 4 1.3.2, SO2 Average number of school days missed by each student due to illness (for each school and in aggregate) 10 8 0 5 5 1.3.5, 1.4.4 Percent of parents in target communities who can name at least three benefits of primary education 40% 70% 0% 70% 6 2.1 Percent of students who can identify at least three key health and hygiene practices 30% 65% 0% 65% 7 2.2 Percent of storekeepers who can identify at least three safe storage practices 30% 50% 0% 50% 9 2.4 Number of 'Little Doctor' students supported by WFP 2,025 2,025 0 2,025 53 USDA Standard Indicator Number Results Framework (indicator no.) Performance Indicator Year 1 Year 2 Year 3 (Only applicable for Capacity Building) Life of project 11 1.4.4, 1.4.1 Number of meetings/workshops/training sessions held for institutional capacity to implement SF as a result of USDA assistance 48 98 0 142 12 1.4.1, 1.4.2, 1.4.4 Number of pilot initiatives supported to design SF modalities as a result of USDA assistance 2 2 0 4 13 1.4.1, Number of government staff trained as a result of USDA assistance 8,040 16,560 20,400 20,400 14 2.2, 1.4.1, Number of technical training for system development for service/food procurement, quality control, supply chain, and strengthened online database system and gender mainstreaming in the programme as a result of USDA assistance 130 260 284 284 15 SO1, SO1 Number of schools supported by the Government with school feeding as a result of USDA assistance 12,356 12,952 13,482 13,482 16 1.4.4 Number of social mobilization/community meetings as a result of USDA assistance 23 33 28 84 17 1.4.4 Number of community mobilization workshops organized as a result of USDA assistance 19 19 0 38 18 1.4.4 Number of teachers, parents and school management committee members attended the community mobilization workshops 584 584 0 1,168 54 Annex D: Activity wise Graduation Timeline Table 6: Activity wise Graduation Timelines Activity Key Milestones Timeline Build Capacity • School Feeding National Policy will be finalized by first quarter 2018 • Follow up of SABER 1-day annual workshop to track progress against the five policy goals will be done by last quarter each year • Draft School Feeding Policy Implementation Strategy will be finalized by early 2020 • Learning sharing workshop based on pilot school feeding modalities will be carried out by last quarter 2019 • Technical support for strengthening the GoB system to establish school feeding supply chain and procurement and quality control (2018- 2020) • Support to asset management online reporting and tracking (2018- 2020) January 2018- September 2020 Distribute Food • Agreement signed, EP and PMP approved, Call forward initiated • Wheat will arrive in Chittagong Port by April 2018, biscuit factory (supplier) receives wheat for production, production ensues, biscuits will be delivered to programme locations (10-week long process) • Biscuits will be served in school by10: 30 am to grade 1-5 July 2018-September 2020 Improve Student Enrolment and Attendance • Programme launch at each school (146 total) • Distribution of student stationaries and school bag for all grades (pencils, crayon, Eraser, Sharpener & kit.) approximately 48,000 students • Library opening ceremony at each school (146) • Quarterly programme review meeting • Annual progress sharing meeting • Student Recognition Day January 2018- September 2020 Promote Improved Health • Reconstruction of latrines and water systems in 35 schools begins in a year, late 2018. • Identification and orientation to “Little Doctors” will be done by first quarter 2018. Approximately 15 little doctors per school (2,190 total). Approximately 146 orientations will be given in total, one per school. January 2018- September 2020 Promote Improved Nutrition • Establish 60 garden demonstration plot at school • Train approximately 5 teachers at each school (730 total) in healthy meal preparation January 2018- September 2020 55 Activity Key Milestones Timeline Support Improved Literacy • Community mobilization orientation once per year at each community • Publication of children's magazine (15 copies per school/once a year) • Library bookshelf distribution (once) for 146 government primary school classes and 55 madrasa classes. • Grade 1-2 teacher training in Bangla reading instruction, approximately 292 teachers • Primary Training Institution and Upazila Resource Center instructor ToT • Head teacher training on school performance, quality improvement, governance, transparency management, monitoring, and evaluation, approx. 146, one per school • Basic training for the programme on instruction and programme implementation January 2018- September 2020 Support Safe Food Preparation and Storage • Initial training at field level (18 total, 25 people each) in year 1 • Central level training for factory officials (25-35 people; 1 training); May 2018 • Yearly refresher training by NGOs at school level approximately 438, three per school January 2018- September 2020 56 Annex E: McGovern Dole Target Beneficiaries and Funding for WFP School Feeding Programme A. McGovern Dole Target Beneficiaries B. Funding for WFP School Feeding Programme Budget Summary Total Amount of Federal Funds Obligated Funding Source Finding Year Commodity Cost (FFPr and MGD Only) Freight Cost (FFPr and MGD Only) Administrative Costs (cash portion) Total Federal Funding Obligated McGovern￾Dole (Food For Education) 2017 $931,000.00 $1,666,000.00 $14,522,720.00 $17,119,720.00 Project Operating Budget Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share Administration Professional Services $0.00 $490,000.00 $0.00 $490,000.00 $490,000.00 Other $0.00 $1,030,777.67 $0.00 $1,030,777.67 $1,030,777.67 Total Administration $0.00 $1,520,777.67 $0.00 $1,520,777.67 $1,520,777.67 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share Activities Build Capacity $0.00 $4,283,073.38 $0.00 $4,283,073.38 $4,283,073.38 Distribute Food $0.00 $482,533.00 $0.00 $482,533.00 $482,533.00 Project Population Participant Approximate number Upazilas 2 Schools 146 Students 47,689 Parents 95,738 School Administrators 146 Teachers 730 57 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share Improve Student Enrolment and Attendance $0.00 $445,483.31 $0.00 $445,483.31 $445,483.31 Promote Improved Health $0.00 $589,473.11 $0.00 $589,473.11 $589,473.11 Promote Improved Nutrition $0.00 $103,188.00 $0.00 $103,188.00 $103,188.00 Support Improved Literacy $0.00 $5,060,509.61 $0.00 $5,060,509.61 $5,060,509.61 Support Improved Safe Food Prep and Storage $0.00 $58,340.25 $0.00 $58,340.25 $58,340.25 Total Activities $0.00 $11,022,600.94 $0.00 $11,022,600.94 $11,022,600.94 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share ITSH Total Activity Cost $0.00 $897,827.00 $0.00 $897,827.00 $897,827.00 Total ITSH $0.00 $897,827.00 $0.00 $897,827.00 $897,827.00 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share Total Direct Costs $0.00 $13,441,205.61 $0.00 $13,441,205.61 $13,441,205.61 58 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share ICR on Administration $0.00 $0.00 $0.00 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share ICR on Activities $0.00 $1,081,514.39 $1,081,514.39 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share ICR on ITSH $0.00 $0.00 $0.00 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share Total Indirect Costs $0.00 $1,081,514.39 $1,081,514.39 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share Anticipated Programme Income $0.00 Expense Type Monetization Proceeds FAS Funds Cost Share Total w/out Cost Share Total w/ Cost Share Grand Total Costs $0.00 $14,522,720.00 $0.00 $14,522,720.00 $14,522,720.00 Total Amount of Federal Award (Total Federal Funds Obligated plus Cost Share): $17,119,720.00 59 Annex F: Role of Partners Table 7: Role of partners Activity Implemented by Partners Role of Partners Build Capacity WFP MoPME and DPE • Receiving capacity support for SF programme design and implementation at the local, regional and national level • Management of the programme with adequate quality control measures by establishing supply chains, food storage facilities etc. • Establishing proper monitoring and reporting systems • Designing and implementing pilots with various combinations of ingredients for the future • Engaging and learning from community awareness activities to make school feeding nutrition sensitive • Independent procurement of biscuits • Establishing a National School Feeding Policy and Strategy • Generating evidence on best practices during the implementation of National School Feeding Policy and Strategy Distribute Food WFP MoPME, DPE • Provision of fortified biscuits during 2018, 2019 and 2020 school years in grades 1-5 • Delivery and distribution of food using WFP’s extensive pipeline and supply chain management system • Monitoring distribution of food and it’s safe storing, tracking and reporting • Providing guidance to school administrators, monitor performance and providing feedback for further improvement Improve Student Enrolment and Attendance WFP RtR, MA, YPSA RtR • Procure bookshelves for libraries for 709 government primary school classes • Initiate Book Checkout Register, Supplementary Reading Material register, visitor Registrar to the schools • Distribute books to the library (both RtR and purchased) and Book leveling sticker • Conduct Library Rating System. RtR will introduce the rating tools but rating conducted by YPSA/MA Literacy Facilitators • Lead teacher training on Bangla • Lead training on Library Management for teachers, primary training institute and Upazila Resource Center • Conduct baseline and midline assessment on reading schools • Lead ToT on introduction at Primary Training Institute and Upazila Resource Center • Lead Quarterly Programme review meeting with implementing partners (3 meetings); for key people from WFP to also attend at the district level 60 Activity Implemented by Partners Role of Partners • Lead Annual Progress Sharing Meeting with WFP and MA/YPSA for key people from WFP to also attend at the district level • Lead Progress Sharing Meeting with Government Officials twice a year in two Upazilas • Lead Technical Review Meetings with Literacy Programme Officer Technical and Literacy Facilitators (literacy team) in two Upazilas MA and YPSA • Support implementation and distribution of fortified biscuits to programme schools. • Liaising with WFP on the distribution of FAO designed health and hygiene posters (5 per school) • Liaise with Room to Read’s Literacy Project Officer on the implementation and facilitation of training, distribution of materials • Identify any potential issues with programme implementation and communicate such issues with WFP • Participate, facilitate and encourage active engagement in programme launch, distribution of programme materials (books, pens, backpacks, library shelves) at the school level • Support student tracking on library usage for grades 1 and 2. • Creating a literate environment in the school in grade 1 and 2 (budget classroom wise). This involves o classroom coloring and decoration o setting display board (for display of students' work) o providing wall clock o displaying materials (grade specific posters on rhymes and poems etc.) on as needed basis. • Initiate wall magazine development – once a year, school based; once a year, Upazila level • Initiate wall magazine competition • Initiate Upazila based recognition event for school teachers, SMCs • Organize Community Mobilizer orientations • Provide WFP with monitoring data such as the amount of food distributed, average student attendance, and number of latrines and water systems rehabilitated Promote Improved Health WFP YPSA, MA • WFP will guide and monitor YPSA and MA in hygiene reconstruction and information communication messaging. The community will be engaged through monthly SMC meetings. • YPSA and MA will support the rehabilitation of water systems and latrines. This will involve identifying what materials are needed for rehabilitation, procuring the materials and rehabilitating the water systems or toilets. 61 Activity Implemented by Partners Role of Partners • WFP will train YPSA and MA on the ‘Little Doctors’ activity which includes lessons on how to measure weight, height, vision test and assist in government distribution of deworming. • WFP will monitor this activity through field monitors and maintain consistent communication between government, schools, MA and YPSA. • YPSA and MA will be trained to provide on the spot guidance, as needed, to teachers and students during routine monitoring. Promote Improved Nutrition WFP YPSA. MA • WFP will monitor the project and train YPSA and MA on basic agriculture skills. • YPSA and MA will implement this activity at the school level, training teachers and overseeing project performance. • YPSA and MA will be trained to provide on the spot guidance, as needed, to teachers and students during routine monitoring. • YPSA and MA will re-enforce “Little Agriculturalist” and the setting up of gardens Support Improved Literacy WFP RtR, MA, YPSA • RtR will develop the training package, provide ToT and co￾facilitate the event with other government trainers at the Upazila level • RtR and WFP will liaise with government to obtain training approvals • MA and YPSA will facilitate school level training Support Safe Food Prep and Storage WFP MoPME, YPSA, MA, DPEO • WFP will impart training to YPSA and MA and government officials (DPEO) on food storage practices. • YPSA and MA will train school teachers, head administrator and SMC members at the local level. • MoPME officials will be trained in monitoring the biscuit manufacturers in food safety standards for safe preparation of biscuits and appropriate biscuit storage practices. • Government officials will be trained in monitoring the biscuit manufacturers. 62 Annex G: Stakeholder Analysis Table 8: Stakeholder analysis Stakeholder Interest in the [Intervention/Project/Operat ion] Involvement in Evaluation and likely use Who (specifically for the Evaluation) Internal (WFP) Stakeholders WFP CO Bangladesh • Responsible for overall management and provide support relating to logistics, procurement, and finance. • Responsible for overall coordination, liaison with stakeholders, implementation oversight and capacity building. • Responsible for the coordination of the activity’s implementation. • Supervising the proper management of warehouses, establishing the supply chain for biscuits and providing capacity support to the Government. • Overseeing the overall financial management of the project and organizing training to Government. • Overseeing collection of data and maintaining data base; data analysis and reporting, capacity support in institutionalizing monitoring and reporting system in MoPME, DPE etc. • Central level coordination, planning, designing, guiding and training for implementation, progress tracking, demonstration of results including reporting • Providing field support, liaison with sub-offices and providing M&E support • The key informant, primary stakeholders, and users of this study. • Involved in using baseline findings for implementation and monitoring of USDA Mc Govern Dole FY17-20 grant • Initial briefing and overview of WFP work in Bangladesh, programme documents, help evaluation team better understand the context of implementation and strategy for the future; • Support the evaluation team through an introduction to key stakeholders; • Review the Inception Report and Draft Baseline Report • CD • DCD (Programme) • DCD (Operations Support) • Head of Programme Planning and Implementation Support • Activity Manager • Evaluation Manager • Senior Programme Officer (SF) • Senior Programme Assistant • Senior Programme Officer (Resource Management) 63 Stakeholder Interest in the [Intervention/Project/Operat ion] Involvement in Evaluation and likely use Who (specifically for the Evaluation) WFP Field Offices Cox’s Bazaar • Coordination between Cox’s Bazar and Dhaka Offices on school feeding programme; management of overall project timeline; ensuring timely submission of good quality deliverables; communication of programme process to relevant stakeholders. • Providing overall programme support for Ukhiya and Kutubdia sub￾districts, ensuring communication flow to Senior Programme Officer; maintaining timelines, reviewing deliverables for submission to donor • Providing support for pipeline management and assisting the implementing agencies in proper tracking supply and utilization of commodities • Overseeing the overall financial management of the project, providing support at field level for biscuits delivery, warehouse management • Coordinating, supervising and guiding implementation, monitoring and reporting at the field level • Sharing feedback to improve programme performance • The key informant, primary stakeholders. • As users of the baseline study will be affected by outcomes. • Senior Programme Officer • School Feeding Officer • Senior Programme Assistant • Logistics Assistant RBB Responsible for both oversight of COs and technical guidance and support, the RB management has an interest in an independent/impartial account of the operational performance. The Regional Evaluation Officer supports CO/RB Key informant and primary stakeholder – involved in planning for next USDA Mc Govern Dole Interested in the independent account of USDA McGovern Dole performance No interviews will be conducted 64 Stakeholder Interest in the [Intervention/Project/Operat ion] Involvement in Evaluation and likely use Who (specifically for the Evaluation) Applying the learning from evaluation to other country offices. WFP HQ Policy and Programme WFP HQ technical units are responsible for issuing and overseeing the rollout of normative guidance on corporate programme themes, activities and modalities, as well as of overarching corporate policies and strategies. They also have an interest in the lessons that emerge from evaluations, as many may have relevance beyond the geographical area of focus. No interviews will be conducted WFP Executive Board (EB) The WFP governing body has an interest in being informed about the effectiveness of WFP operations. The WFP governing body has an interest in being informed about the effectiveness of WFP operations. This evaluation will not be presented to the EB but its findings may feed into annual syntheses and into corporate learning processes. Office of Evaluation (OEV) Provides indirect, independent oversight DE Help Desk No interviews will be conducted External stakeholders SFP beneficiaries They are the ultimate recipients of WFP support beneficiaries and therefore, have a stake in WFP determining whether its assistance is appropriate and effective. As such, the level of participation in the review of women, men, boys, and girls from different groups will be determined and their respective perspectives will be sought. Key informants and primary stakeholder – providing perspective on results, outcomes and emerging impact of WFP’s intervention Will be affected by the decision to continue or to stop the intervention A sample of schools to be selected and within each school, a sample of the following persons will be interviewed: • School-going children in primary schools (equal number of boys and girls) • Teachers, Storekeeper, Headmaster • Parents, SMC members • Efforts would be made to ensure interviews with equal numbers of boys and girls • Individual (structured interviews) Government of Bangladesh Ministry of Primary and Mass Education (MoPME) Directorate of Primary • Responsible for providing guidance on School Feeding priorities and approaches, and ensuring alignment with Government policy. • Receiving capacity support for SF programme design and implementation at the Key informant and primary stakeholder on government policy, priorities, views on support by WFP and on expanding school feeding, on GoB’s commitment to Relevant officials 65 Stakeholder Interest in the [Intervention/Project/Operat ion] Involvement in Evaluation and likely use Who (specifically for the Evaluation) Education (DPE ) Ministry of Health local, regional and national level • Management of the programme with adequate quality control measures by establishing the supply chains, food storage facilities etc. • Establishing proper monitoring and reporting systems • Engaging and learning from community awareness activities to make school feeding nutrition sensitive • Independent procurement of biscuits providing/mobilize resources, issues, and opportunities in handing over of the programme, capacities, and convergence to be explored. Will have a perspective on sustaining the FY17- 20 programme. District & Upazila Level education authorities Responsible for overseeing education sector performance, including implementation of National School Feeding Policy, and liaising with other government departments at a decentralized level Primary stakeholder and key informant – on the implementation of the school feeding components. Will have a perspective on challenges and achievements. Relevant officials Capacity Support Unit (in DPE) (WFP) • Leading the CSU’s team on the development of a national school feeding programme, supporting the Government’s scaling up school feeding and in the formulation of a national school feeding policy and strategy • Administrating and programme implementation support. Support and providing on the job training to the relevant GoB officials • Supporting MoPME for the operationalization of National School Feeding Policy and strategy • Providing transportation support • Providing liaison and program support May be useful in mapping the assessing the existing capacities of MoPME and DPE. • Senior Programme Officer • Senior Programme Assistants • Consultant • Washington DRO UN country team UNICEF Involved in WASH policy and implementation Based on the advice of WFP, no interviews will be conducted 66 Stakeholder Interest in the [Intervention/Project/Operat ion] Involvement in Evaluation and likely use Who (specifically for the Evaluation) DONOR agency USDA Funder of WFP school feeding programme Primary stakeholder and informant - on the relationship with WFP and the priorities moving forward Based on the advice of WFP, a Skype call can be organized if required. NGOs Room to Read • Procure bookshelves for libraries for 709 government primary school classes • Initiate Book Checkout Register, Supplementary Reading Material register, visitor Registrar to the schools • Distribute books to the library (both RtR and purchased) and Book leveling sticker • Conduct Library Rating System. RtR will introduce the rating tools but rating conducted by YPSA/MA Literacy Facilitators • Lead teacher training on Bangla • Lead training on Library Management for teachers, primary training institute and Upazila Resource Center • Conduct baseline and midline assessment on reading schools • Lead ToT on introduction at Primary Training Institute and Upazila Resource Center • Lead Quarterly Programme review meeting with implementing partners (3 meetings); for key people from WFP to also attend at the district level • Lead Annual Progress Sharing Meeting with WFP and MA/YPSA for key people from WFP to also attend at the district level • Lead Progress Sharing Meeting with Government The results of the evaluation might affect future implementation modalities, strategic orientations, and partnerships. • Programme Operations Director • Literacy Director • Research Monitoring & Evaluation 67 Stakeholder Interest in the [Intervention/Project/Operat ion] Involvement in Evaluation and likely use Who (specifically for the Evaluation) Officials twice a year in two Upazilas • Lead Technical Review Meetings with Literacy Programme Officer Technical and Literacy Facilitators (literacy team) in two Upazilas Young Power in Social Action and Muslim Aid • Support implementation and distribution of fortified biscuits to programme schools. • Liaising with WFP on the distribution of FAO designed health and hygiene posters (5 per school) • Liaise with Room to Read’s Literacy Project Officer on the implementation and facilitation of training, distribution of materials • Identify any potential issues with programme implementation and communicate such issues with WFP • Participate, facilitate and encourage active engagement in programme launch, distribution of programme materials (books, pens, backpacks, library shelves) at the school level • Support student tracking on library usage for grades 1 and 2. • Creating a literate environment in the school in grade 1 and 2 (budget classroom wise). This involves • classroom coloring and decoration • setting display board (for display of students' work) • providing wall clock • displaying materials (grade specific posters on rhymes and poems etc.) on as needed basis. • Initiate wall magazine development – once a year, Their respective perspectives will be sought as the engagement of these actors influences the effectiveness of the programme as well as its sustainability. • Programme Manager • Programme Coordinator • Literacy Facilitators • Field Monitors 68 Stakeholder Interest in the [Intervention/Project/Operat ion] Involvement in Evaluation and likely use Who (specifically for the Evaluation) school based; once a year, Upazila level • Initiate wall magazine competition • Initiate Upazila based recognition event for school teachers, SMCs • Organize Community Mobilizer orientations • Provide WFP with monitoring data such as the amount of food distributed, average student attendance, and number of latrines and water systems rehabilitated 69 Annex H: Summary of the methodology presenting the method of data collection, the target groups/sources of information, the nature of the interviews, and the indicative outputs Table 9: Summary of the Evaluation Methodology Methods of data collection Target Group/ Source of Information Primary Survey Secondary Review Quantitative (Structured Interviews) • Student Questionnaire •Parent Questionnaire • School Questionnaire • School Head Questionnaire • School teachers Questionnaire • School Storekeeper Questionnaire •Class Observation •EGRA Questionnaire •Project design document, results framework, annual action plans, and project activities and monitoring reports (quarterly, monthly, yearly) •NPAN-2 •MIS and monitoring data •Report on National Education and School Feeding Policy Stakeholders Engagement Conference Qualitative (Focus Group Discussions and KIIs) •FGD with parents •FGD with students •FGD with SMC members •Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE Indicative outputs of data collection •Establish values at mid-term for various indicators •Compare mid-term values of indicators such as enrolment, attendance and attentiveness (from direct and indirect influence) with baseline values • Identify key enablers and barrier • Status of the performance and progress of the project •Perceptions and overall experiences of the beneficiaries with the project Wherever appropriate sex disaggregated analysis will be provided •Build context and relevance of the programme and track changes (if any) from the baseline study •Establish alignment of the programme with government initiatives and WFP’s country strategy •Analyse the resources utilised •Evaluate the mid-term performance • Identify key design elements that contribute to direct and indirect change •Explore pathways created for sustainability; scope of replicability; key considerations for recommendations 70 Annex I: Data Analysis Methods 1. The questionnaires have been developed in line with the tools used for the baseline study of FY17 project. Information on status of implementation across various indicators will be taken from project monitoring reports of WFP. 2. The indicators specified in the ToR have been mapped (presented in Annex C) with the type of tools, respondent group and the question numbers which facilitate the identification of methods of data triangulation and assess the reliability and validity of the data collected. As indicated earlier, the analysis of primary data will be supported by secondary data. 3. Considering that the Difference-in-Difference analysis was not possible during the baseline, an activity evaluation has been done67 that collected comparative monitoring data from comparative schools. Accordingly, for school feeding indicators, a direct comparison between the intervention and comparison schools was carried out. On literacy indicators, comparison between the intervention and comparison schools was not possible during the baseline, given that the READ programme had ended just prior to the commencement of the baseline survey. It was therefore, hypothesized, that while the performance against the literacy indicators will improve in intervention schools, with the cessation of the literacy intervention in Ramu, higher levels of literacy indicators are unlikely to be maintained. Considering this, for literacy indicators, a comparison between the baseline values and midterm values of intervention schools has been carried out to measure change. 4. The data analysis aimed to highlight how the values established during the baseline study have changed and what progress has been made. This will help WFP gauge whether the project implementation is on track and identify the areas in which delays have happened (if any). The data analysis also provides a descriptive analysis of sex disaggregated data setting the benchmark values in intervention and comparison schools and allow for ensuing the mid-term evaluation to estimate the extent of change that is attributable to the project. 5. Data obtained for the gender-related questions has been analysed comprehensively to report on the gender dimensions of the evaluation. Data on attainment of outputs and outcomes has also been disaggregated by sex and age. Variables such as ethnicity and distance from the school have been used to contextualise the variations in evaluation findings for both sexes. 6. The raw data obtained from the field has been checked by the data analyst for consistency errors, duplicity of cases and missing data. 7. The analysis of qualitative data helped in understanding the reasons that have contributed to changes (if any) in progress indicators established during the baseline study. This may help in determining what corrective actions and adjustments are required to be able to reach the end-term targets of the project in time. The qualitative analysis has also been used to understand the perceptions of beneficiaries about the services they are receiving under the programme, the value they see in these services, their views on what is missing from the school feeding programme and their overall experience with it. Perceived enablers and barriers by the SMCs and school administrations in implementing the programme objectives have also been highlighted through this analysis. Furthermore, best practices and learnings have also been captured through KIIs and FGDs. 8. Primary quantitative and qualitative data will be analysed using the questions set out as per the evaluation matrix and the relevant themes identified under the conceptual approach. Qualitative data will be analysed using content analysis. The analysis will focus on providing the reasons for the change observed. The content of the FGDs and the IDIs, will be classified under the thematic areas of the MTE. Within each category, sub-categories dependent on the qualitative information obtained from the field will be generated. Emerging trends will be noted from the analysis of the main and sub-categories and assessed in response to the mid-term evaluation questions. Case studies will be documented to highlight the best practices, challenges, and learnings to inform the programme. 9. The quantitative data will be analysed using SPSS software post a rigorous review of any errors in the data. Based on the available key information parameters, a data tabulation plan will be finalised in consultation with WFP-CO. In addition, secondary data from relevant sources and qualitative findings will be used to validate the observations. 67 As also advised by USDA and WFP Office of Evaluation during review of baseline report 71 10. Overall, a triangulation of the quantitative data, qualitative data and project documents will be conducted to present conclusions on overall trends and patterns. Wherever, deemed important, viewpoints will be illustrated as quotations from relevant stakeholders. 72 Annex J: Sample Size Covered by Respondent Category Table 10: Sample Size Quantitative (Structured interviews) Details Intervention Comparison No. of schools 30 20 Children 13 per school 390 260 Parents (of selected students) 5 per school 150 100 Total (Mid-term) 540 360 EGRA with Grade 3 students only 14 per school 420 280 School Head 1 per school 30 30 School Teacher 1 per school 30 20 Store keepers 1 per school 30 Total (Mid-term) 90 50 Qualitative – Key Informant Interviews (KII) • Additional Director Department of Agricultural Extension (DAE), Ministry of Agriculture 1 • Assistant Director, Bangladesh National Nutrition Council (BNNC) 1 • Project Director (Joint Secretary), School Feeding Programme, Directorate of Primary Education (DPE 1 • Programme Policy Officer, WFP 1 • Other WFP staff 2 • Project Manager, Room to Read 1 • Senior RLPO Kutubdia, Room to Read 1 • YPSA and MA staff 2 • District and Upazila Education Officers 3 Total (Mid-term) 13 Qualitative- Focus Group Discussions (FGD) Details Total Mothers 6 4 SMC 6 4 Students 6 4 Total (Mid-term) 18 12 73 Annex K: List of Sample Schools S.No Upazila Union School 1 Ukhiya Rajapalong J A Chowdury Government Primary School 2 Ukhiya Rajapalong Tuturbil Government Primary School 3 Ukhiya Rajapalong Dosory Government Primary School 4 Ukhiya Rajapalong Purbo Digholia Government Primary School 5 Ukhiya Rajapalong Hatimura Government Primary School 6 Ukhiya Holodipalong Rumka Hatirghona Government Primary School 7 Ukhiya Holodipalong Sabak Rumka Government Primary School 8 Ukhiya Holodipalong Nolbunia Government Primary School 9 Ukhiya Holodipalong U.Borobil Government Primary School 10 Ukhiya Ratnapalong Ruholladeba Government Primary School 11 Ukhiya Ratnapalong Purchim Ratna Government Primary School 12 Ukhiya Ratnapalong Tulatoli Government Primary School 13 Ukhiya Jaliapalong Sonarpara Government Primary School 14 Ukhiya Jaliapalong Inani Government Primary School 15 Ukhiya Jaliapalong Choankhali Government Primary School 16 Ukhiya Jaliapalong Dailpara Government Primary School 17 Ukhiya Palongkhali Telkhola Government Primary School 18 Ukhiya Palongkhali Lotifunnessa Government Primary School S.No Upazila Union Name of School 1 Kutubdia Uttar Dhurong Char Dhurong Government Primary School 2 Kutubdia Uttar Dhurong Teliakata Government Primary School 3 Kutubdia Uttar Dhurong Baingakata Government Primary School 4 Kutubdia Dakhin Dhurong Dhurong Government Primary School 5 Kutubdia Dakhin Dhurong South Dhurong Habibiya Government Primary School 6 Kutubdia Lemsikhali West Lemsikhali Government Primary School 7 Kutubdia Lemsikhali Shahajipara Government Primary School 8 Kutubdia Koierbil Ghilachari Government Primary School 9 Kutubdia Koierbil Najar Ali Matbar Government Primary School 10 Kutubdia Borghop Middle Aliakbor Dail Government Primary School 11 Kutubdia Borghop North Borghop Government Primary School 12 Kutubdia Aliakbor Dail East Aliakbor Dail Government Primary School S.No Upazila Union Name of School 1 Ramu Pyachar Dweep Government Primary School 2 Ramu Dakshin Chakmerkul Government Primary School 3 Ramu Jaaraitali Government Primary School 4 Ramu Ghonarpada Government Primary School 5 Ramu Shrimura Government Primary School 6 Ramu Nasira Para Government Primary School 74 7 Ramu Merongloya Model Government Primary School 8 Ramu Ramu Khijari Barmij Government Primary School 9 Ramu Ramu Aadorsho Government Primary School 10 Ramu Monirjheel Government Primary School 11 Ramu Shukmoniya Government Primary School 12 Ramu Faakrikata Government Primary School 13 Ramu Majheerkata Government Primary School 14 Ramu Uttor Bodbeel Government Primary School 15 Ramu Haldarkul Government Primary School 16 Ramu Paanerchhada Government Primary School 17 Ramu Khuniyapalong Government Primary School 18 Ramu Eidgarh Government Primary School 19 Ramu Joyariyanala Government Primary School 20 Ramu Ramu Government Primary School 75 Annex L: Evaluation Matrix Table 11: Evaluation matrix Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability RELEVANCE • Is the project implementation strategy designed to reach the right people with the right type of assistance? • Factors driving programme implementation strategy and action plan in the context of Ukhiya and Kutubdia o Prevalent issues and concerns of affected population in light of education, nutrition, social aspects (for boys and girls) o External administrative, social and political environment o Infrastructure availability (water supply, toilets) and access for students (for boys and girls) • Stakeholder perceptions regarding the degree to which needs of different groups were identified appropriately • WFP annual report • Baseline report • NPAN 2 • WFP Bangladesh Country Strategic Plan 2017-2020 • Report on National Education and School Feeding Policy Stakeholders Engagement Conference • Data from Key Informant Interviews • Document review • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE, parents and school staff • FGD with Parents and SMC members • Narrative/ thematic analysis of secondary data • Analysis of Qualitative data (KIIs and FGDs) • Comparing views of GoB, WFP, implementing partners, parents and SMC members • Disaggregation of data by gender (boys and girls) • Comparison of response from WFP, school with response from girls (through FGDs) Strong • Is the project aligned with national government’s policies and strategies including education, school feeding, safety net • Consideration of school feeding programme priorities and interventions within GoB’s policy and planning frameworks such as, National School Meal policy, NPAN 2, Vision 2021, 7th five year plan. • Number of interventions and activities that fulfil the institutional and capacity building needs, supports provision of education, • NPAN 2 • National School Meal policy • Vision 2021 • 7th five year plan • National Nutrition Policy • National Social Security Strategy • Document review • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE • Narrative/ thematic analysis of secondary data • Analysis of Qualitative data (KIIs) • Comparing the views of GoB and WFP. Strong 76 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability and national five years plan? school feeding, safety net services as identified by National Nutrition Policy, National Social Security Strategy. o Does the Capacity Building component address the capacity needs of the functionaries at the MoPME to implement the School Feeding Programme? o Activities strengthening GoB’s ongoing initiatives • Data from Key Informant Interviews • Does the project complement other donor-funded and government initiatives? • Extent to which the school feeding programme activities complement GoB’s ongoing initiatives. • Extent to which programme initiatives complement initiatives undertaken by other development partners. • Action plans and assessment reports on activities undertaken by other development partners • Data from Key Informant Interviews • Document review • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE • Narrative/ thematic analysis of secondary data • Analysis of Qualitative data (KIIs) • Comparing the views of GoB, WFP, implementing partners Fair EFFECTIVENESS • What is the output and the progress of project implementation – is the project on track to carry out all activities as planned? • Progress of activities planned and undertaken (as per action plan) • Mechanisms and processes for engagement with GoB and other development partners • Extent to which the capacities of representatives, GoB have been developed on: o Preparation of operational guidelines for the national school feeding programme • USDA approved work plan • Monthly/quarterly reports from implementing partners • School attendance and enrolment records • Biscuit distribution and receipt records • Programme Implementation Plan (PIP and Action plan) • Baseline data • Document review • Storeroom observation • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE, implementing partners • Focus group discussions with mothers, • Narrative/ thematic analysis of secondary data • Qualitative and quantitative analysis of primary data (KIIs) • Methodological triangulation (Quantitative and Qualitative) • Comparing the data on attendance from schools records with Strong 77 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability o Preparation of monitoring and reporting framework for scaling up national school feeding programme o Use of online database for reporting on school feeding programme for programme strengthening o Operationalizing multi-sectoral committee on school feeding o Integration of complementary components such as WASH, Health, Nutrition and Dietary Practices and Literacy in programme planning (at the govt. level) o Developing community participation strategy and plan for implementation • Programme monitoring reports • SMC minutes of meetings • Data from quantitative survey with children, school staff, teachers and parents • Data from focused group discussions with mothers, students(boys and girls) and SMC members • Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE, implementing partners students(boys and girls) and SMC members • Outcome reports of other similar programmes • Quantitative survey with the children, store keeper, teachers, headmaster and parents information received from parents • Comparing the data received from school on storage management practices with observation • Comparing the response from WFP and GoB • Comparing the information on availability of infrastructure with response from students • Disaggregation of data by boys and girls • Comparison of response from school with response from girls (through FGDs) • Comparison of midterm values with baseline values • Analysis of school attendance, biscuit distribution and receipt records • Analysis of SMC minutes of meetings 78 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability • To what degree have the interventions resulted in the expected results and outcomes – is the project on track to reach set targets? • Comparison of data on performance indicators (standard and custom) with the baseline and targets (disaggregated by sex). • Key enablers and barriers (both internal and external) towards achievement of results? • Stakeholders’ awareness and perceptions about programme interventions o Encouragement of reading habits in school and at home o Perception of students and parents on the read-play festival o Degree to which students perceive that such events have encouraged them to read books (apart from textbooks) o Degree to which students perceive that the use of new teaching techniques has aided their understanding of school curriculum o Percent of parents in target communities who can name at least three benefits of primary education o Perception of parents on adequacy of teaching materials and methods o Degree to which parents are satisfied with the teaching methods in school o Perceptions regarding Read Play Festival, book corners, vegetable gardens (Essential Learning Package) and its effect on children and community; demand for other such provisions • USDA approved work plan • Monthly/quarterly reports from implementing partners • School attendance and enrolment records • Biscuit distribution and receipt records • Programme Implementation Plan (PIP and Action plan) • Baseline data • Programme monitoring reports • SMC minutes of meetings • Data from quantitative survey with children, school staff, teachers and parents • Data from focused group discussions with mothers, students(boys and girls) and SMC members • Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE, implementing partners • Document review • Storeroom observation • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE, implementing partners • Focus group discussions with mothers, students(boys and girls) and SMC members • Outcome reports of other similar programmes • Quantitative survey with the children, store keeper, teachers, headmaster and parents • Narrative/ thematic analysis of secondary data • Qualitative and quantitative analysis of primary data (KIIs) • Methodological triangulation (Quantitative and Qualitative) • Comparing the data on attendance from schools records with information received from parents • Comparing the tasks performed by little doctors, little agriculturists and book captains with the perception of students on their utility. • Participation of women in highlighting and resolving issues during the SMC meetings. • Comparing the response from school and WFP regarding community Strong 79 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability o Degree to which programme activities have influenced parents to demand better quality of education services • Degree to which parents perceive that training on health, hygiene and nutrition has influenced adoption of better health, hygiene and nutrition practices • Degree to which GoB perceives the interventions from WFP have been useful in building their capacities to implement the programme independently • Degree to which SMC members perceive increase in involvement and participation of women in decision-making participation with response from parents • Comparing the response from school and WFP regarding use of new teaching techniques with response from parents • Disaggregation of data by boys and girls • Comparison of midterm values with baseline values • Analysis of school attendance, biscuit distribution and receipt records • Analysis of SMC minutes of meetings EFFICIENCY • What was the efficiency of the processes programme, in terms of transfer cost, cost/beneficiary, logistics, and timeliness of delivery? • Timely and complete disbursement of funds (year wise) as per the programme implementation plan • Timely and complete achievement of the programme outputs (year wise) • Timely and complete utilization of resources by the intervention areas (geographic and thematic) • Timely submission and approvals of monthly/quarterly/yearly action plans • Biscuit distribution and receipt records • Programme monitoring reports(including data from MIS) • Data from quantitative survey with children, school staff, teachers and parents • Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE, implementing partners • Document review • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE • Quantitative survey with the children, store keeper, teachers, headmaster and parents • Narrative/ thematic analysis of secondary data • Qualitative and quantitative analysis of primary data (KIIs), survey and FGDs (disaggregated by gender) • Methodological triangulation (Quantitative and Qualitative analysis) Strong 80 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability • Inventory planning for biscuits and demand forecasting • Timely supply (from WFP and implementing partners) and receipt (by implementing partners and schools) • Establishment of systems and processes for monitoring interventions and feedback • Degree to which headmasters and implementing partners perceive the biscuit distribution process as efficient (timely availability and distribution) • Data from focused group discussions with mothers and SMC members • Focus group discussions with mothers and SMC members • Analysis of biscuit distribution, receipt records and observation findings • How efficient the operation and approach are in terms of capacity building of government toward eventual handover? • Timely completion of planned capacity building activities • Use of online MIS for (resource) planning, implementation and decision-making • Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE. • Document review • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE • Narrative/ thematic analysis of secondary data • Analysis of Qualitative data (KIIs) • Comparing the views of GoB and WFP. Strong SUSTAINABILITY • Is the programme sustainable in the following areas: strategy for sustainability; sound policy alignment; stable funding and budgeting; quality programme design; • Ownership and commitment of government and school officials to take forward or adopt best practices or innovations from the initiative • Institutionalisation of delivery model and approaches within existing administrative structures • NPAN 2 • School Feeding Policy and other policy documents • Programme documents and monitoring reports • Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE • KII with representatives of MoPME and WFP • Desk Review of government document, policies and action plan • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing • Narrative/ thematic analysis of secondary data • Analysis of data through KIIs and FGDs • Comparing views of GoB and WFP Strong 81 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability institutional arrangements; local production and sourcing; partnership and coordination; community participation and ownership? • Capacity of the line departments and functionaries to ensure that the programme functions independently • Institutionalization of coordination mechanisms for the independent implementation of the school feeding programme • Institutionalization and use of online database for real-time tracking • Availability of a post-project sustainability plan • Key enablers and barriers (both internal and external) towards sustainability of the programme • Data from focused group discussions with mothers and SMC members partners, representatives MoPME, DPE • Focus group discussions with mothers and SMC members • What progress has the government made toward developing and implementing a nationally owned school feeding programme? • Drafting and approval of National School Meals Policy • Preparation of action plan for implementation of National School Meals Policy • Budgetary provisions/commitments by GoB to implement the programme independently (inclusion in revenue budget) • Inclusion of programme components in govt. priorities/policies (NPAN2) • NPAN 2 • School Feeding Policy and other policy documents • Programme documents and monitoring reports • Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE • Data from focused group discussions with mothers and SMC members • KII with representatives of MoPME and WFP • Desk Review of government document, policies and action plan • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE • Focus group discussions with mothers and SMC members • Narrative/ thematic analysis of secondary data • Analysis of data through KIIs and FGDs • Comparing views of GoB and WFP Strong 82 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability • Are local communities fully involved in and contributing toward school feeding and education activities? • Preparation and operationalization of community engagement strategy • Community participation in school management and biscuit distribution • NPAN 2 • School Feeding Policy and other policy documents • Programme documents and monitoring reports • Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE • Data from focused group discussions with mothers and SMC members • KII with representatives of MoPME and WFP • Desk Review of government document, policies and action plan • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE • Focus group discussions with mothers and SMC members • Narrative/ thematic analysis of secondary data • Analysis of data through KIIs and FGDs • Comparing views of GoB and WFP • Comparing views of WFP and schools with SMC and parents Strong IMPACT • What is the intermediate impact of the project? Have there been any unintended outcomes, either positive or negative? What internal and external factors affect the project’s achievement of intended results? • Number of students regularly (80%) attending USDA supported classrooms/schools • Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance • Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance • Number of students enrolled in school receiving USDA assistance • Number of public-private partnerships formed as a result of USDA assistance • School attendance and enrolment records • Biscuit distribution and receipt records • Programme Implementation Plan (PIP and Action plan) • Baseline report • Programme monitoring reports • SMC minutes of meetings • Data from quantitative survey with children, school staff, teachers and parents • Data from focused group discussions with mothers, students(boys • Document review • Storeroom observation • Workshop/ Key informant interviews (KII) with WFP programme staff, implementing partners, representatives MoPME, DPE, implementing partners • Focus group discussions with mothers, students(boys and girls) and SMC members • Narrative/ thematic analysis of secondary data • Qualitative and quantitative analysis of primary data (KIIs) • Methodological triangulation (Quantitative and Qualitative) • Comparing the data on attendance from schools records with information received from parents Strong 83 Research Questions Key information areas and Indicators Main Source of Data/ Information Data Collection Method Data Analysis Methods/ Triangulation Evidence availability/r eliability • Value of new public and private sector investments leveraged as a result of USDA assistance • Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance • Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance • Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance • Number of schools using an improved water source • Number of schools with improved sanitation facilities • Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text • Degree to which SMC members perceive the impact of programme activities in improving attendance, reading abilities, increased adoption of better health, hygiene and nutrition practices and girls) and SMC members • Data from Key Informant Interviews with WFP staff, representatives MoPME, DPE, implementing partners • Outcome reports of other similar programmes • Quantitative survey with the children, store keeper, teachers, headmaster and parents • Comparing the data received from school on storage management practices with observation • Comparing the response from WFP and GoB • Comparing the information on availability of infrastructure with response from students • Disaggregation of data by boys and girls • Comparison of response from school with response from girls (through FGDs) • Comparison of midterm values with baseline values • Analysis of school attendance, biscuit distribution and receipt records • Analysis of SMC minutes of meeting Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 84 Annex M: Data Collection Tools 11. The quantitative data collection tools have been attached as a separate document. Mapping of the indicators with the type of tools and the questions is provided in table 10. Table 12: Mapping of the indicators with the type of tools and the questions Standard Indicator Number Performance Indicator Source Type o Questionnaire Question No. 1 Number of students regularly (80% attending USDA supported classrooms/schools Primary survey School Questionnaire Student Questionnaire Questions 85, 86 Questions 7, 8 2 Number of textbooks and othe teaching and learning material provided as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Student Questionnaire Teacher Questionnaire Questions 7,8, 57 Question 26 Question 14 3 Number of school administrator and officials in target schools who demonstrate use of new technique or tools as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Class room observation Headmaster Questionnaire Question 88 Questions 1,2,3,4 Questions 4,5 4 Number of school administrator and officials trained or certified as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Headmaster Questionnaire Questions 88 Questions 2,3 5 Number o teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Primary survey School Questionnaire Teacher Questionnaire Class room observation Headmaster Questionnaire Question 88 Questions 11, 12, 13 Questions 1,2,3,4 Questions 4,5 6 Number o teachers/educators/teaching assistants trained or certified as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Teacher Questionnaire Headmaster Questionnaire Question 88 Questions 9, 10 Questions 2,3 7 Number of educational facilities (i.e school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Questions 14, 20, 21,57 8 Number of students enrolled in school receiving USDA assistance WFP monitoring reports Primary survey School Questionnaire Questions 83, 84 9 Number of Parent-Teache Associations (PTAs) or simila “school” governance structure supported as a result of USDA assistance Primary survey School Questionnaire Questions 63,64,76,77 10 Number of public-private partnerships formed as a result o USDA assistance Secondary; WFP 11 Value of new public and private sector investments leveraged as a result of USDA assistance Secondary; WFP Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 85 Standard Indicator Number Performance Indicator Source Type o Questionnaire Question No. 12 Number of educational policies regulations and/or administrative procedures in each of the following stages of development as a result o USDA assistance: Secondary; WFP Stage 1: Analysed Stage 2: Drafted and presented fo public/stakeholder consultation Stage 3: Presented fo legislation/decree Stage 4 Passed/Approved Stage 5: Passed for which implementation has begun 15 Number of daily school meal (breakfast, snack, lunch)68 provided to school-age children as a result o USDA assistance Primary survey School Questionnaire Student Questionnaire Storekeeper Questionnaire Questions 56,81 Questions 12, 13,16 Question 37 16 Number of school-age children receiving daily school meal (breakfast, snack, lunch)69 as a resul of USDA assistance Primary survey School Questionnaire Student Questionnaire Storekeeper Questionnaire Questions 56,81 Questions 12, 13,16 Question 37 17 Number of social assistance beneficiaries participating in productive safety nets as a result o USDA assistance Secondary; WFP monitoring reports 18 Number of individuals trained in child health and nutrition as a resul of USDA assistance Primary Survey WFP; monitoring reports School Questionnaire Teachers Questionnaire Parents Questionnaire Questions 28,29,39,43,44,45,88 Question 22 Questions 39,40,41 19 Number of individuals who demonstrate use of new child health and nutrition practices as a result o USDA assistance Primary survey Parent Questionnaire Student Questionnaire Questions 33,34,35,37 Question 37 20 Number of individuals trained in safe food preparation and storage as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Store Keepe Questionnaire Question 82 Questions 9,10 21 Number of individuals who demonstrate use of new safe food preparation and storage practices a a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Store Keepe Questionnaire Question 88 Questions 7,8,11 22 Number of schools using an improved water source WFP monitoring reports Primary survey School Questionnaire Question 14 23 Number of schools with improved sanitation facilities WFP monitoring reports Primary survey School Questionnaire Question 20 24 Number of students receiving deworming medication(s) WFP monitoring reports School Questionnaire Questions 57 68 School Feeding Programme in Bangladesh provides biscuits. Hence, data will be obtained is for number of biscuits provided to school age children as a result of USDA. 69 Data will be collected for number of children receiving biscuits as a result of USDA assistance. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 86 Standard Indicator Number Performance Indicator Source Type o Questionnaire Question No. Primary survey 26 Percent of students who, by the end of two grades of primary schooling demonstrate that they can read and understand the meaning of grade level text WFP monitoring reports Primary survey EGRA questionnaire Questions 1,2,3, 4 27 Number of individuals benefiting directly from USDA-funded interventions WFP monitoring reports 28 Number of individuals benefiting indirectly from USDA-funded interventions WFP monitoring reports Custom 1 Percent of teachers in target school who attend and teach school at leas 90 percent of scheduled school day per school year WFP monitoring reports Primary survey School Questionnaire Questions 89 2 Number of classroom libraries (book shelves with books) established as a result of USDA assistance WFP monitoring reports Primary survey School Questionnaire Student Questionnaire Question 6,7,8 Question 26 3 Percent of students in classroom identified as attentive by thei teachers Primary survey School Questionnaire Question 87 4 Average number of school day missed by each student due to illnes (for each school and in aggregate) Primary survey School Questionnaire Student Questionnaire Question 85 Questions 7,8 5 Percent of parents in targe communities who can name at leas three benefits of primary education Primary survey Parent Questionnaire Question 12 6 Percent of students who can identify at least three key health and hygiene practices Primary survey Student Questionnaire Question 37 7 Percent of storekeepers who can identify at least three safe storage practices Primary survey Storekeeper Questionnaire Questions 11, 29-36 9 Number of 'Little Doctor' student supported by WFP WFP monitoring reports Primary survey School Questionnaire Student Questionnaire Question 42 Question 39 11 Number o meetings/workshops/training sessions held for institutiona capacity to implement SF as a resul of USDA assistance Secondary; WFP monitoring reports 12 Number of pilot initiatives supported to design SF modalities as a result o USDA assistance Secondary; WFP monitoring reports 13 Number of government staff trained as a result of USDA assistance Secondary; WFP monitoring reports 14 Number of technical training fo system development fo service/food procurement, quality control, supply chain, and strengthened online database system and gender mainstreaming in program as a result of USDA assistance Secondary; WFP monitoring reports 15 Number of schools supported by the Government with school feeding as a result of USDA assistance Secondary; WFP monitoring reports Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 87 Standard Indicator Number Performance Indicator Source Type o Questionnaire Question No. 16 Number of socia mobilization/community meeting as a result of USDA assistance Secondary; WFP monitoring reports 17 Number of community mobilization workshops organized as a result o USDA assistance Secondary; WFP monitoring reports 18 Number of teachers, parents and school management committee members attended the community mobilization workshops Secondary; WFP monitoring reports Primary Survey Parents Questionnaire Teachers Questionnaire Question 39 Questions 22 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 88 1. SCHOOL QUESTIONNAIRE INTERVIEW INFORMATION A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. Name of the school A4. School code A5. Number of grades in the school Multiple responses possible Pre-school 1 Class 1 2 Class 2 3 Class 3 4 Class 4 5 Class 5 6 A6. Number of children in the school (Please note from school records) Boys Girls Pre-school Primary A7. Number of teachers Male Fema le Pre-school Primary A8. Date of visit (dd/mm/yy) A9. Time of survey (in am/pm) A10. Name of the enumerator A11. Name of the supervisor A12. Designation of the person being interviewed (Multiple responses possible) School principal/head master 1 School vice-principal 2 School teacher 3 A13. Name of the head master A14. Phone number of the headmaster A15. Area type Urban 1 Rural 2 Peri-urban 3 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 89 SCHOOL FACILITIES S.No. Question Options Codes Skip For each facility that is present, OBSERVE the facility and code accordingly. Ask For Both Programme And Comparison Schools 1. Number of classrooms in the school 2. Does the school have a separate classroom for each grade? (considering the 2 shifts separately) Yes 1 Go to 4 No 2 3. If no, then how many such grades are sharing their class room with other grades? Morning Afternoon 4. Does the school have a Library? Yes 1 No 2 Go to 6 5. If yes, how many supplementary books does the school have in the library? (No. of books from the library records) 6. Does the school have Classroom Libraries? Yes 1 No 2 7. Has the school received books from any organization? Yes 1 No 2 Go to 9 8. If yes, what is the name of the organization? RtR/WFP/MA/YPSA70 1 Government 2 Institutes/Colleges 3 Donated by individuals/any other NGOs 4 Don’t Know 5 Others (specify) 9. Does the school have a source o drinking water for students near or at school? Inside schoo premises 1 Outside schoo premises 2 No drinking water facility 3 Go to 13 10. What is the main source o drinking water? Tap water 1 Bottled water 2 RO plant within schoo 3 Borehole 4 Protected dug well 5 Unprotected well 6 Protected spring 7 Unprotected spring 8 Rainwater collection 9 Rivers or ponds 10 70 RtR: Room to Read, WFP: World Food Programme, MA: Muslim Aid, YPSA: Young Power in Social Action Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 90 SCHOOL FACILITIES Vendor-provided water 11 Tanker truck water 12 Other (Specify) 11. Has the water been treated for drinking by boiling etc.? Yes 1 No 2 Don’t know 3 Not required 4 12. . Is this drinking water available during all times of the day as well? (Ask as well as observe by going to the drinking water facility) Only available during lunch hour 1 Available during al times of the day 2 13. If No, how do the majority of studen get drinking water during schoo hours? Buy Bottled water from shops 1 Children carry water from home 2 Get water from neighbours 3 Other (Specify) 14. Has any water supply systems been rehabilitated / constructed with WFP/support? Yes 1 No 2 15. Does the school have a common Hand Washing facility? Yes 1 No 2 16. Do the students wash their hands using the Hand Washing facility at school? (as per your observation) Yes 1 No 2 17. Does the school have a functioning toilet for the students? Yes 1 No 2 18. Are there separate toilets for boys and girls? Yes 1 No 2 19. How many toilets are there? Total toilets / Functional toilets 20. Are there toilets which have been rehabilitated / constructed with WFP support? Yes 1 No 2 21. . Has the school been provided with needed pumps and motors to improve water systems through WFP support? Yes 1 No 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 91 22. For each functioning toilet, ask the following set of questions (WASH) Record through Observation Functioning Toilet Type of toilet 1. Flush or pour/flush facilities connected to a: (Piped sewer septic, pit latrine) 2. pour/flush toilets without a sewer connection 3. Pit latrines with a slab 4. Pit latrines withou slab/open pit 5. Ventilated improved pi latrines 6. Composting toilets 7. Bucket latrines 8. Hanging toilets/latrines Is the toile for boys or girls? Boys...1 Girls…2 Mixed…3 Is the toile locked? Yes…1 No…2 Does the toilet have water inside for use? Yes, piped water connection…1 Yes, water kept in a container…2 No water inside the toilet…3 Does the toilet have a mug? Yes…1 No…2 Is the toilet maintai ned? Yes: 1 No: 2 1 2 3 4 5 23. Do the toilets have hand washing facilities within the school? Yes 1 No 2 Go to 27 24. Does the handwashing facility have continuous water supply? Yes 1 No 2 25. Does the hand washing facility have soap? Yes 1 No 2 26. How often is it a challenge to have sufficient water for the hand washing facility? Never 1 Rarely 2 Sometimes 3 Always 4 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 92 SCHOOL GARDEN and LITTLE DOCTORS S.No. Question Options Code s Skip Only Ask For Programme Schools. Not to Be Asked in Comparison Schools 27. Does the school have a vegetable garden? Ask the school head to show you the school garden. Only after observing the school garden, mark the correct answer Yes 1 No 2 Go to 34 28. Has there been any training on establishing and maintenance o vegetable garden? Yes 1 No 2 Go to 30 29. If yes, who has provided the training? WFP/MA/YPSA 1 Government 2 Any other, please specify 30. Is there any vegetable/fruit sown or growing in the garden Yes 1 No 2 Go to 32 31. Write the names of the vegetables/fruits that you see are planted. 32. If no, What are the reasons? Soil is not fertile 1 Don’t have anyone to take care of the garden 2 Have not receive any training 3 Any other, please specify 33. What is the source of water for irrigating the school garden? Water supply no available 1 River/stream/canal 2 Pond within the school 3 Pond outside of the school premises 4 Rain water 5 Piped water from the district administration 6 Others (specify) 34. How often is it a challenge to have sufficient water for the garden? Never 1 Rarely 2 Sometimes 3 Always 4 35. Have any “Little Agriculturists” been identified for vegetable gardens? Yes 1 No 2 Go to 41 36. If yes, how many? (specify number Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 93 SCHOOL GARDEN and LITTLE DOCTORS 37. Has there been any training provided to the little agriculturists? Yes 1 No 2 38. Who has provided training to Little Agriculturists? WFP/MA/YPSA 1 Government 2 Any other, please specify 39. On what all aspects have they been provided training? Cultivation practices setting up and maintenance of schoo garden 1 Nutritional value o fruits and vegetables 2 Healthy mea preparation 3 Any other, please specify 40. What are the activities that Little Agriculturists perform? (Multiple responses possible) Setting up and maintenance of schoo garden 1 Disseminating information o nutritional value o fruits and vegetables 2 Supporting healthy mea preparation 3 Any other, please specify 41. Have any “Little Doctors” been identified in the school? Yes 1 No 2 Go to 48 42. If yes, how many? (specify number) 43. Has there been any training provided to Little Doctors? Yes 1 No 2 Go to 48 44. Who has provided training to Little Doctors? WFP/MA/YPSA 1 Government 2 Any other, please specify 45. On what all aspects have they been provided training? (Multiple response possible) Health and hygiene practices 1 Delivery of deworming tablets 2 Conducting health check ups 3 Any other, please specify 46. What are the activities that Little Doctors perform? (Multiple response possible) Support in delivery and administration o deworming tablets 1 Support in Health Check-ups 2 Any other, please specify 47. What is the frequency of support? Once a month 1 Once in 3 months 2 Once in 6 months 3 Once in a year 4 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 94 SCHOOL FEEDING PROGRAMME S.No. Question Options Code s Skip Only Ask For Programme Schools. Not To Be Asked To Comparison Schools. 48. How many days was the schoo open in the April? (before Ramadan) ___Days 49. In the last one month (April) (before Ramadan), how many days were school biscuits provided to the students? ___Days 50. What are the main challenges your school has faced in regularly providing fortified biscuits? Multiple responses possible. No biscuits available due to delay in delivery of stock 1 No adequate supply of biscuits 2 Students do not like the biscuits 3 Frequent absenteeism o storekeeper 4 Others (specify) 51. Does your school have a separate storage room/facility to store the fortified biscuits? Yes, within the school 1 Yes, outside the schoo premises 2 No 3 52. If no, then where are the biscuits stored? Multiple response possible In a class room 1 In teacher’s room 2 Open space 3 In the house of the school/ head/teacher 4 Neighbour’s house 5 Others (Specify) 6 53. Does the school have a Pest Management Plan for their food storage facilities? Yes 1 No 2 54. Does the storage room have wooden pallets above the ground for storage of commodities? (through observation) Yes 1 No 2 Could not observe 3 55. Has the storage room/s been rehabilitated / constructed with WFP/USDA support? Yes 1 No 2 56. Since when is the school receiving support from the school feeding programme supported by WFP? Ask for the year in which the WFP support started. a. Mention the year and month when it started: ____ Year ___ Month b. Duration: ___ Years ___ Months Put “99” for years and months, if the school have not received any suppor yet, but will receive Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 95 SCHOOL FEEDING PROGRAMME Accordingly calculate the number of years till the time of survey (that is 2019). 57. What type of support is the school receiving/has received from WFP, RtR, MA, and YPSA in the last 1 year? (Ask the Headmaster about all the options mentioned here, and at the end also check if there is any additional support) Type of support Yes=1 No=2 Source 1: Government 2: WFP/RtR/MA/YPSA 3: Others 1. Fortified biscuits for the students 1 2 2. Textbooks 1 2 3. School gardening support 1 2 4. Infrastructure support (toilet/store room etc.) 1 2 5. Infrastructure suppor (library/playground/classroom etc.) 1 2 6. Teaching materials / guidelines 1 2 7. Water supply for drinking 1 2 8. Water supply for toilets 1 2 9. Water supply for school garden 1 2 10. Training for teachers / administrators / suppor staff 1 2 11. Receiving deworming medication(s) 1 2 12. Training for students / parents / others 1 2 13. Any other, specify 58. Other than WFP/YPSA/RtR/MA, is this school currently benefitting from any other project with a partner, for example an NGO, charity, private company or the government? Yes 1 No 2 59. If yes, then ask: What type of project or projects is the school participating in? Multiple responses possible Water, hygiene and sanitation (WASH)) 1 Education / enrolment (general) 2 Education / enrolment (girls) 3 Nutrition 4 Anti-malaria or anti- 5 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 96 SCHOOL FEEDING PROGRAMME Dengue (mosquito nets etc.) Others (specify) School Management Committee Ask For Both Programme And Comparison Schools 60. Does the school have a School Management Committee (SMC)? Yes 1 No 2 61. How many members are there in the SMC? Male Female Total 62. Has the SMC received any form of training/workshop? Yes 1 No 2 Go to 59 63. If yes, who has provided the training/conducted the workshop? WFP/YPSA/MA/RtR 1 Government 2 Others, please specify 3 64. If provided by WFP/YPSA/MA/RtR, what were the aspects? Health, Hygiene and Nutrition awareness 1 School infrastructure 2 School Feeding Programme 3 Quality of education 4 Environment Education 5 Gender 6 School Management 7 Any others, please specify 65. Does the SMC have plans for regular meetings? Yes 1 No 2 Go to 61 66. What is the frequency of planned meeting per academic year? No. o planned meetings 67. How many times did the committee actually meet in the last one year? 68. How many members attended the last meeting? 69. How many female members attended the last meeting? 70. Do you maintain minutes of the meeting? Please validate it by checking the minutes of the meeting from the record book and collect copy of the same Yes 1 No 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 97 SCHOOL FEEDING PROGRAMME 71. What are the topics discussed during SMC meetings? Health, Hygiene and Nutrition awareness 1 School infrastructure 2 School Feeding Programme 3 Quality of education 4 Environment Education 5 Others (Specify) 72. In general, how would you describe the functionality of the SMC? Only one response Non-functional / Provides no support to the school. 1 Somewhat functional /Provides some support to the school 2 Highly functional / Provides strong support to the school 3 Only Ask For Programme Schools. Not To Be Asked For Comparison Schools 73. What is the kind of support that the SMC provides to the school feeding programme? Multiple responses possible Management of biscuits stock 1 Management of Parent Teacher Association/ (PTA) 2 Coordination with RtR to execute the activities streamlined under the programme 3 Grievance redressal o provision of fortified biscuits related issues 4 Disseminate materials to visualize good hygiene practises 5 School infrastructure 6 No support 7 Others (specify) 74. Does the school have Parent Teacher Association (PTA)? Yes 1 No 2 75. How many members are there? _____Male _____Female 76. Have the PTA members received any training from WFP/RtR/MA/YPSA Yes 1 No 2 77. If yes, on what all aspects have PTA members received training Health, Hygiene and Nutrition awareness 1 School Feeding Programme 2 Quality of education 3 Environment Education 4 Gender 5 School Management 6 Any others, please specify Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 98 SCHOOL FEEDING PROGRAMME 78. What is the frequency of planned meeting per academic year? 79. How many times did the PTA actually meet in the last one year? 80. What are the topics discussed during PTA meetings? Health, Hygiene and Nutrition awareness 1 School infrastructure 2 School Feeding Programme 3 Quality of education 4 Environment Education 5 Grievance redressal o provision of fortified biscuits related issues 6 Others (Specify) ¨ FOOD UTILISATION DATA ONLY FOR PROGRAMME SCHOOLS. NOT TO BE FILLED FOR COMPARISON SCHOOLS S.No. Question Options Codes Skip 81. Does the school have information for biscuit utilization at school level? Yes 1 No 2 If coded 2 in Q.73, then end the questionnaire Enter “888” if data is not available S.No Months Number of students receiving daily school biscuits Number of days in the month when the school biscuits were provided Total Male Female # 82. 82.1 82.2 82.3 82.4 1 October 2018 2 November 2018 3 December 2018 4 January 2019 5 February 2019 6 March 2019 STUDENT ENROLMENT AND ATTENDANCE Students Current Enrolment and Attendance Data To Be Asked To Both Programme And Comparison Schools S.No. Question Options Codes Skip 83. Does the school have records o Student Enrolment for pas academic years? Yes complete records 1 Yes partial records 2 No 3 If coded 3 in Q.75, then skip to the next section 84. ENROLMENT FOR 2018 AND 2019 ACADEMIC SCHOOL YEARS Complete for Pre-school and Primary only (Grades 1-5) Enter "999" if the class does not exist Enter "888" if data is not available Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 99 STUDENT ENROLMENT AND ATTENDANCE Students Current Enrolment and Attendance Data To Be Asked To Both Programme And Comparison Schools Academic year Enrolment at start of academic school year Total number o students presen (headcount) Total number o students absent Transfers or deceased Left withou reason (dropout) Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls 2019 a. b. c. d. e. f. g. h. Pre￾school 1 2 3 4 5 2018 Pre￾school 1 2 3 4 5 Students Attendance Data in the last 1 year TO BE ASKED TO BOTH PROGRAMME AND COMPARISON SCHOOLS S.No. Question Options Code s Skip 85. Does the school have records of Student Attendance for last academic years? Yes complete records 1 Yes partial records 2 No 3 If coded 3 in Q.77 then skip to the next section If coded 1 or 2 in Q.77, then Complete this section for Pre-school and Primary (grades 1-5) only Enter "999" if the class does not exist Enter "888" if data is not available First, enter the total number of school days for each month (not including holidays or school closures) Then, from the attendance records (2018) write the attendance for students selected for the survey Enter the number of days the students attended school each month. March 2018 July 2018 October 2018 1 2 3 86. Number of schools days Grade Student code Sex (male=1 female=2) March 2018 July2018 October 2018 a. b c d d e 2 3 4 5 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 100 STUDENTS ATTENTIVENESS TO BE ASKED TO BOTH PROGRAMME AND COMPARISON SCHOOLS 87. This section is to be filled for students selected for the survey. From each class, students have been pre-selected for the survey. For each student fill their student code carefully and fill the rest of the information after asking the teachers who teach the respective students. S.No Grade Student code Sex (male=1 female=2) According to teachers, is the pupi generally attentive in class? (E.g. not easily distracted, doesn’t have difficulty staying on task, is able to follow directions, appears organized not sleepy in class etc.) Yes=1, No=2 a. b. c. d. 1 1 2 3 2 4 5 3 6 7 8 9 10 4 11 12 5 13 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 101 TEACHERS TRAINING 88. Teachers Training Data in the last 1 year a. b. c. d. e. f. b. c. d. e. f. g. h. Teachers training data (Have the teachers received training from WFP on----- in the last 1 year? (Use of training received) Source Teacher ID Teacher’s Name (start with Head teacher’s information)) Sex 1. Male 2. Female Position in the school 1. Head teacher/ principal 2. Deputy head teacher 3. Teacher . Full time/ Part time 1. Full￾time 2. Part￾time Which grade does the teacher teach? Storage practices Teaching / Learning techniques (at least 2 days or 16 hours) Hygiene and WASH Child health and nutrition If trained, whether the teacher is using storage practices? If trained, does s/he demonstrate use of new and quality teaching techniques or improved literacy instruction identified by his / her supervisor/ head teacher? If trained, does he/she demonstrate use of child health and nutrition practices? Code 1: Government 2: WFP 3:Others 1 2 3 4 5 6 7 8 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 102 TEACHERS ATTENDANCE DATA IN THE LAST 1 YEAR TO BE ASKED TO BOTH PROGRAMME AND COMPARISON SCHOOLS S.No. Question Options Codes Skip 89. Does the school have records o Teacher Attendance for last academic years? Yes complete records 1 Yes partial records 2 0 No 3 If coded 3 in Q.81, then skip to the next section If coded 1 or 2 in Q.81, then Complete this section for teachers of PRIMARY SCHOOL (grades 1-5) only Enter "999" if the class does not exist Enter "888" if data is not available 90. Teacher Attendance History This question is applicable only for those teachers working in the school at least for a year (Put the number of days he/she attended school. Put 888, if data is not available and 999 if not applicable, i.e. if he / she was not working in that certain month) Teachers ID Sex (male=1 female=2) March 2018 July2018 October 2018 No. working days No. days present No. working days No. days present No. working days No. days present a b c. d. e. f. g. h. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 103 2 TEACHER QUESTIONNAIRE Interview Information A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. Name of the school A4. School code A5. Date of visit (dd/mm/yy) A6. Time of survey (in am/pm) A7. Name of the enumerator A8. Name of the supervisor A9. Has the teacher given her/his consent to participate in the interview? Yes 1 No 2 If the teacher says no, then thank them and proceed to the next teacher on your list. S.No. Question Options Codes Skip 1. What is your name? 2. Gender (Observe only) Male 1 Female 2 3. How many years have you been teaching? ________Number of years 4. For how many years have you been teaching in this school? ________Number of years 5. What position do you hold in this school? Head teacher/principal 1 Assistant/ Deputy head teacher 2 Senior Teacher 3 Teacher (permanent/regular) 4 Teacher (paid contract) 5 Teacher (volunteer) 6 Others 6. What is your highest educationa qualification? MSc or higher 1 Bachelor 2 Diploma 3 Higher/Advanced diploma 4 Technical/vocational 5 Secondary (G9-G11) 6 Middle (G6-G8) 7 Primary (G1-G5) 8 Untrained 9 Other 10 7. Which grades do you teach? Multiple responses possible Class/grade 1 1 Class/grade 2 2 Class/grade 3 3 Class/grade 4 4 Class/grade 5 5 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 104 8. What subject do you teach at school? Multiple responses possible/ Literature / Language (Bengali) 1 Literature / Language (English / other foreign language) 2 Science 3 Mathematics 4 Social science 5 History 6 Religion 7 Arts / Painting 8 Life skills 9 Others (specify) 9. Have you received any training from RtR/WFP? Yes 1 No 2 10. On what all aspects have you received training and a manual? Teaching instruction training Library management Leadership and school management Training 1 Yes 2 No Manual 1 Yes 2 No Training 1 Yes 2 No Manual 1 Yes 2 No Training 1 Yes 2 No Manual 1 Yes 2 No 11. Do you find these trainings and manuals useful? Training 1 Yes 2 No Manual 1 Yes 2 No Training 1 Yes 2 No Manual 1 Yes 2 No Training 1 Yes 2 No Manual 1 Yes 2 No 12. Do you apply the learnings (received from the trainings and manuals)? Yes 1 No 2 13. How often do you apply these learnings? All the time (for all students and all topics) 1 Only for certain topics 2 Only for some students 3 14. What proportion of students in your classroom has the required textbook/s for the class? None 1 Less than 25% 2 25-50% 3 50-75% 4 75-99% 5 100% 6 Don’t Know 7 15. What proportion of students in your class has a desk and chair? None 1 Less than 25% 2 25-50% 3 50-75% 4 75-99% 5 100% 6 Don’t Know 7 16. Only to be asked for intervention schools. Do you think that since the schoo feeding programme has started children are more attentive in class? Yes 1 No 2 Don’t Know 3 17. Yes 1 No 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 105 Do you think that the provision o biscuits increases the number o children coming to school? Don’t Know 3 18. For intervention schools, ask According to you, if the provision of biscuits stopped from today, what would be the consequence of students’ attendance? For comparison schools where provision of biscuits is not going on ask, According to you, if the biscuits are provided from today, what would be the result on student attendance? No consequence, attendance will remain the same 1 Attendance will drop by 25% 2 Attendance will drop between 25% and 50% 3 Attendance will go up by more than 50% 4 19. For intervention schools, ask According to you, if the provision of biscuits stopped from today, what would be the consequence on student enrolment? For comparison schools where provision of biscuits is not going on, ask According to you, if the biscuits are provided from today, what would be the consequence on student enrolment? No consequence, enrolment will remain the same 1 Enrolment will drop by 25% 2 Enrolment will drop between 25% and 50% 3 Enrolment go up by more than 25% 4 20. Only to be asked to intervention schools What type of trainings have you received from WFP/RtR on the following Type of training No. o trainings Duratio n Utilizatio n (Y/N) Teaching / Learning techniques (at least 2 days or 16 hours) Storage practices Hygiene and WASH Child health and nutrition Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 106 21. Only to be asked to intervention schools Are you aware of any of these community mobilisation initiatives? 1. Read Play Festival Yes 1 No 2 2. Grade-I Reception Day Yes 1 No 2 3. Healthy Meal Preparation Day Yes 1 No 2 4. Other, please specify Have you attended any of the community mobilisation initiatives? 22. 1. Read Play Festival Yes 1 No 2 2. Grade-I Reception Day Yes 1 No 2 3. Healthy Meal Preparation Day Yes 1 No 2 4. Other, please specify Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 107 3 STUDENT QUESTIONNAIRE Interview Information A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. Name of the school in which the child is studying A4. School code A5. Student Code (Note the code from the code list of children provided for that school) A6. Date of visit (dd/mm/yy) A7. Time of survey (in am/pm) A8. Name of the enumerator A9. Name of the supervisor A10. Has the teacher given her/his consent to interview the student? Yes 1 No 2 Take the permission from the teacher to interview the student. If the teacher says no, then thank them and proceed to the next student on your list. S.No. Question Options Codes Skip 1. What is your name? 2. Gender (Observe only) Male 1 Female 2 3. What is your age? (in completed years) (Confirm age with teacher) 4. Class/grade in which the child is currently enrolled Class/grade 1 1 Class/grade 2 2 Class/grade 3 3 Class/grade 4 4 Class/grade 5 5 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 108 S.No. Question Options Codes Skip 5. How long does it take for you to get to school? (in minutes) 6. How do you commute to school? Walk 1 Motor-vehicle (private) 2 Motor-vehicle (public) 3 Bicycle 4 Motorbike 5 Others (Specify) Absenteeism 7. Today is [Monday, Tuesday etc.]. Since last [Monday, Tuesday etc.] that is in the last 1 week, have you been absent from school? Yes 1 No 2 Go to 11 8. If yes, then how many days were you absent in the last one week? ____ Days If 0 then go to 11 9. What was the reason of this absence? (Do not prompt; probe for reasons) Multiple responses possible Reasons Code Number o days I fell sick/health issues 1 My parents asked me to stay at home to help them out (with household chores, farm work, take care of siblings) 2 My home is far away from school/ I did not feel like walking to school 3 Teacher was not in school 4 I was hungry and sufficien food was not available for breakfast 5 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 109 S.No. Question Options Codes Skip Due to bad weather conditions 6 There was a festival in my village/ family function a home 7 No particular reason 8 Don’t want to say 9 Others (Specify) 10. If coded 1 in para 9 then ask, You said that you were sick. So wha was wrong? Stomach ache 1 Fever 2 Headache 3 Tiredness 4 Cold and Cough 5 Vomiting 6 Diarrhoea 7 Others (Specify) 11. In the last 24 hours, did you have…? Code Circle 1 for Yes, and 2 for No If Yes in column 2, ask if the food was enough? Circle 1 for Yes, and 2 for No If No in column 2, ask what was the main reason? Circle the code 1 2 3 4 Breakfast 1 2 1 2 Not hungry 1 Food was not available 2 Illness 3 Not enough 4 Other (specify) Lunch 1 2 1 2 Not hungry 1 Food was not available 2 Illness 3 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 110 Not enough 4 Other (specify) Dinner 1 2 1 2 Not hungry 1 Food was not available 2 Illness 3 Not enough 4 Other (specify) Mid￾morning/late afternoon snacks 1 2 1 2 Not hungry 1 Food was not available 2 Illness 3 Not enough 4 Other (specify) School Feeding Programme Only to be asked for project schools where school biscuit are provided. S.No. Question Options Codes Skip 12 In the past 6 school days, did you receive any biscuits from the school to eat during the school hours? Yes 1 No 2 13 If yes, how many days did you receive the biscuits during the school hours? __Days 14. Have you eaten the biscuits received during the school hours? Yes 1 No 2 15. If never eaten the biscuits at school during the school hours, what were the reasons? Multiple response possible Was not hungry 1 Didn’t have time 2 Do not like the taste of the biscuits 3 Gave the biscuit to someone else 4 Ate lunch / snacks brought from home 5 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 111 School Feeding Programme Only to be asked for project schools where school biscuit are provided. S.No. Question Options Codes Skip Others (specify) 16. On an average in a school week, how many days does the school provide you biscuits in school? __Days 17. Do you ever feel hungry even after eating the biscuits in the school? Yes 1 No 2 18. On days when the biscuit is provided, do you bring your own lunch/ snacks from home to eat? Yes 1 No 2 Don’t remember 3 19. Does the biscuit provided act as an incentive for going to school every day? Yes 1 It does not matter 2 Can’t say 3 20. What do you eat on days when the school does not provide biscuits? Get snacks from home 1 Go home and eat food 2 Stayed back at home 3 Get money from home to buy food from canteen 4 Go hungry/skip meal 5 Eat with friends 6 Others (Specify) 21. If you become aware that tomorrow biscuits will not be available, will you go to school? Yes 1 No 2 School Facilities and Environment S.No. Question Options Codes Skip 22. What do you like about your school? Being with my friends 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 112 School Facilities and Environment S.No. Question Options Codes Skip Participating in class/schoo activities 2 Working groups 3 Biscuit is provided 4 Physical education/ sports at school 5 Classroom Libraries 6 Learning new things 7 Others (Specify) 8 Can’t say 9 23. Do you like coming to school? Yes 1 No 2 Can’t say 3 24. Do you have a separate classroom for your class? Yes 1 No 2 25. If no, then with how many such grades are you sharing your class? 26. Do you have a library in your school? Yes 1 No 2 27. How often do you visit the library in a week? __Days 28. Does your school have story books other than textbooks for you to borrow? Yes 1 No 2 29. Do you have a separate classroom library? Yes 1 No 2 Go to 36 30. Do you like books available in the classroom library? Yes 1 No 2 31. Yes 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 113 School Facilities and Environment S.No. Question Options Codes Skip Have you issued any books from the classroom library? No 2 Go to 34 32. If yes, how often do you issue books from the classroom library? Once a week 1 Once in two weeks 2 Once a month 3 33. Are you able to read these books comfortably on your own? Yes, always 1 No, I require help reading them every time 2 I only require help sometimes but otherwise can read on my own 3 34. Are you aware of Book Captains? Yes 1 No 2 35. Do you have a Book Captain assigned to your class? Yes 1 No 2 36. How often in the last week did the teacher read a story or a poem which was not in the textbook? Every day 1 Few times 2 Once 3 Never 4 Can’t say 5 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 114 Awareness on Health, Nutrition and Hygiene S.No. Question Options Codes Skip 37. Can you mention a few good nutrition and dietary practices? Hand washing with soap before meal, after defecation, before preparing food etc. 1 Using sanitary latrine 2 Keeping the food away from insects 3 Drinking water from a safe source (e.g. tube well purified safe water from other source) 4 Should give enough food to both boys and girls 5 Eat vegetables and fruits every day to protect our body from diseases 6 Eat fish, meat, egg and drink milk every day for our physical growth 7 Eat rice, bread, potato etc to get energy for our daily work 8 Eat dark color (green yellow and red) vegetable small fish for vitamin to prevent night blindness 9 Eat fortified biscuits in school to get vitamin energy and minerals 10 Wash vegetable before cutting 11 Should not cook vegetables for longer duration or with excessive heat to preserve vitamin 12 Should not spill water during cooking rice 13 38. Yes 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 115 Are you aware of Little Doctors program? No 2 39. Do you have a Little Doctor in your class? Yes 1 No 2 40. What are the activities that Little Doctors perform? (Multiple response possible) Support in delivery and administration o deworming tablets 1 Support in Health Check ups 2 Any other, please specify 41. Do you have a vegetable garden in your school? 1 No 2 42. Are you aware of students being identified as “Little Agriculturists” for vegetable gardens? Yes 1 No 2 43. Are you aware about the Healthy Meal Preparation Day? Yes 1 No 2 44. Have you participated in/attended the Healthy Meal Preparation Day? Yes 1 No 2 45. Do you share the information received on Nutrition and Health practices at home? Yes 1 No 2 Go to 48 46. If yes, with who? Multiple response possible Parents 1 Siblings 2 Neighbours 3 Friends outside school 4 Other family members 5 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 116 Household environment S.No. Question Options Codes Skip 47. Do you have anyone at home (family members / relatives / neighbours but not private teachers) to help you with reading or writing? Yes 1 No 2 Can’t say 3 48. How many days per week do you take extra lessons (paid tuition) after school? Put the number of days. Put “0” if doesn’t take extra lessons 49. During the last week, did anyone in your house encourage you to study? Yes 1 No 2 Can’t say 3 50. During the last week, did anyone in your house read you a story or poem from a book apart from your textbooks? Yes 1 No 2 Can’t say 3 No 2 Can’t say 3 51. Do you read any extracurricular books? Yes 1 No 2 Can’t say 3 52. If yes, did you read any story within last 7 days? Yes 1 No 2 Can’t say 3 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 117 4 PARENT QUESTIONNAIRE Interview Information A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. Name of the school in which the child is studying A4. School code A5 Student Code (Note the code from the code list of children provided for that school) A6. Student name for which the parent is being interviewed. A7. Respondents name A8. Household address A9. Household size A10. Date of visit (dd/mm/yy) A11. Time of survey (in am/pm) A12. Name of the enumerator A13. Name of the supervisor A14. Has the parent given consent for the interview? Yes 1 No 2 If the parent says no, then thank them and proceed to the next parent on your list. Socio-Demographics Characteristics S. No. Question Options Codes Skip 1. Relationship of the respondent with the child Father 1 Mother 2 Both 3 Others (Specify) 2. What is the total number of children in the household? _______Boys Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 118 Socio-Demographics Characteristics S. No. Question Options Codes Skip _______Girls 3. Number of children in primary school in the household _______Boys _______Girls 4. Which of the following items do you have in your household? ITEMS Yes No Radio 1 2 Electricity 1 2 Refrigerator 1 2 Bicycle 1 2 Toilet 1 2 Mobile Phone 1 2 Television 1 2 Motorbike 1 2 Car 1 2 Tractor 1 2 5. What is the main source of drinking water? Piped water into the house 1 Piped water to yard/plot of the house 2 Surface water (river, lake, stream, canal, etc.) 3 Protected well 4 Unprotected well 5 Protected spring 6 Unprotected spring 7 Bore well 8 Rainwater 9 Bottled water/water vendor 10 Tanker 11 Others (Specify) 6. What is your relationship to the head of the household? See code Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 119 Socio-Demographics Characteristics S. No. Question Options Codes Skip 7. What is the highest level of education of the household head? See code 8. What is the main occupation of the head of the household? See code 9. What is his/her average monthly income (either in cash or kind or both)? _______________Amount in taka 10. How many members in the household earn an income (including farming)? S.No. Member Relationship to the child Sex Education (Code) Main Occupation (Code) Monthly income 1 2 3 4 5 6 7 QUESTIONS CODES 9, 13 (3): Relationship: What is you relationship to the household head? 1= head, 2 = spouse, 3 = child, 4 = grandchild, 5 = sibling, 6 = parent, 7 = parent in-law, 8 = son/daughter –in-law, 9 = other (specify) 10, 13 (5): Education - What is the highest level of education? 0= no schooling, 1-12=last Grade passed, 13= higher diploma; 14= technical/ vocational diploma, 15= Bachelor or equivalent, 16= Master or equivalent 17=Pre-primary/Just enrolled, 18= Don’t know, 19=Other (and specify) 11, 13 (6): Main Occupation What is your / their main occupation? 0 = Unemployed, 1=Farmer, 2=Casual, 3=Fishing, 4=Petty trade business 5=Official employee (public /private service), 6=Livestock: Poultry rearing 7=Cottage industry /handicraft 8=Domestic maid, 9=Rickshaw/van push cart 10=Transport worker (e.g. bus/truck), 11=Housewife, 12=Student, 19 = Too old or too young to work, Other (and specify) Questions related to attitude/perspectives about the school/education S. No. Question Options Codes Skip 11. Yes 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 120 Questions related to attitude/perspectives about the school/education S. No. Question Options Codes Skip Do you think education is important and beneficial? No 2 12. If Yes, what according to you are the benefits of primary education? Multiple responses possible Improves literacy rate 1 Improves future opportunities of work for children 2 Helps child’s skill development 3 Helps girls to remain in school and delay early marriage 4 Helps children from different social and ethnic groups to bond 5 Helps children learn more about the world 6 Helps break the cycle of poverty 7 Others 13. Has your child been given textbooks from school? Yes 1 No 2 Go to 39 14. If yes, who provided these textbooks? WFP/RtR/MA/YPSA 1 Government 2 Other, please specify 15. If no, how do you manage? We buy 1 We borrow from others 2 Take used books from senior students 3 Other, please specify 16. At home, do you or someone in your family help (name of the child) in studies or completing the school work? Yes 1 No 2 17. 41. Yes 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 121 Questions related to attitude/perspectives about the school/education S. No. Question Options Codes Skip At home, do you or someone in your family read stories to (name of the child)? No 2 18. 42. Only to be asked for intervention schools According to you what are the benefits of the school feeding programme? Multiple responses possible My child is getting nutritious food for lunch 1 My child stays in school all day/improves attendance 2 The child does not stay hungry in school 3 The child can pay more attention in class 4 Less expense on food 5 Others 19. Is there a Parent Teacher Association (PTA) in your school? Yes 1 No 2 Can’t say 3 20. If yes, then are you a part of the Parent Teacher Association (PTA)? Yes 1 No 2 21. How many meetings have been conducted in the last one year? Number of meetings………… 22. How many meetings have you attended in the last one year? Number of meetings………… 23. Are you aware of the existence of the School Management Committee (SMC)? Yes 1 No 2 24. If yes, what is your perception of the SMC’s involvement in the school activities? High 1 Medium 2 Low 3 25. Are you aware about the responsibilities of SMC? Yes 1 No 2 Go to 51 26. If yes, what all activities do they perform? Improving quality of education 1 Improving school infrastructure 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 122 Questions related to attitude/perspectives about the school/education S. No. Question Options Codes Skip (Do not prompt) Improving management of school 3 Improving school feeding programme 4 Others, please specify 27. What is the perception on the role o SMC in…? Use code: 1 = Useful, 2 = Satisfactory 3 = Not useful Improving quality of education Improving school infrastructure Improving management of school Improving school feeding programme Others 28. 51. Only to be asked for intervention schools What are the existing avenues through which you can ge information or make complaints on the school feeding programme? Meetings with SMC 1 Meetings with School Administrators 2 Suggestion Box 3 Informal communication (verbal) with teachers / SMC members 4 My child (student) 5 None 6 Others Questions related to health and hygiene S. No. Question Options Codes Skip 29. 49. Do you have a toilet in your house? Yes inside the dwelling 1 Yes, inside the plot of the house 2 No 3 30. Do you have access to water inside the toilet? Yes, piped water 1 Yes, a container for water 2 No 3 31. Yes, all members 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 123 Questions related to health and hygiene S. No. Question Options Codes Skip Do your household members use the toilet? Yes, but only some members 2 No 3 32. Do you have a hand washing facility inside your household/in the yard/plot? Yes, piped water with tap 1 Yes, water in a bucket or a container 2 Yes, tippy tap 3 No 4 33. Do you use soap for hand washing in your household? Yes 1 No 2 34. Can you tell me the times when according to you should one wash hands? Multiple responses possible Before eating a meal 1 After eating a meal 2 Before feeding a child 3 Before preparing/handling food 4 After using toilet 5 After handling farm work/animals 6 35. What are the times when you wash hands? (Ask with soap- yes or no for each coded option) Multiple responses possible Options Code s With soap yes-1 no-2 Before eating a meal 1 After eating a meal 2 Before feeding a child 3 Before preparing/handling food 4 After using toilet 5 After handling farm work/animals 6 Others 36. Local health clinic/hospital 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 124 Questions related to health and hygiene S. No. Question Options Codes Skip Where do you get your information related to health, hygiene and nutrition? School Health and hygiene Brochures 2 School teachers 3 Community meetings 4 School PTA meetings 5 NGO/GoB Community health workers 6 Poster and Pamphlet 7 Notice board/ wall magazine / Wal paintings/hording board 8 Radio / Television 9 Video/Documentary Street Drama Show 10 Newspaper / Magazine 11 Others Questions related to the dietary diversity of the child 37. 1=Yes 2=No 9 = Don’t know Food Items (In the last 24 hours (during the day and night), did (CHILD NAME) eat any of these food items? Ask about every single items and record the answer. If any items are consumed less than one tea spoon, record response “2. NO”; Only count them “1.YES” if consumed ≥1 teaspoon.) 1 Cereals, grains, roots and tubers (Food made from grains such as bread, rice, noodles, biscuits, or any other foods made from millet, sorghum, maize, rice, wheat, white potatoes, white yams, manioc, cassava, [other local root crops] or any other foods made from roots or tubers) 2 Legumes/nuts (any foods made from beans, peas, or lentils, or foods made from nuts or seeds such as pumpkin flower seed) 3 Milk and dairy products like cheese, yogurt 4 Meat, fish and eggs 5 Vegetables and leaves (any dark green leafy vegetables such as [local dark green leafy vegetables] 6 Fruits (ripe mangoes, ripe papayas or [other local vitamin A-rich fruits] guava 7 Foods made with red palm oil, red palm nut, or red palm nut pulp sauce (Vitamin A rich oil), etc. Questions related to community mobilisation initiatives (only for intervention school) S. No. Question Options Codes Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 125 Questions related to the dietary diversity of the child 38. Are you aware of any community mobilisation initiatives/the following events? Read Play Festival Yes 1 No 2 Grade-I Reception Day Yes 1 No 2 Other, please specify Yes 1 No 2 39. Have you attended any of the community mobilisation initiatives/events? Read Play Festival Yes 1 No 2 Grade-I Reception Day Yes 1 No 2 Other Yes 1 No 2 40. Have you received any training/workshop on health, hygiene and nutrition? Yes 1 No 2 41. If yes, who provided the training/conducted the workshop? WFP/RtR/YPSA/MA 1 Government officials 2 CHWs 3 Any other, please specify Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 126 5 EGRA Answer Sheet Interview/General Information A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. Name of the school A4. School code A5. Student Code (Note the code from the code list of children provided for that school) A6. Student Name A7. Sex Male 1 Female 2 A8. Current Grade/Class A9. Date of visit (dd/mm/yy) A10. Time of survey (in am/pm) A11. Name of the enumerator A12. Name of the supervisor A13. Has the teacher given her/his consent to interview the student? Yes 1 No 2 Take the permission from the teacher to take the interview of the student If the teacher says no, then thank them and proceed to the next student on your list. Household environment Question Options Code Skip 1 What is the level of your father’s education? No schooling 1 1-9 (last class passed) 2 SSC 3 HSC 4 Degree or equivalent 5 Master or equivalent 6 Only signature 7 Don’t know 8 2 What is the level of your Mother’s education? No schooling 1 1-9 last class passed 2 SSC 3 HSC 4 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 127 Degree or equivalent 5 Master or equivalent 6 Only signature 7 Don’t know 8 3 Do you have anyone at home (family members / relatives / neighbours but not private teachers) to help you with reading or writing? Yes 1 No 2 Don’t know 3 4 How many hours do you do self￾study? Less than 1 hour 1 1 hour 2 2 hours 3 More than 2 hours 4 5 Do you read any extracurricular books? Yes 1 No 2 6 If yes, did you read any story within last 7 days? Please mention how many? Number 7 Have you issued any books from the classroom library? Yes 1 No 2 Go to next section 8 If yes, how often do you issue books from the classroom library? Once a week 1 Once in two weeks 2 Once a month 3 9 Are you able to read these books comfortably on your own? Yes, always 1 No, I require help reading them every time 2 I only require help sometimes but otherwise I can read on my own 3 Student Reading & Understanding skills (EGRA) Achievement in different section of the test (Applicable only for students in grade 3) (Translation) Fluency and Accuracy (Paragraph Reading) Allow the student(s) to read aloud the passage (taken from Grade-2 textbook) written with simple words and ask to answer the questions given below: Stop this section of the test if the child cannot answer anything in the first 60 seconds, and move on the next section 1. Total Words read in 60 Second from the paragraph Number of words 2. Total Correct words Number 3. Total incorrect words in 60 seconds (if the words are skipped then they wil also be counted as incorrect words) Number of incorrect words Reading Comprehension 4. Total number of correct answers given Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 128 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 129 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 130 6 SCHOOL STOREKEEPER QUESTIONNAIRE Interview Information A1. Name of the district A2. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 A3. Name of the village A4. School Name A5. School Code A6. Date of visit (dd/mm/yy) A7. Time of survey (in am/pm) A8. Name of the enumerator A9. Name of the supervisor A10. Has the storekeeper given her/his consent to participate in the interview? Yes 1 No 2 If the storekeeper says no, then thank them and proceed to the next storekeeper on your list. S.No. Question Options Codes Skip 1. What is your name? 2. Gender (Observe only) Male 1 Female 2 3. Since when have you been appointed as the storekeeper? ________Number of years 4. As a store keeper, how much time do you spend on this role in a day? ________Number of hours 5. Do you have a record book where you keep a stock of the food? (Request if you could see the record book.) Yes 1 No 2 If response is 2 Skip to Q7 6. When was the last time that the record book was updated? This week 1 Last week 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 131 S.No. Question Options Codes Skip Verify from the record book Last month 3 Two months back 4 More than 2 months back 5 Last year 6 No record found in the book 7 7. Does the school have a pest/insects management plan? Yes 1 No 2 8. Does the school carry out pest/insects control measures? Yes 1 9. Are you trained in safe food preparation and storage practices? Yes 1 If response is 2 Skip to Q11 No 2 10. If yes, who has provided training? WFP/MA/YPSA 1 Government 2 Any other, please specify 11. What according to you constitute safe storage practices? (Do not prompt) Keeping the store room properly ventilated? 1Yes 2 No Pest Control 1 Yes 2 No Keeping the room dry and free from moisture 1 Yes 2 No Stacking the stocks at an elevated level (using pallets) 1 Yes 2 No Others, please specify S.No. Question Options Codes Skip Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 132 12. How many such trainings did you receive in last one year … Write 99 if does not remember 13. Have you received a book abou Warehouse management within the last one year? Yes 1 No 2 Don’t remember 3 14. Are you paid a salary for being a storekeeper here? Yes 1 No 2 15. Does your school have a dedicated store room for biscuits? Yes, within the school 1 Yes, outside the school 2 No 3 If response is 3, Skip to Q16 16. If the store room is outside the school, then how far is the distance to the store room? _____in km 17. If No, where are the biscuits stored? In a classroom 1 In teacher’s room 2 Open space 3 In the house of head master/teacher 4 Neighbour’s home 5 Others 18. Can the food storage room be locked? (Question and observe) Yes 1 No 2 19. When you are on leave who manages biscuit distribution? Head Master 1 Teacher 2 School Suppor Staff(non-teaching) 3 No one manages, room is locked 4 20. Every week 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 133 When do you receive new stock o biscuits Every 15 days 2 Every month 3 Once in two month 4 Any other please specify 21. How much time does it take to receive the biscuits, once the requisition is made Stock is delivered before any requisition 1 One week 2 15 days 3 One month 4 More than a month 5 22. Has there been an instance when the stock delivered was less than quantity demanded Yes 1 No 2 Go to 23 23. If yes, how many times this has happened in the last one year Enumerator to record reasons Once 1 2 times 2 More than 2 times 3 S.No. Question Options Codes Skip 24. Has there been an instance of stock out Yes 1 No 2 Go to 25 25. If yes, how many times this has happened in the last one year Enumerator to record reasons Once 1 2 times 2 More than 2 times 3 26. What was the common reason for stock out Delayed supply from MA/YPSA/WFP despite timely requisition 1 Delay in requisition 2 Supply of stock not fit for consumption 3 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 134 27. Do you maintain buffer stock Yes 1 No 2 28. If yes for how many days One week 1 15 days 2 One month 3 OBSERVATION OF THE STORAGE FACILITY 29. Is the store room properly ventilated? DIRECT OBSERVATION Yes 1 No 2 30. Is there any evidence of presence of rodents in the store? (Question and observe) Yes 1 No 2 31. Is there any evidence of presence of insects (weevil and others)? (Question and observe) Yes 1 No 2 32. Is there any evidence of mould and excess of humidity? (Question and observe) Yes 1 No 2 33. Is there any evidence of spillage or leakage? (Question and observe) Yes 1 No 2 34. Is the food stored off the ground? (Question and observe) Yes 1 If response is 2, Skip to Q27 No 2 35. If yes, does the school use improvised raised pallets for commodities' storage? (Question and observe) Yes 1 No 2 36. Is the quantity of biscuits provided enough to consume every school day? Yes 1 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 135 37. Biscuit Receipt Record (Demand and Supply) Month Quantity o biscuits required Quantity o biscuits received Date of request Actual delivery date Quantity of biscuits distributed to students* March 2019 February 2019 January 2019 December 2018 November 2018 October 2018 *Please verify from the consumption record book and click a picture of the page Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 136 7 CLASS OBSERVATION Interview Information A1. Name of the district A2. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A3. Name of the village A4. School Name A5. School Code A6. Name of the teacher A7. Class A8. Subject A9. Date of visit (dd/mm/yy) A10. Time of survey (in am/pm) A11. Name of the enumerator A12. Name of the supervisor A13. Has the teacher given her/his consent for the class observation? Yes 1 No 2 If the teacher says no, then thank them and proceed to the next teacher on your list. Please observe the teaching technique of the teacher in the class for 45 minutes S.No. Question Options Code Skip 1. Did the teacher apply any participatory technique during the class? (Multiple Response) No 1 Role play & asking question instead of giving the message straightway 2 Story telling (to make student understand instead of giving the message straightway) 3 Group Discussion 4 Problem solving: by discussion / as a group 5 Other (Specify) 2. Did the teacher use any audio￾visual material? No 1 Pictures from text Book 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 137 Please observe the teaching technique of the teacher in the class for 45 minutes (Multiple Responses) Handmade posters 3 Printed visual aids 4 Drawing and writing at the board 5 Playing video / audio (for teaching/learning purpose) 6 Other (Specify) 3. Did the teacher make effort to engage students? (Multiple Response) No 1 Gave classroom exercise 2 Asking students to write at the blackboard (not as a punishment 3 Identify inattentive students and give special care 4 Ask the students if they have understood the word or topics being discussed 5 Discuss new word or concept with meaning and examples 6 Allowing students to reading loudly / silently at the class 7 Encourage students to ask questions 8 Gave answer nicely if there is a question from students 9 Other (specify) 4. Other classroom observations: (Multiple Response) Teachers made effort to make the class interesting 1 Teacher initiates topics related discussion from the common experiences before start discussing on the main topics 2 Teacher repeatedly and properly discussed difficult topics and words 3 Teacher reviews the content and evaluates the learning of the students at the end of class 4 Teacher follows lesson plan 5 Teacher helps slow learners to ensure their learning 6 Other (Specify) Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 138 8 Headmaster Questionnaire Interview Information A1. Name of the sub-district/Upazila Ukhiya 1 Kutubdia 2 Ramu 3 A2. Name of the village A3. School Name A4. School code A5. Sex Male 1 Female 2 A6. Date of visit (dd/mm/yy) A7. Time of survey (in am/pm) A8. Name of the enumerator A9. Name of the supervisor 1. What does the school authority or administrator do to improve teaching/classroom environment? (Multiple Response) Nothing 1 Regular monitoring of teaching techniques of teachers 2 Provide feedback to teachers for improvement of the quality and techniques 3 Conduct regular training on for teachers 4 Other (Specify) 2. Have the teachers in your school received training from WFP/RtR in the last one year (for Intervention Schools Only) Yes No 3. If yes, on what all aspects have the teachers received training(last one year) (Multiple Response) Teaching / Learning techniques (at least 2 days or 16 hours) 1 Storage practices 2 Hygiene and WASH 3 Child health and nutrition 4 4. How do you support adoption of new techniques by the teachers? (Multiple Response) Do nothing 1 Supplementary reading materials are made available to teachers 2 Encourage teachers to use audio-visual aid during class 3 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 139 Encourage teachers to prepare handmade posters / locally made visual aids 4 Encourage teachers to adopt participatory teaching techniques (e.g. Role play / Story telling /Group Discussion / Problem solving) 5 Reward/recognition for teachers 6 Other (Specify) 5. How do you monitor adoption of new techniques by teachers (Multiple Response) Do nothing 1 Classroom observation 2 Feedback from students 3 Feedback from teachers 4 Feedback from parents 5 6. Only to be asked for intervention schools. Do you think that since the school feeding programme has started, children are more attentive in class? Yes 1 No 2 Don’t Know 3 7. Only to be asked for intervention schools. Do you think that the provision of biscuits increases the number of children coming to school? Yes 1 No 2 Don’t Know 3 8. For intervention schools, ask According to you, if the provision of biscuits stopped from today, what would be the consequence of students’ attendance? For comparison schools where provision of biscuits is not going on ask, According to you, if the biscuits are provided from today, what would be the result on student attendance? No consequence, attendance will remain the same 1 Attendance will drop by 25% 2 Attendance will drop between 25% and 50% 3 Attendance will go up by more than 50% 4 9. For intervention schools, ask According to you, if the provision of biscuits stopped from today, what would be the consequence on student enrolment? For comparison schools where provision of biscuits is not going on, ask According to you, if the biscuits are provided from today, what would be the consequence on student enrolment? No consequence, enrolment will remain the same 1 Enrolment will drop by 25% 2 Enrolment will drop between 25% and 50% 3 Enrolment go up by more than 25% 4 10. Are there measures in place/functional for improvement and maintenance of school infrastructure? No 1 Cleanliness of the school surrounding 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 140 (Multiple Response) Regular cleaning of classrooms 3 Regular cleaning of teacher’s rooms 4 Proper maintenance of school latrines 5 Proper maintenance of classroom and furniture 6 Availability of clean water for teachers and students 7 Other (specify) Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 141 FOCUS GROUP DISCUSSIONS: CHILDREN (Grades 3, 4, 5) Name of the School: Name of the Upazila: No. of boys: No. of girls: S. No Name Age Gender Grade Icebreaker 5 minutes Start with a 3-5 minutes ice-breaker, children have to throw a ball at each other and say their name. After a minute throw another ball so that children have two balls being thrown around, which will create positive confusion and break the ice. Note: Ensure that all children feel comfortable and ask the questions in a conversational manner, without making them feel any pressure to answer. A. INTRODUCTION 1. Please tell us something about your school? What do you like most about your school/what is the favourite part of your day? 2. Which grade would you like to study till? Why? (ask separately for girls and boys) 3. Do you know of anyone who has left (dropped out) the school? If yes, why? (ask separately for every class) 4. Was there any activity/ceremony held when new students joined? If yes, what happened as part of the activity? Can you describe it? 5. Do you eat anything before coming to school? If not, why? 6. When you come to school, do you feel hungry? If yes, what do you do? (Stay hungry or eat something?) • When else do you usually feel hungry? (Probe: ask class period wise to understand time) 7. Do you carry a tiffin to school? Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 142 Only Ask Intervention Schools (Q: 10-Q: 12) 8. Do you receive biscuits? If yes, • What time do you get them? When do you eat them? • Are you still hungry after eating them? • If yes, what do you do? (Probe: eat tiffin, stay hungry, go back home during recess) • Do you get these biscuits every day? • Do you like eating them? 9. If you stop receiving biscuits, would you continue coming to school? 10. Do your parents know you get these biscuits? What do they think/say about them? B. NUTRITION & HEALTH 11. Ideal diet vs. Actual diet There will be two pie charts presented to the participants with different categories of food items required to meet the nutritional requirements of children. From the first pie chart, the participants will choose food items which they perceive to be a part of an ideal on a daily basis. However, from the second pie chart the participants will choose food items that they consumed in the last 24 hours (representative of what they consume of a daily basis). Ask the following questions as part of the activity: 12. From the pie-chart, what all should you eat and why? 13. How many meals have you had in the last 24 hours- 1 meal/ 2 meals/ 3 meals/more than 3? (Check for each respondent and note response) 14. From the pie chart, what all have you eaten in the past 24 hrs as part of all the meals? (To be marked as per the pie chart for each respondent) 15. Who tells you about what to eat? (Probe: family, teachers, peers, television, IEC material, community health workers, others) 16. Are you aware of any vegetable garden? If yes, where is it? • Why do you have a vegetable garden? • Do you know what all is grown in these gardens? Do you like eating the vegetables that are grown? • Do you help grow/maintain the vegetables in the garden? How often? • Have you talked about the vegetable garden at home? • Do you eat any of these vegetables at home? If not, do you then ask your parents to buy these vegetables? 17. Do you eat meals prepared from the vegetables grown in the garden? (Probe: Healthy meal preparation day) • How often? Do you like the food? If no, why? • Is the food eaten as a part of this, different from the food you get at home? How? • Is there any celebration/picnic around this activity? Who all participate? (Probe: mothers, teachers) Nuts and seeds Dairy products (milk/ cheese/ yoghurt) Meat/ fish/ eggs Vegetables including green leafy ones Fruits (ripe mango/ papaya/ guava etc.) Grains/ roots/ tubers Legumes/ beans/ pulses Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 143 • Do you talk to your parents about these meals and the celebration/picnic? • Do you ask your parents to prepare the same kind of meals at home? C. LITERACY 18. Do you have textbooks? • If yes, are these provided by the schools or purchased / borrowed? 19. Do you like reading books? Do you find it easy to read them? If not, why? 20. Does anyone help you read books/stories? Who? Do they read them out? 21. Does your school have a library? • If yes, is it stocked with books? • Do children borrow books from it? If not, why? • Do you like the books available in the library? Do you find them interesting? 22. Does your class have a library corner? • If yes, is it stocked with books? • Do children borrow books from it? If not, why? • Do you like the books available in the library? Do you find them interesting? 23. Do you have a ‘book captain’ in your class? • If yes, what does the captain do? • How does one become a book captain? • Would you like to be a book captain? Why/Why not? • If yes, what are you doing about it? 24. Have you heard about ‘Read Play Festival’? • If yes, what happened in this festival? • Who all participated? • Did you enjoy it? Why/why not? 25. Have these activities increased your desire to read more? 26. Do you know how to speak, read, write Bangla? (record for each student) If no, are you learning how to speak/read and write Bangla in school? 27. Who teaches you? What challenges do you face? 28. What more help do you require? 29. Amongst the options listed, have you seen the adoption of these by your teachers in the last 6 months (Probe for perceptions on use of the following) • Audio-visual tools • Chart papers • Role playing and asking questions • Story telling • Group discussion • Collective problem solving 30. Have these techniques helped you? How? • What other support would you require from the teacher to help you learn better? D. WASH and Health 31. Do you carry a water bottle? Do you carry water from home? 32. Do you drink water in school? If yes, from where? • Is it always available? Has there been a situation when drinking water has been unavailable? If yes, please tell us about it. What was done in such a situation? 33. Do you have access to toilets in school? Do you have a separate toilet for girls and boys? Do you use them? If no, why? (Probe: non-functional toilets (broken/ closed), not clean, not enough toilets, unavailability of water, unavailability of soap) 34. Who gives you information with regard to health and hygiene? (Probe: teachers, family, friends, school education programmes, school education programmes, community health workers etc.) • Do you find this information useful? Why/ why not? • Do you speak about these at home? 35. Do you have ‘Little Doctors’ in your school? Do you have one in your class? • If yes, what do they do? (Probe: checking height, weight, de-worming program). Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 144 • How many Little Doctors are identified? How does one become a Little Doctor? • What information do you get from the ‘Little Doctors’? Do you find it useful? Meet a few ‘Little Doctors’ and ask them about their experience. *End the discussion by clapping for them and asking them to clap for each other and thank them for participating. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 145 FOCUS GROUP DISCUSSIONS: PARENTS Name of the School: Name of the Village: Name of the Upazila: No. of Children in Primary School S. No Name of the Parent Age Gender Boy/s Girl/s Rapport Building 1. Please tell me something about your village. How many households are there? What is the main occupation of men and women in your village? 2. Does this school fall in your village? If no school within the village, then what is the distance to the nearest school? How do children commute to school? 3. Are there any challenges in reaching the school? (Probe: during monsoons, flood situation) 4. Do all children go to school? Are there any households in your village where children do not go to school? If so, then why? 5. Why do you think it is important for the children to go to school? (Probe for: different reasons like understanding the importance of education, or they want to keep them at school as they have to go to work, or they receive nutritious meals in the school, etc.) 6. What are your expectations from the school to fulfil the aspirations for your child? 7. Do you think the school is adequately addressing your expectations? If not, what more is required? Map on the table below. STRENGTHS IDEAS WEAKNESSES  Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 146 8. At home, who decides if a child should go to school or not? How many years of school would you like to see your own children complete and why? (Probe for differences between boys and girls) 9. Have any of your children/someone else’s children joined school in the past year? If yes, was there any activity/ceremony held on them joining? If yes, what happened as part of the activity? Can you describe it? 10. Does the school conduct parent-teacher meetings? Do you attend? What are the benefits? 11. Do you know the teachers who teach your child? Are you satisfied with their teaching? What more do you think they can do to benefit the children? 12. Are you aware of the various activities being undertaken by the school? If yes, what are these? Please describe what you know for each: • Biscuit distribution • Vegetable Gardens • Healthy Meal Preparation Day • Little Doctors • Deworming Program • Book Captains • Library Corners • Remedial Classes in Bangla • Read Play Festival 13. Does your child come home and tell you about what was discussed during the activities? Please give details 14. Can you recall a time when your child came back elated/very excited about an activity held in school? Please elaborate. 15. Where else do you get your information regarding these activities? How often?(Probe: any community mobilisation activities) 16. Do you or anyone in the household help your child read story-books and complete homework? • Does your child ask for this help? 17. Does the school conduct any training or activity for parents or the community particularly? If yes, what are these? Please describe? (Probe: Read Play Festival, awareness on adequate nutrition and hygiene practices, healthy meal preparation day etc.) • How often are these conducted? • What kind of information do you get through these? • Do you find this information useful? Why? 18. Do you participate in these activities in any way? Which ones? How do you contribute to these? 19. Given a chance, would you want to contribute towards the following: • Promotion of health & nutrition messages • Promotion of hygiene & sanitation practices • Demand for better services to improve quality of education • Demand for improving school infrastructure • Managing activities such as, biscuit distribution 20. In order to do these, what additional support would you require? (Probe: training, workshops) NUTRITION & HEALTH 21. Ideal diet vs. Actual diet There will be two pie charts presented to the participants with different categories of food items required to meet the nutritional requirements of their children. From the first pie chart, the participants will choose food items which they perceive to be a part of an ideal diet on a daily basis. However, from the second pie chart the participants will choose food items that their child consumed in the last 24 hours (representative of what they consume of a daily basis). Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 147 Ask the following questions as part of the activity: 22. From the pie-chart, what all should your child eat and why? 23. How many meals has your child had in the last 24 hours- 1 meal/ 2 meals/ 3 meals? (Check for each respondent) 24. From the pie chart, what all has your child eaten in the past 24 hrs as part of all their meals? (To be marked as per the pie chart for each respondent) 25. Does your child come and tell you about what he/she has learnt in school regarding nutrition, child health and hygiene? 26. Have you received any training on child health and hygiene practices? • If yes, on what? Please describe. • Who has provided you with these? • How have these trainings helped you? Can you illustrate any daily practice you may have changed as a result of these trainings? • Has this change in practice caused a change in your environment? If yes, how? • Have you been able to help others (family, neighbours, and relatives) adopt these improved daily practices too? How? Please describe. 27. Do you think the activities initiated by the school has increased awareness on children’s health, nutrition, education and WASH related issues? How? Please give examples. 28. Where else do you get information regarding adequate nutrition and healthy eating? (Probe: children’s school, family, teachers, television, internet, IEC material, community health workers, community programmes etc.) Nuts and seeds Dairy products (milk/ cheese/ yoghurt) Meat/ fish/ eggs Vegetables including green leafy ones Fruits (ripe mango/ papaya/ guava etc.) Grains/ roots/ tubers Legumes/ beans/ pulses Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 148 FOCUS GROUP DISCUSSIONS: SCHOOL MANAGEMENT COMMITTEE (SMC) Name of the School/SMC: Name of the Village: Name of the Upazila: No. of members: S. No Name Age Gender Position A. APPOINTMENT, ROLES & RESPONSIBILITIES 1. In which year was the SMC formed? • How was it formed? • How many members are part of the SMC? Who all does it constitute? (Note numbers separately for men and women) • Can you describe how the members and head of the SMC are selected? What were the criteria for selection? 2. How often do the members meet? • How many times have you met in the last 1 year? • Does anyone beside the members attend these meetings? Who? • Are minutes of the meeting recorded? (if yes, ask for SMC MoM register) 3. What is the importance of forming the SMC? • What are its various functions? • What are the strengths of your SMC? • What all is discussed as part of the meetings? Kindly name the issues. • What were some of the most recent issues raised? (ask to recall) • What was the outcome of the meeting in which the issue was raised? (ask for issue related decisions taken) 4. Has the SMC received any form of training in the last one year/Have there been any workshops for the SMC? (Ask for training from government and training from WFP separately) • If yes, what is the training/workshop on? • Who provides this training? (Probe: WFP/RtR/YPSA/MA) How often? • Do you face any challenges? What are these? • Would you like to receive training on other aspects? Like what? • Have the trainings/workshops helped the functioning of the SMC in any way? Please elaborate. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 149 5. What have been the various challenges faced in the school? (Probe: lack of infrastructure, facilities, high drop outs, absenteeism etc.) • How has the SMC contributed in addressing these? Please explain. • Has there been a change in the school’s environment through the SMC’s efforts? What has changes and how? 6. Through the table presented below, can you describe what according to you are the strengths and weaknesses of the programme and its activities? Please give examples. How do you think the programme can be improved to address these challenges? STRENGTHS (What works) IDEAS (What more can be done) WEAKNESSES  (Wha doesn’t work) B. SMC’S ROLE IN THE PROGRAMME 7. What are the major challenges in the community with regard to; education, health and nutrition? (Probe: absenteeism, drop-outs, malnourishment, illnesses etc.) • What are the reasons for these? • Have these improved or worsened in the past year? What are the reasons? 8. Do you think there was a need for the WFP/SF programme? Why? 9. What activities has WFP implemented as part of their programme in your school? (Probe: vegetable gardens, little doctors, book captains, library corners, remedial Bangla classes, Read Play Festival, Healthy Meal Preparation Day, biscuit distribution, awareness generation campaign on importance of education) 10. When did these activities begin? Have you noticed any changes as a result of the activities? 11. What support does the SMC provide in implementing these activities? 12. Has the SMC’s roles and responsibilities been modified in any way since the inception of the programme? How? 13. Do you face any challenges in the programme’s implementation? What are these? • What steps have been taken to address these? 14. Has the programme been revised/modified in any way in the past year? If yes, how? • Do you know the reasons these changes were brought in? • Have these changes been helpful? Why/why not? 15. How do you think the programme has impacted the children? (Probe: improved health and nutrition, improved attendance, improved reading abilities, improved awareness with regard to health etc.) Please give examples. What has helped bring about this change? 16. Do you see any change in the school environment as a result of the activities? If yes, kindly elaborate. 17. Have you observed any change in the parent’s/community’s attitude in perceiving the importance of education and nutrition of their children? How? Please describe. What has helped bring about this change? 18. Do the parents/community support/participate in the programme in any way? If yes, how and on which occasions? Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 150 Checklist for Capacity Building Activities (GoB) 1. We understand that a nutrition sensitive draft ‘National School Meal Policy 2019’ has been prepared. Kindly elaborate on the following￾a) What was the process of preparing this policy? (various activities undertaken prior to/that led to preparing the policy ) b) How many divisional/high level consultations took place during the preparation? What were the learnings from such consultations? c) What were some of the key considerations kept in mind while developing this policy? Please give examples of specific components incorporated as part of the policy. d) Has an implementation plan for the policy been prepared? e) What challenges are foreseen in the adoption and implementation of the same? f) What support would MoPME, MoHFW and other ministries require from WFP to address some of these challenges? 2. What is their role in addition to provided technical assistance on nutrition standards? (Probe: Monitoring of these standards) 3. Have any learning visits on GoB owned system of SF operations been conducted GoB officials and inter￾agency committee members? a) If yes, how many such visits were held and who all participated? b) What were some key observations and learnings? c) How have these learnings been incorporated in the NSMP? d) Have these visits led to increased ownership or a perception change of certain ministries? Kindly give examples e) Have these visits led to any actions by specific committees/ministries/departments? If yes, please give examples 4. What kind of dissemination of the NSMP has been undertaken so far? Specify platforms. 5. Has the dissemination and wider sharing of NSMP led to any feedback/actions? Please elaborate. 6. What kind of and how many technical workshops for targeting of the school feeding areas for expansion and scale up have been undertaken? a) How has this helped in identification of areas for scale up? b) Have any challenges been faced with regards to scale up and expansion? c) What more support is required for the government to be able to independently undertake such targeting for scale up? 7. Has the monitoring and reporting framework for the scale up areas been created? If yes, has it been agreed upon the various ministries and departments? (Request documents for the same) 8. Have the following trainings/workshops/orientations been undertaken? Technical training on food storage Yes/No Technical training on supply chain management Yes/No Technical training on commodity tracking Yes/No Technical training on programme planning Yes/No Programme planning and Review Workshops Yes/No Orientation on SF implementation modalities Yes/No Refresher trainings (in identified weak areas) Yes/No Training on gender/gender equality Yes/No Online database management system Yes/No Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 151 Any other, please specify 9. How are the learnings from these trainings and workshops being used by MoPME/DPE staff? 10. Online Database and Monitoring a) How many districts are currently using the online MIS? b) How has the online monitoring helped improve the implementation of SFP? c) Are any challenges being faced in use of online MIS? d) What more is required to ensure its institutionalization? 11. What kind of support has WFP provided MoPME in ensuring selection of NGOs/implementing partners independently? a) Has DPE begun identifying implementing partners on their own? b) If yes, what kind of orientation activities have been undertaken and by whom? a) Towards this, what has been the nature of support provided by WFP? b) What are the challenges being faced in improving the learning environment? c) What more support is required from WFP to ensure the same? 12. Have any meetings with the Ministry of Finance been held for sustainable financing of NSMP? What is the progress on its inclusion in the revenue budget? 13. What other initiatives/ activities have been undertaken to strengthen and enhance engagement of local organizations and community groups? (Probe: training manuals for community mobilization/advocacy materials/IEC and media coverage) 14. What have been some of the key learnings from the technical assistance provided by WFP to GoB/MoPME? a) Have these learnings been documented? b) How are these learnings being incorporated to inform future activities? Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 152 Checklist for Capacity Building Activities (WFP) 15. We understand that a nutrition sensitive draft ‘National School Meal Policy 2019’ has been prepared. Kindly elaborate on the following￾a) What kind of support did WFP provide to ensure the same? b) What are the next steps? 16. Have any learning visits on GoB owned system of SF operations been conducted for the cabinet division, GoB officials and inter-agency committee members? f) If yes, how many such visits were held and who all participated? g) What were some key observations and learnings? h) How have these learnings been incorporated in the NSMP? i) Have these visits led to increased ownership or a perception change of certain ministries? Kindly give examples 17. Has a cost-benefit analysis of school meal vs biscuits taken place? If no, when is it planned? If yes, what have been the results and how are the findings being incorporated? 18. Has a review on the capacity of government functionaries for implementing school feeding programme been undertaken? a) If yes, what were some of the key areas reviewed? What were the findings? b) Have any meetings been conducted with ministries and directorates for institutional capacity building? 19. Have any meetings with the Ministry of Finance been held for sustainable financing of NSMP? What is the progress on its inclusion in the revenue budget? 20. What has been the progress on the Revised Development Project Proposal (for the new phase of SFP)? 21. What kind of and how many technical workshops for targeting of the school feeding areas for expansion and scale up have been undertaken? d) How has this helped in identification of areas for scale up? e) Have any challenges been faced with regards to scale up and expansion? f) What more support is required for the government to be able to independently undertake such targeting for scale up? 22. Has the monitoring and reporting framework for the scale up areas been created? If yes, has it been agreed upon the various ministries and departments? (Request documents for the same) 23. Have the following trainings/workshops/orientations been undertaken? Technical training on food storage Yes/No Technical training on supply chain management Yes/No Technical training on commodity tracking Yes/No Technical training on programme planning Yes/No Review Workshops Yes/No Orientation on SF implementation modalities Yes/No Refresher trainings (in identified weak areas) Yes/No Training on gender/gender equality Yes/No Online database management system Yes/No Any other, please specify 24. What were some of the learnings and challenges from these trainings/workshops? How are the challenges being addressed? 25. Online Database and Monitoring e) How many districts are currently using the online MIS? Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 153 f) How has the online monitoring helped improve the implementation of SFP? g) Are any challenges being faced in use of online MIS? h) What more is required to ensure its institutionalization? 26. What kind of support has WFP provided MoPME in ensuring selection of NGOs/implementing partners independently? c) Has MoPME begun identifying implementing partners on their own? d) If yes, what kind of orientation activities have been undertaken and by whom? 27. Have any pilots been undertaken towards mainstreaming WASH, Health and Nutrition or to improve the delivery mechanism or improve the learning environment? 28. How is government integrating complementary components such as WASH, health, nutrition, dietary practices and literacy of school children in the hand-over areas? 29. What kind of support has WFP provided to ensure the same? What more is required? 30. How is the government working towards improving learning environment for children? (including feeding and nutrition support services for pre-school children and partnership with ECD network) Please provide details. a) Towards this, what has been the nature of support provided by WFP? b) What are the challenges being faced in improving the learning environment? c) What more support is required from WFP to ensure the same? 31. What other initiatives/activities have been undertaken to strengthen and enhance engagement of local organizations and community groups? (Probe: Circular of functioning of PTA, training manuals for community mobilization/advocacy materials/IEC and media coverage) 32. What have been some of the key learnings from the technical assistance provided by WFP to GoB/MoPME? c) Have these learnings been documented? d) How are these learnings being incorporated to inform future activities? e) Is there an exit strategy for the Capacity Building activities with GoB? f) If yes, what are the aspects that WFP will continue to provide support on and aspects it will stop supporting as a result of adequate capacities built? Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 154 Annex N: Evaluation Mission Schedule Table 13: Evaluation mission schedule Dates Activity Stakeholders Locations/ Sites Scoping Visit 7th -9th May 2019 • Meeting with WFO – CO team-Evaluation Manager, School Feeding Team , Government Capacity Strengthening Officer, Head of Programme, MGD Policy Officer, Head of M&E unit • Meeting with representatives of Implementation Partners- RtR, MA, YPSA • Discussion with USDA representative • Debriefing to the Evaluation Committee NRMC, WFP CO, Implementation Partners Dhaka Inception Phase 10h May – 10th June 2019 Literature Review Preparation of quantitative and qualitative tools Finalization of evaluation approach and methodology Identification of field enumerators and supervisors for data collection Preparation and submission of Inception Report NRMC New Delhi 11th June - 15th July 2019 Review of Inception report and tools by WFP, DEQAS and USDA Incorporating comments in the report Finalization of the Inception report Translation of tool and coding into CAPI NRMC, WFP CO, DEQAS, USDA Dhaka, New Delhi Training 16th July – 18th July 2019 Classroom training of the survey teams on data collection tools and ethical guidelines; Pilot and incorporation of changes in the tools. NRMC Cox’s Bazar 19th July 2019 Field Practice in three schools in Cox’s Bazar and debriefing with the data collection teams Cox’s Bazar Data Collection 21st July – 23rd July 2019 Primary Qualitative and Quantitative Survey in intervention schools Debriefing of team at the end of each field day Quality Check of all filled questionnaires and data collected with the supervisors NRMC Ukhiya, 24th July – 26th July 2019 Primary Qualitative and Quantitative Survey in intervention schools Debriefing of team at the end of each field day Quality Check of all filled questionnaires and data collected with the supervisors NRMC Kutubdia 27th July- 31st July 2019 Primary Qualitative and Quantitative Survey in comparison schools NRMC Ramu Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 155 Dates Activity Stakeholders Locations/ Sites Debriefing of team at the end of each field day Quality Check of all filled questionnaires and data collected with the supervisors 31st July – 1st Aug 2019 KIIs with representatives GoB (MoPME, DPE) NRMC Dhaka 1st August 2019 Debriefing at WFP CO NRMC, WFP CO Dhaka Data Analysis, Reporting and Dissemination Phase 2nd August – 14th September 2019 Quantitative data cleaning, data analysis, documentation of field notes and findings, development of analysis plan and report structure NRMC New Delhi 19th August 2019 Submission of Data Tabulation Plan to WFP NRMC, WFP CO New Delhi 15th September 2019 Submission of Draft Mid- Term Evaluation Report to WFP NRMC New Delhi 16th September – 10th November 2019 Review of Draft Mid- Term Evaluation Report by WFP, DEQAS NRMC, WFP CO, DEQAS Dhaka, New Delhi 11th November- 14th November Validation workshop in Dhaka and Cox’s Bazar NRMC, WFP CO, MoPME, DPE< DAE, DEO, UEO, SMC members, Dhaka, Cox’s Bazar 15th November- 30th November Incorporating comments in the report Preparation of Final Mid- Term Evaluation Report NRMC New Delhi 1st December 2019 Submission of the Final Mid- Term Evaluation Report to WFP NRMC New Delhi Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 156 Annex O: Documents Gathered 12. WFP CO has shared a host of documents for the evaluation. The table below described the nature of documents reviewed. Table 14: Documents Gathered and Reviewed Document Type Comment / Titles & dates of documents received Received - Y/N (N/A) Link to Evaluation matrix Project related documents Y Appraisal mission report N/A Project document (including Logical Framework in Annex F) Country Strategic Plan, Bangladesh (2017-20), Project Agreement between the FAS and the WFP including the results framework Y Relevance Standard Project Reports Y Budget Revisions N/A Note for the record (NFR) from Programme Review Committee meeting (for original operation and budget revisions if any) N/A Approved Excel budget (for original intervention and budget revisions if any) Programme budget summary and narrative Y Relevance, Efficiency Intervention/Project Plan (breakdown of beneficiary figures and food requirements by region/activity/month and partners) Project Agreement between the FAS and the WFP Y Relevance, Effectiveness Other Country Office Strategic Documents (if applicable) N/A Country Strategy Document (if any) Country Strategic Plan, Bangladesh (2017-20), Y Relevance, Sustainability Other Assessment Reports [if applicable] N/A Baseline Evaluation Report Y Comprehensive Food Security and Vulnerability Assessments N/A Crop and Food Security Assessments (FAO/WFP) N/A Emergency Food Security Assessments N/A Food Security Monitoring System Bulletins N/A Market Assessments and Bulletins N/A Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 157 Document Type Comment / Titles & dates of documents received Received - Y/N (N/A) Link to Evaluation matrix Joint Assessment Missions (UNHCR/WFP) N/A Inter-Agency Assessments N/A Rapid needs assessments N/A Cash and voucher feasibility studies N/A Other Monitoring & Reporting (if applicable) MGD FY17 semi-annual report Y Efficiency, Effectiveness M&E Plan Performance Monitoring Plan and New Monitoring Frameworks Y Relevance, Effectiveness, Impact M&E Policy USDA Monitoring and Evaluation Policy Y Relevance M&E Indicators and Definitions USDA and McGovern-Dole Indicators and Definitions Y Effectiveness, Impact Country Situation Report (SITREP) N Country Executive Brief WFP Bangladesh Country Brief Y Relevance Food Distribution and Post-distribution Monitoring Reports Will be required N Effectiveness, Impact Monthly Monitoring Reports Semi-annual and annual project reports Y Effectiveness, Impact, Efficiency Beneficiary Verification Reports N/A Donor specific reports Bangladesh FY 17 Semi-annual Report Y Effectiveness, Impact Output monitoring reports (if applicable) Quarterly and Monthly Monitoring Reports of RtR and MA Y Effectiveness, Efficiency Actual and Planned beneficiaries by activity and district/ location by year Actual and Planned beneficiaries by activity by year; location wise monitoring data is not available, USDA semi-annual reports Y Effectiveness, Impact, Efficiency Male vs. Female beneficiaries by activity and district/ location by year Would be required to evaluate the gender component of the programme N Beneficiaries by age group N/A Actual and Planned tonnage distributed by activity by year FY17 Semi-annual report Y Effectiveness, Impact, Efficiency Commodity type by activity N/A Actual and Planned cash/voucher requirements (US$) by activity by year N/A Operational documents (if applicable) N/A Organogram for main office and sub-offices BDCO McGovern Dole Annual Work Plan FY2019 Y Effectiveness, Efficiency, Sustainability Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 158 Document Type Comment / Titles & dates of documents received Received - Y/N (N/A) Link to Evaluation matrix Activity Guidelines Will be required. N Mission Reports N Pipeline overview for the period covered by the evaluation N Logistics capacity assessment N/A Partners (if applicable) Terms of Reference Y Annual reports from cooperating partners From RtR and MA Y Effectiveness, Efficiency, Impact, Sustainability List of partners (Government, NGOs, UN agencies) by location/ activity/ role/ tonnage handled List of partners has been provided in ToR. Relevant documents have been shared for their roles. Y Effectiveness, Sustainability Field level agreements (FLAs), Memorandum of Understanding (MOUs) Roles and Responsibilities of various partners Y Effectiveness, Efficiency, Sustainability Cluster/ Coordination meetings (if applicable) N/A Logistics/Food Security/nutrition cluster documents N/A NFRs of coordination meetings Would be required to evaluate the capacity development activities N Sustainability Other Evaluations/ Reviews Y Relevance, Effectiveness, Impact, Sustainability Evaluations/ reviews of past or on-going operation USDA McGovern-Dole International Food for Education and Child Nutrition Programme’s Support (2014-2016) to WFP Bangladesh Country Programme: Baseline and End Term Report FY17 Baseline report Y Relevance, Effectiveness, Impact, Sustainability Resource mobilisation (if applicable) Y Efficiency, Effectiveness Resource Situation This will be required. N Efficiency, Effectiveness Contribution statistics by month N/A Resource mobilization strategy This will be required. N Efficiency, Effectiveness NFRs Donor meetings N Maps (if applicable) Y Relevance Operational Map BDCO McGovern Dole Annual Work Plan FY2019 Y Relevance Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 159 Document Type Comment / Titles & dates of documents received Received - Y/N (N/A) Link to Evaluation matrix Logistics Map N Food/Cash/voucher Distribution Location Map N/A Food Security Map N/A Other documents collected by the team (including external ones) (if applicable) NPAN 2, End line evaluation of Reading Outcomes in Government Primary Schools (USAID’s Reading Enhancement for Advancing Development (READ) Activity Y Templates Y Work plans Y Effectiveness, Efficiency Norms and standards Y Specify Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 160 Annex P: NRMC’s Internal Protocols 13. NRMC’s Internal Quality Assurance procedures adhered to the following three categories of norms as follows: 14. Our internal quality control measures were followed at each stage of our evaluations starting from design of tools, to hiring and training of field teams; from data collection in the field to monitoring of the process; from data analysis till the reporting stage. Some of our key measures undertaken to ensure data quality is as follows: a. A three-stage internal review process of key deliverables: Our evaluation teams was structured in a manner that all deliverables including data collection tools, inception report, data analysis plans, and reports are reviewed by the project coordinator, the team leader and an external technical backstopping expert assigned internally for the project. b. Our evaluation teams travelled extensively to the field work areas to ensure that the data quality is excellent. We engaged our core evaluation team members to travel for field visits especially for qualitative interviews with sensitive populations. We devised efficient and effective methods to deploy coordinators and supervisors to monitor the data collection process continuously. c. Assuring data quality during analysis: Using interactive checking, validation of sample data, and data cleaning by our data analysts, NRMC ensured the quality of data from qualitative or quantitative methods. We ensured that transcripts of the qualitative interviews along with field notes are made available to the evaluation team for better analysis. d. Strong support teams: Training for field teams was conducted by core team members (from the evaluation team). Our professional editor ensured that all reports are edited before sharing with the client. Our internal data processing team ensured that all errors are resolved quickly and thoroughly. The staff on payroll or contract always demonstrates honesty, integrity, and professionalism at all times. The staff is aware of applicable statutes, regulations, practices, and ethical standards governing data collection and reporting. The team reports information accurately and without bias. The team is accountable, and holds others accountable, for ethical use of data. Integrity The team promotes data quality by adhering to best practices and operating standards. The team provides all relevant data, definitions, and documentation to promote comprehensive understanding and accurate analysis when releasing information. Data Quality The team treats data systems as valuable organizational assets and hence data backup is a mandatory affair. The team safeguards sensitive data to guarantee privacy and confidentiality as our servers are accessible to limited staff only. Data Security Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 161 Annex Q: Team Composition and Specific Tasks Table 15: Team Composition and Responsibilities Core Team Members Primary Role Specific tasks within the Evaluation Deliverables Dates Rahul Agrawal Team Leader and Evaluation Manager • Overall project backstopping and coordination of assignment including interaction with WFP CO • Technical inputs on research design, development of research tools and conceptual framework, sampling design analysis of data, and quality assurance of the inception, draft, and final report • Technical inputs on detailing results framework • Oversee quality of deliverables to WFP CO, DEQAS and USDA • Designing evaluation frameworks, key indicators and analytical frameworks • Technical inputs on evaluation methods and tools • Technical backstopping for research design implementation • Field work quality assurance • Training of data collection teams • Tabulation plan, validation of tables and indicators • Data analysis • Report Writing and presentation Inception report Draft report and Final report As per the timelines in section 6.2. Prof. Hafizur Rehman Advisor- Primary Education • Technical inputs and advisory on the development of the evaluation methodology • Technical inputs on quantitative and qualitative tools • Technical backstopping • Technical inputs on detailing results framework • Designing of EGRA tool and classroom observations Inception report, Draft report and Final report As per the timelines in section 6.2. Jayesh Bhatia Quality Assurance Expert • Defining quality assurance protocols for data collection, analysis and deliverables and ensuring its implementation Inception report, Draft report and Final report As per the timelines in section 6.2. Ankita Singh Gender Expert • Development of conceptual framework and evaluation design with a specific focus on gender • Development of evaluation indicators and tools • Development of tools with inclusion of equity and gender components • Contextual analysis of the data with a gender lens • Data collection and monitoring of field work • Tabulation plan, validation of tables and indicators • Development of analytical framework and data analysis • Report Writing Inception report, Draft report and Final report As per the timelines in section 6.2. Mrinalini Research Manager • Development of evaluation indicators and tools • Training of data collection teams • Data collection and monitoring of field work(including quality assurance) • Tabulation plan, validation of tables and indicators • Development of analytical framework and data analysis • Report Writing Inception report, Draft report and Final report As per the timelines in section 6.2. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 162 Annex R: The Training Schedule of the Data Collection Team Table 16: Training Schedule S. No. Time Topic Responsibility Day 1 1. 9:30 am–10:00 am Team Introductions NRMC and DMA 2. 10:00 am-11:30 am Training of data collection teams on context of the programme, about the programme and the various components of the evaluation. Overview of the tools. NRMC 3. 11:30 am-12:00 pm Ethical guidelines to be followed during the evaluation survey NRMC 4. 12:00 pm-1:30 pm Training on School Questionnaire NRMC and DMA 5. 1:30 pm-2:15 pm Lunch break 6. 2:15 pm–4:00 pm Training of school questionnaire contd. NRMC and DMA 7. 4:00 pm-4:15 pm Break 8. 4:00 pm-5:30 pm Mock calls, Role Play and Assessment NRMC and DMA 9. 5:30pm-6:00 pm Formation of groups (4 groups) basis on the exercise conducted to assess their capabilities NRMC/DMA Day 2 1. 9:00 am￾9:30 am Recap of last day NRMC and DMA 2. 9:30 am￾11:00 am Questionnaire for Students Questionnaire for Parents Questionnaire Headmaster NRMC and DMA 3. 11:00 am￾11:15pm Break NRMC and DMA 4. 11:15 am￾1:15pm Questionnaires Teacher, Observation Visits Questionnaire for Students; Practice/Mocks Questionnaire for Parents Practice/Mocks NRMC and DMA 5. 1:15 pm￾2:00 pm Lunch Break 6. 2:00 pm￾4:00 pm Questionnaire EGRA Questionnaire Storekeeper NRMC and DMA 7. 4:00 pm￾4:15 pm Break 8. 4:15 pm￾5:30 pm Mock calls and Revision for the day NRMC and DMA Day 3 1. 9:30 am￾10:00 am Recap of last day NRMC and DMA 2. 10:00 am￾1:00 pm Mock calls of all questionnaires using CAPI NRMC and DMA 3. 10:00 am￾1:00 pm Training on FGDs NRMC and DMA 4. 1:00 pm￾2:00 pm Lunch Break NRMC and DMA Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 163 S. No. Time Topic Responsibility 5. 2:00 pm￾4:00 pm Mock calls of all questionnaires using CAPI NRMC and DMA 6. 2:00 pm￾4:00 pm Mock Calls and Role Play for FGDs NRMC and DMA Day 4 1. 7:30 am￾12:30 pm Field Practice NRMC and DMA 2. 12:30 pm-1:00 pm Lunch break 3. 1:00 pm￾3:00 pm Field practice NRMC and DMA 4. 3:00 pm￾4:00 pm Field enumerators share experiences, lessons and queries NRMC and DMA 5. 5:00 pm￾6:00 pm De-briefing of the team and query resolution NRMC and DMA 6. 6:00 pm – 6:30 pm Finalize field movement plan NRMC and DMA Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 164 Annex S: Midline Values of Key Indicators Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project SO1 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of the grade-level text Students reading and understanding ability were assessed using the EGRA tool. The tool was administered with 420 students in intervention and 280 students in comparison schools in Grade 3. Students were categorised into emergent, beginner and reader categories with comprehension ability based on a sub-test score, as recommended by EGRA guidelines. Students who can read more than 45 words and correctly answer 4-5 questions are indicated here. 35% 6.9% 8.4% 5.5% 28% 25% 32% 71 All midterm values are on the basis of WFP monitoring report and extrapolated on the basis of findings from the sample survey of the mid-term evaluation Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 165 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project 1.1 Number of school administrators and officials in target schools who demonstrate the use of new techniques or tools as a result of USDA assistance No. of trained headmasters who reported using and demonstrating the use of new learning techniques and tools 116 0 0 0 78 68 10 1.1 Number of teachers/educators/teaching assistants in target schools who demonstrate the use of new and quality teaching techniques or tools as a result of USDA assistance Teachers, educators, teaching assistants who have successfully completed a pre- or in￾services training programme to teach in schools or equivalent non￾school based settings over the past one year were then assessed on whether they are using the learned techniques. Use of techniques was assessed by asking their supervisors whether the teacher is using the techniques he/she learned in the classroom. 584 117 40 77 404 204 200 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 166 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project 1.1 Percent of teachers in target schools who attend and teach school at least 90 percent of scheduled school days per school year72 Regular teacher attendance was defined as attending more than or equal to 90 percent of school days; teacher attendance was assessed using monthly school records of all the teacher from sample schools and the number of working days per month during the period July 2017- September 2017 75% 82.3% 90.9% 76.1% -- -- -- 1.2 Percent of students in classrooms identified as attentive by their teachers Attentiveness of students is based on the perception of teachers on the attentiveness of sampled students from each school 70% 70.3% 68.8% 71.7% 75.1% 71.4% 78.9% 1.3 Number of students enrolled in school receiving USDA assistance Student enrolment is based on monthly school records (pre-primary and primary) of the sample schools for 47,689 316 (average per school)73 170 (average per school) 146 (average per school) 322 (average per school) 153 (average per school) 169 (average per school) 72 The data for this indicator could not collected in midterm evaluation due to government’s circular which restricts access to teacher’s attendance data for anyone apart from education department. 73 Comparable values were not available as the presented number was derived from the sample of the baseline (data for all 146 schools was not available). Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 167 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project the academic year commencing January 2018 47586 (from monitoring report) 1.3 Percent of students regularly (at least 80 percent of the school days) attending school Student attendance was measured using the attendance record of fifteen randomly selected students from every sample school for the last academic semester (September 2017- February 2018). 82% 83.3% 84.4% 82.4% 85.2 % 82% 88.3% 1.3 Average number of school days missed by each student due to illness Data was collected by asking students the no. of school days missed in the last 1 week and out of the missed school days, how many were missed due to illness. 5 2 2 2 2 2 2 1.3 Percent of parents in target communities who can name at least three benefits of primary education Data was collected through interviews with parents and asking them about the benefits of primary education. 70% 67.8% 61.3% 72.4% 73.2% 80% 66.4% Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 168 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project 2.1 Percent of students who can identify at least three key health and hygiene practices Students who could identify three or more key health and hygiene practices during the primary survey 65% 25.1% 27.6% 22.7% 39.2% 39.6% 38.9% 2.2 Number of individuals trained in safe food preparation and storage as a result of USDA assistance The data represents teachers, headmasters and storekeepers who reported having received training on storage practices 322 0 0 0 94 66 28 2.2 Number of individuals who demonstrate the use of new safe food preparation and storage practices as a result of USDA assistance This data was collected through observation in the storage facilities during the primary survey. 290 35 22 13 94 66 28 2.2 Percent of storekeepers who can identify at least three safe storage practices This data was collected through the interviews with the storekeepers 50% 96.7% NA NA 100% 100% 100% 2.4 Number of schools using an improved water source Schools that had any of the following sources of water: Tap water, RO plant within the school, Borehole, protected dug well or Protected spring 146 26 NA NA 96 NA NA Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 169 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project Data collected from WFP Monitoring report 2.4 Number of schools with improved sanitation facilities Schools that have any of the following sanitation facilities: Flush or pour/flush facilities connected to a piped sewer/septic/pit latrine, flush or pour/flush toilets without a sewer connection, Pit latrine with a slab, Ventilated improved pit latrines or Composting toilets Data collected from WFP Monitoring report 146 11 NA NA 99 NA NA 2.5 Number of individuals trained in child health and nutrition as a result of USDA assistance74 Teachers and headmasters who received training on health, hygiene and nutrition 730 0 0 0 1480 (from monitoring report) 17 8 74 Individuals consist of headmasters, teachers and parents for this indicator. While none of the individuals have been trained on child health and nutrition practices under this programme yet, the values given in the table represents the number of headmasters and teachers who have received training on health, hygiene, and nutrition from the GoB or any other source. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 170 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project SO2 Number of individuals who demonstrate the use of new child health and nutrition practices as a result of USDA assistance75 Data for handwashing was collected from parents. The values represent those parents who were trained by WFP and among them who reported washing hands for 4 or more out of 6 critical times Adding to this, the number of head teachers and teachers who demonstrated use of child health and nutrition practices. 28,600 114 NA NA 204 (extrapolated) 19 23 Dietary diversity of pupils was measured through interviewing parents using 24 hours recall method. Mean Dietary Diversity Score is presented here. 4.2 4.2 4.3 4.96 4.9 5.02 75 Child health and nutrition practices were assessed using 3 parameters; Handwashing, Dietary Diversity and Deworming (data for deworming has been reported separately) Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 171 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project 2.5 Number of students receiving deworming medication(s) This data was collected from the headmasters 47,689 0 0 0 47,000 22,266 24,734 1.3.3 Number of educational facilities (i.e. water systems and latrines) rehabilitated/constructed as a result of USDA assistance This data was collected from the headmasters through school questionnaires 35 N.A. N.A. N.A. 1876 NA N.A. SO1 Number of individuals benefitting directly from USDA-funded interventions WFP Monitoring reports 48,711 N.A. N.A. N.A. 51,23277 N.A. N.A. 76 Schools that reported to receive support from WFP for rehabilitation/construction of toilets, storage facilities, water systems, library, playground, classroom etc. 77 Individuals included in this indicator are students, headteacher, Grade I and Grade II Bangla teachers, Govt. officials, SRM teachers and SMC members. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 172 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project SO1 Number of individuals benefitting indirectly from USDA-funded interventions WFP Monitoring Reports 190,756 N.A. N.A. N.A. 175,04478 N.A. N.A. 2.4 Number of “Little Doctor” students supported by WFP This was collected through school questionnaire 2,025 N.A. N.A. N.A. 2,093 N.A. N.A. 1.4.4 Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance This is based on an interview with headmasters on the existence of SMCs and PTAs in the school 334 5979 NA NA 189 NA NA 1.4.4 Number of public-private partnerships formed as a result of USDA assistance WFP monitoring reports 1 N.A. N.A. N.A. N.A. N.A. N.A. 1.4.4 Value of new public and private sector investments leveraged as a result of USDA assistance WFP Monitoring Reports $227.1 N.A. N.A. N.A. USD 19 million dollars N.A. N.A. 1.4.2 The number of educational policies, regulations and/or administrative procedures in each of the following stages of WFP Monitoring Reports 2 N.A. N.A. N.A. 2 N.A. N.A. 78 Individuals reported for this indicator includes parents of the students receiving school meals 79 All these schools had school governance structures prior to commencement of activities under the current USDA grant Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 173 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project development as a result of USDA assistance: • Stage 1: Analyzed • Stage 2: Drafted and presented for public/stakeholder consultation • Stage 3: Presented for legislation/decree • Stage 4: Passed/Approved Stage 5: Passed for which implementation has begun 1.4.4, 1.4.1 Number of meetings/workshops/training sessions held for institutional capacity to implement SF as a result of USDA assistance WFP Monitoring Reports 142 N.A. N.A. N.A. 6 N.A. N.A. 1.4.4 Number of community mobilization workshops organized as a result of USDA assistance WFP Monitoring Reports 38 N.A. N.A. N.A. N.A. N.A. N.A. 1.4.1, 1.4.2, 1.4.4 Number of pilot initiatives supported to design SF WFP Monitoring Reports 4 N.A. N.A. N.A. N.A. N.A. N.A. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 174 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project modalities as a result of USDA assistance 1.4.1 Number of government staff trained as a result of USDA assistance WFP Monitoring Reports 20,400 N.A. N.A. N.A. 837 N.A. N.A. 2.2, 1.4.1, Number of technical training for system development for service/food procurement, quality control, supply chain, and strengthened online database system and gender mainstreaming in the programme as a result of USDA assistance WFP Monitoring Reports 284 N.A. N.A. N.A. 3080 N.A. N.A. SO1, SO1 Number of schools supported by the Government with school feeding as a result of USDA assistance WFP Monitoring Reports 13,482 N.A. N.A. N.A. N.A. N.A. 1.4.4 Number of social mobilization/community meetings as a result of USDA assistance WFP Monitoring Reports 84 N.A. N.A. N.A. N.A. N.A. N.A. 80 From WFP monitoring reports Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 175 Result level (Indicator no.) Indicator Source and measurement Targets Base Value Baseline Value by sex Mid Value71 Midline Value by sex Project Male Female Project Male Female Project Project Project Project 1.4.4 Number of teachers, parents and school management committee members attended the community mobilization workshops WFP Monitoring Reports 1,168 N.A. N.A. N.A. N.A. N.A. N.A. 1.2.1.1 & 1.3.1.1 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance WFP Monitoring Reports 47,689 N.A. N.A. N.A. 47,586 22,588 24,998 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 176 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex Project Compar ison Male Female Project Compar ison Male Female Project Compar ison Project Compa rison Project Compar ison Project Comparison SO1 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of the grade-level text Students reading and understanding ability were assessed using the EGRA tool. The tool was administered with 420 students in intervention and 280 students in comparison schools in Grade 3. Students were categorised into emergent, beginner and reader categories with comprehension ability based on a sub-test score, as recommended by EGRA guidelines. Students who can read more than 45 words and correctly answer 4-5 questions are indicated here. 6.9% 13.6% 8.4% 11.7% 5.5% 15.4% 28% 17% 25% 12% 32% 23% 1.1 Number of textbooks and other teaching and learning materials provided as a result of USDA assistance Number of textbooks and other teaching materials provided 0 0 N.A N.A N.A N.A 82,807 0 NA NA NA NA Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 177 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex Project Compar ison Male Female Project Compar ison Male Female Project Compar ison Project Compa rison Project Compar ison Project Comparison 1.1 Number of school administrators and officials trained or certified as a result of USDA assistance No. of headmasters who have successfully completed a pre- or in￾services training programme to teach in schools or equivalent non-school based settings over the past year. 0 0 0 0 0 0 88 0 68 0 20 0 1.1 Number of school administrators and officials in target schools who demonstrate the use of new techniques or tools as a result of USDA assistance No. of trained headmasters who reported using and demonstrating the use of new learning techniques and tools 0 0 0 0 0 0 78 0 68 0 10 0 1.1 Number of teachers/educato rs/teaching assistants trained or certified as a result of USDA assistance Teachers, educators, teaching assistants who have successfully completed a pre- or in￾services training programme to teach in schools or equivalent non-school based settings over the past year. 0 0 0 0 0 0 338 0 175 0 163 0 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 178 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex Project Compar ison Male Female Project Compar ison Male Female Project Compar ison Project Compa rison Project Compar ison Project Comparison 1.1 Number of teachers/educato rs/teaching assistants in target schools who demonstrate the use of new and quality teaching techniques or tools as a result of USDA assistance Teachers, educators, teaching assistants who have successfully completed a pre- or in￾services training programme to teach in schools or equivalent non-school based settings over the past one year were then assessed on whether they are using the learned techniques. Use of techniques was assessed by asking their supervisors whether the teacher is using the techniques he/she learned in the classroom. 117 110 40 37 77 73 83 0 42 0 41 0 1.1 Percent of teachers in target schools who attend and teach school at least 90 percent of scheduled school days per school year Regular teacher attendance was defined as attending more than or equal to 90 percent of school days; teacher attendance was assessed using monthly school records of all the teacher from sample schools and the number of working days per month during the period July 2017- September 2017 82.3% 86.8% 90.9% 92.5% 76.1% 83.1% -- -- -- -- -- -- Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 179 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex Project Compar ison Male Female Project Compar ison Male Female Project Compar ison Project Compa rison Project Compar ison Project Comparison 1.1 Number of classroom libraries (book shelves with books) established as a result of USDA assistance No. of schools that had a library within their premises 0 0 NA NA NA NA 30 0 NA NA NA NA 1.2 Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance This data was collected from the school records NA NA NA NA NA NA 24,88,91 4 0 NA 0 NA 0 1.2 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance This data was collected from the school records NA NA NA NA NA NA 46,791 0 22,211 0 24,580 0 1.2 Percent of students in classrooms identified as by their teachers Attentiveness of students is based on the perception of teachers on the attentiveness of sampled students from each school 70.3% 89.2% 68.8% 89.6% 71.7% 88.9% 75.1% 71.2% 71.4% 71.3% 78.9% 71% Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 180 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex Project Compar ison Male Female Project Compar ison Male Female Project Compar ison Project Compa rison Project Compar ison Project Comparison 1.3 Number of students enrolled in school receiving USDA assistance Student enrolment is based on monthly school records (pre￾primary and primary) of the sample schools for the academic year commencing January 2018 268 (Average per sample school) 364 (Averag e per sample school) 125 (Average per sample school) 163 (Averag e per sample school) 142 (Averag e per sample school) 200 (Averag e per sample school) 322 (Averag e per sample school) 401 (Averag e per sample school) 153 (Average per sample school) 182 (Averag e per sample school) 169 (Averag e per sample school) 219 (Average per sample school) Percent of students regularly (at least 80 percent of the school days) attending school Student attendance was measured using the attendance record of fifteen randomly selected students from every sample school for the last academic semester (September 2017-February 2018). 83.3% 91.8% 84.4% 89.8% 82.4% 93.8% 85.2% 93.6% 82% 92.7% 88.3% 94.6% 1.3 Average number of school days missed by each student due to illness Data was collected by asking students the no. of school days missed in the last 1 week and out of the missed school days, how many were missed due to illness. 2 2 2 2 2 2 2 2 2 2 2 2 1.3 Percent of parents in target communities who can name at least three benefits of primary education Data was collected through interviews with parents and asking them about the benefits of primary education. 67.8% 66.0% 61.3% 63.2% 72.4% 67.7% 73.2% 70% 80% 85.7% 66.4% 69% Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 181 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex Project Compar ison Male Female Project Compar ison Male Female Project Compar ison Project Compa rison Project Compar ison Project Comparison 2.1 Percent of students who can identify at least three key health and hygiene practices Students who could identify three or more key health and hygiene practices during the primary survey 25.1% 34.6% 27.6% 31.7% 22.7% 37.3% 39.2% 38.8% 39.6% 37.2% 38.9% 40.5% 2.2 Number of individuals trained in safe food preparation and storage as a result of USDA assistance The data represents teachers and headmasters who reported having received training on storage practices 0 0 0 0 0 0 94 0 66 0 28 0 2.2 Number of individuals who demonstrate the use of new safe food preparation and storage practices as a result of USDA assistance This data was collected through observation in the storage facilities during the primary survey and the numbers of teachers and headmasters who reported to demonstrate the use of new safe food preparation and storage practices and represents the numbers out of the individuals who are trained 35 NA 22 NA 13 NA 94 0 66 0 28 0 2.2 Percent of storekeepers who can identify at This data was collected through interviews with storekeepers 96.7% NA NA NA NA NA 100% 0 100% 0 100% 0 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 182 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex Project Compar ison Male Female Project Compar ison Male Female Project Compar ison Project Compa rison Project Compar ison Project Comparison least three safe storage practices 2.4 Number of schools using an improved water source Schools that had any of the following sources of water: Tap water, RO plant within the school, Borehole, protected dug well or Protected spring 26 14 NA NA NA NA 30 20 NA NA NA NA 2.4 Number of schools with improved sanitation facilities Schools that have any of the following sanitation facilities: Flush or pour/flush facilities connected to a piped sewer/septic/pit latrine, flush or pour/flush toilets without a sewer connection, Pit latrine with a slab, Ventilated improved pit latrines or Composting toilets 11 17 NA NA NA NA 29 20 NA NA NA NA 2.5 Number of individuals trained in child health and nutrition as a Teachers and headmasters who received training on health, hygiene and nutrition 0 0 0 0 0 0 25 0 17 0 8 0 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 183 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex Project Compar ison Male Female Project Compar ison Male Female Project Compar ison Project Compa rison Project Compar ison Project Comparison result of USDA assistance81 2.5 Number of individuals who demonstrate the use of new child health and nutrition practices as a result of USDA assistance82 Data for handwashing was collected from parents. The values represent those who are trained by WFP and among them who reported washing hands for 4 or more out of 6 critical times. Adding to this, the number of head teachers and teachers who demonstrated use of child health and nutrition practices. 114 43 NA NA NA NA 42(25 head teachers and teachers + 17 parents) 0 19 0 23 0 Dietary diversity of pupils was measured through interviewing parents using 24 hours recall method. Mean Dietary Diversity Score is presented here. 4.2 4.2 4.2 4.2 4.3 4.1 4.97 5.49 4.96 5.5 4.99 5.48 81 Individuals consist of headmasters, teachers and parents for this indicator. While none of the individuals have been trained on child health and nutrition practices under this programme yet, the values given in the table represents the number of headmasters and teachers who have received training on health, hygiene, and nutrition from the GoB or any other source. 82 Child health and nutrition practices were assessed using 3 parameters; Handwashing, Dietary Diversity and Deworming (data for deworming has been reported separately) Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 184 Result level Indicator Source and measurement Base value Value by sex Mid value Value by sex Project Compar ison Male Female Project Compar ison Male Female Project Compar ison Project Compa rison Project Compar ison Project Comparison 2.5 Number of students receiving deworming medication(s) This data was collected from the headmasters 0 0 0 0 0 0 47,000 29,644 22,266 14,257 24,734 15,387 Foundat ional results Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance This is based on an interview with headmasters on the existence of SMCs and PTAs in the school 5983 40 N.A. N.A. N.A. N.A. 189 N.A. N.A. N.A. N.A. N.A. 83 All these schools had school governance structures prior to commencement of activities under the current USDA grant Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 185 Annex T: Terms of Reference for Mid-term Evaluation The Terms of Reference (TOR) presented here are for the Fiscal Year (FY) 2017 – FY2020 McGovern-Dole International Food for Education and Child Nutrition Grant (McGovern-Dole) funded through the U.S. Department of Agriculture (USDA). This programme aims to reach 47,689 primary school aged children per year with fortified biscuits in two sub-districts of Cox’s Bazar, Bangladesh. The sub-districts are Ukhiya and Kutubdia. The programme will target 146 government and non-government organization supported schools. The biscuits will be produced by a local biscuit factory using the wheat donated under this agreement. The biscuits will be fortified with vitamins and minerals to reduce micronutrient deficiencies in students. Complimentary activities aimed at improving literacy by increasing classroom instruction, awareness of health and hygiene practices, and community engagement in education complement the provision of biscuits to students. The three-year programme was developed with support from the Ministry of Primary and Mass Education (MoPME), particularly with the soon to be approval of the School Feeding Programme in Poverty Prone Areas (SFPPA). WFP and MoPME have a close working relationship which entails four WFP staff seconded to MoPME to assist with capacity strengthening for managing the implementation of the School Feeding Programme. WFP and MoPME hold formal bi-monthly meetings with the SFPPA project director and four to five key government staff on the programme. The McGovern-Dole project will engage the sub-recipient Room to Read (RtR) to implement targeted education activities specifically designed to achieve McGovern￾Dole’s Strategic Objectives (SO). RtR will implement their innovative and nationally aligned early grade literacy activities and provide overall technical assistance to two local NGOs. Muslim Aid (MA) and Young Power in Social Action (YPSA) are two additional sub-recipients who will be implementing literacy, school meals, nutrition and capacity strengthening activities at the local and district level. MA will primarily work in Ukhiya whereas YPSA will work in Kutubdia. Both organizations were selected to work on the McGovern￾Dole programme because they were already partners with WFP on school meals programmes prior to the McGovern-Dole award. The purpose of this TOR is two-fold. Firstly, it provides key information to the evaluation team and helps guide them throughout the evaluation process; and secondly, it provides key information to stakeholders about the proposed evaluations. The evaluation process will be managed by an Evaluation Manager (WFP - EM) appointed by the WFP Bangladesh Office. The EM will be the main focal point for day to day contact with the hired evaluation firm during the evaluation period. The WFP – EM will be supported by the M&E Unit not associated with the implementation of the programme in the WFP Bangladesh country office. A competitive bidding process will result in an outside firm to be contracted to carry out the actual evaluation. Appropriate safeguards to ensure the impartiality and independence of the evaluation are outlined within this TOR. The evaluation process entails three deliverables and will be commissioned by the Bangladesh Country Office. They will be undertaken in a single assignment (contract). The specific deliverables (timeframes mentioned are subject to change) include: Evaluation Date Baseline evaluation report January – June 2018 Midterm evaluation report June – December 2019 End-line evaluation report June – December 2020 The first deliverable will provide information about the pre-project situation, establish a baseline and review project targets. The second deliverable will provide an evidence-based assessment of the project outcome, independent evaluation of the operation and associated interventions about halfway through the project so Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 186 that WFP-Bangladesh can adjust the project design/implementation for the remaining project period. The final deliverable will be a final evaluation of the three-year programme to inform any future project design, outcomes and lessons learned. This FY2017 programme is located in a different district than the previous three-year funding support from USDA. As such, programme design, targeting and interventions are unique to this new cycle of funding. Noting these differences, the evaluation firm may wish to review previous programme evaluations and findings to support the evaluation design. Bangladesh’s School Meals Context and Subject Bangladesh is one of the most densely populated and disaster-prone countries in the world. Its population is estimated at over 160 million and it is classified as a least-developed, low-income, food-deficit country. It falls in the low human development category, ranking 142 out of 185 countries on the Human Development Index. Despite significant gains in terms of macro-economic growth and human development over the past decade, Bangladesh continues to experience high levels of extreme poverty, and high rates of food insecurity and under-nutrition. Forty-one percent of children under the age of five are stunted, 16% are wasted, and 36% are underweight; levels that are above public emergency thresholds. Bangladesh is also highly vulnerable to natural disasters, such as flooding and cyclones, which exacerbates food insecurity status of millions of people. Bangladesh also faces the human development challenge of illiteracy. The national literacy rate is 50.5% (11- 45 years) and among 11-14-year old, 19.5% are non-literate and 10.4% are semi-literate. In recent years, Bangladesh has made significant progress in its efforts to address illiteracy, especially with regard to increasing access to education and gender equity at the primary level, and is on track to reach the net enrolment target of Millennium Development Goal 2, universal primary education, by 2015. Targeted Beneficiaries and Regions: In line with USDA and GoB priorities, the most vulnerable and food insecure areas of Bangladesh will be targeted by this McGovern-Dole programme. The programme will operate in two sub-districts of Cox’s Bazar, Ukhiya and Kutubdia. Cox’s Bazar has a poverty rate of 32% and an extreme poverty rate of 17% and is highly food insecure with over 34% of the population below the food consumption poverty line. The region is also experiencing a Level 3 emergency with the influx of over 600,000 ethnic Rohingya refugees from Myanmar. The graduation of USDA funded programmes to GoB District Upazila No. schools No. of children Graduation Year/month Kurigram Kurigram Sadar 177 45,846 12-Jun Ulipur 408 69,978 12-Jun Chilmari 146 24,943 14-Dec Rajarhat 36 1,074 14-Dec Nageshwari 315 59,909 14-Dec Bhurungamari 153 34,819 12-Jun Fulbari 199 29,034 14-Dec Rowmari 142 36,175 14-Dec Rajibpur 69 16,494 14-Dec Barisal Bakergonj 348 86964 12-Feb Gaibandha Gobindhagonj 273 70002 14-Dec Saghata 182 45693 14-Dec Total 2,448 520,931 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 187 The education system in Bangladesh consists of approximately 150,000 institutions, 40 million students, and a million teachers. In addition, there are thousands of non-formal primary education centres and quomi (indigenous) faith-based madrasas not included in education statistics. There are approximately 18 million students and 80,000 institutions in primary education. Participation in education has expanded remarkably over the past two decades. The Government of Bangladesh has implemented several mechanisms for incentivizing school attendance, including a national school feeding programme for 2.5 million children with the support of WFP; the distribution of financial stipends to 13 million poor children and their families (including disabled children) under a cash for education programme; and the provision of free textbooks. As a result, at the primary level, there is over 97% enrolment while gender parity in primary enrolment has also been achieved. The expansion in education (SDG4) has been widely celebrated but the quality of education remains a serious concern as are high drop-out rates, which increasingly affect young girls and boys dropping out of school to pursue economic activities. Significant education challenges remain however relating to attendance, completion, and a prevailing low national literacy rate (51.3%). These challenges are amplified in Cox’s Bazar. WFP has shifted from direct delivery of biscuits to supporting the government in managing the national school feeding programme. In 2007, WFP assisted directly two million children. This number reduced to one million in 2014 and was further reduced to 500,000 in 2016 as a result of the government gradually taking over from WFP. The graduation of McGovern-Dole supported schools to the Government of Bangladesh began in 2012. Since this time, 520,931 students in 2,448 schools have been handed over to the Government of Bangladesh. For specific district and school information, please see the chart above. Since the start of the SFPPA, WFP has been providing technical assistance to the Government to manage school feeding activities through an embedded Capacity Support Unit (CSU) in the Department of Primary Education. Programme Interventions The project will use McGovern-Dole commodities and cash funding to contribute directly towards both of the McGovern-Dole programme’s highest-level Strategic Objectives, McGovern-Dole SO1: Improved Literacy of School-Aged Children and McGovern-Dole SO2: Increased Use of Health and Dietary Practices. WFP has developed a comprehensive programme to provide fortified biscuits at the early hour of school and complimentary educational interventions. The following activities will be undertaken and contribute toward the achievement of SO1 and SO2: • Build Capacity: WFP will provide technical assistance to MoPME to finalize the SFPPA and strategy and support scaling up school feeding. WFP will develop an action plan for policy implementation and will conduct an assessment for the implementation strategy. • Distribute Food: WFP in collaboration with the MoPME will provide fortified biscuits to school children in the Cox’s Bazar District of Bangladesh. WFP will arrange for the fortified biscuits to be produced by local biscuit factories, using the wheat donated under this agreement. • Improve Student Enrolment/Attendance: Through sub recipient, Room to Read, WFP will organize awareness campaigns on the importance of education at both the community and central level. • Promote Improved Health: In cooperation with the NGOs Muslim Aid UK and Young Power in Social Action, WFP will improve water systems and latrine facilities in selected participating primary schools and provide the schools with the needed pumps and motors. The NGOs will organize parent gathering sessions at the school level to discuss topics related to health, hygiene and nutrition awareness; support deworming distribution and develop and disseminate information education materials to help visualize good hygiene practices. • Promote Improved Nutrition: WFP will establish school gardens, school garden clubs, and provide training to SMC’s, teachers, and community members on the establishment and maintenance of school gardens. • Support Improved Literacy: Through the sub-recipient Room to Read, instruction materials will be developed and distributed to teachers and school administrators in grades 1 and 2 in all participating Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 188 schools. School reading corners in all of the project schools will be established in grades 1-5. A variety of additional education related activities will also be implemented. • Support Improved Safe Food Prep and Storage: WFP will train biscuit manufacturers in food safety standards for safe preparation of biscuits and appropriate storage practices. MoPME officials will be trained in monitoring the biscuit manufacturers. WFP will organize training on food storage practices for MoPME, school teachers, and administrators at the local level. Project Population Participant Approximate number Number of Upazila/sub-district 2 Number of Schools 146 Student 47,689 Parents 95,378 School Administrators 146 Teacher 730 District & Upazila/ sub￾district level education officials 10 (Cox’s Bazar district & two Upazila education officials) Evaluation Criteria and Questions The evaluations proposed will use the standard evaluation criteria of Relevance, Effectiveness, Efficiency, Sustainability, and Impact as well as Adequacy, Transparency, and Timeliness. Gender Equality and the Empowerment of Women (GEEW) should be mainstreamed throughout. The FY17 mid-term and end-line evaluation will address the proposed key evaluation questions (see Annex III: Evaluation Criteria and Questions FY17 baseline, mid-term and end-line evaluations). These are only the key indicative questions and sub-questions, in order to provide the background to the evaluation team. The evaluation team is therefore required to further elaborate the questions and sub questions under each criterion during the inception phase of each evaluation. Collectively, the questions aim at highlighting the existing circumstances, performance of school meal activities during the project period and key lessons learnt, which could inform future strategic and operational decisions. Evaluability is the extent to which an activity or a programme can be evaluated in a reliable and credible fashion. A preliminary evaluability assessment will be done by the Country Office at the initial stage of the project cycle, which will be deepened by the evaluation team in each inception package relating to deliverables. The evaluation team shall notably critically assess data availability and take evaluability limitations into consideration in its choice of evaluation methods. In doing so, the team will also critically review the evaluability of the gender aspects of the operation, identify related challenges and mitigation measures and determine whether additional indicators are required to include gender empowerment and gender equality dimensions. The evaluations will take a programme theory approach based on the results framework. It will draw on the existing body of documented data as far as possible and complement and triangulate this with information to be collected in the field. Stakeholders and Users Stakeholders A number of stakeholders both inside and outside of WFP have interests in the results of the evaluation and some of these will be asked to play a role in the evaluation process. Those include WFP Country Office, Regional Bureau, and WFP Washington Office, as well as key headquarters Divisions (Programme Policy Division, the Performance Management, and Monitoring Division and the Office of Evaluation among others). Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 189 Externally, USDA including the Food Assistance Division in Washington, DC and the regional Agricultural Attaché, and other key project partners, including Government of Bangladesh, specifically the Ministry of Primary and Mass Education (MoPME), Directorate of Primary Education (DPE), Local Government Division (LGD), Department of Public Health and Engineering (DPHE), Ministry of Health and Family Welfare. In addition, NGOs and international organizations Room to Read, Muslim Aid-UK, YPSA, BRAC, GAIN (Global Alliance for Improved Nutrition), UNICEF and FAO also have a stake in the evaluation. An Evaluation Reference Group (ERG) will be formed with representation from WFP CO, RBB, OEV HQ, and externally USDA, MoPME/DPE, Room to Read, Muslim Aid-UK, YPSA and UNICEF. The ERG members will review and comment on the draft evaluation products and act as key informants in order to safeguard against bias and influence. Users The primary users of this evaluation will be: • WFP-Bangladesh and its government partner to adjust joint activities to implement a school feeding programme and to inform future project design and implementation • USDA to inform changes in project strategy, results framework, and critical assumptions • NGOs to inform current activities or future project design • The GoB is expected to take over the management and monitoring of the school feeding programme over time, therefore, information on whether the programme is yielding the desired results is of primary importance • The WFP Regional Bureau to provide strategic guidance, programme support, oversight, and extract lessons for sharing across the region • WFP HQ for wider organizational learning and accountability • Office of Evaluation to feed into evaluation syntheses • Other COs may also benefit from the findings, which can contribute to corporate learning on the implementation of capacity development interventions Documents for review could include previous evaluations of the school feeding programme, noting that such evaluations were in a different location and different combination of interventions. These documents may include the McGovern-Dole FY14 baseline study and mid-term evaluation of the School Meals Programme, the Country Programme 2013-2017 mid-term evaluation, and the recently commissioned outcome survey for the USDA funded McGovern-Dole programme in Gaibandha. Additional documents may include monitoring data currently available with the CO and partners. These will be made available separately to the Evaluation Team. Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 190 Annex U: Bibliography 1. Ahmed, A. U. (2004). Impact of feeding children in school: Evidence from Bangladesh. Washington, DC: International Food Policy Research Institute. 2. Ahmed, Akhter. (2017). Strategic Review of Food Security and Nutrition in Bangladesh. 10.13140/RG.2.2.24673.71529. 3. Ahsan, K. Z., Streatfield, P. K., Ijdi, R. E., Escudero, G. M., Khan, A. W., & Reza, M. M. (2015). Fifteen years of sector-wide approach (SWAp) in Bangladesh health sector: an assessment of progress. Health policy and planning, 31(5), 612-623. 4. Alston, M., & Akhter, B. (2016). Gender and food security in Bangladesh: the impact of climate change. Gender, Place & Culture, 23(10), 1450-1464. 5. Bangladesh Education Statistics 2018; BANBEIS 6. FAO, WFP and IFAD. 2012. The State of Food Insecurity in the World 2012. Economic growth is necessary but not sufficient to accelerate reduction of hunger and malnutrition. Rome, FAO. 7. Health SDG Profile: Bangladesh (2018) 8. Helen Keller International and James P. Grant School of Public Health). (2015). State of Food Security and Nutrition in Bangladesh: 2014. BRAC University: Dhaka 9. Howlader, et al. (2012). Investing in Nutrition Now: A Smart Start for Our Children, Our Future. Estimates of Benefits and Costs of a Comprehensive Program for Nutrition in Bangladesh, 2011– 2021. PROFILES and Nutrition Costing Technical Report. Washington, DC: Food and Nutrition Technical Assistance III Project (FANTA), FHI 360 10. Human Development Report 2016-Human Development for Everyone: Briefing note for countries on the 2016 Human Development Report 11. icddr,b, UNICEF, Bangladesh, Global Alliance for Improved Nutrition (GAIN), Institute of Public Health and Nutrition (IPHN) (2011-12). National Micronutrients Survey. Dhaka, Bangladesh 12. International Labour Organization, ILOSTAT database (2017) 13. Islam, M. S., & Hasan, M. N. (2017). Bangladesh. In The World Guide to Sustainable Enterprise (pp. 36-41). Routledge. 14. Ministry of Health and Family Welfare, GOB, 2017. Second National Plan of Action for Nutrition (NPAN-2) 15. Ministry of Health and Family Welfare, Government of Bangladesh, 2017. Second National Plan of Action for Nutrition (2016-2025) 16. Nutrition country profile accessed from www.globalnutritionreport.org Source: UNICEF/WHO/World Bank Group: Joint child malnutrition estimates. (2018) 17. Rahman, M. M. (2017). Biannually school-based deworming by Mebendazole 500mg has reduced the worm load drastically in Bangladesh. EC Bacteriology and Virology Research, 2, 113-114. 18. Terms of Reference (TOR) of Fiscal Year (FY) 2017 – FY2020 McGovern-Dole International Food for Education and Child Nutrition Grant (McGovern-Dole) 19. The ‘Global Water Supply and Sanitation Assessment’ by World Health Organization (WHO), United Nations Children Fund (UNICEF), Water Supply and Sanitation Collaborative Council (WSSCC) 20. The Global Gender Gap Report benchmarks countries on their progress towards gender parity across four thematic dimensions: Economic Participation and Opportunity, Educational Attainment, Health and Survival, and Political Empowerment. 21. UN Renewed Efforts against Child Hunger (REACH), Undernutrition in Bangladesh A Common Narrative. 2014 22. UNEG Indicators on Gender Equality and Human Rights in UNEG Quality Checklist for Evaluation 23. United Nations Development Programme, Human development indices and indicators: 2018 Statistical update. 24. United Nations Development Programme. (2018). Human development indices and indicators: 2018 Statistical update. 25. World Health Organization. (2017). New perspectives on global health spending for universal health coverage (No. WHO/HIS/HGF/HF Working Paper/17.10). World Health Organization. 26. Retrieved from: a. https://www.globalhungerindex.org/bangladesh.html b. http://www.fao.org/3/CA3252EN/ca3252en.pdf c. https://washwatch.org/en/countries/bangladesh/summary/statistics/ Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 191 d. https://reliefweb.int/sites/reliefweb.int/files/resources/operation_presence_map_wash_ sector_august_2018_ta_0.pdf e. http://www.dghs.gov.bd/images/docs/Publicaations/HealthBulletin2017Final13_01_201 8.pdf f. http://app.dghs.gov.bd/localhealthBulletin2016/publish/publish.php?org=10000924&ye ar=2016&lvl=1 g. https://www.wateraid.org/bd/media/wateraid-launches-the-wash4urbanpoor-project￾aiming-to-serve-450000-urban-poor-with-water h. https://info.undp.org/docs/pdc/Documents/BGD/UNGA_Booklet_2017.pdf i. http://documents.worldbank.org/curated/en/857071529206219039/pdf/BANGLADESH -QLEAP-PAD-05252018.pdf Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 192 Annex V: List of Acronyms ASPR Annual Sector Performance Report (prepared by Directorate of Primary Education) AUEOs Assistant Upazila Education Officers BANBEIS Bangladesh Bureau of Educational Information and Statistics BIRTAN Bangladesh Institute of Research and Training on Applied Nutrition BRAC Bangladesh Rural Advancement Committee CSP Country Strategic Plan CWW Children Without Worms DAE Directorate of Agriculture Extension DEQAS Decentralized Evaluation Quality Assurance System DGHS Directorate General of Health Services DHS Demography and Health Survey DID Difference-in-Difference DMA Data Management Aid DPE Directorate of Primary Education DPEd Diploma in Primary Education DPEOs District Primary Education Officer DPHE Department of Public Health and Engineering EGRA Early Grade Reading Assessment EQAS Evaluation Quality Assurance System FAS Foreign Agricultural Service FAO Food and Agriculture Organisation FGD Focus Group Discussion FM Field Monitor FY Fiscal Year FYP Five Year Plan GAIN Global Alliance for Improved Nutrition GDI Gender Development Index GDP Gross Domestic Product Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 193 GEEW Gender Equality and Empowerment of Women GHI Global Hunger Index GNI Gross National Index GoB Government of Bangladesh GPI Gender Parity Index GPS Government Primary School HDI Human Development Index HDR Human Development Report HNP Health, Nutrition, and Population IFAD International Fund for Agricultural Development ILO International Labour Organisation IQ Intelligent Quotient IR Inception Report JMP Joint Monitoring Programme JRP Joint Response Plan KIIs Key Informant Interviews LGD Local Government Division M&E Monitoring and Evaluation MA Muslim Aid MGD Mc Govern-Dole MIS Management Information System MoA Ministry of Agriculture MoHFW Ministry of Health and Family Welfare MoPME Ministry of Primary and Mass Education MPI Multidimensional Poverty Index MUR Monthly Utilization Report NCTB National Curriculum and Textbooks Board NER Net Enrolment Rate NGO Non-Government Organization NNP National Nutrition Policy NPAN2 National Plan of Action for Nutrition - 2 Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 194 NRMC NR Management Consultants India Pvt Ltd NSFPPA National School Feeding Programme in Poverty Prone Areas NSFPS National School Feeding Policy and Strategy NSMP National School Meal Policy NSSS National Social Security Strategy OEV Office of Evaluation, WFP PEDP Primary Education Development Programme PESP Primary Education Stipend Project PTA Parent Teacher Association PTI Primary Teachers Training Institute QLEAP Quality Learning for All Programme READ Reading Enhancement for Advancing Development RtR Room to Read SABER Systems Approach for Better Education Results SDG Sustainable Development Goals SFP School Feeding Programme SFPPA School Feeding Programme in Poverty Prone Areas SHEWA-B Sanitation, Hygiene Education and Water in Bangladesh SMC School Management Committee SOs Strategic Objectives SRM Supplementary Reading Material STH Soil-transmitted helminth SWAp Sector-wide approach ToR Terms of Reference ToT Training of Trainer TSER Transforming Secondary Education for Results UEOs Upazila Education Officer UN United Nations UNEG United Nations Evaluation Group UNHCR United Nations High Commission for Refugees UNICEF United Nations International Children's Emergency Fund Mid-Term Evaluation of WFP School-Feeding USDA Mc Govern Dole Grant for FY 2017-2020 195 URC Upazila Resource Center USAID United States Agency for International Development USDA US Department of Agriculture U5MR Under-five Mortality Rate VAM Vulnerability Assessment and Mapping WASH Water, Sanitation, and Hygiene WFP HQ WFP Head Quarter WFP World Food Programme WFP-CO WFP Country Office WHO World Health Organization YPSA Young Power in Social Action [Dhaka, September 2019, Inception Report, Mid-Term Evaluation Report]