This publication was produced at the request of the United States Department of Agriculture. It was prepared independently by Kathleen Webb, Donald Makoka, Brenda Sopo, and Sophia Dunn. Foreign Agricultural Service, United States Department of Agriculture Malawi McGovern-Dole Food for Education Project Final Evaluation March 2018 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Final Evaluation of the School Meals Programme in Malawi with support from the United States Department of Agriculture, and the Governments of Brazil and the United Kingdom 2013 to 2015 With funding support from the United States Department of Agriculture (USDA) McGovern-Dole Food for Education Program, World Food Programme (WFP) implemented the School Meals Programme, from 2013 to 2016. The aim of the program is to improve the literacy, health, and dietary practices of primary-school-aged children in Burkina Faso. Agreement Number: FFE-612-2013/037-00 Project Duration: 2013-2016 Implemented by: World Food Programme Evaluation Authored by: Team Leader: Kathleen Webb Team Members: Augustine Kamlongera, Donald Makoka, Brenda Sopo Independent Consultant: Sophia Dunn i Final Evaluation of the School Meals Programme in Malawi with support from United States Department of Agriculture, and the Governments of Brazil and the United Kingdom 2013 to 2015 Final Evaluation Report Volume 1 Decentralized Evaluation March 2018 Jointly commissioned WFP Malawi, FAO Malawi and WFP HQ Purchase for Progress Division Evaluation Manager: Grace Makhalira, M&E Officer Prepared by: Kathleen Webb, Team Leader Augustine Kamlongera, Team Member Donald Makoka, Team Member Brenda Sopo, Team Member Reviewed and edited by: Sophia Dunn, Independent Consultant Decentralized evaluation for evidence-based decision making ii Acknowledgements This is the final evaluation report of two World Food Programme (WFP) school meals programmes under implementation in Malawi. These are: the McGovern-Dole funded project and the Purchase from Africans for Africa project. The evaluation team wishes to acknowledge WFP Malawi, for providing guidance and support, specifically Ms. Coco Ushiyama, Country Director, Mr. Mieczylslaw Mietek Maj, Deputy Country Director, Mr. Bernard Owadi, Head of Programs, Ms. Chalizamudzi Matola, Head of School Meals Programme Section, Mr. Martine Mphangwe, Programme Policy Officer for School Meals Programm and Ms. Grace Makhalira, Evaluation Manager and Monitoring and Evaluation Officer, WFP Malawi. Many interviews were conducted with government, donors, and the private sector. The evaluation team expresses gratitude to the national and district Government of Malawi stakeholders. These were: Ms. Thokozile Chimuzu Banda, Chief Director (Basic Education and Secondary Education) and Ms. Virginia Kachigunda, Chief Education Officer, from the Ministry of Education, Science and Technology, Mr. Francis Chalamanda, National Coordinator for Early Childhood Development, Ministry of Gender, Children, Disability and Social Welfare, Ms. Martha Mwale, Nutrition Officer, Mr. Peter Nkoma, Agriculture Officer and Mr. Andrew Chamanza, Agri-business Officer, all from the Ministry of Agriculture, Irrigation and Water Development and Mr. Luke Kalilombe, Deputy Director, Poverty Reduction and Social Protection, Ministry of Finance, Economic Planning and Development. Ms. Florence Rolle, FAO Representative and Ms. Mpumulo Magombo, FAO Nutrition Coordinator, provided insight into the Purchase from Africans for Africa School Meals Program. The evaluation team also thanks Mr. Dallitso Mcheka, Head of Programs, Mary’s Meals and Ms. Catherine Mfitilodze, Technical Advisor, Deutche Gesellschaft Internale Zusammerbeit1 , for taking time to discuss the approaches to school meals programmes in their respective organizations. Finally, we wish to express gratitude to the local primary school heads, teachers, learners, householders and farmers for participating in the study. Disclaimer The opinions expressed in this report are those of the Evaluation Team, and do not necessarily reflect those of the World Food Programme. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. 1 Deutche Gesellschaft Internale Zusammerbeit refers to GIZ (German Technical Development) iii Table of Contents 1 INTRODUCTION 1 1.1 Overview of the evaluation subjects 1 1.1.1 The McGovern-Dole funded SMP 1 1.1.2 PAA SMP funded by the Governments of Brazil and United Kingdom 2 1.2 Context 4 1.3 Summary of findings from previous evaluations 7 1.4 Evaluation methodology and limitations 7 2 EVALUATION FINDINGS 10 2.1 Evaluation Theme 1: Relevance/appropriateness 10 2.1.1 Appropriateness to needs 10 2.1.2 Alignment with national policies and strategies 11 2.1.3 Alignment with WFP corporate strategies, policies and normative guidance 13 2.1.4 Alignment with other development partners 13 2.1.5 Summary of key findings on relevance 14 2.2 Evaluation Theme 2: Effectiveness 15 2.2.1 The McGovern-Dole funded SMP 15 2.2.2 PAA SMP funded by the Governments of Brazil and United Kingdom 24 2.2.3 Additional findings on the effectiveness of both projects 32 2.2.4 Summary of key findings on effectiveness 33 2.3 Evaluation Theme 3: Efficiency 33 2.4 Evaluation Theme 4: Impact and outcomes 35 2.4.1 School feeding component 35 2.4.2 Institutional demand/productive support component 35 2.4.3 Cross cutting outcomes 36 2.4.4 What were the major factors influencing the achievement of results? 37 2.5 Evaluation Theme 5: Sustainability 39 2.6 Evaluation Theme 6: Governance 41 3 CONCLUSIONS AND RECOMMENDATIONS 42 3.1 Overall assessment/conclusions 42 3.2 Lessons learned and best practices 44 3.3 Recommendations 44 4 BIBLIOGRAPHY 48 iv List of Tables Table 1: Literacy of school age children in McGovern-Dole supported schools 15 Table 2: Stationery and teaching supplies in McGovern-Dole supported schools 16 Table 3: Use of improved literacy instructional materials in McGovern-Dole supported schools 16 Table 4: Knowledge of teachers and school administriators in McGovern-Dole supported schools on SMP and HGSM 17 Table 5: Reported attentiveness and short-term hunger in McGovern-Dole supported schools 19 Table 6: Number of school meals provided in McGovern-Dole supported schools 19 Table 7: Enrolment and attendance rates in McGovern-Dole supported schools 20 Table 8: Economic and cultural incentives (THRs) in McGovern-Dole supported schools 21 Table 9: Promotion rates in McGovern-Dole supported schools 21 Table 10: Health-related absence in McGovern-Dole supported schools 22 Table 11: Presence of soap and water at handwashing stations in McGovern-Dole supported schools 23 Table 12: School infrastructure in McGovern-Dole supported schools 23 Table 13: Parents from McGovern-Dole supported schools able to name benefits of primary education 23 Table 14: Participation in school management and other committees in PAA supported schools 25 Table 15: Schools gardens in PAA supported schools 26 Table 16: Numbers of farmers benefiting from PAA inputs 27 Table 17: Enrolment rates in PAA supported schools 28 Table 18: Attendance rates in PAA supported schools 28 Table 19: Dropout rates in PAA supported schools 28 Table 20: Number of school staff trained on nutrition or diversified food basket 29 Table 21: Percentage of households reporting change to their diet 29 Table 22: Percentage of households from PAA schools reporting change to their diet 35 List of Figures Figure 1: Volume of the top ten commodities produced by PAA supported FOs 30 Figure 2: Volume of food commodities procured by PAA schools from local FOs (2014-2016) 31 v Acronyms AECDM Association of Early Childhood Development in Malawi AIDS Acquired Immunodeficiency Syndrome APES Agricultural Productivity Estimate Surveys AU African Union CO (WFP) Country Office CP (WFP) Country Programme CRECCOM Creative Centre for Community Mobilization CSB Corn Soya Blend CSB+ Corn Soya Blend with added micronutrients DEM District Education Manager DEMIS District Management and Information System DEQAS Decentralized Evaluation Quality Assurance System DFID United Kingdom Department for International Development DID Difference in difference analysis ECD Early Childhood Development EGRA Early Grade Reading Assessment EMIS Education Management Information System FAO Food and Agriculture Organization FEWSNET Famine Early Warning System Network FGD Focus Group Discussion FO Farmer Organization GDP Gross Domestic Product GIZ Deutche Gesellschaft Internale Zusammerbeit (German Technical Development) HGSM Home Grown School Meals HHS Household Hunger Scale HIV Human Immunodeficiency Virus IDI/GI In Depth Interview/Group Interview IPC-IG International Policy Centre for Inclusive Growth KII Key Information Interviews M&E Monitoring and Evaluation MDG Millennium Development Goal MGDS Malawi Growth and Development Strategy MoAIWD Ministry of Agriculture, Irrigation and Water Development MoCECCD Ministry of Civic Education, Culture and Community Development MoEST Ministry of Education, Science and Technology MoGCDSW Ministry of Gender, Children, Disability and Social Welfare MT Metric Tonne vi MWK Malawi Kwacha (currency) NAPE Nutrition and Access to Primary education NESP National Education Sector Plan NGO Non-Governmental Organization OECD Organization for Economic Cooperation and Development OVC Orphans and vulnerable children P4P Purchase for Progress PAA Purchase from Africans to Africa PTA Parent and Teacher Association SHN School Health Nutrition SMC School Management Committee SMP School Meals Program SO Strategic Objective SY School Year THR Take Home Ration ToR Terms of Reference UNDP United Nations Development Programme UNEG United Nations Evaluation Group UNESCO United Nations Educational, Scientific and Cultural Organization UNICEF United Nations Children’s Fund USD United States Dollar (currency) USDA United States Department of Agriculture WFP World Food Programme vii Maps Map 1: USDA McGovern-Dole SMP intervention areas in Malawi - 2016 (13 districts) Source: WFP Malawi viii Map 2: The PAA Africa intervention areas in Malawi 2016 (2 districts) Source: WFP Malawi ix Map 3: Acute food security situation (August 2013) Source: FEWSNET x Map 4: Acute food security situation (June 2016) Source: FEWSNET xi Executive Summary 1. This report is the final evaluation of two school meals projects (SMPs) implemented in Malawi. The two SMPs are implemented by WFP Malawi through its five-year Country Programme (2012-2016) with support from the United States Department of Agriculture (USDA), and the Governments of Brazil and United Kingdom. With the support from the USDA and McGovern-Dole, WFP implements a centralized SMP model using food commodities imported from the United States. At the same time, under the Purchase from Africans for Africa (PAA) Project supported by the Governments of Brazil and the United Kingdom, WFP is piloting a decentralized model using locally procured food commodities. Both SMPs are in their second phase.  Phase II of the McGovern-Dole SMP was implemented between January 2013 and December 2016 with a budget of USD 30 million, and covering 586 primary schools and 35 Early Childhood Development (ECD) Centres in thirteen districts. The SMP reached 693,349 students.  The PAA SMP is a smaller project, implemented jointly by WFP and FO in two districts – Phalombe and Mangochi at a cost of USD 560,700. Phase II of the PAA supported 4,798 farmers and 10,350 students in 10 schools in Mangochi and Phalombe Districts. 2. The two projects overlap their implementation areas in Phalombe and Mangochi Districts. The evaluation intends to assess the contribution of each project to the WFP’s overall School Meal Programme in Malawi. Overall, the evaluation results will be used to document best practices, identify challenges and possible solutions. The evaluation results will also be important for informing national policies and as an advocacy tool for the mobilization of resources on the scale up of Home Grown School Meals (HGSM) models to utilize local food items. Both the SMPs are implemented by WFP Malawi in partnership with the Ministry of Education, Science and Technology (MoEST), the Ministry of Gender, Children, Disability and Social Welfare (MoGCDSW) and the Ministry of Agriculture, Irrigation and Water Development (MoAIWD). Each project also has multiple other partners. The evaluation assesses and reports on the performance of the two SMP projects with dual and mutually reinforcing objectives of accountability and learning. Evaluation methodology 3. The evaluation applied the Organization for Economic Cooperation and Development (OECD) Development Assistance Committee (DAC) evaluation criteria of relevance, effectiveness, efficiency, and sustainability and evaluated the outcomes of the second phase of both the McGovern-Dole and PAA supported SMPs. The evaluation was designed to compare achieved results of the two programmes against targets and/or reported monitoring results. The evaluation was designed to answer multiple questions under six key themes: Relevance/appropriateness, Effectiveness, Efficiency, Impacts and outcomes, Sustainability, and Governance 4. The evaluation team comprised four consultants (one international team leader and three national consultants), with combined expertise working in evaluation, education, agriculture and development. The evaluation took place between November 2016 and July 2017, including two separate field trips, one for each SMP.  Field trip 1 (January - February 2017): McGovern-Dole field work covering seven of the thirteen targeted districts  Field trip 2 (April -May 2017): PAA field work covering Phalombe and Mangochi Districts xii 5. Key evaluation methods included document review, quantitative data collection from households, schools, Early Childhood Development (ECD) services, teachers/caregivers and farmers, and qualitative interviews with a range of project stakeholders including WFP, government, schools and community representatives. Key evaluation findings 6. Relevance: The evaluation finds that both SMPs were relevant for the Malawi context and for addressing the needs of the targeted beneficiaries: boys, girls, school personnel and farmers. The SMPs are both coherent with government, WFP and other development partners and the WFP and FAO collaboration for the implementation of the PAA SMP was highly appropriate since HGSM is one of the preferred models of the government and it is also being promoted by the African Union (AU). Capacity building for the government and involvement of the community in SMP implementation were both highly relevant activities employed by both SMPs. The USDA supported SMP on capacity building for the MoEST at national level resulted in the release of the SHN policy and paved the way for an upcoming strategy on SMP. 7. Effectiveness: The McGovern-Dole supported SMP has been effective at improving the capacity of the MoEST at national, district and school levels. The SMP also included effective community awareness campaigns that have contributed to ongoing student enrolment and attendance and reduced dropout. However it was not possible for the evaluation to assess whether the first SO – improved literacy – had been met due to secondary data access challenges. The other targets that were not achieved were due to unrealistic targets such as reducing health-related absence to zero or reducing inattention in class to <10 percent. It is important to note that despite not reaching a target of zero percent, it is a notable achievement that the programme has contributed to reducing the rate of health-related absence from 25 percent to 14 percent. The evaluation also found that households were more food secure at endline, but it is not possible to attribute the SMP to that finding as the overall food security situation in Malawi had improved since baseline. The McGovern-Dole SMP has also been effective at improving school infrastructure, particularly kitchens, although the handwashing facilities present at baseline (with soap) were not adequately maintained over the course of the programme. However, knowledge on importance of using soap for hand washing was found to be high despite the access issues. 8. The PAA supported SMP has been effective at improving market linkages between local farmers and schools. Schools successfully purchased numerous food commodities from FOs and provided children with diverse school meals. However, farmers were not able to provide sufficient food to schools because of the limited size of farmers land, production planning and dry spells. This resulted in schools buying additional food from local markets and farmers who were not members of the targeted FOs. The PAA supported SMP has effectively worked with both male and female farmers to improve their food production. This increased production, and the volume of sales to schools, indicate that farmers have almost certainly increased their income. However, more detailed information on this is needed. Training of school personnel, farmers and households on nutrition and dietary diversity has been effective at improving knowledge and has resulted in many households improving the diversification of their household diet. The evaluation team felt that Phase II was a missed opportunity to increase PAA coverage and iron out implementation issues such as the FOs being unable to supply the required volume of food items by officially allowing schools to purchase from other sources. xiii 9. Efficiency: The approaches of both SMP projects was efficient in terms of cost benefit as the social return to education investment was high. All supported communities benefited by sending their children to school, while girl retention rose due to community awareness. There was also a more active citizenry and improved sense of responsibility of community members towards school development. In terms of cost per child, the HGSM has been estimated2 to be more expensive than the direct provision of CSB, especially the imported CSB. However, full cost benefit analysis is required to determine which model is more cost effective, given the additional benefits of the HGSM that have not been factored into the costing. These benefits include provision of a more diversified meal to children and multiplier effects to the local economy. 10. Impact: There is strong evidence that both projects’ activities created positive impact on the intended beneficiaries. These included empowerment of women, increased community awareness of the value of education and in the case of the PAA supported SMP, improvements in dietary diversification (improving nutrition) during schools meals and at household level. However, there are still some factors that need to be addressed, such as inequity in volunteerism, placing a heavy burden on the workload of women. At national level, both project impacted positively on the capacity of the MoEST. The Ministry developed a SHN policy with support of the McGovern-Dole funded project and a stronger evidence base on HGSM due to the PAA project. 11. Sustainability: The Government has identified HGSM as the best model for sustainability as it addresses both education outcomes and stimulates agricultural production. However, the evaluation found that without clear statements on this, and a clear SMP strategy, along with stable and adequate government funding, the SMP cannot be sustained in the long term regardless of the model employed. 12. Governance: By supporting policy development, training and inputs to the MoEST at national level, the McGovern-Dole project has supported the ministry’s ability to exert administrative authority regarding school meals. However, the financial support to SHN committees was not enough for them to carry out expected monitoring and evaluation activities in schools. The McGovern-Dole support to school data collection supported the DEMIS databases, but district government level data entry has not kept up with data collection. The PAA project has also supported national governance by providing evidence-based experience towards a sustainable national HGSM model. Lessons learned and best practices 13. Community involvement is key factor in improving community ownership and sustainability: Capacity building, sensitization and mobilization of local communities to support the implementation of the two SMPs were major reasons for their success. Community awareness on the importance of education was also helpful for encouraging parents to enrol their children in school. In areas of high dropout, such as Mangochi District, greater emphasis on follow-up is needed to ascertain reasons and determine if the community awareness messaging is correct. 14. A HGSM model can provide wider benefits that increased participation in education: The PAA approach stimulated agricultural production and presumably increased the income of local farmers, although there is no documented evidence. The 2 Imani Development, “School Meals in Malawi-Best Practices Guide.” xiv presence of a stable demand market from the SMP resulted in greater participation of farmers, and more diversification of crops and household diets. This meant greater diversification of school meals, potentially improving child nutrition over time. 15. The opening up of school meals as a stable demand market meant that normal market forces including supply and demand came into play. Unfortunately even though the project was implemented in the most agriculturally productive districts, the local farmers were unable to meet the ongoing demand of the schools. Changes to the model therefore need to be introduced to further enhance local production capacity and/or to enable schools to purchase from multiple local sources. This would increase competition and potentially reduce commodity prices. Evaluation recommendations 16. The evaluation makes the following recommendations for improving the school meals programme in Malawi. They are listed in priority order. Priority 1: To be initiated within 6 months Strategic recommendations 17. Recommendation 1: WFP and other education partners should continue to support the MoEST to articulate a clear School Meals Strategic Plan that can be used as the basis for a nationalization strategy. The Strategic Plan should include an implementation plan and be accompanied by a designated government school meals budget line. 18. The formalization of a School Meals Strategic Plan remains critical for the Government of Malawi and for SMP implementers and donors as it will guide the process of nationalization of school meals and identify technical backstopping. The evaluation recommends that the School Meals Strategic Plan be developed in consultation with other partner ministries particularly those with existing linkages to education and/or school meals, including the MoAIWD, the MoGCDSW, Ministry of Finance, the MoCECCD and others as appropriate. The School Meals Strategic Plan should then include specific responsibilities for each actor. It is also recommended that as part of the development of the Strategic Plan, the social protection benefits of the SMP be strengthened and clearly articulated. This may help form stronger linkages with relevant ministries as appropriate. 19. Recommendation 2: WFP should continue to support the government to implement HGSM models. 20. The government has specified that a decentralized or HGSM approach is the preferred model for school feeding. However, this preference is yet to be formalized. Evaluation interviews with MoEST representatives indicate that the government is looking for a large scale SMP that incorporates both primary school and pre-schools (as per McGovern-Dole model) and that uses locally procured food commodities (as per PAA model). It is also recommended that during the development of the School Meals Strategic Plan, that stakeholders consider alternative HGSM models. Currently, there are multiple SMPs in Malawi implemented by multiple agencies and donors and it is not yet clear which one is most effective and which aspects of each should be replicated. All SMP models therefore need to be considered so that future implementation is clear. Note that a future national SMP model may require different SMP models to be implemented in different locations due to differences in productivity. It is also recommended that WFP and partners undertake a full cost benefit analysis to determine the cost efficiency of centralized and decentralized SMP models, and therefore the potential cost of a future national HGSM programme. xv Operational recommendations 21. Recommendation 3: WFP Malawi should address the causes of delays in their pipeline – both for delivery of CSB+ and for the release of monies to schools for the HGSMP. Priority 2: To be initiated within 1 year Strategic recommendations 22. Recommendation 4: WFP and the MoEST should strengthen the implementation of the Malawi National Girls Education Strategy and develop partnerships to implement complementary programming that make the school environment more appropriate for girls and orphans and vulnerable children (OVCs). 23. The evaluation found multiple examples of harassment of girls in primary schools, as well as high rates of girl dropout in some locations. It is therefore recommended that WFP establish partnerships with other agencies to strengthen the implementation of the Malawi National Girls Education Strategy. This will enable SMP projects to impact better on girls’ participation in education (enrolment, attendance, retention, completion, and their transition from primary to secondary schools). 24. Recommendation 5: WFP and other education actors should support the MoEST to implement the National Strategy for Inclusive Education. 25. The evaluation has found a gap in the provision of pre-primary education and education for children with special needs. It is therefore recommended that WFP advocate for the expansion of more ECDs building on the support provided to them in the past such as food, equipment and training of caregivers. Operational recommendations 26. Recommendation 6: The MoEST with support from WFP and other education partners including UNICEF and UNESCO should continue to address gaps in school infrastructure, teacher numbers and access to clean water. 27. Although both SMPs have contributed to increased participation in education, and the McGovern-Dole SMP has improved school infrastructure, challenges to the provision of quality education in Malawi still remain. Where possible, it is recommended that WFP leverage their relationships with other agencies to establish complementary projects to help address the above challenges. 28. Recommendation 7: WFP and MoEST should provide additional support to the monitoring, technical backstopping and training provided by the SHNs. 29. Recommendation 8: WFP and MoEST should continue to provide community awareness and education on the importance of education. 30. Recommendation 9: WFP and the MoEST should develop some common SMP guidance to ensure that each SMP include some key outputs and outcomes, regardless of implementer or donor. 31. Recommendation 10: MoEST with support from WFP and FAO should increase the use of the school gardens as a nutrition education tool and/or as an opportunity to provide fruits and vegetables for a more diverse school meal. 1 1 Introduction 1. This report is the final evaluation of two school meals projects (SMPs) implemented in Malawi. The two SMPs are implemented by WFP Malawi through its five-year Country Programme (2012-2016) with support from the United States Department of Agriculture (USDA), and the Governments of Brazil and United Kingdom. 2. With the support from the USDA and McGovern-Dole, WFP implements a centralized SMP model using food commodities imported from the United States. At the same time, under the Purchase from Africans for Africa (PAA) Project supported by the Governments of Brazil and the United Kingdom, WFP is piloting a decentralized model using locally procured food commodities. 3. The evaluation intends to assess the contribution of each project to the WFP’s overall School Meal Programme in Malawi. Overall, the evaluation results will be used to document best practices, identify challenges and possible solutions. In particular, the evaluation results will inform the design and implementation modalities of Government￾led initiatives and possible scale-up of a government owned, sustainable school meals model. Lastly, the evaluation results will be important for informing national policies and as an advocacy tool for the mobilization of resources on the scale up of Home Grown School Meals (HGSM) models that utilize local food items. The full evaluation Terms of Reference can be found in Annex 1. 4. Both the SMPs are implemented by WFP Malawi in partnership with the Ministry of Education, Science and Technology (MoEST), Ministry of Gender, Children, Disability and Social Welfare (MoGCDSW) and the Ministry of Agriculture, Irrigation and Water Development (MoAIWD). Each project also has multiple other partners as described ahead. The evaluation assesses and reports on the performance of the two SMP projects with dual and mutually reinforcing objectives of accountability and learning. 5. The rationale for the evaluation is to document the achievements and potential of the two SMP projects. This evaluation covers implementation of the two SMPs from January 2014 until December 2016. The evaluation took place between November 2016 and July 2017. The final report was then revised in February 2018 by an Independent Consultant to ensure that the report met the required quality and standards while at the same time qualifying some of the findings missing in the first version. 1.1 Overview of the evaluation subjects 1.1.1 The McGovern-Dole funded SMP 6. The McGovern-Dole SMP has been implemented in Malawi since 20103 for a total budget of USD 49.2 million. The first SMP under evaluation is Phase II of the McGovern-Dole SMP, implemented between January 2013 and December 2016 with a budget of USD 30 million. Phase II was implemented in 586 primary schools in thirteen districts: Nsanje, Chikwawa, Chiradzulu, Zomba, Thyolo, Mulanje, Phalombe, Mangochi, Dedza, Ntcheu, Salima, Lilongwe and Kasungu districts. In addition, 35 Early Childhood Development 3 Phase I was implemented from 2010-2012 in the same 13 districts. 2 (ECD) centres were supported in Chikwawa and Nsanje Districts (Map 1). In total, Phase II targeted 693,349 students. Phase III of the programme is currently ongoing.4 7. The McGovern-Dole SMP has two strategic objectives (SO): to improve literacy rates through improved quality of literacy instruction, improved attentiveness and improved attendance; and increased use of health and dietary practices through training, increased access to water, sanitation and preventive health practices, and increased access to adequate food preparation and storage tools. These objectives are shown in more detail in the results framework in Annex 2. The McGovern-Dole funded project also includes community sensitization on the benefits of education and more importantly, empowering parents to manage SMP through committees. The project was also designed to provide a conducive school environment by improving school infrastructure such as kitchens and storerooms. These structures are erected through community support. The SMP also included capacity-building for teachers, administrators and those handling foodstuffs on food preparation and storage. Cooking equipment including fuel efficient stoves, pots, plates, and spoons is provided for the preparation of porridge, and preparation of porridge is done by community members on voluntary basis. 8. The SMP has five planned outcomes: improved school attendance, increased learner enrolment, decreased learner drop-outs, increased knowledge of safe food preparation and storage practices and improved teacher attendance. The grant agreement between WFP and USDA/MGD incorporates 12 specific performance indicators and 21 results indicators against which performance of the program is measured. There were no modifications to the design or budget since the original agreement was signed. 9. As a centralized SMP project, USDA/McGovern-Dole provides enriched corn-soya blend (CSB+)5 imported from the United States for the provision of a daily meal. Children receive one meal a day at school (breakfast) comprising 100g of CSB+ per child per day. This comprises 60g from USDA/McGovern-Dole and 40g from other donors. Children in ECD receive 50g per meal. The project also provides take-home rations (THR) to girls and orphaned and vulnerable children (OVCs) in Standard 5 – 8 conditional on a monthly attendance rate of at least 80 percent. THR beneficiaries receive a transfer of 10kg maize/cereals per month during the lean season (January–March) using complementary funding from other donors. 10. The main partners in the implementation of the McGovern-Dole funded project are the MoEST, the MoAIWD, the Ministry of Gender Children, Disability and Social Welfare (MoGCDSW), the Creative Centre for Community Mobilization (CRECCOM) and the Association of Early Childhood Development in Malawi (AECDM). Community partners were school heads, management committees, and school feeding committees, students and householders. 1.1.2 PAA SMP funded by the Governments of Brazil and United Kingdom 11. The second SMP under evaluation is the Purchase for Africa by Africans (PAA) SMP funded by the Governments of Brazil and the United Kingdom to the value of USD $560,700. PAA is a joint initiative of FAO, WFP, local governments, the Brazilian Ministry of Foreign Affairs and the United Kingdom’s Department for International 4 Phase III is currently being implemented (2016-2017) in the same 13 districts targeting 548,000 learners in 456 primary schools with an estimated budget allocation of US$15 million. 5 CSB+ is corn soya blend enriched with multiple vitamins and minerals. 3 Development (DFID). The PAA started as a pilot in 20126 and the phase under evaluation is Phase II (2014-2018). The PAA differs from the McGovern-Dole funded SMP in that it procures food from local traders – Farmer Organizations (FOs) - for the provision of school meals. Phase II of the PAA supported 4,798 farmers and 10,350 students in 10 schools in Mangochi and Phalombe Districts (Map 2). Phase II of PAA is ongoing and will be completed in 2018. 12. The goal of the PAA SMP is to pilot a viable and sustainable school meals model that can be replicated by the government. It has two planned outcomes: community awareness on local production, food and nutrition, and local food procurement for schools. The project activities are drawn from a logical framework (Annex 3) and is jointly implemented by WFP and the FAO. WFP deals with the purchasing process, support to schools, and school feeding, while FAO provides inputs and technical assistance to farmers. 13. The PAA supported SMP is a decentralized SMP model. WFP transfers funds to schools via the district councils every school term. School committees, made up by teachers, parents and community representatives, then procure the food from farmer organizations based in the local community. Schools are then in charge of preparing and distributing the meals. Schools also have the choice of purchasing food items in local markets in cases where FOs are unable to supply specific items, quantities or quality products. The price is determined through negotiations between FOs and schools and aims to reflect local market prices. In addition the WFP provides the necessary infrastructure for storing, cooking and consuming the school meals. The schools procure a wide range of foods including cereals, pulses, fruits, vegetables, dairy products and meat. 14. Phase II targeted six FOs, each of which received training and inputs to build their marketing and management skills. The PAA also provides new infrastructure in the target schools: kitchens, eating areas and storerooms. There were no reports of modifications to the design or budget of the PAA project. Children in the PAA supported schools receive one meal a day at school (breakfast) comprising local food commodities. 15. No baseline survey for the PAA project was conducted but the indicators and classifications used in three mid-term and final monitoring reports reflect how the programme is to be monitored with regards to its outputs and outcomes7. 16. The PAA is implemented in partnership with the MoEST, the MoAIWD and FAO as well as the international NGO WeEffect and district councils, farmer organizations management, school procurement and feeding committees, and farmers. ~*~ 17. The operational areas of the two SMP projects overlap in two districts – Phalombe and Mangochi, and both SMPs work with the district School Health and Nutrition Committees (SHNs) and within the District Education Managers Offices (DEMs). Also, gender issues have also been mainstreamed throughout both programmes. Specific gender design features of the McGovern-Dole SMP include an emphasis on the need for gender balance on schools Boards of Management and improving sanitation facilities in schools to 6 Phase 1 began in February 2012 focusing on the operationalization of local food purchase pilots along with the elaboration of assessments and strategic plans to strengthen local food purchases for food assistance. With a total budget allocation of USD 4.58 million, the project supported 1,587 beneficiary farmers and 9,527 learners in 7 schools. 7 The three reports are: PAA Africa Malawi-Phase II Midterm Monitoring Report, May 2016 and; PAA Africa Phase II, Final Monitoring and Narrative Report, December 2016. The Progress Report of the Purchase from Africans for Africa Project, Malawi, 2013-2015, also served as a reference for comparison of findings related to the project indicators in the logical framework. 4 provide a more gender-sensitive school environment that encourages girls to consistently attend school, including during menstruation. The McGovern-Dole SMP also specifically provides THR to girls to improve girl’s enrolment and attendance in school and reduce dropout rates, and WFP also trained all project committee members, both men and women on the transition to HGSMP, and collected sex disaggregated data during monitoring where possible. The PAA actively prioritized the inclusion of women farmers into the programme. 18. The intended users of the evaluation findings include the WFP Malawi Country Office (CO), the relevant ministries of the Government of Malawi, USDA, the Governments of Brazil and the United Kingdom, the PAA Africa/WFP-FAO Coordination Unit, FAO, and other implementing partners. The USDA will also use the McGovern-Dole evaluation results to provide baseline values for Phase III of their SMP. The evaluation findings will also inform the scale-up of the PAA Africa programme. The FAO and WFP Regional Offices will use the evaluation findings to provide strategic guidance, programme support and oversight. The WFP Executive Board and Headquarters and FAO Headquarters and governing bodies (including the Committee on World Food Security) may use the evaluation to feed into evaluation syntheses and annual reporting. The Government of Malawi, other African governments, the FAO Nutrition and Food System Division and the Division for Emergency Operations and Rehabilitation, UN Agencies in Malawi, the International Policy Centre for Inclusive Growth (IPC-IG) and the general audience will also benefit from this evaluation’s learning component. 19. The CO management will be responsible to respond to the evaluation recommendations by providing actions that will be taken to address each recommendation and estimated timelines for taking those actions. The CO should also ensure that the final evaluation report is shared with all relevant stakeholders. The final evaluation report will be published on the WFP public website, and findings will be disseminated, and lessons incorporated into other relevant lesson sharing systems. 1.2 Context 20. Malawi is a low-income country that ranks 170 on the Human Development Index (2016). With the support of the International Monetary Fund, the World Bank, and other development partners, Malawi has been able to make important economic and structural reforms and sustain its economic growth rates over the last decade. Nevertheless, poverty is still widespread, and the economy remains undiversified and vulnerable to external shocks.8 21. Poverty: Malawi has a population of 17.7 million people of which 80 percent live in rural areas and depend on rain-fed subsistence farming for a living9. An estimated 39 percent of the general population and 60 percent of rural farmers live below the poverty line, while 15 percent of the population are classified as ultra-poor10. According to the 2016 United Nations Development Programme (UNDP) Human Development Report, there is a 19.2 percent contribution of poor education to poverty (evidenced in low enrolment), while 27.2 percent is health gaps (poor nutrition, stunting and high child mortality) and 53.5 percent is low living standards (poor housing and unsafe water sources). 8 http://www.worldbank.org/en/country/malawi/overview 9 PAA Programme Phase II: Country project, Malawi; Comprehensive Africa Agricultural Development Programme. 10 This means living on less than USD 2 per person per day. 5 22. Education: The introduction of free primary education in Malawi in 1994 meant increased school enrolment. However, this raised challenges such as shortage of qualified primary school teachers, inadequate and inferior infrastructure, inadequate teaching and learning materials, poor retention of girls, and poor participation of school committees. In response to the challenges, under the National Education Sector Programme (2008- 2017),11 Malawi is implementing a priority, fast-track programme to strengthen primary education. The programme combines policy measures, quality improvement and investments, through three sets of strategies: improve education quality, improve education access and improve education governance and management. 23. Despite the many challenges, at national level Malawi is nearing universal access to primary education although low attendance and completion rates, as well as high dropout and repetition rates remain in some areas. Data from 2014 shows a gross enrolment rate in primary education of 141% for both girls and boys combined. However, this decreased to 42% in lower secondary, with a student transition rate to secondary school of 73%. The primary net enrolment rate was 97% and the primary completion rate was 74%.12 Data for the following school year (SY 2015/16) shows only 50.9 percent of children completing primary school. Fifty-three percent of these were boys and 47 percent were girls. The repetition rates for the same school year were 23.4 percent (24.1 percent for boys and 22.7 percent for girls)13. In addition, it is estimated that 11 percent of children of official primary school ages are out of school with the rate for boys being slightly higher (11%) than for girls (10%).14 24. For children of primary school age in Malawi, the biggest disparity can be seen between the poorest and the richest children, with 18 percent of children in the poorest quintile out of school, compared with 3% of the children in the richest quintile.15 Similarly, data indicates that gender disparity in education remains. The Welfare Monitoring Survey (2014)16 found that 22 percent of females aged 15 years and above had never attended school compared to 9.8 percent of males17. The Malawi National Girls Education Strategy,18 identified girls to be particularly vulnerable in the primary school setting. This was due to absence of girl-friendly sanitation facilities, parents’ cultural preference for early girl child marriages, lack of safety on the way to school and harassment by boys and teachers in the schools. The same study found that only 25 percent of girls transited to secondary school with an estimated 25 percent of girls dropping out due to the above reasons19. Annex 4 provides information on key bilateral and multilateral donors to the education sector. It is estimated that 13 percent of bilateral aid to Malawi goes towards education20. 25. Literacy: Data from USAID’s Early Grade Reading Assessments (EGRA) indicate that literacy among primary school children in Malawi is poor. Students are not learning how 11 Government of Malawi. National Education Sector Plan 2008-2017. Ministry of Education, Science and Technology. Malawi. 2008 12 https://www.epdc.org/sites/default/files/documents/EPDC%20NEP_Malawi.pdf 13 Ministry of Education, Science and Technology. 2016 Education Statistical Bulletin (Annex 12, Table 3). 14 https://www.epdc.org/sites/default/files/documents/EPDC%20NEP_Malawi.pdf 15 Ibid. 16 Government of Malawi. Welfare Monitoring Survey. 2014 17 Government of Malawi. Welfare and Monitoring Survey 2014 and National School Health and Nutrition Strategic Plan (2009- 2018). 18 Government of Malawi. National Girls’ Education Strategy. 2014. Ministry of Education, Science and Technology 19 Government of Malawi. National Girls’ Education Strategy. 2014. Ministry of Education, Science and Technology; UNICEF, WFP, UNFPA. Joint UN proposal: Improving access and quality of education for girls in Malawi. 2013. 20 According to the Official Development Assistance figures (2014 to 2015) the main donors are the United States, United Kingdom, EU, Germany and France. 6 to read well in the first three years of primary school, with the average student beginning Standard 4 reading only 15 words per minute, and with 13 percent comprehension.21 26. Agriculture and food security: Agriculture is considered the engine of Malawi’s economic growth since it makes up almost 30% of GDP.22 The headline inflation rate has reduced rapidly in recent years falling to 9.3% in August 2017, compared to 22.8% in August 2016.23 This downward trend has largely been due to a sustained decline in food prices resulting from the increased availability of maize, and a stable exchange rate. Map 3 shows that at the end of 2013, much of Malawi was classified as food insecure, due largely to torrential rain and flooding. Since then food security has generally improved due to improved harvest and the subsequent decrease in food prices. Food insecurity is highest in the southern districts of Malawi, mainly caused by overdependence on maize, lack of multiple sources of income and poor market infrastructure. Map 4 shows that as of the end of 2016, the majority of the country is classified as food secure. 27. Health and nutrition: The 2006 National School Health and Nutrition Baseline survey of children between the ages of 5 and 10 years found that 70% of primary school children go to school without taking breakfast.24 Hunger was identified as one of the main obstacles to pre-primary and primary school attendance as hungry and underfed children drop out of school more frequently and face problems in concentration and homework. The general health status of school-aged boys and girls was also an issue: 34 percent of learners had suffered from illness in the two-week period prior to the survey: nine percent had intestinal worms and 19 percent had bilharzia.25 28. School meals programmes: WFP has been supporting SMPs in Malawi since 1999 and is one of more than a dozen organizations providing school meals. Overall SMP coverage in Malawi is estimated at 1,946,590 representing about 40% of total number of pupils in all primary schools. In total, WFP school meal interventions provide daily meals to over 993,000 pupils in about 783 schools and 93 ECD centres in 13 districts. This represents about 50% of the total number of children receiving meals in the country. The meals provided consist mainly of CSB, a large share of which are in-kind donations from international donors including the USDA. The overall SMP in Malawi focusses on addressing school health and nutrition through a multi-sectoral approach applied by key ministries working together with local communities. At first, the SMP approach was highly centralized in that beneficiary schools received imported CSB along with other benefits such as THRs. The Government policy now stresses a decentralized SMP through a HGSM model as it has the potential to stimulate the local economy, promotes greater community involvement in school meals production, and provides a more diversified diet. 29. Humanitarian situation: Malawi experienced recurring drought and flooding since 2002. The government’s ability to respond to both emergency and development activities is limited and has worsened following the 2013 financial scandal dubbed "Cashgate” 26 after which foreign aid was suspended. In 2015, there was massive flooding and subsequent drought, which left 6.5 million people food-insecure during the 2015 to 2016 season. Cotton, groundnuts, maize and rice crops worth USD 17.3 million were damaged 21 USAID. Malawi National Early Grade Reading Assessment Survey. Final Assessment. November 2012. 22 http://www.fao.org/3/a-i4491e.pdf 23 http://www.worldbank.org/en/country/malawi/overview 24 Ministry of Education and Vocational Training, Ministry of Health, National Statistical Office, author. National school health and nutrition baseline survey. Zomba: Malawi Government; 2006. 25 Ibid. 26 Cashgate was the uncovering of millions of dollars in stolen monies by government officials in Malawi. 7 due to the floods. The government instituted a state of emergency and emergency relief supported by donors followed. 30. Additional gender indicators: Women in Malawi face widespread inequalities. Malawi ranks 173 out of 188 on the UN’s Gender Inequality Index and has the eighth highest child marriage rate in the world. Labour force participation of men and women is similar (81.2% for women, and 80.8%) for men but women tend to be involved in the informal and lesser paid sectors than men. 31. Key additional education, equity and budgetary and expenditure data for Malawi (2012- 2016) can be found in Annex 5. 1.3 Summary of findings from previous evaluations 32. The McGovern-Dole funded SMP carried out a midterm review in March 2015. The key recommendations from this centred around improving national ownership and involvement in implementation, including the need for the government to finalize the National School Health and Nutrition Policy and dedicate a budget line towards SMP implementation. It was also recommended that WFP continue building the capacity of the government and other stakeholders for effective management of the SMP and increased ownership. The PAA project had a midterm review (May 2016) and a final monitoring report (December 2016). These reports outlined numerous recommendations mainly related to the school procurement process and improving national ownership. The full list of recommendations from the previous reviews of the McGovern-Dole SMP and the PAA SMP can be found in Annex 6. 1.4 Evaluation methodology and limitations 33. The evaluation applied the Organization for Economic Cooperation and Development (OECD) Development Assistance Committee (DAC) evaluation criteria of relevance, effectiveness, efficiency, and sustainability and evaluated the outcomes of the second phase of both the McGovern-Dole and PAA supported SMPs. The evaluation was designed to compare achieved results of the two programmes against targets and/or reported monitoring results. The evaluation was designed to answer multiple questions under six key themes as per the Terms of Reference (Annex 1). The evaluation findings are presented as per each of these themes.  Evaluation Theme 1: Relevance/appropriateness  Evaluation Theme 2: Effectiveness  Evaluation Theme 3: Efficiency  Evaluation Theme 4: Impacts and outcomes  Evaluation Theme 5: Sustainability  Evaluation Theme 6: Governance 34. To ensure validity and reliability of data, the evaluation questionnaires were designed using an evaluation matrix (Annex 7). This ensured that all aspects of the ToR were included. The evaluation also used an independent team of enumerators, who collected the quantitative data. The evaluation used a mixed-methods approach and triangulated information from different methods and sources to enhance the reliability of findings. The survey enumerators were trained in advance and the field tools were tested prior to field work.. 35. To conduct one evaluation that included two SMP projects, the team developed a study design that would respond to the evaluation criteria in the ToR. In the ToR, each criterion was divided up into components: general (overall project level assessment), school 8 feeding component (applicable to both projects) and institutional demand. Institutional demand focused on productive support, which was only applicable to the PAA project. Both quantitative and qualitative information was collected to respond to questions in the evaluation matrix. 36. The evaluation team comprised four consultants (one international team leader and three national consultants), with combined expertise working in evaluation, education, agriculture and development. The evaluation took place between November 2016 and July 2017, including two separate field trips, one for each SMP.  Field trip 1 (January - February 2017): McGovern-Dole field work covering seven of the thirteen targeted districts  Field trip 2 (May-June 2017): PAA field work covering Phalombe and Mangochi Districts 37. During both field trips, the consultants and research assistants worked as one unit, travelling to each sampled district and completing both quantitative and qualitative aspects of the evaluation before moving to the next district. This approach meant the team could meet at the end of every day to discuss. Triangulation of document reviews, survey and checklist analysis and synthesis of question guides was carried out in the report￾writing phase. This ensured the reliability and credibility of the data and information. The full evaluation schedule can be found in Annex 8. 38. The main stakeholders for the evaluation were drawn from a stakeholder analysis by the evaluation team drawing from the Terms of Reference. A full list of key informants can be found in Annex 9. 39. The key evaluation methods were as follows:  Document review: There was extensive review of documents as per the bibliography at the end of the report. Documents included the McGovern-Dole baseline survey and mid￾term review, as well as monitoring reports from the PAA SMP, as well as various WFP and Government of Malawi strategies and policies.  Quantitative data collection: Six quantitative tools were utilized were used for the evaluation: household survey, school/ECD survey/checklist, teacher/caregiver interview, farmer survey and a market survey. The quantitative survey tools can be found in Annex 10. In total the following quantitative data was collected: o 1,131 household surveys (762 households with children attending primary schools supported with school meals and 369 control schools), o 159 surveys in primary schools (103 McGovern-Dole supported, 10 PAA supported, and 46 control schools), o 14 surveys in early childhood care and development centres (eight supported by McGovern-Dole and six control centres), o 138 interviews with teachers/caregivers, o 288 farmers (139 farmers who were members of FOs and 149 control farmers) and surveys in 12 district markets close to the FOs. 40. Two of the quantitative tools, the school/ECD survey/checklist and the household survey were derived from the templates used for the 2015 McGovern-Dole baseline. The other quantitative survey tools were developed by the evaluation team.  Qualitative data collection: A total of sixteen qualitative tools were utilized during the fieldwork. Qualitative interviews were conducted with 94 persons either as key informant interviews (KIIs), group interviews (GIs) or in-depth interviews (IDIs). This figure does not include boy and girl learners interviewed in focus group discussions. Interviewed 9 stakeholders included government and community representatives as per Annex 11. The tools used paid special reference to gender dimensions, such as higher dropout rates of girls compared to boys, women’s’ heavy home workload and matrilineal and patrilineal systems which perpetuate discrimination against girls and women. To ensure girls’ and women’s participation, men, women, boys, and girls were separated during FGDs in order to gather different perspectives. Many of the tools explored issues of equity, social tensions and social protection issues. In Malawi, the MoEST requires each primary school to collect monthly data on education outcomes. This data was also utilized at school level and then verified with the DEM offices. 41. Sample selection: The selection of schools, ECDs and control groups was based on the original sample of the 2015 McGovern-Dole baseline survey. However the sample size was reduced when compared to 2015 in order to enable the USDA and PAA evaluations to be combined, while still maintaining statistical relevance at national level. Selection of control sites was randomly done by using every third count of the original sample from the McGovern-Dole survey, taking into account that the distance between the control and sampled subjects were in close proximity to provide the necessary comparisons and assume that all other benefits were the same. 42. Control group selection: Control groups were selected for schools, ECDs and households, as those not benefiting from the two SMP projects, or not listed in the WFP list of schools and ECDs as beneficiaries. In the second phase of fieldwork, the team again studied the six FOs and administered a new quantitative field survey tool only to farmers listed as members of the six FOs. Farmer control subjects were those using the same facilities/structures, but not on the FO membership lists, thereby limiting comparison. A more detailed description of the methodology including the panned and actual sample sizes can be found in Annexes 12 and 13. 43. Data management: During fieldwork data was entered directly into tablets provided by WFP. The evaluation team carried out content analysis of qualitative data, grouping data into themes and sub-themes, in line with the OECD-DAC criteria and the evaluation matrix. The evaluation relied on the support of IPC-IG and FAO for quality assurance of the evaluation inception report related to PAA exclusively, while the WFP Evaluation Manager used the WFP DEQAS Quality Assurance Checklists to ensure quality reporting. Analysis of the quantitative data was done using SPSS, STATA and Microsoft Excel software. All results are reported against the Performance Monitoring Plan agreed by MGD and WFP in their grant agreement. A Difference in Differences (DID) analysis was conducted on the survey datasets of 103 McGovern Dole primary schools, eight ECDs, 46 control primary schools and 6 control ECDs using SPSS.27 44. Ethics: The evaluation adhered to the United Nations Evaluation Guidelines (UNEG) Ethical Guidelines for Evaluation (March 2008).28 Utility was addressed by adherence to standards such as the evaluation matrix and a stakeholder analysis. Data collection for the evaluation included training for all enumerators on the survey and survey ethics. Participation in the evaluation was voluntary and participants were informed about the risks and benefits of participating in the research, confidentiality and use of the data. All information received from the interviews was anonymous, so it could not be attributed to a specific person. 27 Statistical Package for Social Science, Version 22. 28 United Nations Evaluation Group. UNEG Ethical Guidelines for Evaluation. March 2008 10 45. Limitations: The evaluation team encountered several challenges that need to be addressed or mitigated to complete the mission.  The two SMPs contained distinct differences in terms of their objectives and approach. There were also different donor requirements and different indicators for each project that had to be included in the data collection. This meant that two periods of field work were required.  The PAA project did not have any baseline survey and reporting which made determination of change as a result of the project difficult.  The datasets made available to the evaluation team by WFP and We Effect on the project were not up to date and primary schools and ECDs were not disaggregated.  The available data on farmer outcomes had gaps which made final evaluation difficult.  Household and farmer control groups were not readily available or unwilling to participate in the evaluation as they were not supported. The evaluation team interviewed some of the other organizations implementing SMPs, however these findings could not be triangulated as quantitative reports and summaries were not provided to the evaluation team. 2 Evaluation Findings 2.1 Evaluation Theme 1: Relevance/appropriateness 46. In assessing the appropriateness of the SMPs, the evaluators examined the extent to which the objectives, targeting, choice of activities, and modalities of transfer were appropriate to the needs of the population, aligned with government and WFP corporate policies and strategies, and coherent with the work of other development actors. 2.1.1 Appropriateness to needs 2.1.1.1 The McGovern-Dole funded SMP 47. The McGovern-Dole supported SMP is implemented in 13 districts that are among the most food-insecure in Malawi. Education outcomes in these districts are also among the lowest in the country. The evaluation therefore finds that these were appropriate districts for implementation. At school level, WFP has a checklist to determine the schools where the SMP will be implemented. This include checking whether the communities were willing to implement the programme, if there was a water source within 500 meters of the school, and the number of teachers and classrooms. However, the application of WFP procedures was constrained by congestion in the intervention schools which have overwhelmed the programme. FGDs with school personnel in McGovern-Dole schools confirmed gaps in terms of water sources and adequate number of teachers. Despite these challenges the evaluation found that the schools chosen were all appropriate for the intervention. 48. The SMP was designed using a centralized approach and prioritizing achievement of education targets. The SMP also converted twenty-two schools to the HGSM model.29 As a result of the centralized approach, project funding and activities were mostly directed 29 These are Mpita School and Kankhande School in Mangochi. There are also 20 McGovern-Dole converted schools in Salima: Chagunda, Chapita, Chikadwe, Chikwa, Kasupe, Katerera, Lifidzi, Lungumadzi, Mchoka, Mkhula, Mkombe, Mpita, Msindikiza, Mtemeyiti, Mtunthama, Naliwomba, Namgogoda, Kapira, Mgwirizano and Pembe Schools. 11 at national and school level support and less at district level support to SHNs. For this model of SMP, the evaluation finds that prioritizing support at national at school level was appropriate. School level activities included training, construction/rehabilitation of school infrastructure, and community awareness on the importance of education and nutrition all of which were appropriate given the context. In addition, the provision of THRs to girls and vulnerable children was highly relevant as a means of encouraging parents to send girls to school and prevent dropout. Likewise, supporting ECDs as well as primary schools was appropriate as the ECDs generally receive less government support than primary schools. The evaluation found that all these activities were appropriate and relevant to improve universal primary education but less appropriate to advance the productive components of SMP. “Our classes are peaceful as the learners are fed and able to concentrate. They now draw straight lines in their notebooks and eat using spoons, not their hands.”30 2.1.1.2 PAA SMP funded by the Governments of Brazil and United Kingdom 49. The PAA supported SMP is implemented in two districts: Mangochi and Phalombe, both of which are particularly affected by poverty. Mangochi has a population of 900,000, of which more than 90 percent are classified as poor. Phalombe has a similar poverty profile: out of 355,000 inhabitants 89 percent are poor. 31 Both districts are also characterized by a high HIV prevalence and more than 10 percent of children in both districts are living without their parents.32 Despite the high levels of poverty, both districts have a greater agricultural potential than surrounding areas. The evaluation finds that these two districts were therefore appropriate locations for piloting the PAA SMP. 50. WFP and FAO jointly implemented the PAA SMP using a decentralized, HGSM model. This model included activities directed at improving agricultural production, which in turn would improve food supply and market access. Aside from the productivity component, the PAA SMP aimed to improve education outcomes including enrolment, attendance and dropout rates. The incorporation of these objectives was appropriate for the context and improved the complementarity between the PAA and MGD projects. 51. The design of the PAA was found to be appropriate for capacity building of farmers. This was done by clustering farmers and enrolling them into registered FOs. The farmers then received inputs and training in marketing and agricultural production which helped them increase production and enable them to sell to schools.33 However, the evaluation found that given the preference of the government for the HGSM model, an expansion in coverage during Phase II would have been appropriate. 2.1.2 Alignment with national policies and strategies 52. WFP Malawi has been implementing SMP through its Country Programme in support of the government’s Growth and Development Strategy II in achieving universal primary education. The evaluation found that both SMPs also align with other key Government of Malawi strategies and with elements of the Government’s National School Health and Nutrition Policy (2013)34. The policy is directed at providing quality education and at the 30 Focus group discussion with teachers on 07.12.16 at a McGovern-Dole supported primary school, Mulanje District, Malawi. 31 In Mangochi 61% of population are poor and 30% are ultra-poor. In Phalombe 62% are poor, and 27% are ultra-poor. 32 In Phalombe 15% of the children live without their parents. In Mangochi it is 12%. See PAA Africa: A comparative case study of a Community-based HGSM and a HGSM based on Local Food Procurement 33 GIs with six FOs December 2016 and farmer survey results. 34 Republic of Malawi, National School Health and Nutrition Policy (2013) 12 same time enabling schools to link with local farmers to source food from local farmers. The new approach is expected to improve the nutrition of children and at the same time enhance sustainable and stable markets for smallholders to enhance community-wide economic and social development. 53. Both SMPs align with the National Education Sector Plan (NESP) for Malawi which sets out a plan for improving the quality, equity, relevance, access and efficiency of basic education. The plan specifically mentions improving the quality of education to reduce dropout, repetition and promotion rates, aligning well with the SMPs stated outcomes. The NESP also highlights the need for improving pre-primary education which aligns well with the inclusion of ECDs in the McGovern-Dole SMP. 54. The McGovern-Dole SMP also aligns to Millennium Development Goal (MDG) 2 “promote gender equality and empower women” as it supported child care and development, both of which are areas traditionally the responsibility of women, and included interventions specifically directed at girls. However, the McGovern-Dole SMP model is less aligned to the Sustainable Development Goal theme of promoting adequate resources at household and community levels, as it did not direct support to agricultural production. The McGovern-Dole SMP also aligns with the National Strategic Plan for ECD (2009-2014)35 as it provides meals to ECDs. Finally, the emphasis of the McGovern-Dole SMP on THRs for girls and vulnerable children is in line with the Malawi Girls’ Education and Communication Strategy36 and the National Girls’ Education Strategy37. Although at the time of the evaluation the Government National Strategy on SMP was not in place, a School Health and Nutrition (SHN) policy had been developed and is expected to be launched in 2018. Since WFP has supported the development of the SHN policy, both SMPs are expected to align with the key strategic directions. 55. Interviews with the MoEST confirmed that the government appreciated both SMP models. However, the HGSM model was preferred as it stimulated the local economy38 as well as providing meals in schools. The MoEST also made reference to HGSM as the preferred approach to increase the diversity of diets.39 HGSM is referenced in the Government of Malawi Growth and Development Strategic Plan II as the best means of providing one nutritious meal per day, as well as providing related health and nutritious services for school-going boys and girls. The MoAIWD also placed much emphasis on HGSM as it encouraged diversified production amongst local farmers and had the potential to reduce the risk of crop failure and improve farmer earnings40. The HGSM is also being promoted by the African Union (AU). 56. The evaluation found the PAA SMP also aligns well with the Government’s National Policy on School Health and Nutrition (2013)41 as the policy specifically mentions providing quality education while enabling schools to link with local farmers as a source of food. The PAA SMP also aligns well with the National Nutrition Policy and Strategic Plan, which in turn is closely linked to its Comprehensive Africa Agriculture Development Program 35 Republic of Malawi, National Strategic Plan for ECCD (2009-2014) 36 Republic of Malawi, Malawi Girls’ Education and Communication Strategy (2012) 37 Republic of Malawi, National Girls’ Education Strategy (2014) 38 KIIs, Ministry of Education, Science and Technology, December 1, 2016. 39 GI, Ministry of Gender, Children and Community Development, December 1st, 2016. 40 KII, Ministry of Agriculture, May 26, 2017. 41 Republic of Malawi, National School Health and Nutrition Policy (2013) 13 plan, and the Agriculture Sector-Wide Approach. These strategies are designed to reduce food insecurity and spur agriculture-led growth. 2.1.3 Alignment with WFP corporate strategies, policies and normative guidance 57. The objectives of the WFP Malawi Country Programme (CP 200287) include “support the development of sustained food and nutrition security programmes through on-site meals at pre-school and primary school levels”.42 The two key WFP SOs for the CP are to “contribute to increasing the proportion of boys and girls accessing and completing pre￾primary and primary education in WFP-assisted schools”; and “enhance the capacity of the Government to design and implement a sustainable school meals programme”. The CP document43 states that WFP will work with other donors and ministries to provide complementary services such as deworming, infrastructure improvement, school gardens and learning materials. There is also emphasis on government capacity building and need for a handover of SMP to the Government over a five-year period. The complementarity of the McGovern-Dole SMP and the PAA, makes an ideal fit with these objectives. In addition to the two SMPs, the WFP Malawi Country Office implements other programmes including a pilot Purchase for Progress (P4P) project which supports local purchase and support to smallholder farmers. The evaluation finds the SMPs to complement the other work that WFP does in Malawi. 58. The two SMP projects also align to several corporate WFP policies including the WFP Updated School Feeding Policy (2013)44 that specifies that WFP will focus increasingly on helping countries establish and maintain nationally owned programs linked to local agricultural production. The policy states that WFP will implement school feeding programs with clear hand-over strategies, where appropriate, and will engage in policy dialogue and provide technical assistance. The WFP Gender Policy (2012)45 also provides guidance to ensure that WFP’s programs promote a gender equality and women’s empowerment. 2.1.4 Alignment with other development partners 59. In general, the evaluation finds that the two SMPs align well with the work of other development partners in Malawi. WFP is one of more than a dozen organizations providing school meals, and all programmes align with the Government of Malawi’s goal of achieving universal education and providing school meals. More detailed information on the SMPs by other agencies can be found in Annex 14. 60. The provision of school meals also aligns with the WFP support directed at eradicating extreme poverty and hunger and achieving universal primary education in Malawi. Both SMP projects also contributed to the FAO SO of “eradicate hunger, food insecurity and malnutrition”46. The PAA project also contributed to FAO’s SO4 to “enable more inclusive and efficient food systems at local, national and international levels”. 42 WFP Country Programme 200287 (2012-2016) 43 WFP Country Programme 200287 (2012-2016) 44 WFP (2013) WFP Updated School Feeding Policy https://www.wfp.org/content/school-feeding-policy 45 WFP (2009) Gender policy: promoting gender equality and the empowerment of women in addressing food and nutrition challenges. WFP/EB.1/2009/5-A/Rev.1 46 FAO Strategic Objective 1 14 61. The HGSM model used in the PAA SMP provides an inclusive food system for local level school meals provision, while the McGovern-Dole SMP complements the CRECCOM strategy on community empowerment. CREECOM collaborated with WFP to sensitize communities to demand rights for girls to attend school (Salima District). Because of this collaboration, CRECCOM has re-aligned their own approach to link SMP principles to their ongoing projects. CRECCOM have also factored in the linkage of school meals to the rights of girls in their projects. This has meant changes to the CRECCOM Star Approach (Societies Tackling AIDS through Rights Processes).47 The results of the four-month collaboration of CREECOM with the McGovern-Dole SMP included sensitization of local communities to return 200 dropout girls to primary school. The outreach process also brought to the forefront harmful practices and gender-based violence cases against girls. The McGovern-Dole SMP has also supported the AECOM to address the gap in ECD coverage in Malawi.48 This support comprised food, equipment and caregiver training in Chikwawa and Nsanje Districts.49 62. The PAA SMP also contributed to other institutions’ strategies, specifically that of the NGO WeEffect.50 WeEffect partnered WFP to build the capacity of the six PAA FOs to join the cooperative movement and provide the PAA schools with food items. A HGSM model remains the main market for the FOs and according to WeEffect, and the FOs now have the legal standing to link in the future with Transglobe, NASFAM commercial and Auction Holdings Commodity Exchange. In addition, the FOs are now able to sell directly to WFP and others, including forming partnerships to address food security in emergencies when required. 63. The FAO collaboration with the MoAIWD has also been directed at promoting diversified production among farmers, guidance for HGSM menus, and promoting school gardens. It was noted that the MoAIWD has planned new programmes that the SMPs could link to in the future: the farm input subsidy programme, and integrated homestead farming and agribusiness programmes of the government. A future SMP HGSM project could benefit from these programmes but would need to factor in more collaboration with the MoAIWD to benefit from them. 2.1.5 Summary of key findings on relevance 64. Overall, the evaluation finds that both projects are relevant to the Malawi context. WFP is the largest government partner for SMP and their support to project implementation and policy development has been much appreciated. Both programmes align well with government and WFP policies, as well as with WFP’s other work in Malawi and with the work and direction of other development partners in Malawi. The large coverage area of the McGovern-Dole SMP and their support to ECDs was noted positively by many stakeholders. The model of local procurement of the PAA SMP was found to be in line with government preference for procurement models. 47 IDI, CRECCOM, 09.12.16, CREECOM implemented support with WFP from November 2014 – February 2015, sensitizing, motivating and mobilizing communities in Salima (Ngozi Zone) and Monkey Bay (Mangochi District). CRECCOM uses the STR approach- Engaging with DEC and ADC, training 30 volunteers per district to lead community discussion groups, the Training of Facilitators (TOF), the latter only done in Salima as funds ran out for Mangochi. 48 Only 40 percent of children under the age of five years (almost 4 million) are supported with ECDs. 49 KII, AECOM, 07.12.16. 50 We Effect is part of a global consortium directed towards the cooperative movement. 15 2.2 Evaluation Theme 2: Effectiveness 65. The following section looks at the specific results of the SMPs as per their results frameworks. Effectiveness findings are presented separately for the two projects except where the ToR makes reference to both projects.51 The findings are derived from four sources:  Data analysis of the final evaluation surveys and checklists, KIIs, GIs and FGDs of the McGovern-Dole SMP  Comparison of evaluation findings of the McGovern-Dole SMP against the baseline survey (2015) and mid-term review (2016)  PAA Logical Framework and PAA assessments and reports52  Government and other secondary sources of data on education and literacy as per the bibliography. 2.2.1 The McGovern-Dole funded SMP 2.2.1.1 Improved literacy of school-aged children 66. One of the key expected outcomes of the McGovern-Dole SMP is improved literacy of school age children. To achieve this, the SMP planned to provide school supplies (stationery), teaching materials and literacy instructional materials as well as training for teachers on the same, as well as supporting a conducive school environment for learning. However, the evaluation found it difficult to evaluate any change in literacy rates due to lack of monitoring of a literacy specific indicator. The programme monitored primary school pass rate, which does not provide a measure of children’s literacy per se. WFP have already recognized this issue and for Phase III have agreed with USDA to include a literacy specific indicator based on Early Grade Reading Assessments (EGRA).53 67. Table 1 shows that the primary school pass rate has increased over the period of the SMP, increasing from 68.5 percent at baseline to 95.6 percent at endline, far exceeding the 70 percent target, and exceeding the national pass rate of 71 percent. Control schools also improved to a similar level (94%). Table 1: Primary school pass rate of children in McGovern-Dole supported schools # Indicator Target Baseline Endline 1a Primary school pass rate Primary school pass increase from 68.5% - 70% 68.5% No details of number (n) TARGET MET 95.6% (increment of 27%) n=244,838 Control group pass rate = 94% 68. Secondary data from USAID EGRAs in Malawi54 between 2012 and 2015 show improvements in students in EGRA treatment schools in both Standards 2 and 4 in oral reading fluency relative to comparison schools although they were not significantly 51 Note: Some statistical findings have been calculated for the PAA project from analysis of McGovern-Dole education indicators. These can be applied to future PAA benchmarking. 52 The three reports are: PAA Africa Malawi-Phase II Midterm Monitoring Report, May 2016 and; PAA Africa Phase II, Final Monitoring and Narrative Report, December 2016. The Progress Report of the Purchase from Africans for Africa Project, Malawi, 2013-2015, also served as a reference for comparison of findings related to the project indicators in the logical framework. 53 “The percentage of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text.” 54 USAID. Midline Report. Impact Evaluation of the Early Grade Reading Activity. Malawi. 2015 16 different to control schools. The USAID midline suggested that high student to teacher ratios were partly to blame. 2.2.1.2 Better access to school supplies and teaching materials 69. The MoEST was expected to complement the McGovern-Dole funded SMP by ensuring supply of stationery and teaching materials. However, supplies of stationery to McGovern-Dole SMP schools have decreased markedly since baseline (Table 2), from 67.2 percent of schools having sufficient materials, to only 27 percent at endline. A third (32.7%) of control schools reported receiving stationery. Schools receiving teaching materials increased slightly from 67.2 percent at baseline to 70.3 percent at endline although this was not enough to meet the 100 percent target. Interviews with teachers in all schools confirmed that schools faced chronic shortages of both stationery and teaching materials. Table 2: Stationery and teaching supplies in McGovern-Dole supported schools # Indicator Target Baseline Endline 2 Better access to school supplies and teaching materials Increase of supplies of stationery from 67.2% to 100% 67.2% No details of number (n) TARGET NOT MET 27% of schools received stationery package (Total sample size =111 responses) Control schools = 32.7% Increase of supplies of teaching materials from 67.2% to 100% TARGET NOT MET 70.3% of schools received learning materials (Total sample size =111 responses) Control schools = 65.4% 70. The evaluation found significant variation in supply of stationery and teaching materials by district ranging from 9.1-47.4 percent of McGovern-Dole school receiving stationary,55 and 55.6-100 percent receiving teaching materials56(Annex 15). 2.2.1.3 Improved quality of literacy instructional materials 71. In addition to the above school supplies and teaching materials, McGovern-Dole supported schools were provided with training and materials on using improved literacy instructional materials. Although the Performance Monitoring Plan does not include a specific target for this activity, the endline survey found an improvement in the number of schools using the literacy materials, from 67.2% at baseline, to 77.5% at endline (Table 3). This is significantly better than control schools, as less than half the control schools (43.6%) were found to be using these materials at endline. Table 3: Use of improved literacy instructional materials in McGovern-Dole supported schools # Indicator Target Baseline Endline 3 Improved quality of literacy instructional materials Schools currently using improved literacy instructional materials (No target set) 67.2% No details of number (n) IMPROVEMENT 77.5% (Total sample size =111 responses) Control schools = 43.6% 55 9.1% of school in Chiradzulu District and 47.4% of schools in Mulanje District 56 55.6% of schools in Mangochi District and 100% of schools in Phalombe District 17 2.2.1.4 Increased knowledge and skills of teachers and administrators 72. The McGovern-Dole SMP included training for teachers and school administrators on a range of topics including SMP and HGSM, and the national primary school instructional materials/ECD curriculum. It was expected that increased training would translate into improved knowledge and skills, improved teacher attendance and an improved understanding of school administrators on SMP and HGSM. 73. Knowledge and skills of teachers: Table 4 shows that teachers reported increased knowledge and skills after receiving training. At baseline, 63.7-65.2 percent of teachers reported having some knowledge of SMP and HGSMP compared with 77.2 percent at endline. This is higher than the 65.2 percent of teachers with the same knowledge in control schools. In addition, 1,351 out of a total of 1,751 teachers (77.2%) had been trained on the national primary school instructional materials and related materials. Table 4: Knowledge of teachers and school administriators in McGovern￾Dole supported schools on SMP and HGSM # Indicator Target Baseline Endline 4a Increased knowledge and skills of teachers No target set 63.7%-65.2% knew about SMP and 60.6% knew about HGSM IMPROVEMENT 77.2% n=1,351 (Total teachers 1,751) Training was on national primary school instructional materials/ECD curriculum and related instructional materials Control schools = 65.2% 4b Teacher attendance 81.6 % of teachers attending school. Primary school teachers = 83% No details of number (n) TARGET MET 94.5% Total teachers 1,751 n=1,655 teachers attending ECD teacher attendance = 79.1% Control ECD school teacher attendance = 65.7% 5 Increased knowledge and skills of administrators No target set 63.7%- 65.2% (Average – 64.45%) knew about SMP and 60.6% knew about HGSM IMPROVEMENT 79.6 % n=274 (total administrators =309) who stated use of new quality techniques 74. The evaluation found that in Chiradzulu and Mangochi Districts, the percentage of teachers reporting increased knowledge was lower in McGovern-Dole supported schools than in the control schools (Annex 15). The Training that WFP conducted was consistent in all the districts however some of the participant teachers did not put into practice what they had learnt. 75. Teacher attendance: Interviews with DEMs indicate that teachers sometimes did not come to class. At baseline, although teacher attendance was 83 percent, there were reported issues of lack of motivation, high workload related to high student numbers and delays receiving their salaries. Since then teacher attendance has increased to 94.5 percent57 (Table 4). Higher teacher attendance rates were noted in ECDs, with 79.1 percent of teachers attending McGovern-Dole supported ECDs compared with 65.7 percent in control ECDs. 57 KII, 09.12.16, District Education Manager, Chiradzulu District. 18 76. United Nations Educational, Scientific and Cultural Organization (UNESCO) data shows an average teacher to student ratio in Malawi in 2014 of 1:73, this is considerably higher than the average teacher to student ratio in OECD countries (1:21). By endline, the evaluation found a teacher to student ratio of 1:75 in McGovern-Dole supported schools, 1:66 in PAA schools and 1:71 in control schools58 while at national level the teacher student ratio had improved to 1:70 by 2015. The NESP outlined a priority to reduce the ratio of pupils to teachers to 1:60 ratio by 2013/14 and below 1:60 ratio by 2017/18. High student numbers are not only a reflection of high teacher workload but of the increased potential for disruption in class.59 77. Interviews with teachers confirmed that SMP helped motivate them as they saw improved pass levels and reduced absenteeism due to the presence of school meals. Teachers were also more motivated once they were given more responsibility of participating in school meals committees and receiving training. Teachers confirmed during interviews that they learned about SMP procedures, such as how to complete monthly return sheets and deliver results to the DEM Offices and the WFP Regional Office in Blantyre District. However, many found this training too time consuming and too complicated, and in some cases, paperwork took them away from classrooms teaching60. The SMP return sheets delivered to the DEM offices for Chikwawa, Mulanje, Chiradzulu and Kasungu Districts were found to be haphazardly stored. 78. Knowledge and skills of administrators: In addition to training teachers, the McGovern-Dole SMP provided training for school administrators on SMP and HGSM. At baseline, 64.4 percent of administrators reported knowledge of SMP and HGSM compared with 79.6 percent at endline (Table 4). In all districts other than Phalombe District, the control sites administrators reported a lower percentage of knowledge and skills than the McGovern-Dole supported schools. The percentage of Phalombe District administrators reporting knowledge was the same as the control group (62.5%) (Annex 15). 2.2.1.5 Changes in student’s short-term hunger and attentiveness in class 79. The SMP was expected to improve student attentiveness by providing a meal in school to reduce short-term hunger. At baseline, teachers reported that 82 percent of children were attentive, and by endline a small improvement to 83.9 percent was found. This means the target of 97% has not been achieved (Table 5). Head teachers stated that the attentiveness indicator target was not met due to continued hunger. However, while this is probably true, the evaluation notes that there are numerous causes of inattention in class, including boredom, lack of appropriate teaching materials, and high numbers of students per classroom. The evaluation therefore finds that the 97 percent target was too optimistic for the context. 58 Final evaluation 2016 to 2017, school survey results: There were 123,293 learners and 1,652 teachers in McGovern-Dole sampled schools; 9,604 learners and 145 teachers in PAA schools and 51, 985 learners and 728 teachers in control schools. 59 National Center for Education Statistics, 1995. 60 Some examples-GI, 08.12.16, School procurement committee, PAA Nalingula school, Phalombe expressed concern that it was time consuming to track commodities, funds, attendance, dropout and status of the learner beneficiaries. 19 Table 5: Reported attentiveness and short-term hunger in McGovern-Dole supported schools # Indicator Target Schools Baseline Endline 6 Improved attentiveness 97% No details of number (n) McGovern￾Dole supported primary schools 82% No details of number (n) TARGET NOT MET 83.9% n=8,608 learners have improved attentiveness (Total pupils 10,260) 7 Reduced short￾term hunger 0-1=little to no household hunger; 2- 3=moderate hunger in the household; 4- 6=severe hunger in the household 0-1 Little to no household hunger on household hunger scale (HHS) McGovern￾Dole supported primary schools 2.7 Moderate hunger in the household TARGET MET 1.88 Little to no household hunger Control schools 3.6 Moderate hunger in the household 2.16 Moderate hunger in the household 80. At baseline, interviews with households with children attending McGovern-Dole supported primary schools showed they experienced moderate hunger,61 the same as found in the control schools (Table 5). This is consistent with the general food security situation at the time (Map 3). The endline evaluation found hunger had reduced in both supported and control households, which again is consistent with the general improvement in food security noted in Map 4. McGovern-Dole supported households reported little to no hunger at endline (HHS of 1.88), while households from control school still reported moderate hunger. This confirms the point above that at least by endline, ongoing inattentiveness in school was likely due to factors other than hunger. “Sometimes we learn while hungry. As such, sometimes we fail to understand what the teachers are saying. Some of our friends have run away from classes as they are hungry.”62 “As of now we do not struggle with hunger. Our bodies are growing very well. We are active in class and listen to the teachers very well.”63 2.2.1.6 Increased access to school feeding 81. Overall, the McGovern-Dole support was intended to provide children with one meal every school day over the course of Phase II (target = 742,121 meals). The McGovern-Dole SMP has provided 7,987 MT of CSB+ for the provision of the school meal. However, monitoring data indicates that WFP provided meals to 699,853 students, which is 72.9 percent of the target (Table 6). The evaluation noted that this was partially due to delays in CSB+ delivery and/or reduction in CSB+ portions in the meals. Table 6: Number of school meals provided in McGovern-Dole supported schools # Indicator Target Baseline Endline 8 Increased access to food (School feeding) Number of meals consumed on a daily basis at target schools - 742,121 to increase to 100% from 61% 61% No details of number (n) TARGET NOT MET 72.9% n=886,901 learners in project 61 Household Hunger Scale (HHS) score of 2.7 62 Source: FGDs, 15.12.16, mixed boy and girl learners, Control school, Kasungu District (Control school). 63 Source: FGD, 05.12.16, mixed girls and boy learners, McGovern-Dole supported primary school, Chikwawa district 20 82. The main causes of delays in the CSB+ pipeline were challenges accessing schools especially during the rainy season and during the floods. There were also delays in arrival of CSB+ shipments into port. In anticipation of these pipeline breaks, schools were informed by WFP to reduce the ration size from the recommended 100g ration of CSB+ to 60g per child per day (the USDA funded portion only). This was also done to stretch the availability of CSB+ in the midst of a hunger crisis in districts such as Chikwawa, Phalombe and Salima. During evaluation interviews in Chiradzulu District, the SHN Committee64 expressed concern that the reduction in ration size has contributed to inattentiveness in class, and meant the meal was less likely to adequately reduce hunger. 83. The evaluation found a few cases where ECD administration stated that they had to close down due to lack of CSB+ but these were not the norm. In Konzere and Thendo Primary Schools in Chikwawa District, schools ran out of CSB+ stock due to higher than expected number of students and the quantity provided was not adjusted to meet higher numbers. Chenjawazi Primary School in Kasungu District also repeatedly ran out of food after a month (instead of a term) due to them not following the appropriate ration size. 84. Most schools reported that the quality of CSB+ was good, but there were some cases in Chiradzulu and Kasungu Districts where the CSB+ tasted bitter and it was withdrawn by WFP.65 2.2.1.7 Participation in education (enrolment, attendance, promotion, completion and retention) 85. The McGovern-Dole SMP was designed to help increase participation in education. This was done through providing daily school meals as an incentive for students to enrol, attend, and remain in primary school until completion. The evaluation found that at baseline, McGovern-Dole supported schools already had a high enrolment rate of 88 percent and an attendance rate of 77 percent (Table 7) due in part to the many years of SMPs. Table 7: Enrolment and attendance rates in McGovern-Dole supported schools # Indicator Target Baseline Midline Endline 9 Improved student attendance School enrolment (no target set) 88% No details of number (n) 92% n=120,398 97.8% compared to midterm Learners enrolled 2016=135,466 Learners enrolled 2015 =127,988 School attendance increases to 90% from 77% at baseline 77% No details of number (n) 90.5% TARGET MET 92% of 10,260 n=9,440 learners School drop￾out (no target set) 6% No details of number (n) 5% No details of number (n) 4.4% n=738 Learners 86. At endline, further improvement was reported to 97.8 percent enrolment and 92 percent attendance, which exceeds the 90 percent target for attendance. In addition, the evaluation found that school dropout rates reduced from 6% at baseline to 4.4% at endline 64 IDI, 09.12.16, SHN Committee, Chiradzulu District. 65 IDI. 14.12.16, Kasungu SHN Committee 21 (Table 7). District dropout rates varied considerably by district, with Chikwawa District having the lowest rate of 2.8 percent, and Mangochi District the highest at 7.5 percent (Annex 15). As a means of encouraging school attendance by girls and OVCs and preventing dropout, THRs were provided to 70,402 students (predominantly girls) during the lean season as an incentive for regular school attendance. Table 8 shows that at baseline, 14.8 percent of girls and other vulnerable children regularly took home THRs, increasing to 20.9 percent at endline. Table 8: Economic and cultural incentives (THRs) in McGovern-Dole supported schools # Indicator Target Baseline Endline 10 Increased economic and cultural incentives (THR) 14.8% of girls and OVCs at target schools who regularly receive take home rations (no target set) 14.8% No details of number (n) 20.9% of girls and OVC 87. During the period 2013-2015, FY13 USDA did not provide WFP with funding for the THR under the regular McGovern Dole tranche to Malawi. However, WFP was able to support some of the McGovern Dole schools with complementary funding from other donors including DFID, USAID and private donors, to ensure that vulnerable students, especially those in upper-grade levels, remained in schools particularly during the lean season. From 2015, THR was no longer provided in McGovern-Dole schools due to funding constraints.66 According to interviews with the SHNs, the suspension of other THR might have affected both school attendance and performance of girls and orphaned boys.67 88. The McGovern-Dole SMP used promotion rates as a measure of school retention. This is a measure of the percentage of children who were promoted from one standard to the next against number of children enrolled in a particular year. At endline, the evaluation found a marked improvement in promotion rates from 68.5 percent at baseline to 95.6 percent (Table 9) surpassing the 70 percent target. Table 9: Promotion rates in McGovern-Dole supported schools Indicator National average Target Baseline Endline 11 Promotion rate National (2014) = 71% Boys =75%, Girls = 72% 70% 68.5% TARGET MET 95.6% (n=244.838 learners) Control schools = 94% 89. The endline promotion rate is substantially higher than the most recent (2014) national average of 71 percent. Schools in all seven sampled districts surpassed the national pass rate, as did the control schools (average of 94% promotion rate). 90. The findings of the DID analysis through mean tests can be found in Annex 16. No significant difference was found between boys and girls in any of the participation in 66 Some examples: GI, Chiradzulu SHN Committee, 09.12.16, confirmed that THR has not been received for the past 3 years: GI, Salima SHN Committee, 13,12.16, stated that no THR and that WFP informed them their pipeline was down; GI, School feeding committee (Chiradzulu District) 09.12.16, stated that THR was very beneficial but was stopped a year back. 67 Nakhundi Primary School, Chiradzulu District, is an example of a school reporting how THR had reduce girls’ dropout rates until it was stopped. 22 education indicators (primary school enrolment, ECD enrolment, promotion rate, transition to primary or secondary school, or dropout rate). 2.2.1.8 Reduced health-related absences 91. It was expected that the provision of nutrient-dense school meals, along with deworming, training and awareness of good nutrition would contribute to improved health of school children, which in turn would reduce health-related absence. Table 10 shows that at baseline, 25 percent of student absences from primary school were reported as health related, reducing to 14 percent by endline. However, even this significant improvement was not enough to meet the target of zero days. The evaluation finds that this target was not realistic due to the numerous causes of illness in children, only some of which are nutrition related. Table 10: Health-related absence in McGovern-Dole supported schools # Indicator Target Baseline Endline 11 Reduced health related absences School days missed due to illness declines from 25% to a target of nil days missed 25% No details of number (n) TARGET NOT MET 14% (Improvement of 11% from baseline) 92. Interviews with DEMs, SHN Committee members, parents and teachers suggest that the improvement is at least partly attributable to the CSB+ and the additional nutrition that the children receive as a result of the school meal. Parents reported their children to be healthier and able to attend school more regularly. Similarly, 44 percent of farmers stated that the more diversified school menu improved the health of their children. The School Feeding Committee at a McGovern-Dole supported primary school in Chikwazulu District stated, “CSB saves lives”. 68 “Our children drop out less due to sickness now. Most of the learners in this school are able to complete Standard 8 now.”69 93. Government data suggests that intestinal worms and bilharzia remain of ongoing concern. These need to be addressed by improving water sources, hygiene and access to treatment. Treatment of intestinal worms is handled by the Ministry of Health in coordination with MoEST, the World Health Organization and other partners through a national deworming programme. This provides treatment to children against intestinal worms in all schools every 6 months, and treats bilharzia in selected districts. 94. According to DEMs, SHNs, teachers, the level of hygiene and sanitation in the schools also contributed to student attendance as children were healthier when hygiene was maintained. However, the evaluation notes that not all schools had soap and water at handwashing stations. Table 11 shows that although at baseline, 72.8 percent of the McGovern-Dole supported schools had handwashing stations with soap and water, this had decreased by endline. At endline, less than half (45 percent) of schools had soap and water suggesting that previously equipped handwashing stations were not being maintained. No statistical difference was found between McGovern-Dole supported schools and control schools in this regard. 68 GI, 14.12.16, Teachers at GoM Bowe Primary School, Kasungu District, providing CSB to learners. 69 Source: FGD, parents, 14.12.16., McGovern-Dole supported primary school, Kasungu District 23 Table 11: Presence of soap and water at handwashing stations in McGovern￾Dole supported schools Indicator Baseline Endline Percentage of schools with soap and water at handwashing stations McGovern-Dole supported schools 72.8% 45% Control schools 68.3% 44.2% 2.2.1.9 Increased school infrastructure 95. In addition to the provision of the school meal, the McGovern-Dole SMP included construction or rehabilitation of new kitchens, dining areas and food stores. Table 12 shows that the target of 681 new structures has been exceeded. Interviews with teachers indicate that they were very satisfied with the new structures., whereas interviews with teachers from non-McGovern-Dole supported schools indicated that many schools were without dining areas. As a result, students ate under trees and a half-built teacher’s home was being used as a kitchen70. Table 12: School infrastructure in McGovern-Dole supported schools # Indicator Target Baseline Midline Endline 12 Increased school infrastructure 681 kitchens rehabilitated 577 681 TARGET MET n=703 (Target surpassed by 22 structures) 3.2% increment of kitchens rehabilitated from baseline 2.2.1.10 Increased community understanding of the benefits of education 96. The McGovern-Dole SMP included activities to sensitize communities to increase their understanding of the benefits of education and therefore contribute to increased enrolment and attendance. Table 13 shows that at baseline, 72.4 percent of parents were able to name three benefits of primary education. Ongoing community sensitization has resulted in this figure increasing to 99 percent of households by endline. The 90 percent target has therefore been met. The evaluation found the community sensitization to be one factor in the increased enrolment and attendance, and reduced dropout discussed earlier. Table 13: Parents from McGovern-Dole supported schools able to name benefits of primary education # Indicator Target Baseline Endline 13 Increased community understanding of the benefits of education Parents able to name three benefits of education increased from 65% to 90% 72.4% No details of number (n) TARGET MET 99% n=700 parents 97. A completed table of all the McGovern-Dole SMP indicators at endline compared to baseline can be found in Annex 17. 70 Note: The team visited Bowe School as an example of a government school (not part of the McGovern-Dole SMP) where parents and teachers were interested in starting HGSM. It was not part of the sample. 24 Key findings re effectiveness of the McGovern-Dole SMP  It was not possible for the evaluation to assess whether the first SO – improved literacy – had been met. WFP and USDA have agreed to change the literacy indicator for the next phase of the programme and monitor literacy rates more closely.  Aside from literacy, the majority of the programme targets have been achieved. Those that have not were due to the MoEST not providing stationery and teaching materials, or unrealistic targets such as reducing health-related absence to zero or reducing inattention in class to <10 percent.  Enrolment and attendance rates were high at baseline and remain so despite school meals not being provided each school day, and THR being suspended in 2015 in some schools due to resource constraints.  Findings of the DID analysis through mean tests found no significant difference between boys and girls in any of the participation in education indicators (primary school enrolment, ECD enrolment, promotion rate, transition to primary or secondary school, or dropout rate).  WFP monitoring data indicates that meals were provided to 699,853 students, which is 72.9 percent of the target. This is mainly due to delays in the provision of CSB+.  The SMP has been effective at improving school infrastructure, particularly kitchens, although the handwashing facilities present at baseline (with soap) were not adequately maintained over the course of the programme.  Increased community awareness on the benefits of primary education has effectively contributed to the ongoing high enrolment and attendance rates and reduced dropouts. 2.2.2 PAA SMP funded by the Governments of Brazil and United Kingdom 98. Unlike the McGovern-Dole supported SMP, the PAA supported SMP did not conduct a baseline survey and no targets were set for project indicators. The following findings on effectiveness are therefore drawn from comparisons of the degree to which indicators were met at the time of the evaluation based on findings of monitoring reports provided to the evaluation team.71 The expected result of the PAA SMP was that local farmers would be able to provide food items to schools to enable each PAA supported schools to provide a diverse school meal each school day. Like the McGovern-Dole SMP, the school meal was then expected to reduce short term hunger, allowing children to concentrate on their studies. It was also anticipated that the provision of the meal would also provide incentive for parents to send their children to school, increasing the enrolment and attendance rates, and reducing dropout. 99. The evaluation notes that some schools experienced delays in receiving monies to procure food. This was partly due to delays by WFP in releasing funds to the PAA supported schools, and partly due to late submission of financial reports from schools. This meant that meals were not provided in all schools on all school days as planned, potentially reducing the effectiveness of the programme. For example, interviews with school personnel at Ching’ombe Primary School in Mangochi District reported experiencing 71 The reports were: PAA Africa Malawi-Phase II Midterm Monitoring Report, May 2016 and; PAA Africa Phase II, Final Monitoring and Narrative Report, December 2016. The Progress Report of the Purchase from Africans for Africa Project, Malawi, 2013-2015, also served as a reference for comparison of findings related to the project indicators in the logical framework. 25 regular food shortages due to delays in release of funds by WFP and missed at least one whole term of meals. 2.2.2.1 Outcome 1 100. The first outcome of the PAA project is that communities are more aware of local production, food and nutrition and how these can be applied at public policy level, community level, school and household level. To achieve this outcome, the project encouraged community engagement in school meals committees, supported the establishment of school gardens, and registered local farmers into farmer organizations. Number of people participating in the PAA School Management Committees or other HGSM committees 101. As with the McGovern-Dole supported SMP, the PAA supported SMP expected community participation in the management of the SMP. The evaluation found that there were Parent Teacher Associations (PTA) in place in schools in Phalombe District and membership comprised 44 percent women. The percentage of women was higher in School Feeding Committees at 55 percent, and higher still in the School Support Committees (62% women) (Table 14). In addition, there were finance and procurement committees, with participation of community members that made decisions about sourcing of food and payments processes. Table 14: Participation in school management and other committees in PAA supported schools # Indicator District Previous assessments 2017 evaluation 5 Number of people participating in the PAA school management committees or other HGSM committees Phalombe No details Parents in School Support committees=47 38% n=18 are men and 62% n=29 are women Parents in School Food Committees=49 45% n=22 are men and 55% n=27 are women Number of parents in PTAs=48 56% n=27 are men and 44% n=21 are women Total people in school committees=144 Mangochi No details Parents in School Support committees=105 41% n=43 are men and 59% n=62 are women Parents in school food committees=100 39% n=39 are men and 61% n=61 are women Number of parents in PTAs=92 56% n=49 are men and 47% n=43 are women Total people in school committees=297 102. Similar findings were noted in Mangochi District, with PTA membership also being 47 percent women, and School Feeding Committees membership of 61 percent women and School Support Committees 59 percent women (Table 14). Overall, Table 14 shows the high level of parental support in the schools with more than 140 parents involved in the 10 PAA schools in Phalombe, and almost 300 parents in the 10 Mangochi schools. The evaluation found only two schools where the School Meal Committees were not active. One of the 10 PAA schools, and one of the 111 McGovern-Dole schools sampled reported that their school meals committee was not operating but neither provided a satisfactory reason. Number of school gardens in place in the 10 PAA schools supported by the communities 103. One of the expected results of the PAA SMP was to have one school garden present in each targeted school. Table 15 shows that at the time of the evaluation, all five schools in Phalombe District had school gardens, although the gardens were not well maintained 26 nor very productive. Nalingula Primary School had recently put a school garden in place, but soon encountered water shortages as they shared a borehole with the local community72. In Mangochi District there were only three school gardens in place as the other two PAA schools did not have enough land to establish gardens. Chingombe Primary School in Mangochi District did not have a water supply but was found to be cultivating maize on one acre of school land at the time of the final evaluation73. Table 15: Schools gardens in PAA supported schools # Indicator District Previous assessments 2017 evaluation 4 Number of school gardens in place in the 10 PAA schools, supported by community Phalombe No details 5 gardens (100% of schools) Mangochi 3 gardens (60% of schools) Number of FOs registered as cooperatives by year 104. The evaluation found that all six FOs included in the PAA were registered as cooperatives and therefore subject to the rules and levies of the Ministry of Trade. In Mangochi District, the Chibwerera FO was registered in 2009, followed by Mthilamanja FO in 2016 and Mvumba FO in 2017. In Phalombe district, Nanguluwe FO was registered in 2004, while the Nkulumbe FO was registered in 2007 and Chakalamba FO was registered in 2017. Data from the mid-term monitoring report indicates that 58 percent of all members were women, surpassing the 30 percent target. 105. In addition to the above indicators, the log frame included measurement of farmers income and the percentage reinvested into the community. The evaluation found no monitoring data related to this. 2.2.2.2 Outcome 2 106. The second PAA outcome focused on the purchasing modalities of SMP to ensure that they have been adapted to suit school capacities and local procurement constraints. The PAA project planned to implement two models: (a) direct procurement from beneficiary farmers and (b) direct procurement from local markets. Most of the project focus was to develop procurement from farmers since it formed the bulk of school food sources. Procurement from local markets was only done for non-food items and other food commodities that are not produced locally such as salt and vegetable oil. The PAA logframe also includes indicators for monitoring pipeline breaks, the quantities of food delivered to schools and whether agricultural techniques have been utilized by local farmers. Number of farmers who have benefitted from PAA inputs by type and gender 107. Table 16 shows that 478 farmers in Phalombe District and 912 farmers in Mangochi District received seeds for planting in 2014, increasing to a total of 3,134 farmers supported by 2017. The seeds received included cowpeas, groundnuts, soya, beans, pigeon peas, cassava and potato seeds. Most farmers received multiple varieties of seed and the Plof seed in both districts were female farmers. Interviews with farmers found that although there were no special allowances for women farmers, they were able to compete fairly. The evaluation notes that one FO was women-owned. 72 GI, 08.12.16, Teachers, Nalingula Primary School, Phalombe district. 73 GI, 12.12.16, School teachers, Chingombe Primary School, Mangochi District. 27 Table 16: Numbers of farmers benefiting from PAA inputs # Indicator District 2014 assessments 2017 assessments 7 Number of farmers benefiting from PAA inputs by type and gender Phalombe 478 farmers Male = 51.7%; Female = 48.3% 30.4% of district farmers Male = 41.7%; Female = 58.3 % Mangochi 912 famers Male = 49.3%; Female = 50.7% 17.7% of district farmers Male = 17.7%; Female = 69.6 % Total 1,390 farmers 4,798 farmers Source: PAA Midterm monitoring report (May 2016) 108. All six FOs were provided with large warehouses to support food storage. However, these remained underused or empty as the FOs reported they did not have enough business yet. Other than the warehouses, the FOs stated that they had not yet acquired additional assets or purchased/rented additional land. Number of farmers who have benefited from PAA training by type and gender 109. The NGO WeEffect together with the Ministry of Trade and Ministry of Agriculture provided 22 types of training for farmers including cooperative management, agribusiness, contract negotiation, price determination, cost calculation, budgeting, fund management, food quality control, crop management (diversification and production), and post-harvest management (including storage) (Annex 18). At the time of the evaluation, the seed loan scheme and warehouse receipt system courses had the highest number of participants. The evaluation found 1,022 persons from Phalombe District and 964 from Mangochi District attended training. Interviews with farmers indicate high satisfaction rates. According to the FO members interviewed, training focussed on many relevant areas, but more training or re-training was needed. Number of smallholder farmers that supplied food for the PAA Africa programme, by gender and level of education 110. Throughout the project, 254 smaller holder farmers supplied food to PAA supported schools in Phalombe. No data was available from Mangochi. Considering there is a total of 4,472 smallholders belonging to the FOs supplying PAA schools in Phalombe, the number supplying the schools is only 5.6 % of FO members. Women farmers have been included in the project although sales from female farmers represent 36% of the total amount sold to schools. This is a significantly smaller share than men. 2.2.2.3 Outcome 3 111. The third PAA outcome relates to supporting access to education and human capital development in assisted schools though local procured food. Enrolment, attendance, and dropout rates are all included in the PAA logframe for this outcome although no specific targets were set. Enrolment of school learners at assisted schools by gender 112. Table 17 shows an increase in student numbers (boys and girls) enrolled in Phalombe since baseline. In 2014, assessments showed that girls made up 50.9 percent of all students, and this has increased to 54.2 percent by endline. In Mangochi however, both girl’s and boy’s enrolment decreased, with boy’s enrolment decreasing by more than 20 percent. This is due to the Government opening a new school to reduce the high student numbers and travel time to Ching’ombe School. The new school was not included in the 28 PAA project. This has changed the proportion of boys and girls in schools, and by endline, girls made up 55.2 percent of all students in the PAA supported schools in Mangochi. Table 17: Enrolment rates in PAA supported schools # Indicator District 2014 assessments 2017 evaluation PAA Schools Control Schools 1 Enrolment of school pupils at assisted schools by gender Phalombe Boys (n) = 2,793 Girls (n) = 2,896 Boys (n) = 3,277 Girls (n) = 3,871 Boys = 49.1% Girls= 50.9% Boys = 45.8% Girls = 54.2% Boys = 45.2% Girls = 54.8% Mangochi Boys (n) = 1,875 Girls (n) = 1,988 Boys (n) = 1,472 Girls (n) = 1,816 Boys = 48.5% Girls = 51.5% Boys = 44.8% Girls = 55.2% Boys = 48% Girls = 52% Attendance of school learners at assisted schools by gender 113. Data obtained for this indicator represents the aggregate attendance rates for all PAA schools participating in Phase II (Table 18). Table 18: Attendance rates in PAA supported schools # Indicator District 2014 assessments 2017 evaluation PAA Schools Control Schools 2 Attendance of school pupils at assisted schools by gender Phalombe Boys = 78.9%* Girls = 82.6%* Boys = 99.5% Girls = 100% Boys = 96% Girls = 88.2% Mangochi Boys = 75% Girls =80% Boys = 78.7% Girls = 84.8% *Data from 2014 did not separate results by district 114. The evaluation found that implementing partners were not collecting data for this indicator the evaluation used data from WFP and it was difficult to establish how this data was obtained. It is therefore difficult to establish the reliability and validity of the data for this indicator. The data shows an increase in attendance in Phalombe but a decrease in Mangochi with no significant difference between PAA and control schools. Dropout rates 115. Table 19 shows that according to PAA reports, the dropout rates have increased significantly in both Phalombe and Mangochi Districts since 2014. The evaluation found higher dropout rates in PAA schools than in control schools, with the exception of girls in Phalombe. Dropout rates varied considerably between schools, between districts and between boys and girls. In Phalombe District, dropout rates of boys were as high as 14.5 percent (in Thema) while there were no dropouts of boys in Kankhande School in Mangochi district. The dropout rate for girls in Phalombe District was as high as 11.8 percent in Tharu School and as low as 0.5 percent in Nalingula School. This inconsistency of findings indicates there are other factors involved in school dropout that school meals are unable to address. Further data on this can be found in Annex 19. Table 19: Dropout rates in PAA supported schools # Indicator District 2014 assessments 2017 evaluation PAA Schools Control Schools 3 Dropout rates Phalombe Boys = 5.3% Girls = 5.5% Boys = 9.3% Girls = 1.9% Boys = 9.8% Girls = 7.1% Mangochi Boys = 1% Girls = 2.5% Boys = 11.4% Girls =7.3% Boys = 7.4% Girls = 6% 29 “Here in Phalombe, we believe that if a Girl Child has reached Standard 5 and knows how to read and write, she should be withdrawn from school.” 74 Number of school staff trained on handling diversified food basket or nutrition education by gender 116. Table 20 shows that at the time of the evaluation 80 percent of teachers in 8 out of 10 PAA supported schools were trained in food handling and nutrition. Table 20: Number of school staff trained on nutrition or diversified food basket Indicator Previous reports (May 2016) 2017 assessments Number of school staff trained on handling diversified food basket or nutrition education 53% of school staff (70 people) trained) 80 percent of school staff in 8/10 PAA supported schools trained. 117. Overall, interviews with trained famers, both male and female, indicated that they gained new nutritional knowledge on diversified diets. Table 21 shows that more than half the households in both districts reported improving the diversity of their household’s diet, and most households felt that the PAA SMP improved the health of their children. In turn, some farmers, particularly the women, applied this knowledge to their households: 40.3 percent of farmers who sold food to the schools stated that they produced an increased quantity which they then consumed at home. Sixty-five percent of these farmers were women and 34 percent were men. Table 21: Percentage of households reporting change to their diet Phalombe Mangochi Percentage of households that reported changing their families’ diets as they felt that the diversified diets would improve the health of their children. 77% 66% Percentage of households that reported that the HGSM improved the health of their children. 81% 63% 118. In addition to the above training, the PAA SMP set up school procurement committees to handle procurement of local foodstuffs. It was noted that the Parent Teacher Committees and School Feeding Committees were very involved in funds management and this ensured transparency and prudent funds management. The skills learned though the SMPs can be used by trainees for other non-SMP related activities in future. 2.2.2.4 Outcome 4 119. Outcome 4 related to increased revenue of smallholder farmers from sales to the HGSMP. This requires an understanding of agricultural practices used by farmers, the volumes of food sold by farmers to PAA supported schools, and the price sold. Volume of food commodities produced by targeted FOs 120. Overall, the PAA monitoring reports show improved production of multiple food crops in both districts since the start of the PAA SMP. Figure 1 shows the changes in production of the ten most produced commodities. In Phalombe District crop production increased for maize, rice, sweet potatoes, soya, millet, green vegetables, fruits, and green bananas by 2016. Cassava, groundnut, bean and tomato production declined. Similarly, production in Mangochi District showed increased production in maize rice, cassava, 74 FGD, Parents, Nalingula PAA school, December 7, 2016. 30 sweet potatoes, groundnuts, beans, soya, millet green vegetables, tomatoes, onions fruits and green bananas. Crop production declined by 2016 for pigeon peas. Overall, production in Mangochi District was higher than in Phalombe. Figure 1 shows that overall, production of maize, rice, sweet potatoes and green beans have increased significantly over the PAA period. Figure 1: Volume of the top ten commodities produced by PAA supported FOs Quantity of food purchased by PAA supported schools (produced by FOs and sold to schools) by types and over three years 121. One of the key results of the PAA SMP was that schools would be able to purchase sufficient food from the local FOs to be able to provide regular, diverse school meals. Dietary diversity was expected after schools purchased multiple food items from local farmers for us in the school meals. 122. To evaluate this indicator, the evaluation team relied on project monitoring data for the years 2014, 2015 and 2016. Monitoring findings indicate that in total, 361MT of more than twenty different food commodities have been purchased by schools from local FOs (Annex 20). In 2016 FOs in Phalombe also began to sell meat, cassava and soya indicating that farmers continued to diversify production in order to meet the demand from schools. The largest volumes of commodities purchased were fruits (104 MT, 29% of total purchases) followed by maize (80MT, 22% of total), rice (42MT, 11.6% of total) and green bananas (26.5MT, 7%). Monitoring data does not make it clear whether the amount procured from all sources was sufficient to provide a meal for all students every school day. 123. Figure 2 shows increases in sales from all the Phalombe FOs, while Mangochi FOs have had inconsistent successes. The differences between the two districts could be explained by the fact that the implementation of production support activities started later in Mangochi. 0 10,000 20,000 30,000 40,000 Maize Rice Sweet potatoes Green bananas Green vegetables Beans Groundnuts Fruits Beef Pigeon peas 1 2 3 4 5 6 7 8 9 10 Volume of commodities (kgs) 2014 2015 2016 31 Figure 2: Volume of food commodities procured by PAA schools from local FOs (2014-2016) 124. Analysis of the monitoring data however, reveals a discrepancy between the amount of food procured by schools and the amount of food sold by FOs. Monitoring data indicates the total quantity of food sold by FOs during PAA Phase II is 260 MT, however as mentioned above, the total quantity of food purchased by schools during Phase II is 361 MT. This difference shows that schools have purchased almost a third of their food from local markets or smallholders who are not members of the FOs targeted by PAA. Interviews indicate that this was because farmers were not able to provide sufficient food to schools because of their limited land size, poor production planning and dry spells. 125. Although the project monitoring data does not provide any information on whether farmers increased their income, the increased production reported, and the volume of sales to schools indicate that farmers have almost certainly increased their income. Overall, the value of the food provided to PAA supported schools was about USD 274,000. 126. A completed table of all the PAA indicators at endline can be found in Annex 20. Key findings re effectiveness of the PAA SMP  Schools have successfully purchased from FOs and provided children with diverse school meals. However, it is not clear if meals have been provided every school day for all attending students.  The targeted FOs were not able to provide sufficient food to schools because of the limited size of farmers land and drought. As a result, schools bought additional food from local markets and other local farmers.  The increased production reported by FOs, and the volume of sales to schools, indicate that at least some farmers have almost certainly increased their income. However, more detailed information on this is needed.  The PAA SMP has effectively worked with both male and female farmers to improve their food production.  Training of school personnel, farmers and households on nutrition and dietary diversity has been effective at improving knowledge and has resulted in many households improving the diversification of their household diet.  School gardens have not been used effectively either as a nutrition education tool or to help provide food for school meals.  Enrolment and attendance have improved over the project period, however dropout rates for both boys and girls were higher in PAA schools than in control schools. 0 10 20 30 40 50 60 Nkhulambe Chakalamba Nanguluwe Chibwerera Mvumba Mthilamanja Phalombe District Mangochi District Volume of sales (MT) 2014 2015 2016 32 2.2.3 Additional findings on the effectiveness of both projects 127. Community participation: The evaluation found that parents and teachers from both SMPs were very active, and chiefs also took an active role in community mobilization. In most cases, the chiefs and other local leaders selected men and women from surrounding villages to carry out the various SMP activities including the heaviest task of cooking. In most cases, over time, SMP cooking shifted solely to women. This has meant women leaving their homes as early as three in the morning to feed up to 1,000 children in the primary schools. In a few isolated cases such as in Mulanje District, the McGovern-Dole supported schools hired cooks to motivate them to regularly attend. Sometimes chiefs asked village heads to bring cooked food to the schools, but this was discouraged as the HGSM schools are expected to purchase all foodstuffs from the FOs.75 Despite cooking being relegated to women, the majority of volunteers, particularly in management or decision-making positions in school PTA committees were men while women took the lead in school meals committees, although this varied by district. 128. The work of the cooks was not only the heaviest in terms of workload but in some schools it was performed in difficult circumstances. The evaluation found several schools where water was in short supply and in many cases, volunteers had to bring water from outside school premises76. Structures such as kitchens, stores and shelters were mostly sound and met minimum standards, especially the food stores. However, in some cases, the kitchens were rudimentary. It was noted that generally, hygiene measures were simple but effective. The communities were also well aware of the minimum infrastructure required for the various structures and tried to maintain them. 129. Lack of clarity on the preferred government SMP model: Group interviews with SHNs found that they worked well with WFP and other SMP providers such as Mary’s Meals and GIZ under both centralized and decentralized SMP models. However, there was some confusion as the MoEST had not yet stated publicly which SMP model is preferred so the district SHNs continued to follow the direction of individual donors. In 2016 WFP and GIZ supported the MoEST to carry out a detailed study of all school meals programmes in Malawi.77 The study was validated in 2017 and is currently pending approval. It provides clarity on the models but does not provide one size fits all solution due to different contexts across the country. However, the Government preference seems to be towards HGSM due to other spill over benefits into the community. 130. Lack of support to SHN committees: The main issue identified by SHNs was that neither the McGovern-Dole or PAA SMP model currently factored in monitoring, technical backstopping and training support by the SHNs. SHNs had limited resources to link with local leaders and organize refresher training for schools, so some planned activities could not be delivered. According to the SHNs, although information re ration size and delays are communicated to schools it was difficult to get information and feedback from the national level government and WFP on how and when the situation would be rectified. Schools also often lacked the farming tools to adopt technologies in 75 There are WFP SMP in Jamaica, Namibia and Swaziland which allow for parents and community members to bring in cash and in￾kind donated to enrich learner meals. 76 An example of a school where there was shortage of water was Chenjawazi Primary School, Kasungu District (McGovern-Dole supported). Teachers and learners expressed concern that they walk a kilometre to find drinking water as they do not have a borehole within the school premises. 77 Imani Development, “School Meals in Malawi-Best Practices Guide 33 water conservation, forestry and agriculture practices, so SHNs could not demonstrate techniques and schools could not apply what they were taught. 2.2.4 Summary of key findings on effectiveness  Both the SMPs have effectively provided school meals to children. However, the McGovern-Dole schools did not provide a meal every school day as planned due to insufficient availability of food due to delayed deliveries and for PAA schools, delays with release of funds were due to administrative and reporting issues from the districts to WFP to receive additional funding.  The meals provided by the PAA SMP have included a wide variety of food items however, farmers were not able to provide sufficient food to schools because of the limited size of farmers land and dry spells. Schools have therefore purchased additional food elsewhere.  Communities involved in both SMPs have effectively contributed to the programmes and participated in the management and decision-making required for implementation.  Primary school enrolment rates and attendance rates were high even at baseline and have generally remained high or improved.  The McGovern-Dole schools reported a reduction in school dropout rate, while the PAA schools have not. The McGovern-Dole SMP had additional follow up activities and sensitizations with communities to reduce dropout, while such activities were not factored for PAA SMP.  The PAA project has effectively resulted in linkages between the school meal and production of food by local farmers. Although the full extent of benefits to farming households remain unclear, the programme has effectively enabled increased production, diversification and sales by local FOs. However, a third of all food used by schools was purchased through other means. 2.3 Evaluation Theme 3: Efficiency 131. The evaluation studied efficiency78 of the SMPs under three parameters: technical efficiency79, economic efficiency80 and cost-benefits81. The three parameters were analysed in terms of inputs and their costs in relation to outputs and outcomes and their costs. Factors such as the rate of achievement against plan, the use of resources in relation to what was planned (burning rate82), timing and the flow of resources were all considered. 132. Technical efficiency: In terms of technical efficiency, increased enrolment weighed against the prevailing dropout and absenteeism rates over time contributed to a ‘below a 100% enrolment target” achievement. High dropout rates in the PAA schools and some non-attendance meant planned volumes of food were not achieved due to lesser number of students present. Although there was increased acquisition and demonstration of new techniques and learning by teachers, there was reduced percentage of schools that 78 “Efficiency is a term used to describe the relationship between inputs and outputs. The UNEG definition of efficiency = assessment of the qualitative and quantitative outputs achieved in relation to the inputs and compares alternative approaches to see whether the most efficient approaches were used. 79 Technical efficiency is the maximum output that can be achieved from a particular input of resources with a given level of technology (Psacharopoulos and Woodhall, 1985 p206) 80 Economic efficiency is achieving a desired level of output at minimum cost. This is associated with cost-effectiveness (Psacharopoulos and Woodhall, 1985 p206). 81 Cost-benefit is social rate of return of an investment for society as a whole taking into account generating externalities/spill overs, such as external costs arising from the investment –pollution, congestion and external benefits - crime reduction, social cohesion technological innovations intergenerational benefits etc. (Psacharopoulos and Woodhall, 1985, p53, 117) 82 Burning rate is the rate at which funding are being expended on a project. 34 received stationery and learning materials to enable teachers to put their learning into practice. It was therefore inevitable for overall efficiency to deteriorate. 133. Economic efficiency: In terms of the SMP projects’ activities, the evaluation found that the design of the two projects stimulated efficient use of resources, despite some gaps in the supply pipeline. These gaps include the decision to reduce the McGovern-Dole CSB+ portion from 100g to 60g in order to stretch the availability of CSB+ due to delays in delivery and supply. In addition, although WFP recommended a 25kg CSB+ per 250 students, an average of 416 students used it due to reduction in ration from 100g to 60g. 134. The Imani Development Best Practice Guide on SMP83 estimates that universal provision of school meals in Malawi could be achieved at a financial cost of approximately USD 60– 75 million per year. The Guide estimates that a HGSM model costs MWK 12,000 per child per year, not including overheads. This is slightly more than the centralized provision of CSB which was estimated to cost MWK 9,000-13,000 per child per year. However, the evaluation notes that these costings fail to take into consideration the multiplier effects of the HGSM model. Further analysis also indicates that the cost of the providing one meal a day through the PAA SMP was higher than expected during this period of implementation due to unprecedented high food prices experienced during a large part of the project implementation period. This was the result of four consecutive seasons of food deficit characterized by inflation/high food prices. While this initially tipped the balance against HGSM, as production increased, commodity prices reduced. In addition, at the start of the project the FO capacity to manage the situation was low. Now that FOs have received extensive training, it will be up to them to manage their FOs and maximize their agricultural productivity and market linkages. Also, the menus for PAA meals have gone through significant changes over the years from high cost menus to standard menus that are based on locally available commodities, reducing the overall cost of the programme. 135. Cost benefits: The study of cost benefits of SMP by the evaluation team found that the social return to education investment was high because of the following factors:  Both SMPs were beneficial as communities, parents and government officials noted that there was an improvement among learners and parents regarding benefitting from education by sending their boys and girls to school. Had sufficient literacy materials been provided, this may have also resulted in improvements in literacy.  In the McGovern-Dole supported schools, the retention of girls in schools went up, because of community awareness and to some extent due to commitment to following up of girls.  Active citizenry and improved responsibility by community members towards school development through Mother Groups, school committees, and PTAs. 136. Overall, the evaluation has found that there were gaps in technical efficiency but social returns to education were high. Under the three parameters of efficiency, technical and economic efficiency were reduced because of high rates of dropout and repetition, particularly in the two districts where both SMPs operate: Phalombe and Mangochi. This resulted in inefficiencies of SMP for logistics, storage, food handling, meal preparation and human resource mobilization. However, in the absence of a full cost benefit analysis, 83 Imani Development, “School Meals in Malawi-Best Practices Guide.” Page 20. 35 it is also difficult to compare and draw conclusions about the potential cost of a national SMP model. 2.4 Evaluation Theme 4: Impact and outcomes 2.4.1 School feeding component 137. Dietary diversity in the PAA SMP: The school meals provided under the HGSM model are intended to be diverse, prepared from at least four food groups following simple one-dish menus developed by the schools. This has the added benefit of enhancing knowledge of the health aspects of the food groups among students.84 Observations of the school menus by the evaluation team indicate that in general, a minimum of two food groups (staples and legumes) were included in a single meal. 138. Although still a pilot, the PAA project has successfully stimulated a more diverse food basket in the 10 schools and surrounding households. Some examples of the crops provided in HGSM meal were maize flour, rice, sweet potatoes, beans, pigeon peas, soya beans, green vegetables, tomatoes, onions, fruits, bananas and beef. This diverse provision was due to successful production of traditional and new crops and their sale to the 10 schools. Interviews with students indicated that students like the meals provided at school. However, they do not always eat the same type of food in their homes as their parents cannot afford the food items85. 139. Although parents confirmed that they could not always afford the level of diversification in the family meals, Table 22 shows that most parents felt that the HGSM improved the health of their children. This is supported by the high numbers of households that also reported changing their families’ diets as they felt that a greater diversity of food items would improve the health of their children. Interviews with farmers from the FOs also found that 44.8 percent of farmers felt that the PAA diversified menu improved the health of their children and 39.6 percent of farmers stated that because of the PAA project they were more motivated to diversify what they ate at home. Table 22: Percentage of households from PAA schools reporting change to their diet Phalombe Mangochi Percentage of households that reported changing their families’ diets as they felt that the diversified diets would improve the health of their children. 77% 66% Percentage of households that reported that the HGSM improved the health of their children. 81% 63% “We have not had food for the last two weeks as our budget is finished. We need alternative sources for perishables as the FOs often do not have what we need to cook six types of food in one pot, and their prices change.”86 140. Comparison of the SMPs to other social protection programmes: Global evidence clearly highlights the role of school feeding as a social safety net.87 In the case of Malawi, the SMP has been implemented in poor, food insecure areas, ensuring that children receive at least one meal per day. In addition, the interventions were directed at 84GI Report, Dec 8, 2016. School Feeding Committee; Nalingula Primary School. Phalombe District 85FGD Report, Dec 8, 2016. Learners; Nalingula Primary School. Phalombe District 86 IDI, 08.12.16 with PAA primary school management and procurement team, Phalombe District 87 One example is Bundy, D. Rethinking School Feeding: Social Safety Nets, Child Development, and the Education Sector. WFP/The World Bank, 2009. 36 increased access to education and in the case of PAA, agricultural productivity. The WFP ‘targeting for inclusion’ process included directing support through district level SHN committees who mapped out where the poor communities were situated. As well as being food insecure and poor, the targeted districts had low school enrolment high dropout rates. While the two programs were directed generally at social inclusion, they did not address specific interventions for disabled children and those living with HIV/AIDS. Complementary funding enabled McGovern-Dole funded schools to provide THR for girls and orphaned boys in Grades 5 to 8 in the lean season and this served as an incentive to keep girls and orphans in school rather than working or undertaking family duties during the lean season and other periods of food insecurity.88 All these design features mean that the SMPs serve as an important social safety net during the school year. 2.4.2 Institutional demand/productive support component 141. SMP as a stable market for smallholder famers: The PAA SMP opened up a new market for both male and female smallholder farmers. Sensitization, training and provision of inputs have stimulated agricultural productivity as evidenced earlier. There was increased productivity with many new crops in the years over 2014-2016. The FOs and their farmer members gained new skills, which they applied to crop diversification. For example, the GIs with the Chibwerera and Nanguluwe FOs confirmed success in diversification of crops. This finding is in line with the findings from the Agricultural Productivity Estimate Surveys (APES) from 2013/14 up to 2015/16 which showed diverse types of crops being produced in both Phalombe and Mangochi Districts. The results of APES surveys for Phalombe and Mangochi Districts confirm the evaluation finding that farmers increased their production (Annex 20). The evaluation found no market price monitoring data so it is unclear whether price increases have affected the ability of schools to purchase their requirements. 2.4.3 Cross cutting outcomes89 142. Empowering local institutions: Although school committee members generally rated their training positively, some members stated that they still did not have sufficient capacity to manage the SMP. This was either due to transfers of trained personnel or lack of refresher training. At the same time, local committee members and other community members were doing their best to take the lead on SMP management.  The community-led management structures under the school, such as SMC, SMP committees, Mother Groups and PTAs became more directly involved in SMP planning, implementation and monitoring at the school level. In other words, the community and various committees were initiating action whilst the DEM and other management structures took on a more advisory role;  In some circumstances, community fund-raising was initiated to maintain programmes such as for purchasing soap and other minor items. However, the communities are poor, and to date, community fund raising has only succeeded in funding minor activities and items and not major works such as rehabilitation and capital equipment, which demands government intervention. 88 WFP Washington (2016). McGovern-Dole Proposal. Washington, DC. USA 2007 89 Outcomes are changes from the intervention products, capital goods and services which result from a development intervention; may also include changes resulting from the intervention which are relevant to the achievement of outcomes. 37 2.4.4 What were the major factors influencing the achievement of results? 143. Several internal and external factors affected the implementation of the two SMP projects. 2.4.4.1 Internal factors 144. WFP M&E systems: The WFP M&E system at school level has contributed to the expansion of the databases currently in place at the MoEST including the DEMIS, and EMIS. For example, many of the SMP monitoring tools are designed to collect information and alert the schools to areas they need to address when dropouts increase. District Council Finance Officers involved with the PAA project stressed that more needed to be done to simplify school level budgeting. In addition, they recommended better linkages between PAA schools and district councils to support budget preparation. 2.4.4.2 External factors 145. The school environment: The evaluation identified many issues in the targeted schools that affected the outcomes of the SMPs. These mainly relate to the school environment: overcrowded classrooms, high student/teacher ratios, and teachers lacking accommodation. In addition, cultural factors still result in some households withdrawing their children from school, so their daughters can get married or their sons can support household and farm activities. At school level there were also reported cases of verbal and physical harassment of girls from boys and male teachers. Although these issues were not intended to be addressed by the SMP, they all affect parental decision-making on whether to send children to a particular school. 146. Lack of clear school feeding budgets: Schools were required to make plans and budgets before they received funding and share this with the district council accounts. The plan and budgets were expected to form the basis for the amount of funds received. However, in most cases, the schools said they received less than the amount they had requested. This created some sense of uncertainty at school level. According to the WFP CO, the school budgets help WFP determine how much money to disburse. However due to internal processes and changes of prices of the commodities on the market, the schools are requested to revise their budgets to suit the environment at the time of procurement. If there is no change in market prices, there is no need for budget revision. In addition, budgets are reviewed by district councils and WFP to ensure that schools adhere to menus that use locally available commodities and meet minimum nutritional requirements. 147. WFP reported that schools can request budget revisions based on enrolment – e.g. at the time of submission versus time of actual feeding. Calculations at WFP level are made based on number of children, number of feeding days and cost per child per day. However, many PAA supported schools did not have adequate time or capacity to reconcile program funds which resulted in further delays for the schools to get funding as current procedures do not allow receipt of new funding until previous funds have been reconciled. Interviews with the Chibwerera FO (Mangochi District) and Nanguluwe FO (Phalombe District) stated that when delays occurred, foodstuffs often perished. Two PAA supported schools, Nalingula and Ching’ombe Schools in Phalombe and Mangochi Districts respectively, indicated that they had not received funding for the September to December 2016 Term despite the evaluation taking place in the final week of that term. As a result, no school feeding had occurred. 148. Challenges to the adequate provision of local food: The PAA SMP faced challenges to provide sufficient food to the schools for regular provision of school meals despite being implemented in two of the most productive agricultural districts. Farmers from the targeted FOs did not have the necessary acreage to grow sufficient crops and 38 schools not always have adequate funding to purchase additional food elsewhere. The PAA SMP had contractual mechanisms to formalize agreements between schools and the FOs, or between villages and WFP. However, FOs did not always have what the schools wanted or overcharged them on some items. As a result, although the present design tried to limit the schools’ sources to FOs rather than other traders or local markets, evaluation findings indicate that more than a third of the SMP food was procured from other sources. 149. The evaluation found some friction between the FOs and the farmer clusters and clubs within the FOs. Although 79.1 percent of the farmers stated that they planned to continue selling goods through their FOs, 17.3 percent did not plan to continue. The evaluation also found several barriers for smaller holders including the inability and unwillingness of some farmers to purchase shares in the FOs. This was even the case for farmers who were part of the FOs before FOs became cooperatives. Other factors preventing farmers from accessing the SMP market were vendors manipulating the market and exploiting farmers. The government requirement to charge cooperatives levies, and illiteracy amongst farmers that contributed to resistance of farmers to join cooperatives. The evaluation also found some cases of poor cooperation between FOs and farmer members. These contributed to cases of mistrust between FOs and schools, such as complaints of FOs overpricing items. At the same time, the PAA design did not include modalities for FOs to work with wider markets beyond the schools, other than training and registration as cooperatives. In future, once FOs reach cooperative stage, it is expected that they will be able to access other markets beyond the schools. “We know more about business management and how to find markets. With registration, many farmers have brought shares but there are requirements on number of shares a member has to purchase. Some farmers cannot afford shares. Another challenge is that in this area, some of the HGSM foodstuffs (rice and beans) do not grow, so we have to source them elsewhere, which raises the prices. When the schools pay us late, the food can go bad.”90 150. Community input: To a limited degree the two projects have changed community attitudes towards women’s involvement in development activities. There was a good representation of women in school feeding committees as chairpersons and vice chairpersons. However, in both programmes, the responsibility of cooking was considered to be women’s work and men have generally shunned taking part in this aspect of the program. This has added considerably to the workload of women. 151. Civil society involvement: Three civil society organizations collaborated with WFP to implement the SMPs: AECDM and CRECCOM collaborated with McGovern Dole project and WeEffect collaborated with the PAA project.  AECDM was associated with the McGovern-Dole SMP through its link with the MoGCDSW. It provided caregiver training in the Chikwawa and Nsanje Districts and strengthening district networks. It was agreed that only community-based child care facilities with caregivers trained by AECDM over a five-week period could be part of the program. This ensured that the boys and girls were well fed and cared for. It was agreed that children could only attend the ECDs run by trained caregivers from AECOM training supported by the McGovern-Dole funded project. Each ECD was also expected to have a communal garden with support from the MoAIWD. This involvement worked well, albeit on a small scale and with gaps in delivery of CSB+. 90 GI, 12.12.16, FO, Mangochi District 39  CRECCOM involvement was limited to community mobilization from November 2014 – February 2015 in Salima and Mangochi Districts. These were the two districts with the highest number of recorded violations of girls rights. CRECCOM engaged with Area Development Committees, conducted awareness and sensitization campaigns and managed to get girl dropouts to return to the 10 schools they were working in. They also took a strong role in reduction of harassment of girls at home and in the schools.  We Effect trained FOs, registered them as cooperatives with the Ministry of Trade, and facilitated their linkage with HGSM schools and other bodies such as the Association of Smallholder Farmers in Malawi. 152. Flood response and logistical challenges: An emergency response to flooding affected implementation of the McGovern-Dole funded SMP. Access to schools during this time was challenging, and logistical support usually provided to the SMP was affected by ongoing emergency response, making it difficult to maintain the SMP pipeline. Even in non-emergency periods, Malawi’s poor road network makes it difficult to transport quantities of food requested for the SMP for individual schools are too small to form an economic load. This makes delivery of food items expensive compared to other programs like emergency responses91 . 2.5 Evaluation Theme 5: Sustainability 153. Progress towards a nationally owned SMP: WFP and the MoEST have a long partnership in Malawi with WFP being the largest SMP partner with wide geographical coverage. WFP has provided ongoing policy development support to the government, which has resulted in clearer policy direction for ministries, school and households to follow. WFP’s overall SMP is gradually shifting from a centralized model to a decentralized one in line with government preference. However, the MoEST says that although there have been discussions on how to hand over the SMP to them, the modalities have not yet been formally agreed upon. As a result, a clear SMP handover strategy has not been formalized. Similarly, financial constraints in the last two years have constrained the handover of schools to the government as planned. Previously, the government was able to allocate a percentage of its education budget to support handover, but limited resources in recent years have made this impossible92. Stable and adequate government funding specifically allocated for education (including infrastructure and adequate numbers of teachers) as well as support to the district SHN committees is needed. A full cost benefit analysis would also help inform the direction of the programme. 154. Interviews with key stakeholders also indicate a gap in government technical expertise to enable district level monitoring and evaluation data to be captured in the EMIS and DEMIS. Similarly, there is currently no avenue to link education data with other sectoral data such as agriculture and health.93 Each of these issues reduce the possibility of programme sustainability without additional external support. 155. Greater involvement from other government ministries: The MoEST is the lead government partner on SMP. Other than involvement of the MoAIWD and the Ministry of Health for the deworming programme, the evaluation found there was insufficient 91KII Report, November 29, 2016, WFP Logistics. Lilongwe 92 The government faced economic challenges due to the humanitarian response required after the floods and drought and changing priorities to address humanitarian needs 93 Group interview (GI), Ministry of Education, Science and Technology, Maseru, December 1st, 2016. 40 involvement and coordination with other ministries. For example, the work of the Ministry of Finance, Economic Planning and Development’s, Malawi National Social Support Programme (MNSSP) is directed towards school meals programmes, but also have components that lean more towards improvement of infrastructure and income generation.94 Better integration of the Early Childhood Strategy through the MoGCDSW to the early grades of targeted primary schools will do much to address performance. More participation from the MoGCDSW will also do much to bring out interventions for the disabled and boys and girls affected by HIV and AIDS. Finally, although the role of the Ministry of Civic Education, Culture and Community Development (MoCECCD) was considerable in Phase I of the McGovern-Dole funded SMP, their involvement reduced during Phase II. This ministry is well placed to support community sensitization and is linked to community groups carrying out cooking demonstrations and tree planting. Involvement from additional government ministries would help improve SMP sustainability. 156. Collaboration between stakeholders: The evaluation established that there was good collaboration between WFP and stakeholders implementing both SMPs in the districts and at local level. In most cases, the community was ready and willing to work on a voluntary basis, assist financially where warranted and take active roles in the management of the activities. Most important was that those volunteering felt appreciated by their communities, which motivated them further and improves the chances of sustainability. 157. Quality of the schools and ECDs: Neither of the two SMPs were expected to address the gaps that are present in schools such as insufficient classrooms, qualified teachers, latrines and access to clean and safe water. It is also clear that SMPs alone cannot address the issue of quality of education. The evaluation found that few children learned in structured classes – with most classes had 100 students per teacher and there was one case where there were 185 students. These factors compromise the quality of education provided in SMP targeted schools. The quality of the new SMP infrastructure of both projects was good but could be improved to last longer. An example is the McGovern-Dole funded project start-up SMP structures (shelter, shed and kitchen). The structures and technologies (to manage gardens, orchards and woodlots) were found to be simple and practical, which people could easily accept. However, there needs to be alternatives to rocket stoves, and alternative fuel sources. Similarly, triangulation of the survey results with FGDs and IDIs confirm gaps in terms of government service delivery to McGovern￾Dole, PAA and control schools95. Provision of safe drinking water remains an issue, with none of the surveyed schools (cases or controls) having full access to safe drinking water (Annex 21). Similarly, not all schools have separate functioning latrines for boys and girls. Separate latrines are one of the factors demotivating girls from attending school. Additional data on school infrastructure can be found in Annex 21. Evaluation findings also indicate gaps in facilities for children with special needs despite PAA schools having 94 KII, 02.06.17, Ministry of Finance Economic Planning and Development, Poverty reduction and social protection. The MNSSP (Now in its second phase) has five key pillars- Public works, School meals, Cash transfer, Village savings and loans and Microfinance. 95 FGD, Teachers, McGovern-Dole Nankhunda Primary Schools, Chiradzulu District, 09.12.16: Teacher stated that they lacked capacity to handle physically challenged children, due to shortage of special needs teachers and teachers in general. There was also a lack of teachers’ houses and late payment of salaries. FGD, Parents, PAA Nalingula Primary School, Phalombe District, 08.12.16: parents stated that classrooms were in poor shape, with many lacing windows and few desks, the latter forcing children to stand for long periods. An IDI at the same school on 08.12.16 with teachers: They started a school garden, but it quickly dried up as there was lack of water. They rely on a borehole shared with the community. FGD, Teachers, Control school, St. Augustine primary school, Mangochi Districts 12,12.16: Teachers stated there are lack of classrooms and teachers 41 a high proportion of disabled children in the school (30 per school compared to 14 per school in McGovern-Dole and control schools (Annex 21). 158. Sustainability of capacity building of farmers: The PAA SMP integrated school meals, institutional procurement and agricultural support to smallholder farmers in a single intervention. However, PAA supported farmers were expected to reinvest some of their earnings into the surrounding communities to help sustainability but did not happen. The marketing capacity of the PAA-supported FOs and farmers is also not yet well developed enough for them to participate in institutional and non-institutional markets beyond the schools. The PAA supported schools therefore remain their main clients. If funding is withdrawn from the PAA SMP program, it is unlikely that the FOs will be able to continue working with the schools. 159. Overall, the evaluation finds that some of the socio-economic benefits of the PAA program may not be sustainable without continued donor or Government funding. If funding from the donors ends, it is not clear where the funds will come from and the likelihood of continued socio-economic benefits is low. 2.6 Evaluation Theme 6: Governance96 160. Extent to which the programmes have addressed lessons learned from monitoring and evaluation: The midterm evaluation of the McGovern-Dole SMP and the monitoring reports of the PAA SMP both found that additional government input and capacity building is required. In addition, a common, reliable database was important to document and analyse information, but this was not available. Furthermore, when information is collected, dissemination to ministries and district SHN committees is inconsistent. This meant there was a risk of duplication of interventions or interventions not reaching project targets on time. 161. Government of Malawi involvement: The evaluation finds that the Government of Malawi through the MoEST is best placed to lead SMP but with additional support and input from other ministries. The government is best placed to set policy and identify strategic interventions to address the link of education to nutrition and food security, and the new SMP work plan spells out national and district roles of the ministry and other key ministries. According to the SHN committees, the legal frameworks including clearer policies and statement district level strategies, contributed to better SMP implementation. WFP supported the development of the SHN policy and remains best placed to support the MoEST’s expansion of national level policy statements and strategy. Greater linkages between government ministries would be beneficial. 162. The WFP and FAO partnership strategy for HGSM: The collaboration between WFP and FAO with respect to the PAA project has shown to be successful in that skills of FOs and school procurement committees have increased. Farmers learned how to grow a wide variety of crops and in many cases fed a more diversified diet to their families. At the same time, due to small landholdings and lack of management experience, the FOs were still not able to produce enough to meet the needs of the PAA schools. FOs and farmer members could also not afford irrigation measures97. 96 OECD, “Glossary of Statistical terms” defines “Governance” as the exercise of political, economic and administrative authority to manage a nation’s affairs (in this case national school meals program). It is the process within government by which decisions are made. 97 GI, December 12, 2016, SHN Coordinator and Committee, Mangochi District. 42 163. Gender: The McGovern-Dole SMP included specific activities with CRECCOM to bring awareness and other measures on girls education. This has been alluded to in other sections. In addition, sensitization activities in the communities are ongoing led by SHN coordinators. For example, in some PAA schools it was reported that girls faced sexual harassment from boys and teachers, and poverty has meant girls taking up prostitution with men in lodges or pushed into early marriage. Although outside the current scope of the SMPs, these issues are highlighted within the Malawi National Girls Education Strategy and will require ongoing work. 3 Conclusions and recommendations 3.1 Overall assessment/conclusions 164. Relevance: The evaluation finds that both SMPs were relevant for the Malawi context and for addressing the needs of the targeted beneficiaries: boys, girls, school personnel and farmers. The SMPs are both coherent with government, WFP and other development partners and the WFP and FAO collaboration for the implementation of the PAA SMP was highly appropriate since HGSM is the preferred model of the government. The McGovern-Dole model is implemented on a much larger scale than the PAA SMP, targeting numerous districts and covering primary and pre-primary schools, and these are both elements that the government would like to see replicated. Capacity building for the government and involvement of the community in SMP implementation were both highly relevant activities employed by both SMPs. WFP capacity building for the MoEST at national level resulted in the release of the SHN policy and paved the way for an upcoming strategy on SMP. 165. Effectiveness: The McGovern-Dole SMP has been effective at improving the capacity of the MoEST at national, district and school levels. The SMP also included effective community awareness campaigns that have contributed to ongoing student enrolment and attendance and reduced dropout. However it was not possible for the evaluation to assess whether the first SO – improved literacy – had been met. Programme monitoring has not been sufficient in this regard and the expected complementary support from MoEST for the provision of stationery and supplies has not occurred. The other targets that were not achieved were due to unrealistic targets such as reducing health-related absence to zero or reducing inattention in class to <10 percent. The evaluation found that households were more food secure at endline, but it is not possible to attribute the SMP to that finding as the overall food security situation in Malawi had improved since baseline (Maps 3 & 4). The McGovern-Dole SMP has also been effective at improving school infrastructure, particularly kitchens, although the handwashing facilities present at baseline (with soap) were not adequately maintained over the course of the programme. 166. The PAA SMP has been effective at improving market linkages between local farmers and schools. Schools have successfully purchased from FOs and provided children with diverse school meals. However, farmers were not able to provide sufficient food to schools because of the limited size of farmers land. This resulted in schools buying additional food from local markets and farmers who were not members of the targeted FOs. The PAA SMP has effectively worked with both male and female farmers to improve their food production. This increased production, and the volume of sales to schools, indicate that farmers have almost certainly increased their income. However, more detailed information on this is needed. Training of school personnel, farmers and households on nutrition and dietary diversity has been effective at improving knowledge and has 43 resulted in many households improving the diversification of their household diet. The evaluation team felt that Phase II was a missed opportunity to increase PAA coverage and iron out implementation issues such as the FOs being unable to supply the required volume of food items by officially allowing schools to purchase from other sources. 167. Despite the above successes, the PAA SMP has seen inconsistent results in improving school participation. In Phalombe District, school attendance increased from 78.9 percent in 2014 to 99.5 percent for boys and 100 percent for girls by 2017. Likewise, dropout rates decreased. However the opposite result was found in Mangochi District. The DEMs attributed this to the lower community value of education in some communities. The project has also been ineffective at incorporating school gardens into the programme. Of the ten schools that had school gardens, only one was well maintained or productive. 168. Efficiency: The approaches of both SMP projects was efficient in terms of cost benefit as the social return to education investment was high. All supported schools benefited by sending their children to school, while girl retention rose due to community awareness. There was also a more active citizenry and improved sense of responsibility of community members towards school development. In terms of cost per child, the HGSM has been estimated98 to be more expensive than the direct provision of CSB+, especially the imported CSB+. However, the HGSM model provides additional benefits, including a more diversified meal to children and multiplier effects to the local economy which have not yet been factored into the costing. 169. Impact: There is strong evidence that both projects’ activities created positive impact on the intended beneficiaries. These included empowerment of women, increased community awareness of the value of education and in the case of the PAA SMP, improvements in dietary diversification (improving nutrition) during schools meals and at household level. However, there are still some factors that need to be addressed, such as inequity in volunteerism, placing a heavy burden on the workload of women. There was also an impact on girls when the THR was suspended in some schools and some missed opportunities to support more pre-schools. At national level, both project impacted positively on the capacity of the MoEST. The Ministry developed a SHN policy with support of the McGovern-Dole funded project and a stronger evidence base on HGSM due to the PAA project. 170. Sustainability: The Government has identified HGSM as the best model for sustainability as it addresses both education outcomes and stimulates agricultural production. However, the evaluation found that without clear statements on this, and a clear SMP strategy, along with stable and adequate government funding, the SMP cannot be sustained in the long term regardless of the model employed. A cost benefit analysis would also help inform the direction of the programme. 171. Governance: By supporting policy development, training and inputs to the MoEST at national level, the McGovern-Dole project has supported the ministry’s ability to exert administrative authority regarding school meals. This support has enriched the scope of school meals provision, school training and infrastructure in the schools supported. However, the financial support to SHN committees was not enough for them to carry out expected monitoring and evaluation activities in schools. This has meant less 98 Imani Development, “School Meals in Malawi-Best Practices Guide.” 44 development of orchards, woodlots and school gardens than expected. The McGovern￾Dole support to school data collection supported the DEMIS databases, but district government level data entry has not kept up with data collection. The PAA project has also supported national governance by providing evidence-based experience towards a sustainable national HGSM model. This support can be applied towards design and management of SMP in the future. Finally the MoEST and MoGCDSW need to address the quality of schools and ECDs in terms of classrooms, latrines, skilled teachers/caregivers and access to safe water to be able to provide a more conducive learning environment to students to complement the role of the school meals. 3.2 Lessons learned and best practices 172. Community involvement is key factor in improving community ownership and sustainability: Capacity building, sensitization and mobilization of local communities to support the implementation of the two SMPs were major reasons for their success. Community awareness on the importance of education was also helpful for encouraging parents to enrol their children in school. In areas of high dropout, such as Mangochi District, greater emphasis on follow-up is needed to ascertain reasons and determine if the community awareness messaging is correct. 173. A HGSM model can provide wider benefits that increased participation in education: The PAA approach stimulated agricultural production and presumably increased the income of local farmers, although there is no documented evidence. The presence of a stable demand market from the SMP resulted in greater participation of farmers, and more diversification of crops and household diets. This meant greater diversification of school meals, potentially improving child nutrition over time. 174. The opening up of school meals as a stable demand market meant that normal market forces including supply and demand came into play. Unfortunately even though the project was implemented in the most agriculturally productive districts, the local farmers were unable to meet the ongoing demand of the schools. This meant that despite the project design, schools purchased additional food elsewhere. Changes to the model therefore need to be introduced to further enhance local production capacity and/or to enable schools to purchase from multiple local sources. This would increase competition and potentially reduce commodity prices. 3.3 Recommendations 175. The evaluation makes the following recommendations for improving the school meals programme in Malawi. They are listed in priority order. Priority 1: To be initiated within 6 months Strategic recommendations Recommendation 1: WFP and other education partners should continue to support the MoEST to articulate a clear School Meals Strategic Plan that can be used as the basis for a nationalization strategy. The Strategic Plan should include an implementation plan and be accompanied by a designated government school meals budget line.  The formalization of a School Meals Strategic Plan remains critical for the Government of Malawi and for SMP implementers and donors as it will guide the process of nationalization of school meals and identify technical backstopping. The evaluation recommends that the School Meals Strategic Plan be developed in consultation with other partner ministries particularly those with existing linkages to education and/or school 45 meals, including the MoAIWD, the MoGCDSW, Ministry of Finance, the MoCECCD and others as appropriate. The School Meals Strategic Plan should then include specific responsibilities for each actor.  It is also recommended that as part of the development of the Strategic Plan, the social protection benefits of the SMP be strengthened and clearly articulated. This may help formation of linkages with relevant ministries such as the MoGCDSW and others as appropriate. Recommendation 2: WFP should continue to support the government to implement HGSM models.  The government has specified that a decentralized or HGSM approach is the preferred model for school feeding. However, this preference is yet to be formalized. Evaluation interviews with MoEST representatives indicate that the government is looking for a large scale SMP that incorporates both primary school and pre-schools (as per McGovern-Dole model) and that uses locally procured food commodities (as per PAA model).  It is recommended that during the development of the School Meals Strategic Plan, that stakeholders consider alternative HGSM models. Currently, there are multiple SMPs in Malawi implemented by multiple agencies and donors and it is not yet clear which one is most effective and which aspects of each should be replicated. All SMP models therefore need to be considered so that future implementation is clear. Note that a future national SMP model may require different SMP models to be implemented in different locations due to differences in productivity.  It is also recommended that WFP and partners undertake a full cost benefit analysis to determine the cost efficiency of centralized and decentralized SMP models, and therefore the potential cost of a future national HGSM programme.  The PAA SMP was implemented in two of the highest agricultural production areas, and yet local FOs were unable to provide sufficient food. There therefore needs to be more allowance for schools using a HGSM model to utilize multiple local markets so that they can access a greater range and volume of commodities and pay a reasonable price. This would also enable a more competitive approach to markets, which is good for both the market and for project sustainability. Another option would be to consider a hybrid model that allows HGSM in the more productive areas and/or where market linkages are already established. During emergencies and/or in areas where market linkages are limited, or production is low, then a centralized approach may be needed. Operational recommendations Recommendation 3: WFP Malawi should address the causes of delays in their pipeline – both for delivery of CSB+ and for the release of monies to schools for the HGSMP to ensure timely procurement and provision of school meals.  The government relies heavily on operational support from WFP so WFP has a responsibility to quickly develop solutions to implementation problems. Measures to rectify the situation should be inclusive of feasible action plans that result in commodities and funding being delivered to schools in a timely manner so that school meals can start from the first day of each school term. Priority 2: To be initiated within 1 year The evaluation recommends that WFP, the MoEST and other education partners consider the following when determining the most appropriate national SMP model. 46 Strategic recommendations Recommendation 4: WFP and the MoEST should strengthen the implementation of the Malawi National Girls Education Strategy and develop partnerships to implement complementary programming that make the school environment more appropriate for girls and OVCs.  The evaluation found multiple examples of harassment of girls in primary schools, as well as high rates of girl dropout in some locations. It is therefore recommended that WFP establish partnerships with other agencies to strengthen the implementation of the Malawi Girls Education Communication Strategy. This will enable SMP projects to impact better on girls’ participation in education (enrolment, attendance, retention, completion, and their transition from primary to secondary schools).  Actions towards this recommendation will require mapping of institutions and organizations working to include girls and women in social protection programming. It may also require partnerships for specific activities such as sensitization of teachers, construction of separate latrine blocks for girls and the provision of sanitary supplies at primary schools. At the same time, WFP should lobby the MoEST to engage more female teachers into schools and ECDs. Additional measures are also needed to address the safety of women volunteers travelling at night to reach targeted schools and means found to reduce heavy workload of women cooking meals. Recommendation 5: WFP and other education actors should support the MoEST to implement the National Strategy for Inclusive Education.  The National Strategy for Inclusive Education includes social protection practices that can address the needs of disabled children, and orphans attending primary schools and early childhood care and development centers. The evaluation has found a gap in the provision of pre-primary education and education for children with special needs. It is therefore recommended that WFP advocate for the expansion of more ECDs building on the support provided to them in the past such as food, equipment and training of caregivers.  There needs to be more targeted identification, mapping and assessment of pre-primary and primary school learners with special needs with respect to teaching methodologies and support they may need during school feeding. Organizations with experience in teaching methodologies and supplying aids (for movement, eating) should be identified and their participation factored into future interventions. At the same time there needs to be lobbying for an acceleration of government recruitment of special needs teachers and government financing of inclusive education. Operational recommendations Recommendation 6: The MoEST with support from WFP and other education partners including UNICEF and UNESCO should continue to address gaps in school infrastructure, teacher numbers and access to clean water.  Although both SMPs have contributed to increased participation in education, and the McGovern-Dole SMP has improved school infrastructure, challenges to the provision of quality education in Malawi still remain. Some of the challenges identified during this evaluation include high teacher to student ratios, inadequate teaching materials and stationery in schools, and insufficient access to clean water. Where possible, it is recommended that WFP leverage their relationships with other agencies to establish complementary projects to help address the above challenges. 47 Recommendation 7: WFP and MoEST should provide additional support to the monitoring, technical backstopping and training provided by the SHNs.  The evaluation has found that it is necessary to improve the management and coordination of the district School Health and Nutrition Committees so that their roles are well defined and able to be implemented. Currently SHNs have limited resources to link with local leaders and organize refresher training for schools so some planned activities could not be delivered. The SHN Committees carry out their roles through District Councils, so both will need to be strengthened through capacity building, so that they can support school feeding. The recommended roles and responsibilities of other key partners in this regard can be found in Annex 22. Recommendation 8: WFP and MoEST should continue to provide community awareness and education on the importance of education.  The evaluation has found that the community awareness campaigns implemented under McGovern-Dole SMP have contributed to the increased enrolment and attendance rates in those supported schools as well as decreased dropout. However, in some PAA locations dropout rates have increased. It would therefore be useful to carry out activities to follow up school dropouts and ascertain the reasons for dropout. This would help ensure that the community awareness messaging is appropriate. Recommendation 9: WFP and the MoEST should develop some common SMP guidance to ensure that each SMP include some key outputs and outcomes, regardless of implementer or donor.  In including two SMPs into a single evaluation it was notable that aside from participation in education, the two SMPs had few overlapping outputs and outcomes. The evaluation therefore recommends that perhaps as part of the Strategic Plan, the MoEST consider developing some minimum standards and/or outputs and outcomes that all SMP implementers should follow. This would also help harmonize the monitoring and data collection of school meals. Recommendation 10: MoEST with support from WFP and FAO should increase the use of the school gardens as a nutrition education tool and/or as an opportunity to provide fruits and vegetables for a more diverse school meal.  The evaluation notes that although the PAA objective of establishing school gardens was largely achieved, only one of the gardens was productive and used. School gardens can be a useful education tool for multiple subjects including agriculture, horticulture, and nutrition but schools need to actively engage with the garden and include related activities into their curriculum. 48 4 Bibliography A. Munthali. March 2016. Mid-Term Report-The McGovern-Dole International Food for Education and Child Nutrition in Malawi. WFP, Malawi, Centre for Social Research. Baxter, P. Undated. Is school feeding a distraction? Bundy, D. Rethinking School Feeding: Social Safety Nets, Child Development, and the Education Sector. WFP/The World Bank, 2009. CRECCOM. Undated. CRESSOM’S Capacity Statement. Malawi CRECCOM. Undated. Self-Sponsored School Feeding Program-Chankharamu School Community. Mwazisi Zone, T/A Chikulamayembe in Rumphi. Malawi Downen, J., L. Richardson, J. Chima, M. Vallet & M. Mueller. 2014. A mid-term evaluation of WFP’s operations from January 2011 to March 2014. Lilongwe: WFP Country Office. Drake, L., Woolnough, A., Burbano, C. & Bundy, D. Undated. Global School Feeding Sourcebook; Lessons from 14 Countries. Imperial College Press. http://www.fao.org/3/a-i4491e.pdf http://www.worldbank.org/en/country/malawi/overview https://www.epdc.org/sites/default/files/documents/EPDC%20NEP_Malawi.pdf Imani Development. School Meals in Malawi-Best Practices Guide. International Policy Centre for Inclusive Growth. 2016. PAA Africa Malawi-Phase II Midterm Monitoring Report Draft. Malawi, International Policy Centre for Inclusive Growth, UNDP, PAA Africa. Kadzamira, E. & P. Rose. 2001. Educational Policy Choice and Policy Practice in Malawi: Dilemmas and Disjunctures. IDS Working Paper 124. Brighton: University of Sussex Institute of Development Studies. Kadzamira, E., K. Nthara & F. Kholowa. 2004. Financing Primary Education for All: Malawi. IDS, University of Sussex. Kholowa, F. & O. Nkhoma. 2015. Baseline survey report – McGovern-Dole supported school feeding program in Malawi. Lilongwe: World Food Program. Ministry of Education and Vocational Training, Ministry of Health, National Statistical Office, author. National school health and nutrition baseline survey. Zomba: Malawi Government; 2006. Ministry of Education, Science and Technology. 2009-18. National School Health and Nutrition Strategic Plan: School health and nutrition response in the education sector. Lilongwe 3 Ministry of Education, Science and Technology. 2016 Education Statistical Bulletin (Annex 12, Table 3). Ministry of Education, Science and Technology. 2016. Towards an inclusive education system: National strategy on inclusive education. Malawi Ministry of Education, Science and Technology. May 2012. Malawi Girls Education Communication Strategy. Lilongwe 3. 49 Ministry of Finance and Development Planning. 2010. Malawi Growth and Development Strategy II 2011-2016. Ministry of Gender, Children, Disability & Social Welfare. (Undated). Integrated Early Childhood Development(IECD) in Malawi. Malawi Ministry of Gender, Children, Disability and Social Welfare. 2006. National Policy on Early childhood Development. Lilongwe, Malawi: Ministry of Gender, Children, Disability and Social Welfare. MoEST. 2014. Education statistics 2014. Lilongwe: MoEST. MoEST. 2015. Education statistics 2015. Lilongwe: MoEST. MoEST. 2008. National Education Sector Plan 2008-2017. 2008. Lilongwe: Ministry of Education, Science and Technology. Munthali, A.C. & Ali, S. 2000. Adaptive strategies and coping mechanisms: the effect of HIV/AIDS on the informal social security system for OVC. Lilongwe: National Economic Council. PAA Africa Malawi. Phase II Midterm Monitoring Report. May 2016 PAA Africa Phase II. Final Monitoring and Narrative Report. December 2016. PAA Africa, Food and Agriculture Organization of United Nations (FAO) & World Food Programme (WFP). Purchase from Africans for Africa PAA Africa Programme Phase II: Malawi, Promoting Local Food Purchases for food assistance in the African Continent. Malawi, PAA Africa PAA Programme Phase II: Country project, Malawi; Comprehensive Africa Agricultural Development Programme. Progress Report of the Purchase from Africans for Africa Project, Malawi, 2013-2015 Psacharopoulos, G. and Woodhall, M. 1985. Education for Development: An analysis of investment choices. A World Bank Publication. Oxford University Press. p206. Republic of Malawi. Malawi Girls’ Education and Communication Strategy. 2012 Republic of Malawi. National School Health and Nutrition Policy. 2013 Republic of Malawi. National Strategic Plan for ECCD (2009-2014) Republic of Malawi. National Education Sector Plan 2008-2017. Ministry of Education, Science and Technology. Malawi. 2008 Republic of Malawi. National Girls’ Education Strategy. 2014. Ministry of Education, Science and Technology; UNICEF, WFP, UNFPA. Joint UN proposal: Improving access and quality of education for girls in Malawi. 2013. Republic of Malawi. Welfare and Monitoring Survey 2014 and National School Health and Nutrition Strategic Plan (2009-2018). Republic of Malawi. Welfare Monitoring Survey. 2014 Riddel, A. 2003. The introduction of free primary education in sub-Saharan Africa. Paris: UNESCO. The World Bank. 2015. Country Overview, Malawi. http://www.worldbank.org/en/country/Malawi 50 UNDAF. United Nations Development Assistance Framework (UNDAF) Action Plan 2012-2016. UNDP in Malawi. 2015. About Malawi; Country Information. http://www.mw.undp.org/content/malawi/en/home/countryinfo.html UNICEF. 2010. The State of the World’s Children: Celebrating 20 years of the convention on the rights of the child. UNICEF United Nations Educational Scientific and Cultural Organization. 2010. World data on education. VII Ed. 2010/11, IBE/2010/CP/WDE/MW.UNESCO-IBE (http://www.ibe.unesco.org) United Nations Evaluation Group. UNEG Ethical Guidelines for Evaluation. March 2008 United States Department of Agriculture, Foreign Agricultural Service. 07/15/2014. Food for progress and McGovern-Dole Indicators and Definitions, Food Assistance Division, Office of Capacity Building and Development. USAID. Malawi National Early Grade Reading Assessment Survey. Final Assessment. November 2012. USAID. Midline Report. Impact Evaluation of the Early Grade Reading Activity. Malawi. 2015 WFP (2009) Gender policy: promoting gender equality and the empowerment of women in addressing food and nutrition challenges. WFP/EB.1/2009/5-A/Rev.1 WFP (2013) WFP Updated School Feeding Policy https://www.wfp.org/content/school￾feeding-policy WFP Washington (2016). McGovern-Dole Proposal. Washington, DC. USA 2007 WFP. 16 December 2011. Country Programme Malawi 200287 (2012-2016). Distribution General WFP/EB.1/2012/7/1. Malawi, WFP. WFP. 2015. Baseline Survey Report-McGovern-Dole Supported School Feeding Programme in Malawi. Malawi, WFP. WFP. 2015a. World Food Programme support to education in Malawi. Lilongwe: WFP. WFP. 2015b. McGovern-Dole semi-annual report: narrative progress report. Lilongwe: WFP. WFP. April 2016. Fighting Hunger Worldwide, Purchase for progress. Malawi. WFP. June 2016. World Food Programme Support to education in Malawi. WFP. World Bank & World Food Programme. 2013. Scaling up School Feeding. Washington, World Bank.