This publication was produced at the request of the United States Department of Agriculture. It was prepared independently by Jean Christophe Fotso, PhD; Mouftaou Amadou Sanni, PhD; Paul Kouete Jimmy, PhD Candidate; Ashley Ambrose, MPH Foreign Agricultural Service, United States Department of Agriculture Benin Local and Regional Food Aid Procurement Project Final Evaluation November 2019 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Evaluation of Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Project Endline Evaluation and Comparison Study Report The Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) project, funded by the United States Department of Agriculture (USDA) and implemented by Catholic Relief Services (CRS) Benin, supports the establishment and functioning of school canteens in government schools of N’dali, Perere, Parakou and Tchaourou districts in the Borgou Region. The purpose of the project is to develop a replicable and scalable model for decentralized school feeding through effective local procurement and distribution of commodities with the aim of providing students with nutritious and locally procured meals (rice, shea, cowpea and maize) from Producer Associations. CRS and its sub-recipient Union Régionale des Producteurs Borgou-Alibori (URP) worked with school management committees(referred to in French as Comité de Gestion des Ecoles(COGES)), commune authorities, central government agencies and private partners to achieve project goals. The specific goals of the Bèsèn Diannou LRP project were to: • Increase the capacity of schools, school management committees, and the Government of Benin to effectively and efficiently procure local commodities to supply school feeding programs, promoting the sustainability of school feeding; • Strengthen farmer groups' ability to provide high-quality, nutritious commodities and connect them to school feeding programs; and Improve nutrition of students by increasing access to and use of various, high-quality, nutritious, and culturally appropriate foods in school meals. Agreement Number: Benin LRP-680-2017-034-00 Project Duration: 2017-2020 Implemented by: Catholic Relieve Services Evaluation Authored by: EVIHDAF 3 Evaluation of Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Project Endline Evaluation and Comparison Study Report Prepared for: Catholic Relief Services (CRS) Benin Cotonou, Benin Prepared by: Jean Christophe Fotso, PhD; Mouftaou Amadou Sanni, PhD; Paul Kouete Jimmy, PhD Candidate; Ashley Ambrose, MPH EVIHDAF, BP 35328 Yaoundé, Cameroon www.evihdaf.com October 28, 2019 i Evaluation Team This study was conducted by EVIHDAF (Evidence for Sustainable Human Development Systems in Africa). EVIHDAF’s core evaluation team consists of Dr. Jean Christophe Fotso, EVIHDAF Executive Manager; Dr. Mouftaou Amadou Sanni, former Director of the Ecole Nationale de Statistique de Planification et de Démographie (ENSPD) at the University of Parakou; Mr. Paul Kouete Jimmy, University of Parakou, Benin; and Ms. Ashley Ambrose, Global Health Research Officer. Acknowledgements With Dr. Mouftaou Amadou Sanni’s leadership, the l'Observatoire Démographique et Statistique des Populations Locales (ODeSPoL) at the Ecole Nationale de la Statistique, de la Planification et de la Démographie (ENSPD), University of Parakou, Benin served as the field partner for data collection. We would like to acknowledge the unique role played by MM. Djima Baranon and Elihou Adje of ODeSPoL, and Ms. Nadia Tefouet from EVIHDAF HQ in Yaounde, Cameroon, for their immense contribution during the inception phase, data collection, data analysis and report writing. We also thank M. Fred Akoha who programmed the tablets used for data collection, and Dr. Nicaise Laleye of University of Parakou for his contribution. Finally, data collection was made possible by 58 enumerators and seven supervisors for the quantitative component, and nine qualitative researchers (See Annex 5). At Catholic Relief Services (CRS) Benin, our grateful thanks are due to Mr. Thierry Yabi, MEAL Coordinator for Benin-Togo, and Mr. Brice Denakpo, MEAL/LRP, for their valuable support and guidance before, during and after data collection. Our special thanks to Ms. Blandine Ekpodile, the LRP Project Director, for her support throughout the study. Most importantly, EVIHDAF would like to acknowledge the hospitality and enthusiasm displayed by individuals who were involved in the study. Participation in questionnaires, focus groups and interviews was demanding of their time, yet it was willingly given. All evaluation team member declare they have “no conflict of interest”. Disclaimer: The views expressed in this document do not necessarily reflect the views of the United States Department of Agriculture (USDA) or the United States Government. i Contents Evaluation Team .............................................................................................................................. i Acknowledgements.......................................................................................................................... i List of Tables ...................................................................................................................................iv List of Abbreviations.......................................................................................................................vi Executive Summary........................................................................................................................vii 1. Introduction .............................................................................................................................. 10 1.1. Project Theory of Change................................................................................................... 11 1.2. Results Framework............................................................................................................. 12 2. Purpose of the Evaluation......................................................................................................... 14 2.1. Impact Evaluation............................................................................................................... 14 2.2. Comparison Study .............................................................................................................. 16 2.3. Use of the Evaluation’s Findings ........................................................................................ 16 3. Methodology............................................................................................................................. 17 3.1. Impact Evaluation: Quantitative Component .................................................................... 17 3.1.1. Design and key indicators............................................................................................ 17 3.1.2. Sampling strategy ........................................................................................................ 18 3.2. Impact Evaluation: Qualitative Component....................................................................... 20 3.3. Direct Observation and Infrastructure Inventory Component.......................................... 21 3.4 Comparison Study ............................................................................................................... 22 3.5. Data Collection and Quality Assurance.............................................................................. 24 3.5.1. Data collection tools.................................................................................................... 24 3.5.2. Data collection team.................................................................................................... 25 3.5.3. Data collection challenges and solutions.................................................................... 25 3.6. Data Processing and Analysis............................................................................................. 26 3.6.1. Sampling weights......................................................................................................... 26 3.6.2. Impact evaluation estimates....................................................................................... 28 3.6.3. Comparison study estimates....................................................................................... 29 3.7. Ethical Considerations........................................................................................................ 29 3.8. Timelines ............................................................................................................................ 30 4. Results: Impact Evaluation........................................................................................................ 30 4.1. Producer Associations........................................................................................................ 31 4.1.1. Sample characteristics................................................................................................. 31 ii 4.1.2. Organizational capacity ............................................................................................... 33 4.1.3 Use of improved production techniques...................................................................... 35 4.1.4. Use of URP MIS............................................................................................................ 37 4.1.5. Relevance, efficiency and sustainability...................................................................... 38 4.2. COGES................................................................................................................................. 41 4.2.1. Sample characteristics................................................................................................. 41 4.2.2. Capacity in school canteen and stock management................................................... 42 4.2.3. Organizational capacity ............................................................................................... 44 4.2.4. Application of project promoted recipes.................................................................... 46 4.2.5. Good practices in terms of hygiene in food preparation............................................ 46 4.2.6. Quality and quantity of available food....................................................................... 47 4.2.7. Direct observations...................................................................................................... 48 4.2.8. Satisfaction .................................................................................................................. 51 4.2.9. Lessons learned and sustainability.............................................................................. 51 4.3. Students.............................................................................................................................. 53 4.3.1. Sample characteristics................................................................................................. 53 4.3.2. Food consumption patterns........................................................................................ 54 4.3.3. Foods at school that they consume at home .............................................................. 56 4.3.4. Minimum acceptable diet............................................................................................ 57 4.3.5. School attendance....................................................................................................... 59 4.3.6. Effects on students’ families........................................................................................ 61 4.3.7. Lessons learned ........................................................................................................... 61 4.4. Government and commune level partnerships................................................................. 62 4.4.1. Government level actors............................................................................................. 62 4.4.2. Commune level actors................................................................................................. 62 4.4.3. Satisfaction .................................................................................................................. 64 4.4.4. Challenges and opportunities...................................................................................... 65 4.4.5. Lessons learned and sustainability.............................................................................. 66 5. Results: Comparison Study ....................................................................................................... 68 5.1. Cost differences.................................................................................................................. 69 5.2. Time differences................................................................................................................. 72 5.3. Food supply mechanisms................................................................................................... 73 5.4. Appreciation of meals by children ..................................................................................... 75 iii 5.5. Externalities on local economy .......................................................................................... 76 5.6. Satisfaction of project beneficiaries................................................................................... 76 6. Conclusions and Recommendations......................................................................................... 77 6.1. Conclusion .......................................................................................................................... 77 6.2. Lessons Learned ................................................................................................................. 79 6.3. Limitations.......................................................................................................................... 79 6.4. Recommendations ............................................................................................................. 80 7. Appendix Tables – Summary Results........................................................................................ 85 Tables AX1. Producer Associations ........................................................................................... 85 Tables AX2. COGES .................................................................................................................... 85 Tables AX3. Students................................................................................................................. 85 Tables AX4. Cost and Time Differentials between LRP and the MGD and WFP systems......... 85 8. Annexes..................................................................................................................................... 85 Annex 1. Data collection tools................................................................................................... 85 Annex 4. Highlights of key review meetings............................................................................. 85 iv List of Tables Table 1. Impact evaluation quantitative target indicators........................................................... 17 Table 2. Intervention and control schools.................................................................................... 18 Table 3. Survey sample size and description ................................................................................ 19 Table 4. Individual interviews and group discussions .................................................................. 20 Table 5. Observations and infrastructure inventory.................................................................... 22 Table 6. Comparison study quantitative target indicators........................................................... 22 Table 7. Schools for comparison study from the WFP Project..................................................... 23 Table 8. Schools for comparison study from the MGD Project.................................................... 23 Table 9. Data collection schedule for the comparison study ....................................................... 23 Table 10. Sample characteristics – Producer Associations........................................................... 31 Table 11. Sample characteristics - Producer Association individual members............................ 32 Table 12. Producer Association organizational capacity scored .................................................. 33 Table 13. Percentage of Producer Associations demonstrating increased organizational capacity ....................................................................................................................................................... 34 Table 14. Sale of produce in the last 12 months through producer association.......................... 34 Table 15. Percentage of Producers who demonstrate the use of improved production techniques..................................................................................................................................... 35 Table 16. Producer Associations’ use of URP's Market information system................................ 37 Table 17. Sample characteristics - Schools and COGES ............................................................... 41 Table 18. Capacity and training of cooks and store keepers ....................................................... 42 Table 19. COGES scores in canteen and stock management........................................................ 43 Table 20. COGES organizational capacity score............................................................................ 45 Table 21. Good practices in terms of hygiene in food preparation ............................................. 47 Table 22. COGES report on delivery of commodities for school canteen .................................... 48 Table 23. Kitchen inventory in the sampled LRP schools ............................................................. 49 Table 24. Store inventory in the sampled LRP schools ................................................................ 50 Table 25. Sample characteristics - Students surveyed ................................................................. 54 Table 26. Top 10 meals consumed by students based on one day recall .................................... 55 Table 27. Foods consumed at school and at home ...................................................................... 56 Table 28. Percentage of students receiving a minimum acceptable diet .................................... 58 Table 29. Students regularly attending USDA supported classrooms/schools............................ 59 Table 30. Differentials in commodity cost per MT ($/MT) between LRP and MGD systems....... 69 Table 31a. Quantities of commodities in the weekly school canteens' rations (in grams) in LRP and MGD systems......................................................................................................................... 69 Table 31b. Number of school days during the 2018-2019 school year…………………………………..…70 Table 31c. Differentials in commodity cost per student during the school year 2018-2019………70 v Table 31d. Differentials in average commodity cost per meal and per student between LRP and MGD systems ($/meal)…………………………………………………………………………………………………………….70 Table 32. Differentials in commodity cost for specific, comparable meals between LRP and MGD systems ($/meal)........................................................................................................................... 71 Table 33. Time differentials between LRP and MGD systems days)………………………………………… Table 34. Appreciation of meals among students who declared eating at school canteen….……76 vi List of Abbreviations AKB Association Karité Bénin AME APE CLOC COGES Mother Educators Association Parents Association Commune Level Oversight Committee Comité de Gestion des Ecoles CPC CRS Commune Procurement Committee Catholic Relief Services DAS DDAEP Direction de l'Alimentation Scolaire Departmental Directorate of Agriculture, Livestock and Fisheries DPO Data Processing Officer ENSPD Ecole Nationale de Statistique, de Planification et de Démographie EVIHDAF Evidence for Sustainable Human Development Systems in Africa FFE Food For Education FGDs Focus Group Discussions HIV/AIDS Human Immunodeficiency Virus/ Acquired Immunodeficiency Syndrome INRAB Institut National de la Recherche Agronomique du Bénin KAP Knowledge, Attitudes and Practices KIIs Key Informant Interviews MAEP Ministère de l'Agriculture de l'Elevage et de la Pêche MDES Minimum Detectable Effect Size MEMP Ministère des Enseignements Maternel et Primaire MGD McGovern Dole MOH Ministry of Health ODeSPoL Observatoire Démographique et Statistique des Populations Locales PASDeR PME-FCB Programme d’Appui au Secteur du Développement Rural Global Partnership for Education of the Common Budget Fund PSI Population Services International RA Research Assistant RO SAS Research Officer School Food Service UCP Union Communale des Producteurs UNICEF United Nations Children's Fund UNFPA United Nations Population Fund URP Union Régionale des Producteurs USAID United States Agency for International Development WFP World Food Program vii Executive Summary Introduction The Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) project, funded by the United States Department of Agriculture (USDA) and implemented by Catholic Relief Services (CRS) Benin, supports the establishment and functioning of school canteens in government schools of N’dali, Perere, Parakou and Tchaourou districts in the Borgou Region. The purpose of the project is to develop a replicable and scalable model for decentralized school feeding through effective local procurement and distribution of commodities with the aim of providing students with nutritious and locally procured meals (rice, shea, cowpea and maize) from Producer Associations. CRS and its sub-recipient Union Régionale des Producteurs Borgou-Alibori (URP) worked with school management committees(referred to in French as Comité de Gestion des Ecoles(COGES)), commune authorities, central government agencies and private partners to achieve project goals. The specific goals of the Bèsèn Diannou LRP project were to: • Increase the capacity of schools, school management committees, and the Government of Benin to effectively and efficiently procure local commodities to supply school feeding programs, promoting the sustainability of school feeding; • Strengthen farmer groups' ability to provide high-quality, nutritious commodities and connect them to school feeding programs; and Improve nutrition of students by increasing access to and use of various, high-quality, nutritious, and culturally appropriate foods in school meals. 3. Methodology The endline evaluation uses a mixed-methods approach involving quantitative data, qualitative investigationsin the form of individual interviews and group discussions, and direct observations. Impact Evaluation: Quantitative Component • Schools: The same schools sampled during the baseline study were kept; that is 35 intervention schools and 35 control schools. • COGES: The COGES questionnaire was administered to all 70 sample schools even though we did not expect to have formal COGES in schools without a canteen. • Students: The selection of students followed the same procedure as during baseline. In each of the 70 sampled schools, 50 students were randomly selected among all grade 2-6 students. • Producer Associations: The sample consisted of 24 Producer Associations in the intervention group, and 10 Producer Associations in the control group. Impact Evaluation: Qualitative Component: The qualitative component consisted of individual interviews and group discussions. viii Direct Observation and Infrastructure Inventory Component: Direct observation of COGES cooks and storekeepers were conducted to evaluate the level of application of good food management practices. School infrastructure inventory was also conducted by direct observation. Comparison Study: Quantitative and qualitative data was collected from the LRP, MGD and WFP staff. 4. Results: Impact Evaluation Producer Associations: About 75% of Producer Associations in the intervention group demonstrated an increased organizational capacity, against 50% in the control arm. Regarding the use of improved production techniques, 17 individual producers (78.7%) received training on production practices for example, and out of those, 15 (84.5%) put the training into practice. The percentage of Producer Associations who used URP's Market information system (MIS) remained unchanged over time in the control group (at 30%), but increased from 11.8% to 31.8% in the intervention group. However, the resulting impact estimate of 20 percentage points is not statistically significant at the level of 10%. COGES: Close to 50% of COGES achieved a score of 80 or higher in canteen and stock management; 36% achieved a score of 80 or higher in organizational capacity; and close to half of COGES achieved a score of 80 or higher in adoption of good practices in terms of hygiene in food preparation. Students: The percentage of students who consumed the same foods at school and at home increased from baseline to endline in both the intervention schools (+11.6 percentage points) and in the control schools (+5.3 percentage points). The impact estimate of 6.4 percentage points is statistically significant at the level of 5%. The percentage of students receiving a minimum acceptable diet (MAD) decreased from baseline to endline in both arms, and the impact estimate, though negative is not statistically significant at the level of 10%. The percentage of students who attended school at least 80% of time is roughly the same across the two study groups, at around 98.5%. 5. Results: Comparison Study Cost differences: In terms of commodity cost per MT, the LRP system has higher average commodity cost per MT on three out of the four target commodities, compared to the MGD system. The average commodity cost per meal and per student is lower in the LRP model, at $0.14 (or 84 CFA Francs), against $0.19 (or 109 CFA Francs) in the MGD system. For specific comparable meals, commodity cost of Dambou/Zampiti and pasta (maize/soy pasta) is about 50% lower in the LRP system, while the commodity cost of wake and rice is lower (by 7% and 15%, respectively) in the MGD system. Time differences: The total time (from contracting to delivery to beneficiary schools) is substantially shorter in the LRP model, compared with the WFP and MGD systems. Purchase ix contracting time is the shortest in the WFP system and the longest in the MGD system. Time for delivery to schools is almost the same in both the LRP and MGD programs, and substantially longer in the WFP, presumably because of its larger coverage of schools. 6. Conclusions and Recommendations The results of this impact evaluation, as described in this report, demonstrate that the LRP model that relies on local commodities and procurement processes for school canteen meals is a viable method for feeding students quality meals while contributing to the local economy. Through the LRP project, capacities of Producer Associations and COGES were significantly strengthened, contributing to timely delivery of high-quality commodities that were adequately delivered to students during the school day. While there was no measured increase in minimum acceptable diet or attendance among students receiving these meals, qualitative data indicates that the provision of school meals to students improved their day-to-day lives. The evaluation put forwards the following recommendations: 1. Evaluate PA training needs with members of these associations and involve all members during training. 2. Consider contractual models that incorporate access to financial resources for producers before the sales process. 3. Design Contracts with PAs in a format of 1 to 2 pages maximum and in simple and understandable terms that allow producers to better control contract processes. 4. Respect the deadlines set in the contracts, particularly with regard to the purchase and payment of invoices. 5. Consider duly signed partnership protocols between the main institutional actors of the project. 6. Transfer supervisory functions to government entities to increase likelihood of sustainability. 7. Involve community leaders in the management of school canteens (COGES). 8. Encourage governmental authorities to review current plans, programs or school feeding strategies at national level and to allocate financial resources to the town halls for the management of school canteens. 9. Promote an integrated approach including the WASH component. 10. Reduce financial contributions and promote in-kind contributions from parents. 11. Identify means of motivation for cooks and storekeepers to maintain compliance and quality. 12. Consider a greater diversity of dishes, combining local meals with some imported products for school canteens. 13. Obtain student feedback and opinion regarding meal plans at the local level to ensure greater satisfaction in regard to meal quality. 14. Solicit a sufficient number of cooks to avoid school girls’ involvement in cooking activities. 10 1. Introduction The Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) project supports the establishment and functioning of school canteens in government schools of N’dali, Perere, Parakou and Tchaourou districts in the Borgou Region. Its purpose is to set up a local mechanism for the procurement and distribution of the commodities with the aim of providing students with nutritious and locally procured meals (rice, shea, cowpea and maize) from producer associations in the same districts. Funded by the United States Department of Agriculture (USDA), LRP is implemented by Catholic Relief Services (CRS) and its sub-grantee, the Union Régionale des Producteurs du Borgou Alibori (URP/BA) who facilitate local procurement and strengthen the capacity of producers. The intervention areas are highlighted in the opposite map of Benin. This intervention also complements CRS’ existing school feeding program under McGovern Dole (MGD) Food for Education (FFE), currently being implemented in schools of Borgou and Alibori Regions. Both programs operate in different schools. This complementarity allows CRS to compare timeliness and cost-effectiveness of rations distributed as part of the LRP program with those imported and distributed under MGD and other partners including the World Food Program (WFP). The LRP project was implemented with the support of both government and non-governmental actors, considered “key stakeholders,” including Direction de l’Alimentation Scolaire (DAS), Ministere de l’Agriculture, l’Elevage et de la Pêche (MAEP), Ministère des Finances, Ministère de l’Enseignement Maternel et Primaire (MEMP), Ministère du Plan et du Développement, Commune administrations of Parakou, Perere, N’Dali, and Tchaourou, Union Communale des Producteurs (UCP), Association Karité Bénin (AKB), Union Régionale des Producteurs de Riz (URPR), GIZ’s ProAgri project, Cooperation Swiss’ Programme d’Appui au Secteur du Développement Rural (PASDeR) and Association Nationale des Communes du Benin (ANCB). Other stakeholders include relevant civil society organizations (OSC). The specific goals of the Bèsèn Diannou LRP project were to: • Increase the capacity of schools, school management committees, and the Government of Benin to effectively and efficiently procure local commodities to supply school feeding programs, promoting the sustainability of school feeding; 11 • Strengthen farmer groups' ability to provide high-quality, nutritious commodities and connect them to school feeding programs; and • Improve nutrition of students by increasing access to and use of various, high-quality, nutritious, and culturally appropriate foods in school meals. The ultimate aim is to test the efficiency of local procurement for school canteens and explore the scale-up of the model by key stakeholders. The major activities of the project include: • Purchase, delivery and distribution of food; • Construction and rehabilitation of stores and kitchens; • Engagement with the government; • Training and support to producers and school canteen management committees; • Promotion of nutritious local foods. The two-year LRP project officially started on September 29, 2017 and ends on December 31, 2019. Its registration number in Benin is LRP-680-2017/034-00. 1.1. Project Theory of Change LRP’s overall theory of change is structured around the following sequence of assumptions: • IF commune level authorities and community groups have increased capacity to plan, manage and oversee school canteens (Pathways A and B); and • IF central government agencies support the local purchase and canteen management approach (Pathway C); and • IF commodity procurements are structured to leverage existing agricultural support projects and to reinforce the agricultural sector in the Borgou Department (Pathway D); • THEN a replicable and scalable model for a locally managed and sourced school feeding system, allowing school children to receive nutritious, culturally acceptable and diverse quality meals efficiently and cost-effectively, will be developed. Pathway A • IF commune authorities are trained in local procurement of school feeding rations, canteen planning, management, budgeting, and oversight; and • IF school management committees (COGES) are trained in canteen ration planning, ration storage, and ration management; and • IF commune authorities and COGES receive supportive supervision as they carry out activities directly; • THEN the capacity of commune level authorities and COGES to manage school canteens locally will be increased. 12 Pathway B • IF the COGES, mother associations and cooking teams are trained in hygiene, nutrition and food preparation safety and demonstrate application of knowledge and skills; and • IF mother associations and cooking teams are trained in a variety of culturally acceptable recipes; and • IF the cooking team is provided with a hygienic cooking area and cooking materials; and • IF culturally appropriate foods are procured for consumption at school canteens; • THEN school children will receive nutritious and culturally acceptable food that meets quality standards. Pathway C • IF the Département d’Alimentation Scolaire (DAS), Ministère d’Enseignement Maternelle et Primaire (MEMP), Ministère de l’Agriculture, de l’Elevage et de la Pêche (MAEP), and the Ministère de Finance (MF) are engaged early in the LRP process; and • IF the DAS, MEMP, MAEP, and MF receive training in key aspects of local procurement and canteen management; and • IF the DAS, MEMP, MAEP, and MF visit schools with locally procured meals and provide oversight and guidance on project implementation; and • IF the National Association of Communes of Benin and agricultural stakeholders engaged through the project actively advocate for the decentralization of school feeding responsibility with the DAS, MEMP, MAEP, and MF; • THEN central and local government buy-in of localized procurement and canteen management will be created. Pathway D • IF school rations are procured as locally as possible (“Farm to Fork”); and • IF the project leverages the rice, shea, cowpea and maize value chain investments and initiatives of other projects operating in Borgou department; and • IF the project leverages the investments of private sector partners; and • IF the project supports and invests in umbrella organizations to reinforce producer capacity and quality; and • IF the project addresses gaps in current activities; and • IF the project improves producer production capacity, market access, and access to market price information systems for producers not currently targeted through existing projects; • THEN the project will contribute to strengthening local and regional food market systems in the Borgou Department. 1.2. Results Framework The expected results included: • Improved cost-effectiveness of food assistance 13 • Improved timeliness of food assistance • Improved utilization of nutritious and culturally acceptable foods that meet quality standards • Increased capacity of government institutions • Improved policy and regulatory framework • Improved capacity of relevant organizations • Increased leverage of private sector resources The project level results framework is presented below. SO: Improved Effectiveness of Food Assistance through Local and Regional Procurement IR 1: Improved cost￾effectiveness of food assistance IR 2: Improved timeliness of food assistance IR 3: Improved utilization of nutritious and culturally acceptable food that meet quality standards IR 4: Foundational results IR 1.1: Improved cost￾effectiveness of procurement IR 1.2: Improved cost￾effectiveness of delivery IR 1.3: Improved cost￾effectiveness of distribution IR 2.1: Improved timeliness of procurement IR 2.2: Improved timeliness of delivery IR 2.3: Improved timeliness of distribution IR 3.1: Improved access to culturally acceptable foods IR 3.2: Strengthened local and regional food market systems IR 3.3: Improved access to nutritious foods IR 4.1: Increased capacity of government institutions IR 4.2: Improved policy and regulatory framework IR 4.3: Improved capacity of relevant organizations IR 4.4: Increased leverage of private sector resources A 1.1.1: Commodity management (procurement; market monitoring) A 1.1.2: Capacity building (commune gov’t training and support; umbrella organization training and support) A 1.2.1: Commodity management (delivery, construction and rehabilitation [storerooms and kitchens]) A 1.3.1: Commodity distribution (distribution, vouchers) A 1.3.2: Capacity building (community training and support) A 2.1.1: Commodity management (procurement; market monitoring) A 2.1.2: Capacity building (commune gov’t training and support; umbrella organization training and support) A 2.2.1: Commodity management (delivery, construction and rehabilitation [storerooms and kitchens]) A 2.3.1: Commodity management (distribution, vouchers) A 2.3.2: Capacity building (community training and support) A 3.1.1: Capacity building (community training and support) IR 3.2.1: Increased agricultural productivity A 3.2.1.1: Capacity building (umbrella organization training and support; producer organization training and support) IR 3.2.2: Increased value added to post production agricultural products A 3.2.2.1: PASDeR and ProAgri support rice, shea, cowpea and maize value chains with value addition activities IR 3.2.3: Increased access to market to sell agricultural products A 3.2.3.1: Capacity building (producer organization training and support) IR 3.2.4: Improved transaction efficiency A 3.2.4.1: Capacity building (umbrella organization training and support) A 3.2.4.2: Private sector leverage A 3.3.1: Capacity building (community training and support) A 4.1.1: Capacity building (central gov’t engagement; commune gov’t training and support) A 4.2.1: Capacity building (commune gov’t training and support; advocacy) A 4.3.1: Capacity building (community training and support; umbrella organization training and support; producer organization training and support) A 4.4.1: Private sector leverage 14 Through the baseline and endline assessments, the LRP project were rigorously evaluated, and the learnings and recommendations will be shared with MEMP for the adoption, replication and/or scale up of a model for decentralized school feeding using local and regional procurement. 2. Purpose of the Evaluation 2.1. Impact Evaluation The baseline study was implemented in May-June 2018 to: a) Establish benchmark values against which Bèsèn Diannou’s progress on each of the impact and outcome indicators pertaining to the project results framework can be tracked; b) Verify project design assumptions and identify potential threats to implementation; and c) Inform decision-making about early project implementation planning and execution. In addition, the baseline evaluation supported the refinement of the project’s metrics and helped strengthen relationships between the project team and key stakeholders. In particular, participants to the baseline evaluation recommended the alignment of program implementation with community needs and expectations. Students expectations included having free access to quality, nutrient-rich foods from school canteens without exclusions; schools hoped to be supported through the provision of storerooms and quality food products on a regular basis; COGES’ members expected to receive improved trainings on meal preparation, nutrition and canteen management; and communities, producers and schools encouraged strong agreements between key stakeholders to allow for organized and reliable transactions that would facilitate improved school canteen management. They also recommended that improved procedures for school attendance be put in place by school directors and teachers, with support from the LRP project. The endline impact evaluation focuses on assessing the relevance of the interventions, the effectiveness of the implementation strategies and activities, the efficiency of the project, the progress on impact indicators, and achievements related to sustainability, impact and learning. In this regard, the final evaluation: • Assesses whether the project has achieved the expected results as outlined in the project￾level results framework and compared to the baseline evaluation; • Assesses areas of project design, implementation, management, lessons learned and replicability; • Generates lessons learned and recommendations for CRS, the Ministry of Education, Ministry of Agriculture, partners, USDA and other stakeholders for future school feeding programs which emphasize local and regional food procurement. Key evaluation questions for the impact evaluation include: Relevance: • To what extent were the implementation strategies relevant enough to improve effectiveness of food assistance through local and regional procurement? • Are the activities and outputs of the project consistent with the overall goal? 15 • How well does the project complement and link to activities and other donors at the local level? • Are stakeholders (producers, umbrella organizations, COGES, national, communal and local authorities, private partners) satisfied with the impact that their participation in the project has created on the local school feeding model? Why or why not? Effectiveness: • To what extent have members of producer, umbrella organizations, COGES, national, communal and local authorities increased their knowledge and application of different concepts they received training on? • Have targeted producers applied the demonstrated agricultural techniques taught? • Have cooks and COGES demonstrated improved understanding of nutrition and hygiene practices? How often were locally available, nutritious foods from the scoping study included in school canteen meals? Did the project increase the diversity of foods being consumed? • Were contractual obligations respected by umbrella organizations and producer groups supplying commodities? Why/why not? Were quality requirements met? • How well did small lot procurements work at the commune level? What lessons were learned or challenges identified? Efficiency: • Were activities cost efficient? • Were objectives achieved on time? • Was the project implemented in the most efficient way compared to alternatives? • Were there internal and/or external factors that hindered the efficient implementation of project activities? Sustainability: • What activities and/or outcomes (both expected and unexpected) of the project are likely to be sustained? What evidence is there to suggest this? • What are the major factors that can influence achievement or non-achievement of sustainability? • How does the government’s capacities, policies, procedures and priority contribute to sustainability? • What is the attitude of central government, departmental government, and commune level authorities towards the idea of continued local procurement and local management of school feeding? What steps have been taken to integrate it in national and communal plans? Impact: • Did the project achieve a feasible and replicable model for school feeding in Benin? • Were local procurements cost-effective, efficient and timely? • What impact, if any, did procurements have on local market prices and supplies? • What impact did the school feeding have on attendance of students? 16 • Has the project improved the nutritional state of school children? If so, in which extent? • Have the project participants taken ownership of the project concept and approach? • What are the key success factors of the project? What are the main limiting factors? • What are the unintended effects of the project (positive or negative)? • What are the key lessons that can be learned from the project? • (Producers) What are the positive / negative effects of capacity building on climate change on your productivity? Learning: • What are the application and replicability capacities of the different stakeholders for the implementation of the school canteen through the local procurement approach? • What lessons can be learned from the school feeding through the local procurement approach and what measures to increase the chances of extension and success of that model in Benin context? 2.2. Comparison Study The endline evaluation also consists of a comparison study which estimates cost and time differentials between the LRP project and the MGD program that is based on imported commodities and the WFP/MEMP program. Key evaluation questions for the comparison study include: • To what extent is the local procurement approach developed by the project more cost effective and timely than the MGD and WFP approaches? • What are the positive or negative externalities resulting from the local procurement approach on the local economy compared to the MGD and WFP approaches? • What is the appreciation of the meals served in LRP’s canteen schools by the children, compared to those served in the MGD and WFP canteen schools? • What is the level of satisfaction of the project beneficiaries (producers, students, COGES, cooks, parents and teachers) with the LRP program? 2.3. Use of the Evaluation’s Findings We expect that the findings from the present evaluation could contribute to redefine the strategy of implementation of school canteen activities in Benin, and to revitalize the local agricultural sector. The stakeholders of interest of that include the Government of Benin, the local and communal authorities, the implementation partner URP, the National Institute of Statistics and Economic Analysis (INSAE), CRS, and USDA (the funder). 17 3. Methodology The endline evaluation uses a mixed-methods approach involving quantitative data, qualitative investigationsin the form of individual interviews and group discussions, and direct observations. 3.1. Impact Evaluation: Quantitative Component 3.1.1. Design and key indicators The quantitative component adopts a quasi-experimental design with 77 schools across the four target districts (N’dali, Perere, Parakou and Tchaourou) identified as intervention schools, and 148 identified as comparison schools. A total of 24 Producer Associations in Parakou, N'Dali, Perere and Tchaourou who actively worked with the LRP project were selected as the intervention group, and 10 Producer Associations in the comparison districts of Kalale and Niki (see Section 3.1.2 on the sampling strategy). The target indicators presented in Table 1 were measured through surveys with students, Comite de Gestion des Ecoles (COGES) and Producer Associations. Table 1. Impact evaluation quantitative target indicators Indicator Data source Percentage of Producer Associations demonstrating increased organizational capacity Survey with Producer Associations Percentage of Producer Associations using Union Regionale des Producteurs (URP) Market Information Systems (MIS) Percentage of targeted producers who demonstrate the use of improved production techniques in USDA-supported areas Percentage of COGES achieving a minimum 80% score in adoption of good practices related to canteen and stock management Survey with COGES Percentage of COGES demonstrating increased organizational capacity Percent of school cooks demonstrating the application of project promoted recipes Percentage of cooks achieving a score of 80% adoption of good practices in terms of hygiene in food preparation. Percentage of students reporting having consumed foods at school that they consume at home, based on one-day recall Survey with Students Percentage of students receiving a minimum acceptable diet, based on one-day recall Number of students regularly (80%) attending USDA supported classrooms/schools 18 Indicator Data source Percentage of key stakeholders0F 1 (COGES-school mgt committees, parents, teachers and students) who reported having received on time, in quantity and quality, commodities for school canteen (disaggregated by criteria) Other stakeholders Level of functionality of the Commune Purchase Committees (CPCs) and Commune Level Oversight Committees (CLOCs)1F 2 3.1.2. Sampling strategy Schools The same schools sampled during the baseline study were kept; that is 35 intervention schools and 35 control schools, as shown in Table 2. For sample size calculations, we used the Minimum Detectable Effect Size (MDES) method which provides the minimum effect that can be detected by considering the number of students selected from each cluster, for given values of confidence level, margin error and power. The method yielded a sample size consisting of 60 schools (30 intervention schools and 30 comparison), with 50 students from each school (3,000 students in total). During baseline data collection the target of 30 students was not reached in some of the schools, due to the smaller size of these schools or the absence of students. Ten additional schools were randomly selected (five comparison schools and five intervention schools) following the same sampling principles at both baseline and endline (bringing the total of students in endline to 3,500). To qualify as control, a school needs to have no canteen (run by LRP, MGD, WFP or other partners). Three control schools at baseline (in Tchaourou) ended up with canteens supported by WFP. They were replaced with three other schools without a canteen that were randomly selected using the Probability Proportional to Size (PPS) method. On the other hand, one school in N’Dali selected at baseline as part of the intervention group ended up not having a canteen. Likewise, it was replaced with another school. Table 2. Intervention and control schools District Parakou N'Dali Perere Tchaourou Total Intervention Schools 11 19 25 22 77 Sampled schools 5 8 12 10 35 Urban 5 2 1 2 10 Rural 0 6 11 8 25 1 In subsequent discussions with the CRS team, it was agreed to focus on school principals only. 2 In subsequent discussions with the CRS team, it was agreed to address this indicator qualitatively. 19 District Parakou N'Dali Perere Tchaourou Total Control Schools 54 19 9 66 148 Sampled schools 12 5 3 15 35 Urban 13 1 1 2 17 Rural 0 4 1 13 18 COGES The COGES questionnaire was administered to all 70 sample schools (35 in the intervention arm and 35 in the control) even though we did not expect to have formal COGES in schools without a canteen (i.e. the control arm). In each school, data was collected from the COGES representative, one cook representative and one storekeeper representative. Students The selection of students followed the same procedure as during baseline. In each of the 70 sampled schools, 50 students were randomly selected among all grade 2-6 students, using class registers to number students from 1 to N (N being the total size of grade 2-6 classes). As at baseline, first-graders were deemed emotionally and cognitively unable to answer our survey questions. Producer Associations As noted above, the original sample consisted of 24 Producer Associations in the intervention group, and 10 Producer Associations in the control group. While the original sample size was 34 Producer Associations and 136 individual producers in total, data was actually collected from a total of 32 Producer Associations and 128 individual producers. Data was collected at two levels. First, a questionnaire was administered to the chair (or representative) of the Producer Association. Second, another questionnaire was administered to the Producer Association’s representative and three other individual farmers selected randomly from the list of members. The sample size and description for the surveys is summarized in Table 3. Table 3. Survey sample size and description Survey Final sample size Description Respondents 1. COGES Representatives 70 35 intervention schools and 35 comparison schools School Principals as COGES representatives 2. COGES - Cooks 35 35 intervention schools (none in comparison schools) Head of the cooks 20 Survey Final sample size Description Respondents 3. COGES - Storekeepers 35 35 intervention schools (none in comparison schools) Usually only one storekeeper per school. If more than one, select the Head of storekeepers 4. Students 3,500 50 students per school (intervention and comparison) Students selected randomly 5. Student attendance 3,500 From school registers, and for the months of Oct 2018 to April 2019: number of school days, and number of days attended. 6. Producer Associations (PA) 34 All 24 PAs in the intervention arm that were involved in LRP; and the 10 comparison PAs surveyed at baseline Representative/President of the PA. 7. Individual Producers 136 4 individual producers per PA - List of the active members of from the representative - Simple random draw of 3 members from the list. - The representative with be the 4th member 3.2. Impact Evaluation: Qualitative Component The qualitative component consisted of individual interviews and group discussions as detailed in Table 4. Table 4. Individual interviews and group discussions Interview Num￾ber Description Respondents Individual interviews (n = 40) 1. School Principals 8 2 intervention schools per district Schools that have been well involved in their canteen. Same schools to serve for #1 and #2. 2. Parents Associations 8 2 intervention schools per district 3. Government Authorities 2 DDAEP, SAS or DAS/DDEMP One representative each 21 Interview Num￾ber Description Respondents 4. Commune Authorities 4 1 representative per commune A representative familiar with economic and social issues. 5. Communes’ CPC and CLOC 8 1 interview with CPC, and 1 interview with CLOC 2-3 CPC members together; the 2 CLOC members together 6. Producer Associations 3 1 Association per district Among those that sold produce to LRP (from list provided by CRS) 7. PA Faitières 3 URP, URPR et AKB One representative each 8. PASDeR & ProAgri 2 PASDeR, ProAgri One representative at regional level, each 9. LRP program staff 2 2 persons from CRS/LRP 2 persons from CRS/LRP Group discussions [n = 23] 1. Students’ parents 4 1 per district, from intervention schools In each district, three schools that are close to each other to be selected. Respondents for #1, #2, #3 and #4 drawn from these schools. 2. Students’ mothers 4 1 per district, from intervention schools 3. Students 4 1 per district, from intervention schools 4. Teachers 4 1 per district, from intervention schools 5. COGES 4 1 per district, from intervention schools 6. Producer Associations 3 1 per district – none in Parakou In each district, PA that are close to each other to be selected. Respondents drawn from PAs. 3.3. Direct Observation and Infrastructure Inventory Component Direct observation of COGES cooks and storekeepers were conducted to evaluate the level of application of good food management practices. Another observation was made to determine the level of community involvement throughout the food supply chain. School infrastructure inventory was also conducted by direct observation to determine the need to meet set standards 22 or the minimum viable when there is a gap of standard of storerooms and kitchens, and if rehabilitation is possible, or new construction is required (see Table 5). Table 5. Observations and infrastructure inventory Observation Number Description 1. Direct observation (Cooks/ Storekeepers) 35 Sampled intervention schools 2. Direct observation (community involvement) 35 3. School infrastructures inventory 35 3.4 Comparison Study The endline comparison study comparing the LRP project to the MGD and WFP projects utilized a mixed-methods approach, relying on both quantitative and qualitative data. Quantitative and qualitative data was collected from the LRP, MGD and WFP staff to calculate the indicators listed below in Table 6. In addition, information about appreciation of the canteen meals was collected in student questionnaires. The quantitative component was based on the 35 LRP intervention schools, and 36 other MGD/WFP schools. According to the database provided, we had 144 MGD schools and 136 WFP schools in the LRP intervention communes, which gave us a ratio of 18 sampling schools selected covered by the MGD intervention and 18 sampling schools selected covered by the WFP project. Table 6. Comparison study quantitative target indicators Indicator Cost per meal differential between local school feeding approach and MGD and WFP ($/meal) Cost differential between local market commodity and MGD and WFP (from purchase to distribution) Time differential between purchase order and distribution in targeted schools for commodities locally procured versus MGD and WFP (# of days, by commodity) Time differential between purchase of commodities locally procured versus MGD and WFP (# of days, by commodity) Time differential between delivery of commodities locally procured versus MGD and WFP (# of days, by commodity) Time differential of distribution of commodities locally procured versus MGD (# of days, by commodity) In order to ensure a consistent selection method and to keep equiprobability in the selection of students to be sampled, schools were drawn with probability proportional to their student size, 23 in accordance with the distribution of schools of each project in each of their intervening communes (see Tables 7 and 8). Table 7. Schools for comparison study from the WFP Project District Parakou N'Dali Perere Tchaourou Total Intervention Schools 10 30 22 74 136 Sampled schools 2 4 3 9 18 Table 8. Schools for comparison study from the MGD Project District Gogounou Kalalè Kandi Malanville Total Intervention Schools 30 15 67 32 144 Sampled schools 4 2 8 4 18 Data on procurement cost and time was collected from relevant LRP, MGD and WFP staff; data on cost per meal was collected from COGES representatives; and data on the appreciation of meals served was collected from students. To complement these quantitative data, we conducted in-depth interviews with project staff, communes and government authorities, and focus group discussions with COGES members and community leaders, so as to compare the positive or negative externalities resulting from the varying procurement approaches, and the level of satisfaction of the beneficiaries under each project (see Table 9). Table 9. Data collection schedule for the comparison study Data collection tool Sample size Description Respondents Quantitative component 1. LRP/ MGD /WFP Staff Form 3 LRP, MGD and WFP staff in charge of procurement and logistics One person from each program 2. COGES survey 71 35 sampled LRP intervention schools, 18 sampled MGD schools, and 18 sampled WFP schools School principals 3. Students survey 3,550 50 students in each of the 71 schools described above Same procedure as in Table 3 24 Data collection tool Sample size Description Respondents Individual interviews (n=9) 1. LRP/MGD /WFP Staff interviews 3 LRP, MGD and WFP staff in charge of procurement and logistics Same persons as in the quantitative component above 2. Communes 4 2 communes for LRP; 1 commune for WFP; and 1 commune for MGD selected from Table 8 For the LRP/WFP arm: Same persons in Table 4/Individual interviews. For MGD arm: Similar persons as in the LRP communes 3. Government Authorities 2 DDAEP, SAS or DAS/DDEMP Same persons in Table 4/Individual interviews Group discussions (n=4) 1. COGES 4 2 group discussions for LRP; 1 group discussion for WFP; 1 group discussion for MGD For the LRP: Same group discussions as in Table 4/Group discussions For the WFP and MGD: Follow the same procedure as described in Table 4 3.5. Data Collection and Quality Assurance 3.5.1. Data collection tools The study tools are listed on the cover page of Annex 1. For the endine evaluation, there were seven questionnaires related to the surveys with Producer Associations and individual producers (2), COGES (3), and students (2). Nine interview guides were developed for individual interviews, and five for group interviews. Three tools were used for the direct observations of COGES and infrastructure inventory. For the comparison study, one quantitative tool and one guide for individual interviews were developed. Two questionnaires (with COGES and students), two individual interview guides (communes and government authorities), and one group interview guide (with COGES) used in the endline evaluation were expanded to cover key aspects of the comparison study. All questions included in the data tools were formulated in accordance with those implemented under the MGD project; all of which were informed by local stakeholders and considered culturally relevant and appropriate in character. 25 3.5.2. Data collection team Data collection was conducted by our local partner, the l'Observatoire Démographique et Statistique des Populations Locales (ODeSPoL) at the Ecole Nationale de la Statistique, de la Planification et de la Démographie (ENSPD), University of Parakou, Benin. The data collection team consisted of 58 enumerators and seven supervisors for the quantitative component, and nine qualitative researchers and one supervisor for the qualitative component. These team members underwent a five-day training ending with the pre-test and revision of the tools. The survey data was collected using tablets provided by CRS and programmed by EVIHDAF/ODeSPoL. Two overall supervisors oversaw the data collection, providing administrative, logistic and technical support to supervisors, enumerators and the qualitative research team and conducting routine spot checks for data quality. A Data Processing Officer (DPO) was in charge of downloading the quantitative data and generating quality control tables to be used by the overall supervisor and the team supervisors. The DPO also converted the datasets from the tablets into STATA and performed data cleaning. The audio recordings from the qualitative data collection were transcribed verbatim (and translated into French when applicable). 3.5.3. Data collection challenges and solutions There were several challenges we faced during the data collection. The first and most important one has to do with the data collection period which was around the time of preliminary final exams (“Examen blanc du CEP”, in French) for grade 6 students. Consequently, some of the sampled schools that were acting as exam centers were not available. In other schools, principals and teachers were not available because they were part of the team supervising the exams, or grade 6 students were busy for the exams and thus not available for the interviews. The initial plan was to have a reserve list of 10 students per school. Given the challenge above, we ended up with a reserve list of 20 students per school. Also, because the data collection period coincided with Ramadan, some sampled students were fasting. In order to avoid bias in the data, these students were replaced by other Muslim students in the sampling order who were not fasting. In a few schools, some community-funded teachers were not being paid from the end of April, and as a consequence, their classes were closed. In a few others, the total school size was below 50 (our target sample size). These two factors contributed to our not reaching the total target sample size for students. Another major challenge concerns the non-availability of up-to-date registers in some schools. In total, 8 classes out of the 293 classes in the 70 target schools (or 2.7%) had no registers, while 30 (or 10.2%) had only partially updated registers. For data on student attendance, a decision was made to only use the periods for which registers were available. 26 Long distances, poor road quality, and poor directions in locating Producer Association members also posed challenges in data collection. Because it was the start of the farming season, farmers were also not readily available during the day. Data collection teams therefore rearranged their schedules to accommodate longer journeys to villages, spent extra time locating Producer Association members with the help of village leaders, and worked by night to survey those farmers who were unavailable during the day. In almost all qualitative interviews in the intervention arm, producers first refused to take part in the study, protesting against delays in their payments by CRS or against the fact CRS sometimes did not respect agreement on quantities (e.g. both parties agreed on say, 5 tones, but CRS decided to only purchase say, 3 tones). Further details on these complaints are described in the results sections. Once the supervisors clarified that the study aimed to identify problems faced by different actors and help resolve them, they agreed to participate. Importantly, the post-electoral crisis and violence faced by the country around the period of data collection negatively impacted the work. In Tchaourou for example, several schools or commune offices were closed some days of the week. The data collection teams had to adjust the plans, and this created delays in the completion of data collection. Finally, it was quite difficult to conduct interviews with representatives from MGD and WFP for the comparison study. The MGD representative, based in Kandi was working between Cotonou and Kandi, and it took time to finally secure an interview with him. The WFP representative, based in Cotonou was not responsive to our requests, until CRS had to chime in. She eventually sent us the materials by email, but we did never get the opportunity to talk to her. This had implications on the time it took to complete data collection, and to some extent, on the quality of the cost and time data received from WFP. 3.6. Data Processing and Analysis 3.6.1. Sampling weights We used the same school weights for the evaluation study as at baseline, and used the same approach to generate sample weights for the comparison study. Schools Since we used the same intervention and comparison schools, the probabilities of inclusion of schools were same as in baseline; that is probabilities proportional to size (PPS). 𝜋𝜋𝑖𝑖 𝑎𝑎,𝑏𝑏,𝑝𝑝𝑝𝑝𝑝𝑝 = 𝑔𝑔𝑖𝑖 𝑎𝑎,𝑏𝑏 𝐺𝐺𝑎𝑎,𝑏𝑏 ∗ 𝑛𝑛𝑎𝑎,𝑏𝑏 Where: - gl a,b is the total enrollment of school i in district a, zone b (for Grades 2-6) – zone is the urban-rural stratum in each district; - 𝐺𝐺𝑎𝑎,𝑏𝑏is the total enrollment of all schools in district a, zone b (for Grades 2-6); 27 - 𝑛𝑛𝑎𝑎,𝑏𝑏 is the total number of schools sampled in district a, zone b. Then the sampling weight of each school was computed as: ωi a,b = 1 πi a,b,pps COGES, Cooks and Storekeepers There is only one COGES, one head cook and one head storekeeper per school. Therefore, the sampling weights for these units were the above school weights. Since the sampling of schools was undertaken with probabilities proportional to size, our design implicitly allocates higher inclusion probabilities to COGES, cooks and storekeepers in larger schools. Students The probability of inclusion of a student j in school i is: πij a,b = πi a,b,pps*πj i ⁄ a,b ; where πj i ⁄ a,b = 𝑆𝑆𝑎𝑎,𝑏𝑏 gi a,b - 𝑆𝑆𝑎𝑎,𝑏𝑏 is the number of students sampled per school in district a, zone b πij a,b = 𝑆𝑆𝑎𝑎,𝑏𝑏∗ 𝑛𝑛𝑎𝑎,𝑏𝑏 𝐺𝐺𝑎𝑎,𝑏𝑏 Then the sampling weight of each student was computed as: ωij a,b = 1 πij a,b Producer associations No sampling weights were needed, as we surveyed all eligible Producer Associations in both the intervention and the control arms. Individual producers The probability of inclusion and sampling weight for the target Producer Association’s representative was 1. The probability of inclusion of the three other sampled individual members from the producer association i was: 𝜋𝜋𝑖𝑖 = 1 𝑀𝑀𝑖𝑖 − 1 28 Where 𝑀𝑀𝑖𝑖 is the total number of active members of the Producer Association i. The sampling weight for these three members (inverse probability of selection) was: 𝐴𝐴𝑖𝑖 = 1 𝜋𝜋𝑖𝑖 = 𝑀𝑀𝑖𝑖 − 1 Sampling weights for individual producer at baseline was not taken into account in the baseline report. They have been incorporated in the current analyses. 3.6.2. Impact evaluation estimates We used difference-in-differences (DID) estimates to determine the impact of the intervention on the target indicators. We estimate the simple DID for a given outcome Y as follows: 𝐷𝐷𝐷𝐷𝐷𝐷(𝑌𝑌) = �𝑌𝑌𝐼𝐼𝐼𝐼 − 𝑌𝑌𝐶𝐶 � − �𝑌𝑌𝐼𝐼𝐼𝐼 − 𝑌𝑌𝐶𝐶 � (1) where 𝑌𝑌𝐼𝐼𝐼𝐼 and 𝑌𝑌𝐶𝐶 represent the average outcome at endline in the intervention site and control area, respectively, and 𝑌𝑌𝐼𝐼𝐼𝐼 and 𝑌𝑌𝐶𝐶 represent the average outcome at base line in the intervention site and control area, respectively. DID (Y) can also be estimated through a multivariate regression as follows: Y (or Prob[Y=1]) = 𝛼𝛼 + 𝛽𝛽𝛽𝛽 + 𝛾𝛾𝛾𝛾 + 𝛿𝛿( ∙ ) + 𝜀𝜀 (2) Where: • is a dummy variable which equals 1 if the observation is in the intervention group and zero otherwise; • is a dummy variable which equals 1 in the follow-up year and zero in the baseline year; • ( ∙ ) 𝑖𝑖𝑖𝑖 𝑡𝑡ℎ𝑒𝑒 interaction term which equals 1 if the observation is in the intervention group and at endline and zero otherwise (i.e., for comparison group members in both the baseline and endline, and for the intervention group in the baseline year). The estimate of captures the impact of the project on the outcome variable (𝑖𝑖. 𝑒𝑒. 𝛿𝛿 = DID (Y)) In practice, we used (1) to generate the estimates, and (2) to determine their level of statistical significance. Analyses were done using STATA. A number of outcomes are computed as a score, with values ranging from 0 to 100. These scores are calculated by averaging related variables recoded as binary variables (0/1), with value 1 representing the favorable outcome (e.g.: Does your school have a handwashing place?). In the case of variables with three categories, the values were coded as 0; 0.5; 1, with higher values representing more desirable outcomes. There are a number of 4-category variables. These were coded as 0; 0.25; 0.75; 1. An example is the variable on whether the students washed their hands: 0=No; 0.25=after meals; 0.75=before meals; 1=both before and after meals. The averages of relevant variables are multiplied by 100. In many instances, the overall score consists of two or more sub-scores. In this case, the overall score is obtained as s simple average of the constituent sub-scores. As a matter of illustration, let’s take the producer associations’ organizational capacity sub-score derived from individual members. Assuming the indicator is made up of n 29 variable items (j=1, 2, … n). Let’s denote k an individual respondent (k=1, 2 …4. The sub-score of producer association i is: 𝐶𝐶𝑖𝑖 = 100 ∗ 𝐴𝐴𝐴𝐴𝑗𝑗=1,...𝑛𝑛�𝐴𝐴𝐴𝐴𝑘𝑘=1,...4�𝐶𝐶𝑗𝑗 ��; 0 ≤ 𝐶𝐶𝑖𝑖 ≤ 100 Where 𝐴𝐴𝐴𝐴𝑘𝑘=1,...4�𝐶𝐶𝑗𝑗 � is the average of responses to the item variable j across the four individual respondents; and 𝐴𝐴𝐴𝐴𝑗𝑗=1,...𝑛𝑛[ ] is the average across the n item variables. The questionnaire for students contains questions on students’ household possessions and characteristics, similar to the variables used in the demographic and health survey (DHS). Students were asked: “Does your household have any of the following items?”. Items considered included electricity, radio, TV, phone, bicycle, motor bike (see Q11 in Students’ questionnaire). We construct an asset-based indicator of socio-economic status (SES) by summarizing these items using principal component analysis (PCA). The first PCA factor, referred to as wealth index (a continuous variable) is used to classify the population in three groups of equal size or tertiles: Poor (lowest 33.3%); Middle; and Rich (highest 33.3). Tabulation outcomes by SES allows us to measure the extent of inequities by SES. 3.6.3. Comparison study estimates Simple differences were used to compare LRP indicators with those of McGovern-Dole, WFP and the average McGovern-Dole and WFP. 3.7. Ethical Considerations We sought answers to questions related to the ethical guidelines that were to be followed in Benin. At the very minimum, the following ethical principles were adhered to during the interviews with children: - Invited freely given consent from all children, and from the parent s/guardians, ensuring that they understood they can refuse any question or withdraw at any time. - Informed children and parents about the research. - Kept all data confidential, removing all personal identifiers and assigning pseudonyms where appropriate. In addition, consistent with CRS’ “Responsible and open data policy”, the report doesn’t contain any information that should not be made publicly available (e.g. school names). - Informed participants that if they divulge information suggesting that they or others are at risk of harm, the researcher has a duty of care to report this and to ensure support for the child (and to inform the participant that this is occurring). - Left all study participants with a written record of the researchers' names and contact information. Specific strategies were put in place when interviewing children. Appropriate measures were put in place to protect the privacy and wellbeing of young people surveyed and interviewed in 30 research. The interview team was carefully screened and selected according to their suitability for interviewing young people. Surveys with students were carried out within the protected environment of the school setting. Aside from obtaining informed consent from students themselves, the permission of the school principals and teachers were obtained before conducting the interview. The subject and general nature of the interview was explained in a simplistic manner in the local language and formal consent was obtained from both the student and an overseeing adult. For focus group discussions, additional consent was obtained from parents and guardians, and the venue and time for the discussions was carefully selected to provide safety and confidentiality. 3.8. Timelines The inception phase began on April 3, 2019 and ended with the submission of the first draft of the Inception report on May 11. It included the draft study tools, sampling strategy and analysis plan. It was further revised and resubmitted on May 29, 2019. The training of data collectors took place on May 9-14, 2019. The pre-test of the tools was conducted on May 13, 2019 as part of the training process. Data collection began on May 15, with the bulk of interviews completed on June 8th, and the remaining on June 14th, though data from WFP was only received on June 24th. Data analysis and report writing began in mid-June and draft reports were submitted on July 25, August 18, and August 25, 2019. The review of preliminary findings from the comparison study prompted a meeting between the evaluation team and representatives from the thee programs (LRP, MGD and WFP). The meeting took place on September 9, from 2:30pm to 5:30pm, local time. The highlights of the meeting are in Annex 5 (Highlights of key review meetings). Following the meeting, data for the comparison study was updated (by the three programs), the analysis was revised, and the evaluation report was submitted on September 17, 2019. A webinar was organized on September 19 to share the evaluation methodology and results to a wider group within CRS,and receive comments and suggestions on the methods and interpretation of results. The highlights of the webinar meeting are presented in Annex 5 (Highlights of key review meetings). Following the webinar, the CRS team sent additional written comments on September 23. These were examined and addressed, and the final evaluation report submitted on September 24, 2019. 4. Results: Impact Evaluation In the section we present the results of the impact evaluation. Apart from results about producer associations (for which there are no sampling weights), all results, including those about individual producers are weighted. 31 4.1. Producer Associations 4.1.1. Sample characteristics Table 10 shows the sample characteristics. Two of the targeted producer associations could not be reached. As shown, while the share of maize and rice as the most important crops remained unchanged in the control group, the percentage of Producer Associations in the intervention area whose most important crop is rice rose from 8.8% at baseline to 31.8% at endline, and the percentage who most important crop is maize decreased from 88.3% at baseline to 50% at endline. It should be noted that the target Producer Associations in the intervention group changed markedly from baseline to endline, as further detailed below in the section devoted to the indicator on the percentage of Producer Associations demonstrating increased organizational capacity. Table 10. Sample characteristics – Producer Associations Period Control districts Intervention districts Difference Number of Producer Associations Endline 10 22 12 Baseline 10 34 24 Most important crop (%) Maize Endline 60.0 50.0 -10.0 Baseline 60.0 88.3 28.3 Rice Endline 40.0 31.8 -8.2 Baseline 40.0 8.8 -31.2 Others (Soy) Endline 0.0 18.2 18.2 Baseline 0.0 2.9 2.9 2nd most important crop (%) None Endline 50.0 77.3 27.3 Baseline 60.0 20.6 -39.4 Maize Endline 40.0 18.2 -21.8 Baseline 10.0 2.9 -7.1 Rice Endline 0.0 22.7 22.7 Baseline 0.0 5.9 5.9 Cowpea Endline 10.0 0.0 -10.0 Baseline 0.0 2.9 2.9 Others Endline 30.0 67.7 37.7 Baseline 0.0 0.0 0.0 Mean number of members Endline 32.4 22.1 -10.3 Baseline 43.5 44.3 0.8 Mean number of female members Endline 7.7 15.9 8.2 Baseline 15.2 9.1 -6.1 32 Also noteworthy in Table 10 is the fact that over time, the mean number of female members increased noticeably in the intervention group, and decreased in the control group. Qualitative data supports the idea that that this phenomenon was due to changing dynamics in group product sale that was associated with the LRP project. A rice producer in Pèrèrè states: The LRP project has revolutionized the associative life in Pèrèrè. Prior to the project, many women left the association because they earned virtually nothing. But with the LRP project that now accepts our production, there are many who want to join our association. In Table 11, we show the description of the sample of individual producers interviewed (four individual producers per association). The proportion of female producers interviewed more than doubled (from 13.4% at baseline to 29.6% at endline), presumably reflecting the higher number of female members noted above. While the change in the intervention arm may be the result of the difference between producer associations interviewed at baseline (n=34) and at those covered at endline (n=22), the change in the control group is difficult to explain, suggesting a possible quality issue of self-reported data. Table 11. Sample characteristics - Producer Association individual members Period Control districts Intervention districts Difference Number of members interviewed Endline 40 88 48 Baseline 40 136 96 % of Female Endline 20.0 29.6 9.6 Baseline 27.5 13.4 -14.1 Crops produced last season (%) Maize Endline 92.5 78.4 -14.1 Baseline 85.0 94.9 9.9 Rice Endline 47.5 38.6 -8.9 Baseline 57.5 77.9 20.4 Cowpea Endline 22.5 27.3 4.8 Baseline 80.0 51.5 -28.5 Shea collection Endline 7.5 28.4 20.9 Baseline 80.0 74.3 -5.7 The percentage of farmers who produced each of the target crops declined from baseline to endline, suggesting that farmers tended to concentrate on few crops. The only exception is with maize in the control group (the proportion of farmers increased from 85.0% to 92.5%). 33 4.1.2. Organizational capacity Focus group discussions with Producers Associations provide insight into the composition of Producer Associations, which varies from one association to another. The executive office of these associations usually has 7 to 11 members. The most important positions are: the president (presides over all assemblies and ensures the implementation of the decisions resulting from these assemblies); the secretary (assists the president in the preparation of meetings, handles the secretariat during meetings and maintains the group documents); and the treasurer (keeps up to date the accounting and financial documents of the group). These three main members are co-signers of the group's bank account. Most CRS partner cooperatives had a high level of formalization, as the selection criteria for suppliers involved verification of their formal and physical existence. Most of them had statutory texts (statutes and internal regulations). While cooperatives are themselves responsible for and guarantee the execution of agricultural product delivery contracts obtained from CRS, the URP, as a regional umbrella organization, serves as an interface between the cooperatives and CRS, and is responsible for implementing training for the benefit of cooperatives and their members. As at baseline, a score for organizational capacity of Producer Associations was created from relevant questions in the surveys with Producer Associations’ management and individual members. We used the same variables as at baseline. Two sub-scores for Producer Association management and members were created by averaging the corresponding variables, and the overall score obtained by averaging the two sub-scores. The results are presented in Table 12. Overall, the LRP project seems to have had a noticeable effect on Producer Associations’ organizational capacity, with an impact estimated at 30 percentage points and statistically significant at the level of 1%. The magnitude of change was more pronounced from individual producers’ perspectives (+43.0 percentage points, significance level of 1%), than from association management’s perspectives (+17.0 percentage points, statistically non-significant at the level of 10%). Table 12. Producer Association organizational capacity scored (n=44 at baseline; n=32 at endline) Indicator Period Control districts Intervention districts Difference Difference in differences Overall Score Endline 27.5 41.5 14.0 30.0*** Baseline 32.0 16.0 -16.0 Score derived from Management Endline 24.0 31.0 7.0 17.0 Baseline 30.0 20.0 -10.0 Score derived from Members Endline 31.0 52.0 21.0 43.0*** 34 Indicator Period Control districts Intervention districts Difference Difference in differences Baseline 34.0 12.0 -22.0 For Difference in differences: *p<0.1; **p<0.05; ***p<0.01 To determine the percentage of Producer Associations demonstrating increased organizational capacity (the first indicator), the analysis was restricted to the sub-sample of Producer Associations interviewed at both baseline and endline. There were 10 (out of 10) in the control area, and eight (out of 22) in the intervention area. As shown in Table 13, 75% of Producer Associations in the intervention group demonstrated an increased organizational capacity from baseline to endline, against 50% in the control arm. Table 13. Percentage of Producer Associations demonstrating increased organizational capacity (n=18 at baseline and at endline) Indicator Control districts Intervention districts Difference Overall Score 50.0 75.0 25.0 Score derived from Management 50.0 62.5 12.5 Score derived from Members 50.0 75.0 25.0 An important aspect of Producer Associations’ organizational capacity is the extent to which their members sell their products through the associations. Table 14 shows the percentage of individual producers who sold their products through their association in the 12 months preceding the survey. Table 14. Sale of produce in the last 12 months through producer association (n=44 at baseline; n=32 at endline) Indicator Period Control districts Intervention districts Difference Difference in differences Sold produce through producer association Endline 45.1 64.5 19.5 33.4*** Baseline 37.5 23.5 -14.0 Satisfied with the sale Endline 95.3 98.7 3.4 -0.1 Baseline 93.3 96.9 3.6 For Difference in differences: *p<0.1; **p<0.05; ***p<0.01 35 While at the baseline the percentage was 14 percentage points lower in the intervention group, at endline it was 19.5 percentage points higher, for an impact estimate off 33.4, which is statistically significant at the level of 1%. For those who sold their products through their associations, the level of satisfaction did not seem to vary over time or across the two study arms. Data collected from interviews and focus group discussions reinforces that the LRP project has had a great impact on the organizational capacity of Producer Associations. It is with the LRP project that some cooperatives were remobilized as leaders were able to present the interests of group sales in general and in the framework of the LRP project in particular. During a FGD, a member of an association of maize producers in Tchaourou states: "With the LRP project, we were able to showcase our warehouse of warrantage which was not used. The CRS allowed us to use our warrantage stores, which reorganizes the sale of our products". By providing an opportunity for the supply of agricultural products to school canteens, the LRP project has reinvigorated the collective activities of cooperatives. Qualitative data also indicates that with the LRP project, cooperatives have become the platform for regular meetings, sharing of information and exchange of experiences, in accordance with existing statutes and internal regulations. This is confirmed by the words of a producer of maize, during a FGD in Tchaourou: "Our grouping in association also facilitates the sharing of information, experiences and allows us to benefit from training". 4.1.3 Use of improved production techniques Another important indicator for Producer Associations is the percentage of producers who demonstrate the use of improved production techniques. Individual producers were asked: Do you think that the LRP project has enabled you to acquire technical and management skills for your farm? (Q20); If yes: What capacity did you acquire? What training did you receive? (Q21) Which of the trainings received did you put into practice? (Q22) The results are presented in Table 15. It shows that 17 individual producers (78.7%) received training on production practices, and out of those, 15 (84.5%) translated what they learned into practice. Results also show that among associations interviewed, 64.2% have received training on harvest practices; 73.9% on post-harvest practices; and 39% on storage practices. For these trainings, respectively 59.3%, 76.3% and 75.6% have translated what they learned into practices. Table 15. Percentage of Producers who demonstrate the use of improved production techniques (n=22) Training received through LRP on production techniques Production practices 78.7 Harvest practices 64.2 Post-harvest practices 73.9 Storage practices 39.0 36 Training received through LRP on production techniques that were put in practice Production practices 84.5 Harvest practices 59.3 Post-harvest practices 76.3 Storage practices 75.6 As indicated in the quantitative data, not all individual producers were involved in LRP training. Interviews and focus group discussions explain that while the LRP program intended to only train leaders of Producer Associations, members of Producer Associations who participated in the FGD in N’Dali who were not involved in the trained would have like to have received the training information as well. Qualitative data from all communes but Parakou further explains the nature of the trainings received by Producer Associations as part of the LRP project. In Tchaourou, these trainings mainly focused on: agricultural techniques, group sales, market access and contract farming. In these trainings producers learned new techniques for producing, storing and marketing products. A rice producer says during a FGD: Apart from the marketing of our products, we have benefited from several trainings that make us experienced producers, so that we can now deal with other partners. Indeed, we benefited from LRP; the food storage techniques we hadn’t seen before. We are able to do annual conservation without insecticide. We did not know how to do it. We now know how to do group sale and contractualization. This is the case of soy that we sold together this year while we are corn producers in the project. In Pèrèrè, the main training themes for Producer Associations included: agricultural production and processing techniques, the market information system (MIS), climate change, the production of good quality butter, hygiene, the constitution of tender documents and the implementation of a contract. Apart from training on the constitution of tender documents, members of these associations indicated that these topics were review for them, as they had completed previous training on the same themes from other projects. However, with the LRP project, producers in Pèrèrè have been able to access certain markets for the sale of their products, and producers perceive the trainings to contribute to the advertisement of their products. Two of three of associations in N’Dali (a rice and maize Producer Association), on the other hand, criticized the selection of training modules provided by the project. This is apparent from the following statement from a maize producer: We have requested training on the management of financial resources. We also emphasized modules such as entrepreneurship and the business plan. But instead of all that, we will be trained on climate change and the keeping of administrative documents. 37 These are trainings that we have already benefited from and many times. So, in terms of training, it's not too good. Therefore, CRS’ approach to rely mainly on ProAgri and PASDeR (without involving farmers themselves) in order to gauge the need for training of farmers may have limited potential training impact. Despite these criticisms, qualitative data further indicates that knowledge from the training received is implemented. For example, the requirement of laboratory analysis by project managers before buying a product has reinforced application of knowledge gained during trainings. One participant of a focus group discussion with producers states: Indeed, the project takes a sample of each product to analyze at the laboratory in Cotonou before to authorize the purchase. This has meant that each cooperative or association is committed to producing better quality by applying the lessons learned during the training courses. 4.1.4. Use of URP MIS The third indicator for Producer Associations concerns the use of URP's Market information system (MIS). Table 16 shows that the percentage of Producer Associations who used the system remained unchanged over time in the control group (at 30%), but increased from 11.8% to 31.8% in the intervention group. However, the resulting impact estimate of 20 percentage points is not statistically significant at the level of 10%. In the intervention arm, the use of the system to monitor local prices increased over time slightly from 75.0% to 85.7%. Table 16. Producer Associations’ use of URP's Market information system (n=44 at baseline; n=32 at endline) Period Control districts Intervention districts Difference Difference in differences Used URP MIS Endline 30.0 31.8 1.8 20.0 Baseline 30.0 11.8 -18.2 Purpose of using the URP Producer prices Endline 33.3 14.3 -19.0 -10.7 Baseline 33.3 25.0 -8.3 Local market prices Endline 66.7 85.7 19.0 10.7 Baseline 66.7 75.0 8.3 For Difference in differences: *p<0.1; **p<0.05; ***p<0.01 38 4.1.5. Relevance, efficiency and sustainability Relevance of the LRP project for Producer Associations The LRP project with its local procurement principle is undoubtedly a positive market outlet for their agricultural production. In Tchaourou, a maize producer says: "Our products flow quickly and in a grouped way". In Pèrèrè, a producer of shea butter declares: "As an association we have facilities to sell our products in bulk, we do not walk anymore in the villages or in the markets for the sale of shea butter in detail”. In particular, for these women producing shea butter, the benefits are noticeable, since they previously lacked opportunities to access larger markets. A leader of the umbrella organization states: Associations affiliated to the AKB are struggling to find a sustainable market for their products, particularly shea butter. The LRP in its approach comes to offer this marketing market of shea butter in a significant amount. The project is therefore an opportunity that women in the group seize to produce and sell with an assurance of profitability. Producers Organization umbrellas and Producer Associations are satisfied with recognition and use of local producers in school feeding. But because of delays in purchase of products and payment by CRS, the level of satisfaction remains partial. Also, the expected economic benefits are slow to be perceptible. Reducing the chain of sales or streamlining the process will allow producers to get back their income from selling products quickly. Efficiency of Producer Association processes Overall, the most important objectives of the Producer Associations were achieved. 100% of all purchases of agricultural products described in the contracts were made. When the evaluation survey took place, there were still some trainings that the URP had to conduct, but according to CRS, these activities remained well within the original timeframe set. In relation to the efficiency of the project, the local approach to the supply of agricultural products for school canteens bears witness to the efficiency of the LRP project. Most other school canteens obtain supplies on the regional or international market, and generally use imported products. The LRP participation models values contributions from the local community as well as the minimization of costs. To ensure the minimization of costs while also keeping commodities local, the LRP model is supported by in-kind contributions from local communities (e.g. firewood, donation of site for the construction of shops, sand picking, etc.) and financial contributions from parents. Furthermore, the LRP project designed a specific training program for producers, which teaches new techniques for growing, harvesting, preserving and selling agricultural products that allow producers to have better yields. Similarly, training to respond to calls for tender and support in group sale was also received. One producer in Tchaourou states: “Training is very useful because 39 it has made it possible to know how to keep food without insecticides in the long term. They have allowed us to sell our soybeans using grouped sale". However, there are some aspects that reduce the efficiency of the local approach. For example, some cooperatives (especially associations in the agricultural product processing sector) that have won contracts are financially weak in producing the expected quantities on time and of quality. Cooperatives that have won market contracts can find it difficult to produce the expected quantities on time and under conditions that guarantee quality in accordance with the standards required by the project. A URPR member in Parakou states: The CRS in its first tender asked for three tons of parboiled rice from a selected cooperative. In order to deliver this demanded quantity, processing purpose requires 5 tons of paddy rice at a cost of one to two million CFA francs [$3,418.21 USD]. All cooperatives do not have access to credit; CRS does not pay in advance nor does it facilitate access to credit to fulfill the supply contract. In addition, rice processing is traditional in cooperatives and to release 3 tons of rice it takes a minimum of one month. After production, a sampling and a laboratory analysis are required, the result of which, if it is good, authorizes the cooperatives to deliver the product. The lack of equipment support has also had a negative impact on the efficiency of production units in rice processing on time. Furthermore, administrative delays in the signing of contracts, the execution of activities, the purchase and delivery of products impacts efficiency. Relatively weak capacities of Producer Associations in drafting offers also inhibits efficiency of the project. It is important to note that farmers are affected by the delays in the purchase process under the LRP project. For example, a maize producer in N'Dali states: There is some slowness in the purchase of products, the period between the announcement of the purchase and the purchase itself is very long. As a result, the corn that was once winnowed is left with waste and it is necessary to sort again before the purchase five to six weeks later. All of this is huge losses for the producer. There is also slowness not only at the level of purchases but also at the level of payment. Conversations with producers also indicate frustrations by producers with the time it takes for them to get paid after invoicing. After the delivery, it can take several months before invoices are paid. This delay in payment prevents cooperatives from filling other contracts because of lack of available finances. A focus group discussion participant in Pèrèrè states: “The members of the associations contracted credit for the production at the CLCAM. But the money of the products, was paid much later, while on the other hand, the deadlines with the CLCAM do not wait. This causes a lot of damage to the producers”. Producers therefore had difficulties mastering the contracting process and were unable to effectively plan due to changes in market prices. 40 Sustainability of Producer Association participation With a specific training program for producers under the LRP project, producers have increased their capacity to respond to project requirements and needs and have become more professional and self-reliant. These acquired capacities contribute to sustainability of the achievements of the LRP project that could also be used for any other form of institutional sale. A producer from a focus group in Pèrèrè states: The producers are today able to supply the products to other canteen projects or in the institutional market led by WFP. Similarly, there are other associations working to join the group of associations that deliver products to the LRP project. And all this will only increase the delivery capacity of agricultural products according to the demand of the wholesale market or institutional market which will challenge the associations of producers. Another maize producer in Tchaourou indicates the obvious interest of the producers for the perpetuation of such an approach: "The producers are very interested in the project and campaign for its sustainability which constitutes an opportunity for the sustainability of the project. We want this partnership to grow and multiply". Producer Associations (especially maize and rice) believe they can use experiences acquired with the LRP project in other similar initiatives or institutional purchases. Although producers of shea butter do not yet seem able to continue with the activities of the project without support, as these producers needed a lot of help from the AKB to comply with the project procedure. In moving forward, to ensure sustainability of the project among Producer Associations, it is necessary to develop contractual models which allow the associations of producers to find the necessary financial resources before and during the sales process to guarantee their compliance with the contractual obligations. With this support, URP believe it has enough experience that allow it to retain the achievements of the LRP project as well as respond to similar solicitations. But, on the other hand, Producer Associations do not yet seem to have the capacities and resources needed to continue project activities without any outside support. For example, in Pèrèrè, the shea butter Producer Associations relied on support from their umbrella organization (i.e., AKB) to complete their first group sales contract, and seeing that they only have one group sale under their belt, they feel that they don’t yet have sufficient experience in the process to complete it on their own. External support is still needed to mobilize resources to further support this process. According to the AKB, the weak financial capacity that characterizes Producers Associations, as well as their poor organization, do not guarantee the sustainability of this type of project. In order to remedy this situation, it would be important to train the AKB on the project account, taking into account the related constraints. The URPR also recognizes that it has difficulties in structuring its many members in such a way that gives them the right to structured markets. The few cooperatives that have formalized themselves do not have the financial base 41 for services that meet the requirements of the LRP project. Difficulties in accessing bank credit and lack of pre-financing or start-up loans weaken Producer Associations, thus compromising the sustainability of the project. It would be important to consider contractual models that allow Producer Associations to find the necessary financial resources before and during the sales process to ensure their effectiveness in complying with the contractual obligations. The URP, on the other hand, believes that it has enough experience that allows it to perpetuate the achievements of the LRP project. Its aspiration for sustainability is to renew this project and to extend its geographical reach beyond the current four municipalities and continually improve the activities of the project. Producer Associations members also indicate that, given that beneficiaries would be unable to perpetuate project initiatives without external support, continuation of CRS support is necessary. The LRP project is a source of motivation for producers who now have easy access to the markets of agricultural producers, and for school children who feel more comfortable in school. An abrupt stoppage of the project may provoke a demotivation of beneficiaries and the communities will again face the problems that the project seeks to solve. Respondents noted that stakeholders still need additional time and resources to be able to ensure the sustainability of project initiatives. This is clear from the words of a producer in N'Dali: "In order for parents' associations to be able to take over, we need a gradual apprenticeship for at least five years, as was the case with mutual health insurance companies". 4.2. COGES 4.2.1. Sample characteristics Baseline data was collected at the beginning of the 2018-2019 school year and the end line data at the end of the same school year. Out of the 35 control schools surveyed, only three ever had a canteen, and consequently a COGES which is directly tied to the presence of a canteen. Among the intervention districts, only five intervention schools had ever had a school canteen, and out of these, four ever had a COGES. For this reason, the analyses for this section will focus on the intervention schools at endline, as well as the 35 cooks and the 35 store keepers interviewed. Table 17 shows that cooks were all female, while store keepers were largely male. Table 17. Sample characteristics - Schools and COGES (n=70) Period Control districts Intervention districts Number of schools Endline 35 35 Baseline 35 35 Mean school size (number of students)a Endline 191.7 146.3 Baseline 260.4 224.1 Endline 0 35 42 Period Control districts Intervention districts Number of schools that have a Canteen Baseline 0 0 Number of schools that ever had a Canteen Endline 3 5 Baseline 3 5 Number of schools that have/had a COGES Endline 3 35 Baseline 3 4 Number of Cooks at endline Total 0 143 Female 0 143 Number of Store keepers at endline Total 0 35 Male 0 32 a Baseline values are for grade 1-6, while endline values are for grade 2-6. This largely explains the observed drop over time. 4.2.2. Capacity in school canteen and stock management An important factor likely to precede (and perhaps influence) COGES’ practices related to canteen and stock management is the capacity and training of cooks and store keepers. This factor was measured through the following questions: • Q34-35 in the Cooks questionnaire: In your opinion, did the LRP project help you acquire cooking capacity? If so, which capacity did you acquire? which training did you receive? • Q24-25 in the store keepers questionnaire: In your opinion, did the LRP project help you acquire store keeping capacity? If so, which capacity did you acquire? which training did you receive? Table 18 shows that a large majority of the 35 cooks and the 35 store keepers reported having acquired capacity in various aspects of their work. Table 18. Capacity and training of cooks and store keepers (n=35) LRP training received Capacity acquired (among those trained) Cooks Cooking food 96.5 100.0 Hygiene 96.5 100.0 Nutritious foods 88.0 67.6 Planning of canteen activities 90.7 77.1 Ration measurement 77.9 70.4 Monitoring of commodities 84.5 80.5 Store keepers 43 LRP training received Capacity acquired (among those trained) Stock management 93.6 79.0 Hygiene 91.9 89.4 Storage of commodities 88.9 87.6 Interviews with COGES members in all of the intervention communes confirm that COGES members received training under the LRP project. A series of two trainings were organized in the selected sites in the respective communes. The training focused mainly on stock management, the measurement of the quantities to be served, and the operating rules of the COGES for the canteen functioning. COGES members received specific training according to their role (cook, storekeeper, president or teacher), and the COGES members were able to apply the learning outcomes, each according to the roles and tasks assigned to them. Training was conducted in the local language, facilitating the mastery of the concepts taught to the participants. In addition, all COGES members were invited to all of the training workshops, allowing participants to learn about each other's roles. During a focus group, a COGES member states: “With the LRP project, we all [the other COGES members of his school and him], were at this training. This allowed us to master these concepts well. In addition, these training were taught in the local language, it was accessible even for us farmers”. In Table 19, we present the score of COGES in canteen and stock management, and the percentage of COGES who achieved 80% score or higher (the first indicator). The following three sub-scores were computed: • Students hygiene, computed from Q71-74 of COGES questionnaire • Meal service, computed from Q75-75.5 of COGES questionnaire • Store management practices, computed from Q12-18 of Store keeper questionnaire The overall score, calculated by averaging the three sub-scores, was estimated at 77.3, with higher value on student hygiene (85.2), and lower value on the sub-score for cooks. In total, close to 50% of COGES achieved a score of 80 or higher (out of 100). Table 19. COGES scores in canteen and stock management (n=35) Score % Achieving 80% score Overall Score 77.3 49.8 Student hygiene 85.2 97.0 Score derived from Cooks 68.0 69.5 Score derived from store keepers 78.7 71.7 44 In practice, COGES members are dedicated to providing good services in school children feeding. In preparation for this service, meetings are held to decide on recipe choices, purchase of condiments, timing in meal preparation, etc. In general, COGES plans and executes a weekly organization schedule for the canteen. In Tchaourou, a COGES member reported: "A calendar is developed weekly for the meal menu. It's the whole office that meets every Friday to do it. This is done according to the food available in the store’’. The role of teachers in canteen management is quite remarkable. Teachers support COGES members such as the storekeeper, the president or even the cooks to evaluate the quantity of products to be used and meals to serve to students. In addition to supporting COGES members, teachers mobilize and organize students as they get in line for lunch in the canteen. The storekeepers have also been able to improve their inventory management capacity in order to guarantee the good quality of the products kept. In N'Dali, during a FGD, a storekeeper reports being congratulated by the project team during a monitoring field visit: "Last time, the project people said that we are among those who have maintained the store". At Pèrèrè, it was reported that "many improvements are noted in the filling of management books, and the conservation of stocks". In Tchaourou, it is evident that the storekeeper is proactive in managing stock to avoid stockouts. The storekeeper described his strategy stating: "When our stock tends to finish, we send a request for refueling well before it finishes". Furthermore, LRP project activities also included the installation of handwashing devices in the intervention schools, contributing to the high score in student hygiene. The project has purchased a total of 3,012 five-liter cans and 3,280 meters of rope cut on a one-meter basis to make up the tippy tap system, on behalf of the project's intervention schools. The project then led school canteen management committees to install tippy taps in all intervention schools of the project. The school management committees were trained on-site and assisted in the installation of this system. The implementation of the canteen reinforced the use of these devices as it is recommended that every student wash their hands before receiving a meal from the canteen. 4.2.3. Organizational capacity Three sets of questions in COGES questionnaire were designed to capture three different aspects of COGES’ organizational capacity (the evaluation’s second indicator). These are: • Rules and regulations (Q60-64) • Monitoring of activities (Q65-67) • Resource mobilization (Q68-69) Three corresponding sub-scores were computed, and the overall score was calculated by averaging the sub-scores. As indicated previously, the main score and the three sub-scores range from 0 to 100. The results in Table 20 show that 36% of COGES achieved a score of 80 or higher. 45 The overall score was 62.1. COGES scored higher on resource mobilization (73.5 score; 65.4 of COGES achieving 80 or higher), and very low on activity monitoring (43.0 score; 14.3% of COGES achieving 80 or higher). Table 20. COGES organizational capacity score (n=35) Score % Achieving 80% score Overall Score 62.1 36.0 Rules and regulations score 69.9 46.2 Activities monitoring score 43.0 14.3 Resource mobilization score 73.5 65.4 Interviews and focus group discussions with COGES members elucidate the organizational capacity of COGES. In general, the COGES is composed of five members: 1) a president, 2) a storekeeper, 3) an accountant, 4) an advisor who is the principal or one of the school principals for school complexes, and 5) one of the school teachers who serves as the monitoring-evaluator. The president, storekeeper and accountant can be members of the Parents Associations (e.g. some schools sampled in the commune of Parakou such as Dabou), or parents that are not members of Parents Associations, but are selected within the community (e.g. sampled schools in the commune of N'Dali, and in Parakou). In Dabou (Parakou), COGES members unanimously agreed to choose a woman as president. The reasons were, among other things, that "cooking and feeding children is a women's affair. She knows what is good for children…, and as a mother will also be able to assess the needs and aspirations of everyone in terms of food to serve”. All of the cooks or their representatives are also considered members of the COGES. With regard to the five main members mentioned above, apart from the advisor and the teachers, the configuration of the committee varies in terms of the provenance of the members. In addition to the criteria recommended by CRS for the selection of COGES members, some criteria are valued at the local level, including availability, ability to collaborate in decision￾making, and prioritization of COGES work. In general, COGES has operated according to statutory texts which define the attributions and roles of the members. At Pèrèrè, COGES members have demonstrated a knowledge of their roles by stating them on demand. Members meet regularly as needed. In all the intervention communes, the communication between the members of the COGES is primarily through telephone calls. However, in Parakou and N'Dali for example, it appears that the storekeeper and the president are generally more active with in-person communication; demonstrating a greater commitment to the school canteen. Regarding the monitoring of food management and the performance of the canteen, a monitoring committee is set up to allow and ensure all students are served in an orderly fashion. This committee is composed of three grade six students, a cook and a teacher. Another level of 46 monitoring is associated with monthly meetings between the storekeeper, the monitoring officer (the teacher member of COGES) and the project managers. For transparency in the use of food, the president or the storekeeper cannot open the store alone without the permission of the other. COGES members have also demonstrated a good capacity for community mobilization of resources for the purchase of condiments for sauce. Even if some contributions for condiments are requested from students, they are not obliged to pay for them before eating. In Tchaourou, the contributions for condiments come from the Parents Associations committee in schools as expressed in these terms: "We request the Parents Associations committee to help us in the event of lack of resources, because there is a reciprocal agreement between the COGES members and the Parents Associations committee". Contributions for condiments also come from local and traditional authorities as expressed by a respondent: "We went to see the local authorities who supported us in cash, even the king gave us money to support the LRP canteen initiative". In Pèrèrè, if further resources are needed (for example financial contributions), a meeting of all the student parents of the village is convened and a favorable solution is generally found for the good functioning of the canteen. 4.2.4. Application of project promoted recipes The indicator on the percentage of school cooks demonstrating the application of project promoted recipes was not calculated quantitatively. Rather, cooks were asked by qualitative interviewers to list their menus based on recall; a method that could be improved in the future by requiring that cooks record and save all menus and ingredients used for each meal. As confirmed through qualitative data, the LRP model using local products for school meals was adhered to throughout the project. The use of local products is very much appreciated by multiple stakeholders. Using the local products reduces doubts on quality and security of products. During a focus group, a COGES member states: “it is a relief because we no longer have to doubt the quality of food, unlike imports food; for imports food, we do not always know their provenance and do not have means to guarantee their storage conditions”. During the weekly menu planning, COGES members ensure that the meals recommended by the LRP project are highlighted. However, complaints are increasingly raised concerning use of shea butter exclusively in school canteens. And while some COGES (for example in Tchaourou) have provided tofu or fish for some school meals, complaints about lack of protein-based foods like meat or fish have been voiced. 4.2.5. Good practices in terms of hygiene in food preparation Cooks were observed to measure COGES’ practices in terms of hygiene in food preparation in the six following areas: • Body and clothing hygiene (Q1-7) • Food hygiene (Q8-12) 47 • Dishes (Q13-17) • Food cooking (Q18-23) • Food conservation and service (Q24-27) • Store and environmental hygiene (Q28-33) The observation could not take place in one school (in Tchaourou), because the canteen was closed due to stock out in commodities. The overall score was obtained as a simple average of the six sub-scores listed above, and the percentage of COGES that achieved a score of 80% or higher was derived. As shown in Table 21, close to half of COGES achieved a score of 80 or higher, with higher values in food hygiene, followed by store and environmental hygiene, and lower values on body and clothing hygiene. Table 21. Good practices in terms of hygiene in food preparation (n=34) Score % Achieving 80% score Overall Score 80.1 49.8 Body and clothing hygiene 62.7 12.8 Food hygiene 90.3 91.3 Dishes 67.2 48.9 Food cooking 85.3 54.8 Food conservation and service 87.8 53.8 Store and environmental hygiene 87.5 80.4 Qualitative data reinforces how, by having the cooks participate in multiple training courses, including those unrelated to their direct work, mutual monitoring of rule compliance is possible. If bad practices arise, the other members of the COGES can question the person concerned. Thus, the collaborative environment of all COGES members underlies the improvement of the attitude and the capacity of the cooks in their service for the school canteen. 4.2.6. Quality and quantity of available food While it was agreed to qualitatively measure timeliness, quantity and quality of commodities delivered for school canteens, the following relevant questions were included in the COGES questionnaire: • Do you receive on time commodities for the school canteen? (Q57) • Are the quantities received adequate to cover the needs of the canteen? (Q58) • Are the commodities received of good quality? (Q59) 48 As shown in Table 22, almost all COGES reported a level of satisfaction with the quality of commodities received, and close to 90% reported the commodities were delivered on time and with adequate quantities. Table 22. COGES report on delivery of commodities for school canteen (n=35) % Timely delivery of commodities 88.6 Adequacy of quantities 90.4 Quality of commodities 98.1 Discussions with stakeholders help to clarify the quality and quantity of food delivered. As indicated in interviews and focus group discussions, contracts with producers' cooperatives specifies the schools in the intervention communes and their location, with specification on the types of agricultural products to deliver. In addition, the project field staff are also available to assist producers in navigating the delivery process. This promotes timely delivery and avoids stockouts in school stores. Laboratory control in the process of purchasing products ensures the receipt of good quality products. The quantities received are also sufficient to cover the needs of all students according to COGES members. Two strategies have been developed to avoid or manage stockouts. These include: 1) proactively informing the project managers of products that are expected to be finished in order to restock the product before more is needed, and 2) the loan approach that consists of the provision of products from peers (other LRP canteen in the same locality) and reimbursement after provisioning from LRP. Furthermore, the LRP project contributes to improved quality control of school meals. A producer in Pèrèrè states: The project gives canteens to students and the canteen of this project is different from that of other projects. With the LRP project, it is not possible to provide children with what is not good. Even our shea butter is sent to the laboratory and if it is of poor quality, we return that and then we cannot serve that to children. The LRP project is therefore able to feed schoolchildren with local products that are healthy and of good quality. 4.2.7. Direct observations Direct observations help to indicate how well the canteen kitchens are equipped with the materials needed to provide quality meals to school children. Table 23 indicates this kitchen inventory. As shown in Table 23, none of the 35 sampled LRP schools had refrigerators, and 28 of the kitchen areas were in the open air. Furthermore, 28 of the 35 schools did not have a 49 cafeteria for students. It should be noted that refrigerators, enclosed kitchens and cafeterias are not required by the project, or by the government concerning school canteens for that matter. This data rather describes the conditions in which the meals were prepared and served. It is important to note that 31 schools of the 35 schools observed had hand washing places for students. Table 23. Kitchen inventory in the sampled LRP schools (n=35) Number % weighted Location of the kitchen Within school compound 31 91.0 Outside school compound 4 9.0 Type of kitchen area Open air 28 78.1 Tent/shed/hangar 6 20.0 Well/finished 1 1.9 Type of cooking stove used Traditional cooking stove (stone) 32 93.5 Metal cooking stove 3 6.6 Number of cooking stove used 1 cooking stove 5 15.1 2 cooking stoves 17 54.6 3 cooking stoves 11 27.3 4 cooking stoves 2 3.1 Presence of a refrigerator Yes 0 0.0 No 35 100.0 Existence of refectory/cafeteria for students Yes 7 14.8 No 28 85.2 Existence of handwashing place for students Tap 31 93.4 Hand washing system (Dispositif de lavage des mains 8 28.5 Table 24 below describes the construction of and location of the stores of the 35 sampled LRP schools, along with the food stored at each location. As shown, the majority of the stores had brick walls or cement floors with sheet metal or tile roofs. The majority had maize, pulses, rice and shea butter in store. 50 Table 24. Store inventory in the sampled LRP schools (n=35) Number % weighted Location Outside school compound - in a community store 2 4.0 Outside school compound - at home 12 41.0 Inside - in a classroom 8 14.5 Inside - in the principal's office 13 40.6 Type of wall Clay 6 26.0 Brick 27 68.6 Sheet metal and other 2 5.5 Type of floor Ground 5 22.3 Cement 29 74.6 Tile 1 3.2 Type of roof Sheet metal/Roof tile 32 93.1 Slab/No roof 3 6.9 Food in store Maize 33 91.1 Pulses 33 95.2 Rice 32 90.6 Shea butter 26 80.4 It should be noted that the construction of the stores had not yet been started at the time of the evaluation. Several stakeholders interviewed in the field reported the non-construction of stores and kitchens as an unfulfilled expectation or promise held by the project. A member of COGES in Pèrèrè states: "At startup, the project promised the construction in each school, storage store and kitchen. This expectation has remained unsatisfied until now. The sites to house the shops and kitchens were donated. The sand is even picked up”. There were several reasons for the delay in construction of the kitchens. Firstly, the responsibility of kitchen constructions was left with the community, but when assessments indicated that this would not be possible, savings were identified in the project budget, which was realigned to include kitchen and warehouse construction. Secondly, the initial plans did not include the expertise of an engineer, and once it was decided that this expertise was needed, accommodations were made, albeit with a delay. Lastly, given the project commitments and work load, the procurement of local commodities was prioritized over the construction activities. 51 Despite these delays, CRS management indicates that everything is now ready for construction to start in schools: With the inadequacy of the budget line provided to cover the actual costs of the construction of stores and kitchens, we must go back to the lessor each time to agree on the new amounts. Subsequently, his physical agreement was obtained. Currently, the contract with the staff is signed and the start order is given, pending the final agreement with the finalization of the administrative documents. Thus, considering that the arrangements are made at the level of the CRS to carry out this important activity, we can likely say that the LRP Project is in the process of achieving 100% in terms of the achievements of the activities. 4.2.8. Satisfaction The COGES are satisfied with their role in the project and the local supply approach. The relative independence of COGES from existing structures in schools such as EPAs and the representativeness of community members gives it legitimacy within the community, and COGES members are committed to the success of the school canteen. While the satisfaction among students remains high, difficulties have been raised at the school level. In some villages, there is a lack of motivation of the cooks in the canteen, and there is a lack of supplies or condiments due to insufficient contributions. Taking action against these difficulties or adapting certain principles of the approach will only improve its relevance on these points. 4.2.9. Lessons learned and sustainability Qualitative data indicates lessons learned from the LRP school canteen experience. It was noted that local agricultural products serve well in the school canteen, as is reflected in the statements of COGES member in Tchaourou: "We always thought that it is all that comes from outside that is well. But with the LRP, we realized that our local products are better compared to what is imported to us. Local consumption has become a reality for us". Aside from benefiting students, the use of local products in school canteens also provides a financial gain for producers. Another lesson learned is that the more actors involved in the COGES (e.g. teachers, parents of students, Directors), the greater the transparency and improved operation of the canteen. Moreover, the opening and the increase of the number of members in the COGES to involve local elected representatives and other officials at various levels of the community structures were noted during the focus groups. Throughout the project there were several examples of actors outside of the COGES who served as integral contributors to the school canteen, as was the case in Tchaourou where the king contributed financially. Stakeholders therefore suggested that COGES be expanded to ensure a diversity of sources of support for the school canteen, and consequently a greater improvement in the quality of the meals that will be served to students. One COGES member states: "The COGES must be expanded to all the structures that exist in the 52 villages and especially increase the number of its members. It is the participation of all that will need to expand the sources of funding of this canteen and participation in kind". In the light of these lessons learned, several schemes for organizing school canteens based on local products are proposed by the interviewees. A gradual withdrawal of CRS is considered appropriate so that the COGES can be well prepared to manage school canteens (resource mobilization, canteen services and canteen and stock management). For autonomous operation of the school canteen without support from CRS, some have recommended that local farmers who are parents of students supply the food bank or school stores under management of COGES. This should involve a substantial reduction of any request for financial contribution to parents of students. Others propose initiating collective school fields that students or their parents will cultivate for the benefit of the school canteen. It should be noted that the first option is considered stronger than the second. A COGES member of EPP Dabou involved in the Parakou focus group states: It could be easier to ask the farmers for food than to ask them for money or tell them to go work in another field. After the assessment of our needs in staple foods/products or agricultural products, we will define the share to be given by each farmer. The condiments will be managed by the women of the village. In terms of sustainability, most of the COGES members interviewed in all of the LRP communes felt that continuation of the LRP project would be a good thing for the entire community. One stakeholder in N’Dali states: "The renewal of this canteen is a necessity for parents, for teachers as well as for students". Another in Pèrèrè says: "It is very important to renew the project given the many benefits it provides to students, students’ parents and school’’. Members of COGES expressed the need for additional time and resources to be able to sustain project initiatives. This is clear from the following statements from Student Parents FGD in N’Dali: "For the student parents’ associations to take over, requires a gradual learning over at least five years as was the case with mutual health". Parents of students also expressed concern in continuing to make payments for school meals, especially given the many other problems whose resolution requires additional efforts on the part of parents. This is reflected in a statement from a member of the Parents' Association in Parakou: Every year parents must be solicited through subscriptions to recruit community teacher deficit of teachers made available by the State. In these conditions other subscriptions are often poorly perceived by parents of students. This is the case with the contributions requested from parents to ensure the feeding of their children in the canteen. Despite the acceptable cost, parents often say they are out of breath to insure these contributions. This compromises the sustainability of the project which may stop as soon as the bully withdraws. Another member of the Parents’ Association in Tourou states: 53 The parents are already too much solicited in various subscriptions to make work the school by ensuring the effectiveness of the courses to through the recruitment of teachers. So, the contributions for a canteen entirely in charge of the school are not possible. The evidence despite the external contribution that reduces the cost of meals, some parents do not give. The height is that the APE office has been removed from the management of the canteen; how will we convince parents of the need for an additional contribution to the canteen? Some beneficiaries consider that the project was only at an experimental stage. Therefore, it is important that it be implemented over a longer period than that planned to consolidate learning. A parent of says: It is necessary to continue because, despite these limitations, it has been useful for many layers of society. People have benefited, even the project leaders have won a job. It's also our children. The project must continue; we have already done the most difficult, the experimental phase To ensure the continuity of activities, CRS must gradually withdraw in order to give time to COGES to consolidates their capacities of canteen management (resource mobilization, stock management and management of canteen services). Similarly, sustainability will require a strong participation and community contribution. The following proposals were made by stakeholders: - Implement collective school fields whose harvests will be used to supply school canteens. For that, a proposal must be made to the government. - Promote in-kind participation of student parents to supply the school canteens with the products of their own harvest. - Involve community and communal leaders in the management of school canteens. For example, they can provide a budget line for women cooks and storekeepers, or for the canteen supply with condiments. - Advocate with partners other than CRS for support. 4.3. Students 4.3.1. Sample characteristics As shown in Table 25, a total of 3,216 students were surveyed at endline (1,668 in the control group and 1,548 in the intervention group). While at baseline a slightly higher majority of students were girls, at endline, a slight majority were boys. 54 Table 25. Sample characteristics - Students surveyed Period Control schools Intervention schools Average number of students surveyed per school Endline 47.7 44.2 Baseline 28.5 28.3 Number of students surveyed Endline 1,668 1,548 Baseline 988 1,001 % Female students Endline 49.4 48.1 Baseline 51.9 51.5 By grade (%) CP (Grade 2) Endline 26.9 27.1 Baseline 21.7 21.8 CE1 Endline 24.8 22.3 Baseline 20.2 24.4 CE2 Endline 20.7 21.6 Baseline 21.5 21.6 CM1 Endline 17.1 18.2 Baseline 20.1 17.6 CM2 (Grade 6) Endline 10.5 10.8 Baseline 16.5 14.7 By socioeconomic status (%) Poor Endline 33.8 36.0 Baseline 32.5 32.7 Middle Endline 30.2 35.2 Baseline 31.2 35.2 Rich Endline 36.0 28.8 Baseline 36.3 32.2 4.3.2. Food consumption patterns This section presents the top 10 foods cited by the students, in response to the following question: “Now I would like to ask you about meals you ate yesterday, during the day or at night. What meals did you eat yesterday? Did you eat this meal at home, at school, somewhere else? [FOR EACH MEAL, ASK SUBSEQUENT QUESTIONS ABOUT MEAL COMPOSITION AND LOCATION]”. The first step to process this open-ended question consisted of harmonizing manually the names of meals cited by students (they used different names for the same meal). Each reported meal was then multiplied by the number of times it was mentioned, whether it was eaten at home, at school or somewhere else. The results are shown in Table 26. 55 Table 26. Top 10 meals consumed by students based on one day recall Foods Cited Period Control schools Intervention schools Difference 1. Pate (Made of maize, sorghum, yam, cassava, rice) Endline 38.7 34.4 -4.3 Baseline 41.4 43.9 2.5 2. Boiled Rice Endline 14.2 13.5 -0.7 Baseline 13.7 12.9 -0.8 3. Mango Endline 7.6 8.2 0.6 Baseline 11.8 13.7 1.9 4. Porridge (Made of maize, sorghum, millet) Endline 7.3 10.4 3.1 Baseline 6.6 9.8 3.2 5. Boiled Yam (Pounded or not) Endline 4.3 3.4 -0.9 Baseline 4.0 5.9 1.9 6. Macaroni Endline 3.9 1.8 -2.1 Baseline 4.2 2.2 -2.0 7. Wake (Made of rice and cowpea) Endline 6.6 11.1 4.5 Baseline 3.8 1.2 -2.6 8. Beignet (Made of cowpea, maize, wheat) Endline 4.3 2.8 -1.5 Baseline 2.1 2.4 0.3 9. Orange Endline 0.5 0.4 -0.1 Baseline 2.0 1.0 -1.0 10. Akassa (Made of Maize) Endline 2.7 1.4 -1.3 Baseline 1.6 1.2 -0.4 11. Others Endline 9.9 12.6 2.7 Baseline 8.8 5.8 -3.0 Total Endline 100.0 100.0 Baseline 100.0 100.0 Number of occurrences (n) Endline 5,947 5,958 11 Baseline 1,512 1,596 84.0 In general, students participating in the focus-groups recognized that local foods are served in the school canteen. In N'Dali for example, students report that local meals mainly consist of: Wake (rice + beans); Gberewake (corn + beans); Zampiti (beans + maize flour); rice; akassa; and pate. And in Tchaourou, local foods such as Wake (rice + beans); Gberewake (corn + beans); and Zampiti (beans + maize flour) are served in the school canteen. As part of the LRP Project, school children are served a meal once a day; in this case for lunch between noon and 1pm. Most of the students believe that they are served on time, except in Parakou commune were some students suggest that service is often delayed. One student states: 56 "The cooks do not finish cooking early and we are stuck. It is late that we are served food and that we are in school sometimes until 1:30pm or 2:00pm". There is a general consensus among students that the quantity of meals is sufficient, but student reporting on quality of meals varies. In Tchaourou, most students declared their satisfaction as much for the quantity as the quality: "The meal of the canteen is served only once at noon in large quantity and quality. It's very very very nice”. However, there are meals that are not appreciated by students. Unanimously, the food ''Zampiti'' made with beans + maize flour is not liked by children because: "If we eat this meal, the belly becomes bloated”. In Tchaourou, all students agreed that that they do not like the "Gbéréwake" dish made from maize and beans. Also, the use of shea butter is appreciated by some but not others. For some (like children in the Baatonou area), the introduction of this oil does not cause physical discomforts, but children rather dislike the taste of this oil due to the cooking technique used at school. A student from N'Dali commune states: “It is this oil that we consume at home, but that of the school is often bitter. Sometimes it sticks in my throat and smells unpleasant when eaten. Apart from that, I do not find any problem eating this oil”. For other students who are not from Baatonou, shea butter oil is more for cosmetic use rather than consumption use. Because the various preferences among students varied by community, it is important for COGES to ensure that student feedback concerning meals is taken into consideration for future meal planning. 4.3.3. Foods at school that they consume at home The percentage of students who consume the same foods at home as they do at school is shown in Table 27. The percentage of students who consumed the same foods at school and at home increased from baseline to endline in both the intervention schools (+11.6 percentage points) and in the control schools (+5.3 percentage points). The impact estimate of 6.4 percentage points is statistically significant at the level of 5%. It did not vary by sex, but varied greatly by socioeconomic status, with higher value of +18.4 (p<0.05) among the middle class. Table 27. Foods consumed at school and at home Period Control schools Intervention schools Difference Diff. in differences Consumed foods at school that they consume at home Endline 25.3 34.0 8.8 6.4** Baseline 20.0 22.4 2.4 By sex Female Endline 24.7 34.6 9.8 5.9* Baseline 18.9 22.8 3.9 Male Endline 25.8 33.5 7.7 6.7* Baseline 21.1 22.1 1.0 57 Period Control schools Intervention schools Difference Diff. in differences By socioeconomic status Poor Endline 23.3 28.6 5.4 3.3 Baseline 21.3 23.4 2.1 Middle Endline 25.1 38.2 13.1 18.4*** Baseline 20.1 14.7 -5.3 Rich Endline 26.7 36.6 9.9 -1.1 Baseline 19.0 30.0 11.0 For Difference in differences: *p<0.1; **p<0.05; ***p<0.01 Although the LRP approach prioritizes the consumption of similar meals at home and at school, it is important to point out that there is a tendency in students to prefer the meals that are consumed rarely at home. In Parakou, the most popular foods for students in this canteen are food taken at home in special occasion such as '' Amiyo''. In Tchaourou, when children are asked for their preferences, they tend to choose what is rarely served at home. Students during a focus￾group state: "We do not want beans with maize, because we eat beans at home. We do not want the dough because the dough is not good, because we eat the dough at home’’. Based on these responses, it may be helpful to consider a greater diversity of dishes, combining local meals with some imported products for school canteens. 4.3.4. Minimum acceptable diet Table 28 below refers to the percentage of students receiving a minimum acceptable diet (MAD). In both the control and intervention schools, this percentage actually decreased from baseline to endline, no matter the sex or socioeconomic status of students. The impact estimate is not statistically significant at the level of 10% in the general population. To further understand the decrease, we explored the two components of MAD calculation. A MAD is defined as eating four or more of the seven target food groups2F 3, and a minimum frequency of three times the previous day. The percentage of students who ate at least three times decreased from baseline to endline in both arms by about 6 percentage points. Similarly, the percentage of students who ate four or more of the food groups dropped by 5 percentage points in the control schools and by 10 percentage points in the intervention schools (results not shown). Table 28 further shows a negative and statistically significant impact among girls (-10.4, p<0.05). 3 Foods were classified into the seven food groups as specified by USDA: 1) Grains, roots and tubers, 2) Legumes and nuts, 3) Dairy products (milk, yogurt, cheese), 4) Flesh foods (meat, fish, poultry and liver/organ meats), 5) Eggs, 6) Vitamin-A enriched foods, including vegetable oil, fruits and vegetables and 7) Other fruits and vegetables 58 Table 28. Percentage of students receiving a minimum acceptable diet Period Control schools Intervention schools Difference Diff. in differences Received a minimum acceptable diet Endline 49.1 54.3 5.3 -4.3 Baseline 54.7 64.3 9.6 By sex Female Endline 50.5 49.3 -1.2 -10.4** Baseline 55.5 64.7 9.2 Male Endline 47.7 59.6 11.9 1.9 Baseline 54.0 64.0 10.0 By socioeconomic status Poor Endline 39.6 42.1 2.5 -6.1 Baseline 45.1 53.8 8.6 Middle Endline 47.8 60.8 12.9 2.2 Baseline 56.6 67.4 10.8 Rich Endline 56.4 64.1 7.8 -3.6 Baseline 60.4 71.9 11.4 For Difference in differences: *p<0.1; **p<0.05; ***p<0.01 The quantitative results suggest that the students feeding patterns did not satisfy a minimum food diversity quality, particularly within LRP intervention schools. This result is supported by the above discussions reporting a wide range of opinions regarding the quality of the meals served in the LRP school canteens as opposed to collective appreciation of quantity of foods served. While rules requiring that school canteens make meals available to students regardless of availability to pay are intended to promote improved nutrition among all students, this practice may limit resources to pay for a diversity of foods, like fish which is more expensive. Although this ensures that all students have access to the canteens, it does have implications for diversity of ingredients and the perceived quality of meals. Furthermore, the decrease from baseline to endline in the indicator relating to minimum acceptable diet does not necessarily mean that children were fed better last year compared to this year. In contrary, LRP school canteens have rather made student eating patterns more consistent (with consistent ingredients as opposed to an occasional mango or cassava root here or there). This greater consistency could be a contributor to this indicator demonstrating lower diversity of foods, as opposed to greater access to consistent meals. Qualitative data supports this idea that the nutritional status of children actually improved. For example, a COGES member in Tchaourou states: “I noticed that there are fewer cases of diseases this year in the school; no case of child fallen because of hunger. Also, the children have fattened with big cheeks. I say it's because they get to eat balanced all week long in the canteen”. 59 Through this interpretation of the data, it is understood that the meals provided by the canteen are not detracting from the nutritional status of children, but are rather contributing to healthier diet patterns, given that school canteen meals are valued as an important meal for students. 4.3.5. School attendance School attendance shows little difference between control and intervention schools when it comes to students’ regular attendance, although this information was only collected at endline. This data is provided in Table 29. The percentage of students who attended school at least 80% of the time is 0.7 percentage point higher in the intervention arm. Since different schools or classes within schools may have different schedules, the number of school days in the year show a very small variation across the two study groups. Table 29. Students regularly attending USDA supported classrooms/schools Control schools Intervention schools Difference Average number of school days during the year 113.4 112.7 -0.7 Percentage of students who attended school at least 80% of time 97.9 98.6 0.73 Number of students who attended school at least 80% of time 25,734 8,750 NA Total number of students for whom attendance was recorded 26,289 8,872 NA Total number of students for whom attendance was not recorded 3,885 1,456 NA Total number of grade 2-6 students 30,174 10,328 NA On the other hand, qualitative data shows that stakeholders perceive student attendance to have improved in LRP intervention areas. Teachers that were interviewed seemed to think that students’ attendance and attention in classes improved considerably in the LRP program areas. A teacher from a N'Dali intervention school states: In the school, the project was able to make changes to the attendance at class and on time, the attendance in class because the children ate well at noon and this is reflected in their behavior in the classroom especially if it is their favorite dish that is prepared. The children are happy and there is a good atmosphere in class. For schoolchildren, the project has made it possible to improve their nutrition and indirectly improve their school experience. Indeed, many schoolchildren have an irregular school life because they cannot afford to eat when they are in school. A representative of APE of Suanin in N’Dali recounts examples of how the LRP project addressed student challenges: 60 Many pupils have feeding difficulties. When their parents leave in the morning, they do not return until the evening and these children are left to their fate for lunch. Some are forced to sell a few items first before finding something to eat. They come back to school late at night. Others do not even come back because they did not find anything to eat. As a result, evening staffing was always lower than in the morning. The LRP is necessary for us to counter these problems School staff report the greatest improvements in attendance among students whose homes are far from school. According to interviews, prolonged absence that turns into school dropout decreased because students no longer had to leave school before the end of the day to eat. A School Principal in Thanks to this canteen, the absenteeism of students from farms in a village located about 3.5km has dropped considerably. The students are diligent and the dropouts have dropped. For the year passed on the same date there were a hundred long￾term absentees who ended up being classified as dropouts at the end of the school year. But this year on the same date there are only about thirty long-term absentees. Other stakeholders who were interviewed reported a reduction in illnesses due to poor nutrition and poor hygiene practices thanks to the provision of school meals. The school Principal explains: The formerly frequent vomiting is no longer observed in schoolchildren. Hygiene practices have become more prevalent. Washing of hands with soap and water has become widespread among students. The quantity and quality of students' meals improved with regard to unsatisfactory rations to which their scholarship at the level of private catering was entitled. Finally, the school canteen has created motivation and interest among schoolchildren to go to school, whether or not they have constraints. Here is the statement of a teacher in Tchaourou: Without a canteen, you had to scold the child several times before going to school. But now they are more eager to go to school. Especially when they know that the Watche will be prepared. In Pèrèrè, it is reported that children are excited to go to school. However, it is necessary to note how the activities of the school canteen have the potential to encroach on the learning time of some schoolchildren. For example, students in N’Dali ( reported that they spend time searching for firewood for the kitchen after school, encroaching on their studying time. And some girls from N’Dali are pulled out of class to help with cooking (e.g. wash bowls and pots and sometimes to clean the kitchen). While these roles for students were not part of the original project plans, COGES members identified these tasks as student contributions. These cases can be detrimental to the achievement of academic performance outcomes and contrary to the positive effects of the school canteen. 61 4.3.6. Effects on students’ families It is necessary to note benefits to both students and their families. In describing his report, the head of the SAS states: During our field trip to the in N'Dali commune, I had the opportunity to attend a lunch time for school children. The students who eat at this LRP canteen are happy to participate in the meals and show a very happy air. This emotional satisfaction that emerges from the eyes of these students is very important for their life in general and their school life in particular. Also, the presence of these canteens in schools allows parents to be less concerned about the food of their children at school and to be able to go about other occupations. A parent of a student of the in Pèrèrè expresses it in these terms: The canteen LRP too relieved the parents in the cooking of the lunch. Thanks to this canteen, the mothers of school children no longer bothered at noon to provide food for the children. Fathers can go about their business freely. They can go to the field, or travel without worrying about what the schoolboy will eat at noon. Further, the provision of school meals by the canteens has contributed to a reduction in disputes among parents in regards to the meals they are able to provide for their children. Families feel more comforted knowing that all children are better cared for with the addition of the school meal. 4.3.7. Lessons learned For some schoolchildren, the LRP school canteen provides at least one well-balanced meal per day. A parent in Tchaourou says it in these terms: When I take the case of at noon, all children go home to a single parent who still has no means to support needs of these. The child comes to school with a yam tuber. That's what he eats at 10H and sometimes at 12H at his host. This happens every day, without variation. This exposed children to diseases related to chronic malnutrition. Today all this is finished. Because the LRP project has brought the solution. CRS management has also concluded that this project created joy in students school life, in these terms: “On the students’ side, this project makes joy in the students school life; knowing that they will have access to foods in school canteens taking them to school with joy”. In the opinion of all the children in the intervention communes, it is necessary to renew the LRP school canteen, considering the numerous advantages and the solutions that it has brought to them. However, some lessons are learned from the process and should be taken into account in the future. Students suggest that their involvement in the preparation of meals for the supply of wood, the washing of pots and other utensils during school hours is disadvantageous for them, particularly girls, and can impede valuable learning time. Furthermore, while students appreciate 62 that school meals are prepared with local products, there are some specific meals or methods of preparation that are unfavorable, and their rotation out of the school menu should be considered. 4.4. Government and commune level partnerships 4.4.1. Government level actors The LRP project brought together several governmental actors; the Department of School Feeding (DAS), the School Food Service (SAS) of the Departmental Directorate of Maternal and Primary Education (DDEMP), the Ministry of Planning, and the Division of Regulation and Control of Vegetable Products (DRCPV) of the Departmental Directorate of Agriculture, Livestock and Fisheries (DDAEP). Indeed, the involvement of the different actors in the LRP Project responds to a specific and strategic need for the successful implementation of its two intervention arms, namely school feeding and the strengthening of agricultural producers for the supply of quality products at school canteens. The DRCPV is solicited on behalf of the DDAEP for its expertise in the field of quality control. The control is done at the time of purchase by the samples that are made and sent to the lab for toxicity analysis (presence of pesticides, germs, etc.). Control is also periodically done in the field by the DDAEP technicians who visit the LRP canteen school stores every three months. Appropriate recommendations are made according to the findings. The DDEMP through the SAS was involved in the project steering committee in all phases of the project, namely the preparatory phase and the implementation phase. The SAS was represented at project framing workshops, launching the project, selecting offers from producer cooperatives for the supply of agricultural products, and evaluation missions during the implementation of the project. This made it possible to make proposals and recommendations as much as necessary. Interest in the LRP project's local procurement approach has increased at the level of government structures and this is proven with the Ministry of Planning, which has also joined the steering team. 4.4.2. Commune level actors The LRP project has been strongly deployed at the local level in close relation with communes. In particular, two committees were formed to be the functional arms of communes in the LRP project; a Procurement Committee (CPC) and a Supervision Committee (CLOC) were set up in each of the four intervention communes. The creation of these committees was ratified by decrees taken at the commune levels. The Procurement Committee (CPC) is composed of three members from the commune, namely the Responsible of Planning Service, the Chair or his representative (e.g., The First Deputy in most of the four intervention centers) and the Responsible of Social Affairs Service. The CPC is set up to take ownership of the local process of purchase of agricultural products of the LRP project in order to support its continuity following completion of the LRP project. To this end, the LRP project has considered their involvement in all stages of the selection process as well as their 63 responsibility for the purchase of small lots of local agricultural products, in this case maize, at the commune level. The Supervision Committee (CLOC) is composed of two members, namely a representative of the civil society and a head of the school district intervening at the commune level. The CLOC is set up to oversee monitoring and control of the local supply system with the intention of continuing the local supply approach for the school canteen. CRS management summarized the responsibilities of the CPC and CLOCs by stating: The municipal procurement committees and the municipal supervision committees were set up in the various intervention communes to participate in the LRP project processes to strengthen their capacity to implement a local initiative in this direction. The procurement committees should be empowered to buy maize at the local level while supervision committees are involved in the process of controlling and ensuring its proper functioning. The supervision committees ensure compliance with the conditions of organization and implementation, regarding the purchase of agricultural products by procurement committees and the use of the products by the COGES. Supply Committees and Supervisory Committees collaborated with the municipal authorities of the four intervention communes, the rural and agricultural development projects working in the intervention areas (ProAgri and PASDeR), and the Organizations of Agricultural Producers (Alliance Karité Benin, Regional Union of Producers of the Borgou and Alibori Departments, and Regional Union of Rice Producers) who formed the project management team. At the communal level, the creation of the Procurement Committees and the Supervision Committees in the intervention area is the signal of the local anchoring of the project. These two communal structures were involved in the implementation of the project according to a practical learning process based on training and experience (e.g. participation in the LRP process steps). On the other hand, sharing the role with the URP as a sub-recipient of the project in carrying out the capacity building activities of producers has strength. Likewise, because of their proximity to producers, the involvement of umbrella organizations (in general, in the pre-selection of cooperative suppliers of agricultural products, in the audit of the existence of these cooperatives, and the assistance and advice in the different stages of the sales process of the LRP project) has supported their ability to honor their commitment. PASDER and ProAgri have contributed their expertise in the field of agricultural education and training for the reinforcement of the capabilities of the producers with regard to the interventions already carried out at their level. The various trainings have reinforced the capacities of stakeholders and have helped to fill in the remaining gaps (e.g. the writing of bidding documents). In practice, the CPCs and CLOCs did not have autonomy throughout the time of the LRP project; these two committees do not have internal regulations, statutes, or operating budgets. It is therefore difficult to assess the level of functioning of these committees and their achievements. 64 Members meet as needed to prepare for participation in meetings initiated by the LRP. Members' expenses for attending these meetings are directly funded by the LRP project. These committees usually do not make decisions, but execute the CRS guidelines. A CPC member at Parakou states: The effectiveness of its activities is closely linked to initiatives coming directly from CRS. Given the way the committee operates, we cannot talk about decisions coming from this body. The committee executes orders that reach it in accordance with the invitation and guidelines of the LRP Project. In this same line, a member of the Supervision Committee states: We cannot talk about a budget for a committee that does not even have a work schedule. I think things work according to the invitations and choices of CRS. We intervene just to help solve a social problem and our expertise is solicited as a specialist in a field in the service of the state that employs us. Nevertheless, through their involvement in the LRP project, these committees played several roles in familiarizing LRP stakeholders with the reality of the local food procurement approach for school canteens. For the CPCs, these activities or roles include: - Participation in training on the supply approach of school canteens with local products; - Publication of calls for tenders in the partner radios of the town halls; - Participation in the workshops of the offers of suppliers of agricultural products; - Participation in the verification of the existence of the pre-selected cooperatives; - Participation in the delivery of food to the LRP project. When it was still necessary to deliver to the communal stores retained by the LRP project before distributing it to the schools, the presence of the members of the supply committee was required. But after, when the delivery was made directly to schools, this role was removed. These roles and activities were similar among the Supervision Committees’ members who also participated in monitoring and evaluation with the CRS team. 4.4.3. Satisfaction Various institutional actors (governments, communes, project staff) find a certain pride with their participation in this model of school feeding. However, it is important to stress the difficulty of ensuring a balanced sharing of roles that has occurred in such a multi-stakeholder environment. In particular, with the value chain approach promoted by the government, producer organizations linking to specific value chains are better suited to provide technical support for their members. While the LRP project considered the URP as key project partner for the implementation of activities concerning all producers, the URPR would be more appropriate to carry out the activities in favor of its members that include the Producer Associations of rice. This has created some frustrations and has somewhat disrupted the participation of the URPR and ProAgri (which also supports the URPR). On the other hand, some difficulties in relations are reported, particularly in connection with communication (e.g. SAS/DDEMP and ProAgri staffs reported a late sending of meeting invitations to them) or the absence of partnership 65 agreements. For example, the absence of partnership agreement has enabled actors from DDAEP involving in the LRP project to use official cars for field trips. 4.4.4. Challenges and opportunities The short implementation period for the LRP project posed challenges in strengthening local capacity of CPCs and CLOCs. For example, the procurement of small lots of maize, which are planned to be carried out by the CPCs at the communal level to give them more opportunity for practical experience, was not effective. One key aspect is the fact that the Government is not supporting an active role of the Mayor’s offices in the management of school canteens, and the Town Halls are not involved in canteen management. The short implementation time of only one year limited possibility for the transfer of function to these committees, and an extended implementation timeline may foster improved partnership with government entities. A project timeline of three years would be recommended to achieve increased sustainability. As indicated in qualitative interviews, local stakeholders suggest an extended timeline of five years. Furthermore, the low level of institutionalization and the current financial difficulties at the level of the communes explains the reluctance of CRS to entrust financial resources to these committees during its pilot stage. A member of CRS management states: It's a learning project; it was not clear at the outset what the roles of these committees were. Given the temporary nature of these committees, which also do not have a strong institutional base combined with the financial difficulties facing our municipalities, it was difficult to keep the project money by these procurement committees for the purchase of the products as planned. Opportunities to overcome such challenges include the review of current plans, programs or school feeding strategies at national level to consider the inclusion of these committees in larger plans. The LRP project has contributed to and complemented the interventions of the Beninese government, local governments and even other partners in support of rural development in general. The head of the SAS states: School feeding in the form of school canteen is a concern of the Beninese government that is currently investing itself. It is therefore a need of the State - to be accompanied by other structures to cover the largest number of students in our Department (Borgou￾Alibori), why not all the students of Benin - that the project LRP came to fill. In addition, the Director of the Facilitation and Management Unit of PASDeR explains: These government policies place greater emphasis on market access for small producers. Likewise, the LRP project, through its approach based on the supply of local agricultural products to school canteens, contributes. Further, all stakeholders approve of the project mission, which has also been revered by others not directly involved in the project. For example, the Head of the SAS states: "The school canteen 66 model of LRP based on the local supply of products is in itself the innovation of the project; this approach will not be long in schooling all over the country and beyond in the next years ". CRS management also reports that the project has already received visits and solicitations at national and regional level to share its successes. They state: Several delegations have been here already in the intervention area to learn about the local procurement approach. The CRS Burkina-Faso has already contacted us and several Skype discussions are being conducted with them and it is preparing a visit mission, after which the CRS Burkina-Faso also plans to experience. 4.4.5. Lessons learned and sustainability When asked about lessons learned, committee members noted how the local supply approach is relevant for school feeding, and transparency in the selection of local suppliers contributes to the success of the local procurement approach. A member of a Procurement Committee states: The LRP allowed us to have a very useful procurement experience in terms of quality standards. transparency and impartiality. We are happy with the way the process was conducted. We have also acquired procurement know-how that will serve us in similar initiatives in the future. Furthermore, the training has shown to be effective in developing local capacities. A committee member explains: We, too, a member of the committee, received some training that has now made us able to make the supply in a professional way, which is an asset for the mayor who can exploit the resource people we have become in this area through the LRP. In terms of sustainability, the involvement of the Procurement and Supervision Committees in the LRP project gives them the experience necessary for continuation of these roles in future endeavors. A member of a Procurement Committee in Pèrèrè states: The training sessions were done on the mechanisms for the sustainability of school canteens through the learning of supplier selection procedures and the examination of offers. The appropriation of this practice by the purchasing committee allows the continuation of this activity. On the other hand, a lack of a funding strategy for these committees to ensure their participation in the local procurement of school canteens following completion of the LRP project threatens sustainability. While all costs were covered by the LRP project, there is no source of sustainable funding for the support of these committees. In particular, it should be noted that the Town Halls to which these committees are affiliated are less inclined to support a school canteen initiative that would entail a financial burden, because of other priorities and limited resources. A member of a communal authority interviewed in Parakou states: "The LRP is an initiative external to the city council but which fills a void left in the state of project. The Town Hall has other priorities, and also the resources are limited. So, there is no budget reserved for school canteens". 67 Currently, the Ministry of Education does not have a blanketed approach requiring Town Halls to support school canteens. Rather, government involvement in school canteens is according to the diverse initiatives of different agencies that support government involvement in school canteens, and that vary by area of implementation. According to government authorities, a second phase of the project is needed so that producers and parents can improve their experience in the supply and management of school canteens. Government structures, such as School Food Service (SAS) suggest that CRS intensify training and ensure that all equipment, including kitchen and storage facilities and devices for accessing drinking water in certain schools, are in place to improve the functioning of these canteens. Government structures, such as the School Food Service (SAS), has a good attitude towards the sustainability of the local supply approach for school canteens, and conveyed their intention to scale the program in the coming years. Likewise, the communal authorities are favorable for the continuity of the project. But government structures such as these rely on the renewal of the project with the accompaniment of the partner. The communal authority in Parakou states: "The aspiration of the communal authority is the renewal of this project with always the accompaniment of the partner. Otherwise the LRP will not be sustainable”. Government entities have not reserved resources or developed concrete strategies to continue to support the LRP school canteens. Furthermore, the local supply committees and the supervisory committees that would play a pivotal role in the replication of the LRP model have not had sufficient practical experience to autonomously procure commodities. These committees have witnessed the LRP / CRS project play a unifying role in intervention communities, but need additional experience acting autonomously at the municipal level. Hence, additional efforts to support government entities in scaling up this approach are needed. To ensure the sustainability of the project, CRS has advocated for sustainability measures with stakeholders and the funder. CRS has advocated with national actors (Government and WFP), requesting that the operation of the 80 intervention school canteens continue this next school year. CRS has also advocated among donors for the generalization of this local approach to supply school canteens in existing school canteen projects in Benin. One CRS staff member states: “Since the same donor is financing other school canteens with the approach of imported products from the United States, we try to convince him that the local approach is by far favored by the government and local actors”. In addition to advocating for sustainability measures, CRS is capitalizing on its learning and opportunities for scale up by documenting the project approach. The main hope is that the government or WFP can draw inspiration from this documentation to develop improved school feeding strategies. A statement from a CRS staff member illustrates this: "The LRP project within it is very motivated to share the experience. Currently, everything is documented about the 68 approach, and its implementation to serve as a guide to very specific actors from the Government, PTF supporting the Government school feeding program, and the WFP. Factors that may impact the sustainability of the project include: - The renewal of the project for the consolidation of achievements; - The participation / contribution model of the beneficiaries; - The institutional environment of producers (access to support services for producers); - The inclusion of the approach by government in the national strategy of school feeding and the communal plans and budgets respectively. The sustainability of this project depends on the government mainly on two levels: - Taking into account advocacy to take charge of LRP canteens as soon as the project ends; - The adoption and institutionalization of the local procurement approach as part of the development of the National School Feeding Strategy. 5. Results: Comparison Study While the study was originally designed to compare the LRP and the McGovern-Dole (MGD) systems, we ended up including the WFP system. Since the three programs do not necessarily use the same commodities, we are using the following categories: • Maize in LRP and WFP, and SFCM (soy-fortified cornmeal commodity) in MGD • Rice: in all three systems • Legumes: Cowpea in LRP; peas and lentils in MGD; beans in WFP • Cooking oil (Shea butter in LRP; VEGO in MGD; Vitamins A &D enriched oil in WFP). During a meeting on September 9, 2019 that brought together staff members from WFP, MGD and LRP to review the preliminary results of the comparison study, the scope of the study was somewhat revised. The discussions revealed among other things, that cost data is not comparable between WFP on the one hand, and LRP and MGD on the other hand. Tables on costs will therefore feature LRP and MGD only. The cost data collected from the LRP and MGD relates to: • Purchase of commodities; • Delivery of commodities to the programs’ stores • Distribution of commodities to beneficiary schools. We have slightly modified the label of the first indicator, from “cost per meal” to “commodity costs per meal”, as the original label may be misleading (we are estimating not a cost per meal, but a commodity cost per meal. All costs were provided in US dollars ($). Where applicable, an exchange rate of 1$=580 CFA Francs (the local currency) is used in the analysis. We also collected data on the number of days corresponding the following three procurement phases: 69 • Purchase contracting time; • Purchase contracting to delivery to the programs’ central stores; • Delivery to beneficiary schools. 5.1. Cost differences The first indicator of interest is the cost (from purchase to distribution) differential between local market commodity and MGD. Table 30 shows the differentials in commodity cost per MT, across the MDG and LRP programs. As can be seen, the LRP system has higher average commodity cost per MT on three out of the four target commodities, compared to the MGD system. For example, LRP’s rice is about 10% more expense than MGD’s fortified rice , and cowpea is about 30% more expensive than the equivalent in the MGD system. Table 30. Differentials in commodity cost per MT ($/MT) between LRP and MGD systems MGD System LRP System Differentials SFCM/Maize 887.68 287.60 -600.07 Legume (Cowpea, beans, lentils) 848.82 1,099.34 250.52 Rice 817.74 903.91 86.17 Cooking oil (Shea butter, VEGO …) 1,402.18 1,874.94 472.76 The different steps for the calculation of the second indicator, the cost per meal differential between local school feeding approach and MGD, are captured in Tables 31a-d and Table 32. Table 31a shows the weekly (Monday-Friday) composition of canteen menus, as it relates to the target commodities. For example, in LRP supported school canteens, 175 grams of maize are included in each student’s meal on Mondays, Wednesdays and Fridays, and 175 grams of rice included on Tuesdays and Thursdays. The schedule is similar in MGD schools with regard to SFMC and rice. Table 31a.Quantities of commodities in the weekly school canteens' rations (in grams) in LRP and MGD systems MGD System LRP System Days in the week SFCM/Maize 175 175 Monday, Wednesday, Friday Legume (Cowpea, beans, lentils) 30 30 All five days Rice 175 175 Tuesday, Thursday Cooking oil (Shea butter, VEGO …) 8 10 All five days 70 From the Minister of Education’s order fixing the schedule for the school year 2018-20193F 4, we derived the number of school days as shown in Table 31b. Table 31b. Number of school days during the 2018-2019 school year Monday Tuesday Wednesday Thursday Friday total First Term 14 14 14 13 14 69 2nd Term 13 13 13 12 13 64 3rd Term 8 9 8 8 9 42 Total 35 36 35 33 36 175 Table 31c presents the differentials in commodity cost per student during the school year 2018- 2019. The commodity cost per student for the school year is obtained by multiplying the corresponding unit cost in Table 30 by the corresponding quantity in Table 31a and by the corresponding number of days in Table 31b. Overall, commodity cost per student and per school year is estimated at $25.30 in the LRP systems, compared to $32.76 in the MGD systems. The cost is therefore lower in LRP by $7.46 (or 4,325 CFA Francs). Table 31c. Differentials in commodity cost per student during the school year 2018-2019 MGD System LRP System Differentials SFCM/Maize 16.47 5.34 -11.13 Legume (Cowpea, beans, lentils) 4.46 5.77 1.32 Rice 9.87 10.91 1.04 Cooking oil (Shea butter, VEGO …) 1.96 3.28 1.32 Total in $/student 32.76 25.30 -7.46 Total in CFA Francs/student 19,001 14,675 -4,325 Note: Exchange rate of 1$ = 580 CFA Francs Finally, the indicator of interest is derived by dividing the total row in table 31c by the total number of school days (175 days), as shown in Table 31d. The average commodity cost per meal and per student is lower in the LRP model, at $0.14 (or 84 CFA Francs), against $0.19 (or 109 CFA Francs) in the MGD system. 4 Arrêté Interministériel no 2018/081 fixant le calendrier de l’année scolaire 2018-2019 dans les établissements des enseignements maternel, primaire, secondaire général, technique et professionnel. 71 Table 31d. Differentials in average commodity cost per meal and per student between LRP and MGD systems ($/meal) MGD System LRP System Differentials In $ 0.19 0.14 -0.04 In CFA Francs 109 84 -25 Note: Exchange rate of 1$ = 580 CFA Francs To further illustrate the differentials in cost per meal, we identified from the full list of meals served and their composition, the following meals that appear to be comparable content-wise across the two systems. These are: • Dambou: Maize (175g), Cowpea (30g) and shea butter (10g) in LRP; SFCM (175g), lentils (30g) and VEGO (8g) in MGD • Zampiti: Maize (175g), Cowpea (30g) and shea butter (10g) in LRP; SFCM (175g), lentils (30g) and VEGO (8g) in MGD; • Wake: Rice (175g), Cowpea (30g) and shea butter (10g) in LRP; Rice (175g), lentils (30g) and VEGO (8g) in MGD; • Pasta: Maize (175g) and Cowpea (30g) in LRP; SFCM (175g) and LENTILS (30g) in MGD; • Rice: Rice (175g) and Cowpea (30g) in LRP; Rice (175g) and peas (30g) in MGD. Table 32 shows the commodity cost for these five meals. As can be seen, commodity cost of Dambou/Zampiti and pasta (maize/soy pasta) is approximately 50% lower in the LRP system. The commodity cost of wake and rice is slightly lower (by 7% and 15%, respectively) in the MGD system. Table 32. Differentials in commodity cost for specific, comparable meals between LRP and MGD systems ($/meal) MGD System (US$) LRP System (US$) Differentials US$ CFA Francs Dambou/Zampiti 0.21 0.10 -0.11 -61 Wake 0.20 0.21 0.01 8 Pasta 0.20 0.08 -0.11 -66 Rice 0.16 0.19 0.03 17 Qualitative data indicates that the supply mechanism in the MGD project was based on the high level of profitability of agriculture in the United States that makes US food products cheaper than those in Benin. But by including transportation costs from the United States to Benin and the cost of distributing these products to the beneficiary schools, this method of supply proves to be quite expensive; hence the following statement by a MGD staff member: 72 The level of profitability of agriculture in the United States makes the value of food more advantageous than local production. However, this advantage is reduced or even lost in some cases depending on the costs that will be added from the United States to the beneficiaries' plates. However, WFP Staff in charge of program who combine three methods of supply (International, Regional and Local) think they have developed a strong mechanism to make supplies cheaper and quite faster. One staff member states: "We have no elements of appreciation and comparison to answer this question. However, we know that WFP's strategy enables cost￾effective and time-saving procurement; WFP has developed proven expertise in the field ". Finally, for the LRP staff in charge of program, the local procurement method favoring local cooperatives that has been adopted considerably reduces costs. 5.2. Time differences The terms of reference for the study requested that time estimates be divided by the quantities to account for the fact that it takes longer to purchase and deliver larger quantities, suggesting that the underlying indicator is time per MT. The indicator time per MT may be questionable, since it implies that for example, it would take about five times longer to say, deliver 5 MT of rice to schools, compared to delivering 1 MT. Table 33 presents both the average time (in days) and average time per MT (in days/MT), for the total time and for each of the three segments (purchase contracting, delivery to central stores, and delivery to beneficiary schools). These four variables correspond to the four target indicators on time differentials. Table 33 shows that the total time (from contracting to delivery to beneficiary schools) is substantially shorter in the LRP model, compared with the WFP and MGD systems. Purchase contracting time is the shortest in the WFP system and the longest in the MGD system. There is no delivery to central stores in the LRP system. Time for delivery to schools is almost the same in both the LRP and MGD programs, and substantially longer in the WFP. Table 33. Time differentials between LRP and MGD systems (days) WFP System (1) MGD System (2) LRP System (3) Differentials (3)-(1) (3)-(2) 1a. Total time from purchase contracting to delivery to schools (days) SFCM/Maize 120 198.0 120.5 0.5 -77.5 Legume (Cowpea, beans, lentils) 120 198.0 106.3 -13.7 -91.7 Rice 120 198.0 108.3 -11.7 -89.7 Cooking oil (Shea butter, VEGO …) 120 198.0 87.0 -33.0 -111.0 1b. Total time from purchase contracting to delivery to schools per MT (days/MT) SFCM/Maize 0.03 5.08 1.49 1.46 -3.59 Legume (Cowpea, beans, lentils) 0.07 6.45 7.42 7.34 0.97 Rice 0.03 3.77 3.92 3.90 0.15 73 WFP System (1) MGD System (2) LRP System (3) Differentials (3)-(1) (3)-(2) Cooking oil (Shea butter, VEGO …) 0.09 7.67 5.85 5.75 -1.83 2a. Purchase contracting time (days) SFCM/Maize 90.0 180.0 118.3 28.3 -61.8 Legume (Cowpea, beans, lentils) 90.0 180.0 102.8 12.8 -77.2 Rice 90.0 180.0 106.2 16.2 -73.8 Cooking oil (Shea butter, VEGO …) 90.0 180.0 85.0 -5.0 -95.0 2b. Purchase contracting time per MT (days/MT) SFCM/Maize 0.02 4.62 1.46 1.44 -3.16 Legume (Cowpea, beans, lentils) 0.05 5.86 7.17 7.12 1.31 Rice 0.02 3.43 3.85 3.83 0.42 Cooking oil (Shea butter, VEGO …) 0.07 6.98 5.71 5.64 -1.26 3a. Delivery to central stores time (days) SFCM/Maize 5 15.0 Not applicable Not applicable Legume (Cowpea, beans, lentils) 5 15.0 Rice 5 15.0 Cooking oil (Shea butter, VEGO …) 5 15.0 3b. Delivery to central stores time per MT (days/MT) SFCM/Maize 0.00 0.38 Not applicable Not applicable Legume (Cowpea, beans, lentils) 0.00 0.49 Rice 0.00 0.29 Cooking oil (Shea butter, VEGO …) 0.00 0.58 4a. Delivery to beneficiary schools time (days) SFCM/Maize 25 3.0 2.3 -22.8 -0.8 Legume (Cowpea, beans, lentils) 25 3.0 3.5 -21.5 0.5 Rice 25 3.0 2.2 -22.8 -0.8 Cooking oil (Shea butter, VEGO …) 25 3.0 2.0 -23.0 -1.0 4b. Delivery to beneficiary schools time per MT (days/MT) SFCM/Maize 0.01 0.08 0.03 0.02 -0.05 Legume (Cowpea, beans, lentils) 0.01 0.10 0.24 0.23 0.15 Rice 0.01 0.06 0.08 0.07 0.02 Cooking oil (Shea butter, VEGO …) 0.02 0.12 0.13 0.12 0.02 5.3. Food supply mechanisms Supply mechanism for MGD school canteens The food supply process for MGD canteens is done through an order addressed to the sponsor in the United States. Effective delivery times in CRS central stores are 6-7 months. Following consumption forecasts, a quarterly distribution plan is then developed to route these products to the beneficiary schools. 74 Supply mechanism for LRP school canteens As part of the LRP project, the process of supplying school canteens is done in several stages, namely: - Needs assessment: it is based on a calculation of the rations of the children by day of class for the different products. School principals of intervention schools are encouraged to review and provide information on the total enrollment of their schools. On this basis, the need for each targeted product (rice, maize, beans, shea butter) are defined for the school, the municipality, and all the intervention schools. - The drafting of tender documents: For the drafting of the call for tender, the needs assessment data are used. The call for tender presents the quantities to be provided for each school, according to communes. - The publication of tender documents on the partner radios and by posters in communes: The call for tenders is broadcast on the partner radios (Deman FM, Fraternité FM, Nord FM, Nonsina FM, Béké FM, Suti dera FM Nikki) with the support of communes who have benefit contracts with the radios in their area. - The reception, the counting and the analysis of the offers: A mixed counting and analysis committee made up of the managers of CRS, the members of the communal committees of supply and the members of the communal committees of supervision is put in place. The members of this committee meet in a workshop for the pre-selection of suppliers. - Pre-selection: Several criteria are taken into account, particularly whether the tenderers belong to the project area. In cases where the supply capacity of the cooperatives belonging to the intervention zone is considered insufficient to satisfy the demand (for products such as beans and rice for example), the selection may be extended to cooperatives not belonging to the project area. - Verification of the credibility of shortlisted suppliers (formal existence, not belonging to money laundering networks): After the pre-selection of the cooperative suppliers, 2-3 members of the committee of counting make a field trip for the investigation of real and formal existence of these cooperatives. This step is recorded by filling in the characterization forms of each cooperative (formal existence, capacity and resources). These forms are sent to CRS headquarters in Cotonou. CRS then assigns a vendor code number to identify the cooperatives with CRS. - Verification of product quality and price negotiation: The Central Laboratory for Sanitary and Food Safety is asked to sample products from different cooperatives for quality tests. - Contract signature and launch of the purchase: When the test results are conclusive, the contract is signed and the purchase is launched for corn and cowpeas. For rice and shea butter, the contract is first signed, then the transformation or processing started. Upon delivery, the DDAEP packaging agents are on site to check the compliance of the delivered products with the samples taken. After delivery, the cooperative keeps three copies of the delivery slip and three copies of the receipt slip. The delivery man has the different vouchers 75 signed and two copies are deposited to the LRP project facilitators who are in the field. A copy of the contract, a copy of the delivery note and a copy of the receipt slip is then sent to the collection receipt for payment. After processing the files, the payment is made via the CLCAM or by bank transfer. As recognized by a Project Manager at CRS, the administrative procedure at the level of CRS creates a big delay and causes a lot of problems to producers who sometimes took loans from microfinance institutions. Similarly, delays between withdrawals and the signing of contracts mean that when the market price is better, producers are tempted to sell their product on the market. Supply mechanism for WFP school canteens WFP purchases at 3 different levels: international, regional (e.g. iodized salt in Dakar), and local (maize and beans). After the purchase the food is stored in their different stores in the departments. Distribution in beneficiary schools is on a quarterly basis and in several stages: - Estimate of food requirements for schools from school enrollment and daily ration per child. - Development of food allocation plans by quarter, taking into account the number of working class days in the quarter. - Development of Food Allocation Vouchers from the allocation plans developed. - Signing of plans and food allocation vouchers. These benefit vouchers are drawn up by municipality, and take into account the quarterly food needs for the municipality. - Delivery of allocation plans to the Supply Chain Unit for the execution of the plans. For transportation, the private carrier service is used for distribution in schools (carrier shortlist, tender, contract, transparent process, etc.). At the level of school canteens, the COGES receive the food by doing the usual checks and sign the car letters that attest that the service was rendered. 5.4. Appreciation of meals by children In this section we quantitatively analyze the appreciation of the quality of the meals and the services of the canteen among students who reported eating at the school canteen. As can be seen in Table 34, the percentage of students who reported liking the meals served at the canteen was close to 95% in the two groups. The percentage of students who reported feeling full after eating at the school canteen was also high, at around 92%+, with little variation across the three groups. About 23% of students in MGD schools and 21% of students in LRP schools declared there was often a delay in meal service hours. 76 Table 34. Appreciation of meals among students who declared eating at school canteen MGD System LRP System Differentials 1. Liked the canteen meals (%) 93.9 94.9 1.0 2. Felt full after eating at the school canteen (%) 91.1 92.0 1.0 3. Delays in meal service hours (%) 23.4 21.4 -2.0 Sample size 850 1,511 NA Note: Figures are weighted. 5.5. Externalities on local economy The municipal authorities do not perceive a direct influence of the MGD and WFP projects on the local economy, while stakeholders perceive a positive effect of the LRP project on the local economy. The communal authority of Gogounou (MGD zone) explains that the commune has no interaction with the MGD project: "The food is brought from elsewhere without the town hall knowing the supply circuits”. In contrast, the positive impact of the LRP project on the local economy is quite perceptible, particularly compared to the PAM project. A communal authority states: WFP uses imported food while LRP favors local food for the benefit of parents and local producers. The community is more motivated by the idea that children eat what is produced by them. LRP canteens also contribute to the local economy because of the opportunities to sell the products it creates to local producers. Under LRP, beneficiaries are doubly relieved and motivated to produce well and wholesale. This is good for their activities, and for the economic fabric of the municipality. The local supply approach of school canteens of the LRP project is therefore an initiative with strong potential for promoting the local economy. 5.6. Satisfaction of project beneficiaries As described by municipal authorities, the beneficiaries' satisfaction after the implementation of the LRP project is quite noticeable among producers as well as parents and schoolchildren. Producers have easy access to agricultural markets and sell their products faster; parents are less concerned about feeding their children at school and can attend to other occupations; and students have access to healthy food, quality and quantity. However, there are some shortcomings. In N'Dali, for example, "Producers feel that the grouped sale process and the related procedure are embarrassing. Moreover, they deplore the impossibility of modifying the selling price once adopted in the contract of service provision whatever the fluctuation of the price on the market”. In Parakou, a fundamental question remains at the center: "Why should 77 children pay when we talk about poor areas?”. Similarly, the use of shea butter for cooking meals is not much appreciated by schoolchildren, which slightly reduces the rate of adherence to LRP canteen meals. Finally, the project promised to build kitchens and storage warehouses in some schools, but this expectation has not been filled. To ensure higher satisfaction with the program among beneficiaries, these aspects of the project would need to be improved. Stakeholders and beneficiaries of WFP and MGD school canteens recognize that these canteens provide a solution to school feeding problems for schoolchildren, but the lump-sum contributions requested from parents, especially in the MGD project area, is not feasible. Indeed, the PME-FCB canteen (Global Partnership for Education of the Common Budget Fund) which preceded MGD in this area, was free. As a result, the MGD canteen in Gogounou is facing serious difficulties in terms of membership due to the contributions required to access meals. More and more students no longer eat in the canteen for lack of contributions. This has implications for school attendance and academic performance. A municipal authority expresses it in these terms: "Free access is a central factor so that canteens in rural areas can effectively play the role of maintaining and improving the performance of schoolchildren". The disappearance of free access to school canteens has been a central contributor to the lack of satisfaction among beneficiaries of MGD canteens. 6. Conclusions and Recommendations 6.1. Conclusion The LRP final evaluation was a comprehensive measure of the relevance, effectiveness, efficiency, impact, and sustainability of the LRP project. In addition to comparing quantitative indicators compared to the baseline evaluation, a significant amount of qualitative data from interviews and focus group discussions provided an in-depth understanding of the project design, implementation and management; contributing to greater clarity of lessons learned. The LRP endline evaluation results indicate that through the LRP project, Producer Associations were strengthened significantly. Organizational capacity increased among 75% of all Producer Associations in the intervention group, compared to 50% in the control arm. The LRP project trainings provided producers with important skills on production, harvest and storage practices that were implemented in the field, and more individual producers chose to sell their products through Producer Associations. This high level of organizational capacity results in timely delivery of commodities, adequate quantity of food, and high-quality commodities, as reported by COGES members. Aside from strengthening the production side of school canteens, the LRP project also contributed to improved capacity of COGES. A large majority of the cooks and store keepers surveyed reported having acquired capacity in various aspects of their work, and COGES scores in canteen and stock management reached 77.3%. The overall COGES organizational capacity score reached 62.1%. Quality of the services provided by COGES was enhanced with mutual 78 monitoring of role compliance, thanks to the training of COGES members on multiple areas of work. The resulting meals served to students through school canteens contributed to little or no positive changes in student diets. While the percentage of students who consumed the same foods at school and at home increased from baseline to endline by 11.6% (compared to 5.3% in the control schools), students reported that they prefer to eat foods at school that are notserved at home. Furthermore, the percentage of students receiving a minimum acceptable diet actually decreased from baseline to endline in both arms, and there was no significant change in school attendance among students. As demonstrated through qualitative data, some students were actually detracted from school to assist with the operation of the canteen in some circumstances. When comparing the LRP model to the MGD system, the LRP system has higher average commodity cost per MT on three out of the four target commodities, compared to the MGD system. The average commodity cost per meal and per student is lower in the LRP model, at $0.14 (or 84 CFA Francs), against $0.19 (or 109 CFA Francs) in the MGD system. In terms of time differentials, the total time (from contracting to delivery to beneficiary schools) is substantially shorter in the LRP model, compared with the WFP and MGD systems. The success factors of the LRP project include: - The approach is appropriate in addressing a dual concern within the beneficiary community: market access and school feeding; - The project approach is anchored in national development strategies such as promoting access to markets for producers, promoting local consumption, and supporting school feeding by school canteens in poor people; - The innovative collaborative system is based on a network of institutions capable and supported by the principles of complementarity, participation and learning; - Training is developed and implemented (in local language) according to capacity needs for beneficiaries, especially COGES members; - There is great mobilized interest of the local population and the different actors involved at various levels. Factors limiting the success of the LRP project are: - Administrative delays in signing contracts with producers; - The heaviness of standard contracts offered to producers; - The delay in the payment of bills; - Lack of capacity of cooperatives to operate under a formal contract; - Short timeframe for program implementation that limits the deployment of program strategies. 79 6.2. Lessons Learned Many stakeholders have learned from the LRP experience. Overall, the LRP project has demonstrated that the local sourcing approach is feasible and preferred by all actors, and various stakeholders agree that local products can provide quality, culturally accepted meals to schoolchildren. One SAS member also notes that mobilization of resources and collaboration between various sectors can lead to successful school canteen projects. As explained by PAsDeR, the integrated local approach of the LRP project should be considered for other projects concerning rural communities. When considering the overall LRP approach, CRS staff suggest that, given the many challenges faced by schools, an integrated school canteen program with improved access to safe drinking water, basic sanitation (WASH) and health care would be more appropriate for these populations in general, and for schoolchildren in particular. Finally, lessons regarding administrative processes were also learned. For example, CRS staff note that the administrative process for contracting with producers needs to be revised to alleviate slowdowns. Also, the DRCPV has learned that written agreements are important in developing agreements between parties. Agreements should be well documented to improve collaborations and further support positive relationships. With regard to the approaches or content of the approach, the lessons learned are: - The exclusive use of shea butter is not appreciated by students; - Payments of contributions not made compulsory for access to the canteen do not exclude the poorest; but some poorer schools require additional support for the purchase of condiments to improve the quality of their meals - Heavy standard contracts (length with a lot of information in the content) are difficult to operate for producer associations; these types of contracts with producers should be revised to a simple 1-2 pages format to alleviate the problem of administrative slowness - The contractual models with producer associations that consider the possibilities of advances to producers would enable them to implement training and to respond more effectively to contractual obligations - Specific training according to the needs of the actors and their roles make them more effective for the success of the intervention. Local actors such as the village chief or district chiefs are more likely to be interested in sustaining the initiative than communes that are on a larger scale and are less interested in school canteens. 6.3. Limitations The following limitations should be acknowledged. First, it is generally accepted by the scientific community that a project like LRP needs some time, at least 18-24 months to generate impact at the population level. Conducting an endline evaluation about 11 months after the baseline is certainly not ideal. Second, the very short time for data collection (to meet the deadlines set by 80 CRS), coupled with the variety of questionnaires and interview guides (10 individual interview guides and 6 group interview guides), may have had some impact on data quality. The training and the supervision mechanism put in place were designed to mitigate the risks on data quality. It appears there were few missed opportunities for probing during the qualitative interviews, for example on the positive and negative externalities of the LRP, MGD and WFP systems on the local economy. Third, it appears that out of the 24 Producer Associations identified in the intervention arm (22 were interviewed), only eight were part of the 34 Producer Associations interviewed at baseline. We may have two different populations at baseline and endline, which may limit the relevance of our estimate change over time. In particular, the indicator on the “Percentage of Producer Associations demonstrating increased organizational capacity” is based on only 8 Producer Associations in the intervention group (and 10 in the control group). This small sample size of Producer Associations could have implications for external validity of the relevant indicators. Fourth, collecting data on cost and time is always challenging, as a given cost or time may have different meanings for different persons. For example, what does the cost of transportation and distribution mean if commodities are delivered using project vehicles? We also noticed above that transportation and distribution costs were not available at WFP. Lastly, the quantitative methods do not control for possible cofounders, and did not explore possible threats to the integrity of the evaluation (such as non-equivalence at baseline, non￾compliance, spillover, systematic attrition) or common biases (e.g. confounding bias, selection bias, experimenter bias). 6.4. Recommendations Considering these results at the Producer Association, COGES and student levels, with support from additional qualitative insight, the following is recommended for future consideration of school canteen models: 1. Evaluate PA training needs with members of these associations and involve all members during training. The basis for the LRP training plan was a consultation with GIZ and Suiss Coop to understand what training topics had been covered through their projects to ensure that the same topics were not addressed. The LRP project approach did not take into account information from Producer Associations and opinions of their members to assess training needs. As a result, some training topics didn’t fit the needs of producers. For example, some trainings were replicated seeing that some associations had previously self-financed trainings offered under the LRP project to strengthen the performance of their producers and to be competitive. To avoid replication in the future, training needs should also be evaluated with members of associations. This evaluation can be done at the level of each association, and a training plan considering the needs of all of the associations can be developed. 81 Similarly, CRS’ approach in the LRP project is to train only Producer Association leaders. But this approach has limitations because these leaders do not relay the information received to other members of their associations. To remedy this, one of these strategies can be used: either to involve all the members of the associations during the training sessions and to schedule the training sessions by producer association; or set up a monitoring system which obliges the leaders who participated in the trainings to organize training sessions at the level of their associations. Finally, value chain approaches should be considered in the training program because producer organizations linking to specific value chains are better suited to provide technical support for their members. Rather than to select one representative Producer Association umbrella, such as URPBA to train all the Producers Associations involved in the project, it would be preferable for each Producer Association umbrella to organize training for its members. 2. Consider contractual models that incorporate access to financial resources for producers before the sales process. Most Producer Associations have difficulty accessing credit at the bank level, and the project approach does not allow advance payments to producers. So, due to lack of financial resources, some Producer Associations have not been able to produce the expected quantities on time and under conditions that guarantee quality in accordance with the standards required by the project. For this purpose, CRS can contract with the banks so that they grant credit to the Producer Associations that will be selected after a call for tenders; or proceed with a payment advance that will allow these associations to acquire all the resources necessary to comply with the requirements of the call tender. These pre-financing methods may be specified in the calls for tender and it will be up to each Producer Association to choose the method that suits it. The chosen method will then be specified in the contract signed between CRS and the association itself. 3. Design Contracts with PAs in a format of 1 to 2 pages maximum and in simple and understandable terms that allow producers to better control contract processes. The contract sale process is quite new for most Producer Associations. The contracts offered to them are generally very long and difficult to manage. It would be better to design rather short and precise contracts, explaining in simple terms the different procedures that will be adopted for the order, the purchase, the delivery and the payments. 4. Respect the deadlines set in the contracts, particularly with regard to the purchase and payment of invoices Delays by CRS are reported regarding the purchase and payment of invoices. This causes enormous damage to producers: they face price fluctuations on the market, but also struggle to repay the credits they have contracted to meet the requirements of the call for tender. To avoid these delays, project managers must ensure that all administrative and financial measures are in place before issuing a tender. 82 5. Consider duly signed partnership protocols between the main institutional actors of the project The partnership agreement sustains relations between partners by clarifying the roles and procedural arrangements at each institutional level. In contrast, the absence of agreement may hinder and compromise effective participation. For example, the absence of agreement has prevented actors from DDAEP technicians involved in the LRP project from using official cars for field trips, seeing justification and written documentation is needed prior to arranging such resources. It is therefore important to consider duly signed protocols between the main institutional actors of the project. 6. Transfer supervisory functions to government entities to increase likelihood of sustainability The implementation period of the project is considered very short by the various stakeholders. Producer Associations, COGES, CPC and CLOC are not yet sufficiently equipped to take charge of the various activities without external support. To increase the likelihood of sustainability, supervisory functions should be transferred to government and community entities. Concrete support measures that can be provided to Producer Associations and COGES should be discussed with government and community leaders to ensure the appropriate transfer of responsibility. 7. Involve community leaders in the management of school canteens (COGES) COGES should be open to a maximum of actors and if possible, reflect the diversity of stakeholders (parents, school principal, teachers, farmers, managers of companies or institutions, mayors or their representatives, community leaders/local elected representatives, religious leaders ...). This diversity of actors makes it possible to guarantee a diversity of sources of support for the school canteen, and therefore an improvement in the quality of meals served to students. Community leaders such as village chiefs, district chiefs and religious leaders are key people to consider for better management of canteens. Being close to the community, it is easier for them to mobilize the community and encourage them to get involved in the smooth operation of school canteens. Similarly, these community leaders are likely to provide financial support that could solve the problem of the provision of condiments that many school canteens face. 8. Encourage governmental authorities to review current plans, programs or school feeding strategies at national level and to allocate financial resources to the town halls for the management of school canteens. All government and community entities have appreciated the LRP model for school canteens, and are in favor of its continuity and scope. Current school feeding plans and programs at the national level must therefore be revised to include an approach based mainly on the supply of school canteens with local products. For this approach to be fully functional, it must be managed by a local entity such as town halls. But, the current political framework unfortunately does not allow the town halls to have a certain financial autonomy to manage the school canteens of their localities. All costs of the committees (procurement and supervision) formed within the town halls under the LRP project were covered by the project itself. Town halls have no sustainable 83 source of funding to support these committees at the end of the project. Advocacy should therefore be addressed to government authorities for the review of school feeding programs and for the decentralization of the school canteen management process. 9. Promote an integrated approach including the WASH component Improving child nutrition should be accompanied by complementary measures that strengthen the nutritional status of children. The WASH component is a very important element for the smooth operation of school canteens. It is therefore appropriate to associate with any school canteen project the establishment of drinking water points accessible to students, and necessary for the preparation of meals, for the dishes, cleaning and sanitation of the kitchen. In addition, awareness activities on hygiene during preparation and consumption of meals and on environmental sanitation should be incorporated into future school canteen efforts. 10. Reduce financial contributions and promote in-kind contributions from parents Being in rural areas, parents are often solicited to contribute financially to the operation of schools. This makes it difficult to get another financial contribution from them for school canteens. The adoption of a school canteen model promoting an in-kind contribution from parents can be a solution to this problem. In addition, the in-kind contribution of parents is also part of a vision of sustainability for school canteens. Two different models can be adopted: - A supply of canteens by parents with the products of their own harvests. This model has the advantage that it allows a direct involvement of parents of students in the project and that the food supplied will be directly consumable. But this model does include a risk of poorly diversified food supply, irregular supply, and high dependence on the quality and quantity of the families' crops. - The establishment of collective school fields that will be cultivated by parents of students. For this, it is first necessary to discuss this possibility with the local administrative authorities. With this model, it is possible to couple the food crop to the cash crop (for the purchase of unproduced food). This gives some autonomy to the school in school canteen management. Small spaces can also be reserved in these fields for the cultivation of condiments necessary for seasoning meals in the canteen. But these fields will require trained and available staff to organize and monitor activities to ensure optimal production. Parents can also participate in the construction of infrastructure (kitchens, shops, improved stoves, handwashing devices, and school fields), collection of firewood and water. 11. Identify means of motivation for cooks and storekeepers to maintain compliance and quality The current means of remuneration for cooks and storekeepers pose important problems. They are paid from weekly or monthly students/parents subscription for access to school canteen. However, in most cases, students/parents do not assume it. In practice, cooks and storekeepers work without pay. Although these tasks are done on a voluntary basis, regular motivation of these actors is an important factor in sustaining the activities of the canteen. The design of school canteen models should include this aspect. Here are different possibilities: 84 - Inclusion of canteen fees in school fees and the elimination of canteen-specific payments; - Partial payment of the costs of remuneration of cooks and storekeepers by local authorities (village chiefs, district chiefs, mayors ...). Should this option be chosen, these local authorities should be included in the COGES and budgeted as soon as the project is designed; - Establishment of income-generating activities by schools (cash crop in school fields, small livestock). These different possibilities can be combined to ensure not only a long-term commitment of these actors, but also a better quality of meals served in the canteen. 12. Consider a greater diversity of dishes, combining local meals with some imported products for school canteens The diversity of meals in the school canteen seems to be an essential aspect for canteen utilization. The LRP approach prioritizes the consumption of similar meals at home and at school, however the quantitative survey reveals that some students have preferences for meals that they do not consume regularly at home. It is therefore useful to consider a supply model combining local products with some imported products for school canteens. This diversification of food products would also ensure a more o complete nutritional intake for students. 13. Obtain student feedback and opinion regarding meal plans at the local level to ensure greater satisfaction in regard to meal quality The quality of meals was considered poor in some municipalities for two reasons: the regular use of shea butter for cooking meals; and the absence of condiments and foods containing animal protein (meat, fish). In general, student preferences vary from one community to another. It is therefore important for COGES to ensure that student feedback concerning quality of meals is taken into consideration for future meal planning. To improve the quality of meals, a local oil (peanut oil, coconut oil, palm oil) can be used alternately with shea butter. With regard to the intake of protein-containing foods, two possibilities are presented: - Sources of input can be developed in schools such as fish farming and small livestock. This will also be an income-generating activity that can be mobilized for the functioning of the canteen (motivation of the cooks, supply of other foodstuffs); - Contracts can be signed with local breeders (as in the case of Producer Associations). 14. Solicit a sufficient number of cooks to avoid school girls’ involvement in cooking activities Qualitative interviews with students indicate that some school canteens are obliged to involve young girls in the preparation of meals in the canteen so that meals are ready and served on time. This has the potential to negatively impact their academic performance. Contracts with schools benefiting from school canteens must clearly specify the prohibition of any participation of students in the management of such canteens. All the operating and management procedures of the canteen should be discussed with school principals before the implementation of the canteen. Similarly, a preliminary evaluation must be made to determine the number of people 85 needed for good kitchen management (preparation, cooking, dishes, hygiene, wood …). Project leaders must ensure that the number determined for each school is available through community volunteers and respected throughout the project implementation period. 7. Appendix Tables – Summary Results Tables AX1. Producer Associations Tables AX2. COGES Tables AX3. Students Tables AX4. Cost and Time Differentials between LRP and the MGD and WFP systems 8. Annexes Annex 1. Data collection tools Annex 4. Highlights of key review meetings i 8. Annexes Table of contents Annex 1: Data Collection Tools.................................................................................................................1 Part I: Impact Study (Endline) ...................................................................................................................1 I. Quantitative Surveys......................................................................................................................1 1. COGES - Representatives............................................................................................................1 2. COGES - Cooks............................................................................................................................7 3. COGES – Storekeepers..............................................................................................................10 4. Students...................................................................................................................................13 5. Students – Attendance.............................................................................................................16 6. Producer Associations..............................................................................................................17 7. Individual Producers.................................................................................................................22 II. Individual Interviews....................................................................................................................27 1. School Principals.......................................................................................................................27 2. Parent Associations..................................................................................................................30 3. Government authorities...........................................................................................................32 4. Municipal authorities (Mayor / Representative in charge of economic/social affairs).............34 5. Local Supply Committees (CPC) and Local supervision committees (CLOC) .............................37 6. Producer Associations..............................................................................................................38 7. OP Faitières..............................................................................................................................40 8. PASDeR/ProAgri .......................................................................................................................43 9. LRP Staff in charge of program.................................................................................................45 III. Focus group discussions...........................................................................................................46 1. Parents.....................................................................................................................................46 2. Mothers....................................................................................................................................48 3. Teachers...................................................................................................................................50 4. COGES ......................................................................................................................................52 5. Producer Associations..............................................................................................................55 6. Students...................................................................................................................................58 IV. Direct Observations..................................................................................................................60 1. COGES Observation (Cooks and Storekeepers) ........................................................................60 2. COGES Observation (community involvement in the buying process) .....................................62 3. Inventory of infrastructures .....................................................................................................63 ii Second Part: Comparative study on costs and times..............................................................................64 I. Quantitative Surveys....................................................................................................................64 1. Sheet for LRP/FFE/PAM Staff in charge of procurement and logistics......................................64 2. COGES - Representative ...........................................................................................................67 3. Students...................................................................................................................................67 II. Individual Interviews....................................................................................................................67 1. LRP/FFE/PAM staff in charge of procurement and logistics .....................................................67 2. Communes...............................................................................................................................67 3. Government authorities...........................................................................................................67 III. Focus Group Discussions..........................................................................................................67 1. COGES ......................................................................................................................................67 Annex 4. Highlights of key review meetings...........................................................................................72 September 19th Webinar.....................................................................................................................72 Discussion with FFE and PAM staff......................................................................................................72 1 Annex 1: Data Collection Tools Part I: Impact Study (Endline) I. Quantitative Surveys 1. COGES - Representatives Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) COGES - Representatives’ Questionnaire (LRP, FFE2, PAM) Hello. My name is ___________________, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou". CRS implemented school canteens in some Borgou schools (mechanisms designed to provide food to students on school days). I would like to speak with you as a member of the committee that manages the school canteen. We are interested in the management capacity of the canteen by the COGES, as well as your appreciation of any changes caused by the project. The survey will take about 20 minutes to complete. Answers you give to the questions will be recorded using a smartphone. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Q01. Do you wish to participate in this survey? (0 = No; 1 = Yes) Q02. Can we start now? (0 = No; 1 = Yes) Q03. Date: _____________________ Start time of the interview: ___H / ___Min Q04. Type of school: 1 = LRP Intervention, 2 = LRP control, 3 = FFE2/PAM Comparison Q05. Commune: Parakou 1; N'Dali=2; Pèrèrè=3; Tchaourou=4; Kalalé=5; Gogounou=6; Kandi=7; Malanville=8 Q06. School Name and ID: _________________________________________ Section A. Respondent Identification Q07. Respondent’s sex: 0=Female; 1=Male Q08. Respondent’s name: ______________________________________________________ Q09. Respondent’s age: __/__ Q10. Respondent's Level of Education: 0 = none, 1 = Alphabetization in local language, 2 = Primary, 3 = Secondary first cycle, 4 = Secondary second cycle, 5 = University Q11. If Q10 = 2-5, last class completed: 11 = CI, 12 = CP, 13 = CE1, 14 = CE2, 15 = CM1, 16 = CM2, 21 = 6 ème, 22 = 5ème, 23 = 4ème, 24 = 3ème, 31 = 2nde, 3 2= 1ère, 33 = Tle, 41 = Bac+1, 41 = Bac+2, 43 = Bac+3, 44 = Bac+4, 45 = Bac+5, 46 = Bac+6, 47 = Bac+7 [IF Q04 (TYPE OF SCHOOL) = 3 (FFE2/PAM COMPARISON), GO DIRECTLY TO SECTION H] Section B: Knowledge of canteen management before the implementation of the LRP project (Type of school = LRP Intervention, LRP Control) Q12. Does this school have a canteen right now (aside from food vendors)? 0 = No, 1 = Yes (if Yes, skip to Q16) Q13. Has this school ever had a canteen in the past? 0 = No, 1 = Yes (if No, skip to Q44) Q14. Why did the canteen stop activities? 1 = end of the supporting project, 2 = Bad management, 3 = other reasons Q15. If other reasons, please explain. Q16. Is/Was there a COGES? 0 = No, 1 = Yes Q17. Did any person involved in the canteen management receive training on cooking food? 0 = No, 1 = Yes (if No, skip to Q20) Q18. When was the last training on cooking food? 2 Q19. How many people have been trained for this school? Q20. What are the topics on which they were trained? a. How to cook some commodities to ensure good nutritional meal quality b. Quantity of commodities c. How to make a traditional firebox to ensure comfort during cooking d. Other (specify) e. Don’t remember Q21. Did any person involved in the canteen management receive any training on stock management? 0 = No; 1 = Yes (If No, Skip to Q24) Q22. When was the last training on stock management? Q23. How many people have been trained for this school? Q24. What are the topics on which they were trained? a. How to fill in the stock registers b. How to organize stock products c. How to handle spoiled/expired foods d. Other (specify) e. Don’t remember Q25. Did any person involved in the canteen management receive any training on hygiene? 0=No; 1=Yes (If No, Skip to Q29) Q26. When was the training on hygiene? Q27. How many persons have been trained on hygiene for this school? Q28. What are the topics on which they were trained? a. Hand washing b. Kitchen cleaning c. Cleaning utensils d. Garbage management in kitchens e. Food preparation f. Other (specify) g. Don’t remember Q29. Did any person involved in the canteen management receive any training on nutritious foods? 0 = No; 1 = Yes (If No, Skip to Q33) Q30. When was the training on nutritious foods? Q31. How many persons have been trained on nutritious foods for this school? Q32. What are the topics on which they were trained? a. Energy foods (carbohydrate and lipids) b. Health foods (fruits and vegetables) c. Mental force food (protein) d. Other (specify) e. Don’t remember Q33. Did any person involved in the canteen management receive any training on food storage? 0 = No; 1 = Yes (If No, Skip to Q37) Q34. When was the training on food storage? Q35. How many persons have been trained on food storage for this school? Q36. What are the topics on which they were trained? a. Cleaning the storeroom and its environment before storage b. Commodity piling techniques c. Hygiene during storage d. Commodity pests and rodents e. Other (specify) 3 f. Don’t remember Q37. Did any person involved in the canteen management receive any training on ration measurement? 0 = No; 1 = Yes (If No, Skip to Q41) Q38. When was the training on ration measurement? Q39. How many persons have been trained on ration measurement for this school? Q40. What are the topics on which they were trained? a. Measurement unit b. Rationing c. Other (specify) d. Don’t remember Q41. Did any person involved in the canteen management receive any training on canteen data collection? 0 = No; 1 = Yes (If No, Skip to Q45) Q42. When was the training on canteen data collection? Q43. How many persons have been trained on canteen data collection for this school? Q44. What are the topics on which they were trained? a. Student satisfaction b. Quality of the meals c. Quantity of meals d. Other (specify) e. Don’t remember Q45. Do you understand why school canteens are important? 0 = No; 1 = Yes (If No, skip to Q47) Q46. If yes, what do school canteens contribute to? a. Improved school enrolment b. Improved school attendance c. Improved students’ nutritional status d. Improved performance e. Poverty reduction f. Other (specify) g. Don’t know If Q12 = No, stop Q47. Names of the cooks and the storekeepers Name Role Mobile phone number [IF Q04 (TYPE OF SCHOOL) = 2 (LRP CONTROL), STOP] Section C. Knowledge of canteen management after the implementation of the LRP project (type of school = LRP Intervention) Q48. When was the LRP canteen installed (or started their activities) in this school? (2017, 2018 or 2019) Q49. In what month did the LRP canteen start activities in this school? (January = 1, February = 2, March = 3, April = 4, May = 5, June = 6, July = 7, August = 8, September = 9, October = 10, November = 11, December = 12) Q50. What are the objectives / results of the LRP canteen? (Multiple responses possible) a. Use of local products for school feeding programs (01) 4 b. Increased local capacity for food production (02) c. Improvement of students’ nutrition (regularity, quantity, quality, hygiene, etc.) (03) d. Variation of meals of students (04) e. Promoting the quality of culturally accepted meals for students (05) f. Reduction of student absenteeism (or delay) (06) g. Reduction of school drop-out (07) h. Increased academic success (08) i. Strengthening the organizational and operational capacities of associations (09) (COGES 901; OPA 902; APE 903; Directors 904; Teachers 905; Parents 906; Mothers 907; Students 908; Local authorities 909; Other authorities 910; Other 988 (specify) ………) j. others (specify) : ………………………………………………….. Q51. In your opinion, has the school canteen: a. reduced school absenteeism? b. reduced student delay? c. reduced school drop-out? d. improved the number of students enrolled in the school? Q52. Has the school canteen improved the school performance of students? (0 = No, 1 = Yes) (If No, skip to Q53) Q52.1. If Yes, what has improved? a. Punctuality b. Attendance (active participation in class, discipline, etc.) c. Punctuality and diligence d. Report card e. Frequency (number or percentage) of transition to higher class Q53. Has the school canteen improved student nutrition? (0 = No, 1 = Yes) (If No, skip to Q54) Q53.1. If Yes, what has improved? a. the meal schedule is regular b. meals are sufficient (quantity) c. meals are varied and well prepared (quality) d. the meals are hygienic Q54. Has the school canteen helped to reduce poverty in the community? (0 = No, 1 = Yes) (If No, Skip to Q56) Q55. If Yes, why? a. Flow of agricultural food products b. Improvement of economic activities c. Increase in jobs d. Revenue improvement e. Disease reduction in school f. Increase in school attendance g. Other (specify): …….. Q56. Was it necessary to implement a canteen in the school? a. Yes; Main reason: ………………. b. No; Main reason: ………………. Q57. Do you receive products for the school canteen on time? (Yes = 1, No = 0) Q58. Are the quantities received sufficient to cover the needs of the children in the school? (Yes = 1, No = 0) Q59. Are the food products of good quality? (Yes = 1, No = 0) Section D. Internal functioning of COGES (type of school = LRP intervention) 5 Q60. Do you have rules and regulations for COGES? (0 = No, 1 = Yes) Q61. Are COGES meetings scheduled? (0 = No, 1 = Yes) (If No, skip to Q63) Q62. If yes, what is the frequency of COGES meetings? 1 = occasionally, 2 = regularly; specify the regularity of the meetings: .......... Q63. How do you call meetings? (Multiple responses) a. Regularly scheduled b. Invitation letter c. Phone call d. Social networks / SMS e. Oral communication Q64. What are the decision-making modes? a. Consensus b. Restricted committee c. President of COGES d. Other (specify): ………… Q65. Is there a mechanism for monitoring and evaluating the objectives/results of the canteen? (0 = No, 1 = Yes) (If No, skip to Q67) Q66. What are the means of monitoring? a. satisfaction survey with students b. inventory control c. monitoring the distribution of meals Q67. Is there a permanent mechanism for collecting complaints, observations and suggestions? a. suggestion box b. focal points (class delegates, teachers, nurses) c. monitoring and evaluation sheets(daily or weekly) d. other (specify): ……… Q68. Do COGES members mobilize the entire community for the smooth operation of the canteen (loading, unloading of products, volunteers in various activities, local contributions)? (0 = No, 1 = Yes) Q69. Do the COGES members ensure the mobilization of resources (contributions, provisions to supplement the contributions of the LRP, provision of infrastructures, etc.)? (0 = No, 1 = Yes) Q70. Do all COGES members have the same level of information? (0 = No, 1 = Yes) Section E. Implementation of learning outcomes in canteen management Q71. Does your school have a hand washing device? (0 = No, 1 = Yes) (If No, skip to Q73) Q72. Which ones: a. Tippy-Tap b. Handwashing device (DLM) c. Other (specify): …………………… Q73. When do students wash their hands? [Unique choice] (1= before each meal, 2 = after each meal, 3 = before and after each meal, 8 = other (to be specified) ...............) Q74. What items do students use to wash their hands at school? a. soapy water b. simple water c. disinfected water d. Other (specify)………………… Q75. Do you have fixed hours for the service of daily meals to students? (0 = No, 1 = Yes) (If No, skip to Q75.2) Q75.1. If YES, are these schedules regularly respected? (0 = No, 1 = Yes) Q75.2. Is there any menu programming? (If No, skip to Q74.5) 6 Q75.3. If YES, how often is this programming established? (1 = per day, 2 = per week, 3 = per month, 8 = other (specify) ........................ Q75.4. Are the planned menus regularly respected? (If yes, skip to Q80) Q75.5. If No, why? a. Sold out b. Leftover meal c. Other (specify)……………………….. Section F. Lessons Learned and Sustainability of the Project (School Type = LRP Intervention) Q76. According to you, is it necessary to renew this school canteen experience? 1 = No, 2 = Yes If yes, why? a. The canteen relieves parents b. The canteen promotes children's nutritional balance c. The canteen promotes attendance and improves academic performance Other (specify)……………………….. [WE WILL NOW ASK YOU SOME QUESTIONS TO MAKE A COMPARISON WITH OTHER PROJECTS WITH SIMILAR OBJECTIVES]. Section G. Evaluation of costs per canteen meal (School type = LRP intervention, FFE2/WFP comparison) Q77. How much does an average meal cost in the canteen? _/ _ / _ / Q78. Is the cost different for different meals? (0 = No, 1 = Yes) (If No, skip to Q80) If Yes, a. What is the minimum price? _/ _ / _ / b. What is the maximum price? _/ _ / _ / Q79. If yes, why? a) The price of foods b) The time taken to prepare meals c) Other (specify) Q80. Is the cost of a meal in the canteen different from that of other school food vendors? (0 = No, 1 = Yes) (If No, skip to Q82) Q81. If yes, is it higher or lower? (0 = higher, 1 = lower) Q82. Is the cost of a meal affordable for all students in the school? (0 = No, 1 = Yes) Q83. Do you receive products for the school canteen on time? (0 = No, 1 = Yes) Q84. Are the quantities received sufficient to cover the needs of the children in the school? (0 = No, 1 = Yes) Q85. Are the food products of good quality? (0 = No, 1 = Yes) End date and time Thank you for your collaboration 7 2. COGES - Cooks Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) COGES - Cooks’ Questionnaire (LRP) INTRODUCTORY FORM Hello. My name is ___________________, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou". CRS implemented school canteens in some Borgou schools (mechanisms designed to provide food to students on school days). I would like to speak with you as a cook with the school canteen. We are interested in the management capacity of the canteen by the COGES, as well as your appreciation of any changes caused by the project. The survey will take about 20 minutes to complete. Answers you give to the questions will be recorded using a smartphone. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. 1. Do you wish to participate in this survey? (0 = No; 1 = Yes) 2. Can we start now? (0 = No; 1 = Yes) 3. Date: |__|__|.|__|__|.2019 Start time: |__|__|h|__|__|min 4. Type of school: LRP Intervention = 1; LRP Control = 2 5. Commune: Parakou = 1 ; N'Dali = 2 ; Pèrèrè = 3 ; Tchaourou = 4 ; Kalalé = 5 ; Gogounou = 6 ; Kandi = 7 ; Malanville = 8 6. School: |__|__|__| Section A. Respondent Identification Q1. Respondent’s sex: (0 = Female; 1 = Male) Q2. Respondent’s name: …………………………. Q3. Respondent’s age: |__|__| Q4. Respondent’s education: 0 = none, 1 = Alphabetization in local language, 2 = Primary, 3 = Secondary first cycle, 4 = Secondary second cycle, 5 = University Q5. If Q4 = 2-5, last class completed: 1 1= CI ; 12 = CP ; 13 = CE1 ; 14 = CE2 ; 15 = CM1 ; 16 = CM2 ; 21 = 6 ème ; 22 = 5ème ; 23 = 4ème ; 24 = 3ème ; 31 = 2nde ; 32 = 1ère ; 33 = Terminale ; 41 =Bac+1 ; 41 = Bac+2 ; 43 = Bac+3 ; 44 =Bac+4 ; 45 =Bac+5 ; 46 = Bac+6 ; 47 = Bac+7 Section B. Knowledge of food and canteen management (School type = LRP intervention, LRP control) Q6. Do you know some commodities which provide energy? 0=No; 1=Yes (If No, skip to Q8) Q7. If yes, give one example Q8. Do you know some commodities which provide mental force (protein)? 0=No; 1=Yes (If No, skip to Q10) Q9. If yes, give one example Q10. Do you know some commodities which provide health? 0=No; 1=Yes (If No, skip to Q12) Q11. If yes, give one example Q12. Do you know some consequences of malnutrition? 0=No; 1=Yes (If No, Skip to Q14) Q13. If yes, give one example Q14. Does your school have a device with a faucet for hand washing? 0 = No, 1 = Yes Q15. What do students use for hand washing at school? Q16. How often should a kitchen be swept to ensure proper hygiene? 1= Before each utilization, 2= After each utilization, 3= At the end of each day, 4= when it is needed, 5= other (specify), 6= Don’t know 8 Q17. How often should a kitchen be washed to ensure proper hygiene? 1= Before each utilization, 2= After each utilization, 3= At the end of each day, 4= when it is needed, 5= other (specify), 6= Don’t know Q18. How often should pots and pans be washed to ensure proper hygiene? 1= Before each utilization, 2= After each utilization, 3= At the end of each day, 4= when it is needed, 5= other (specify), 6= Don’t know Q19. Is it important to boil the eating utensils (spoons, forks) after each use? 0=No; 1=Yes (If No, skip to Q21) Q20. If yes, why? 1 = to prevent the transmission of sicknesses from one user to another, 2 = other (specify), 3 = Don’t know Q21. How many times does (did) the canteen feed the students per day? 0=No; 1=Yes Q22. Does (did) the canteen serve breakfast? 0=No; 1=Yes (If No, skip to Q24) Q23. If Yes: Does (did) the quantity given to each student depend on their age? 0=No; 1=Yes Q24. Does (did) the canteen serve lunch? 0=No; 1=Yes (If No, skip to Q25) Q25. If Yes: Does (did) the quantity given to each student depend on their age? 0=No; 1=Yes Q26. Does (did) the canteen serve afternoon tea? 0=No; 1=Yes (If No, skip to Q28 Q27. If Yes: Does (did) the quantity given to each student depend on their age? 0=No; 1=Yes Q28. Does (did) the canteen serve dinner? 0=No; 1=Yes (If No, Stop) Q29. If Yes: Does (did) the quantity given to each student depend on their age? 0=No; 1=Yes [IF TYPE OF SCHOOL = LRP CONTROL, STOP]. Section C. Knowledge of project and practices in food, nutrition and hygiene (School type = LRP intervention) Q30. In what year was the LRP canteen installed (or started their activities) in this school? (2017, 2018 or 2019) Q31. In what month did the LRP canteen start their activities in this school? (January = 1, February = 2, March = 3, April = 4, May = 5, June = 6, July = 7, August = 8, September = 9, October = 10, November = 11, December = 12) Q32. What are the objectives / results of the LRP canteen? (Multiple responses possible) a. Use of local products for school feeding programs (01) b. Increased local capacity for food production (02) c. Improvement of students’ nutrition (regularity, quantity, quality, hygiene, etc.) (03) d. Variation of student meals (04) e. Promoting the quality of culturally accepted meals for students (05) f. Reduction of absenteeism (or delay) of students (06) g. Reduction of school drop-out (07) h. Increased academic success (08) i. Strengthening the organizational and operational capacities of associations (09) (COGES 901 ; OPA 902 ; APE 903 ; Directors 904 ; Teachers 905 ; Parents 906 ; Mothers 907 ; Students 908 ; Local authorities 909 ; Other authorities 910 ; Other 988 (specify) ………) j. others (specify) : ………………………………………………….. Q33. Has the school canteen improved student nutrition? (0 = No, 1 = Yes) (If No, skip to Q35) Q34. If yes, what has improved? (Multiple responses possible) a. the meal schedule is regular (1) b. the meals are sufficient (quantity) (2) c. the meals are varied and well prepared (quality) (3) d. the meals are hygienic) (4) e. Other (specify): …………. (8) Q35. Was it necessary to implement a canteen in the school? 9 a. Yes=1; Main reason: ………………. b. No=0; Main reason: ………………. Section D: Training received in canteen management (Type of school = LRP Intervention) Q36. Has the project (LRP) led to the development of your cooking skills? (0 = No, 1 = Yes) If yes, what capacity did you acquire? What training did you receive? (Multiple responses possible) Theme Effectiveness Relevance Cost-effectiveness Sustainability Ac qui red cap aci ty Have you receiv ed traini ng? Total numb er of traini ngs Numb er of peopl e traine d Did you need this training? Do you think the training could have had other content? What other content would you have liked? (list them) Is it necessary to continually renew this training? (yes, no) Mai n reas on a. cooking food b. hygiene c. nutritious food (02) d. canteen planning (03) e. ration measuremen t (04) f. food monitoring (05) g. other (specify) (88) Q37. How often do cooks prepare meals promoted by the LRP project? (List the dishes you have prepared during the last week) Monday Tuesday Wednesday Thursday Friday 1 st meal 2 nd meal 3 rd meal Section E. Lessons Learned and Sustainability of the Project (School Type = LRP Intervention) Q38. In your opinion, is it necessary to renew this school canteen experience? (0 = no, 1 = yes) (If No, skip to Q39) Q39. If Yes, why? ……………………………………………………………………. Q40. Names and roles of individual cooks Q47. Names of the cooks and the storekeepers Name Role Mobile phone number End date and time Thank you for your collaboration 10 3. COGES – Storekeepers Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) COGES – Storekeepers’ Questionnaire (LRP) INTRODUCTORY FORM Hello. My name is ___________________, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou". CRS implemented school canteens in some Borgou schools (mechanisms designed to provide food to students on school days). I would like to speak with you as a storekeeper with the school canteen. We are interested in the management capacity of the canteen by the COGES, as well as your appreciation of any changes caused by the project. The survey will take about 20 minutes to complete. Answers you give to the questions will be recorded using a smartphone. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently Q01. Do you wish to participate in this survey? (0=No; 1=Yes) Q02. Can we start now? 0=No; 1=Yes Q03. Date:: |__|__|.|__|__|.2019 Start time: |__|__|h|__|__|min Q04. Type of school: LRP Intervention =1; LRP Control =2; Q05. Commune: Parakou = 1 ; N'Dali = 2 ; Pèrèrè =3 ; Tchaourou = 4 ; Kalalé=5 ; Gogounou=6 ; Kandi=7 ; Malanville=8 Q06. School: |__|__|__| Section A. Respondent Identification Q07. Respondent’s sex: 0=Female; 1=Male Q08. Respondent’s name: …………………………. Q09. Respondent’s age: |__|__| Q10. Respondent’s education: 0=None, 1= Alphabetization in local language, 2=Primary, 3= Secondary first cycle, 4=Secondary second cycle, 5=University Q11. If Q10 = 2-5, last class completed: 11= CI ; 12= CP ; 13 = CE1 ; 14 = CE2 ; 15=CM1 ; 16= CM2 ; 21= 6 ème ; 22= 5ème ; 23= 4ème ; 24= 3ème ; 31= 2nde ; 32= 1ère ; 33= Terminale ; 41=Bac+1 ; 41=Bac+2 ; 43=Bac+3 ; 44=Bac+4 ; 45=Bac+5 ; 46=Bac+6 ; 47=Bac+7 Section B. Management of a storage warehouse Q12. How often should a food storeroom be swept to ensure proper hygiene? 1=After each utilization, 2= At the end of each day, 3= when it is needed, 4= other (specify), 5= Don’t know Q13. How often should a food storeroom be washed to ensure proper hygiene? 1=After each utilization, 2= At the end of each day, 3= when it is needed, 4= other (specify), 5= Don’t know Q14. Do you use salt water to wash the storeroom? 0 = No, 1 = Yes Q15. Does (did) the school have a register to record the type and quantity of commodities received for the canteen? 0=No; 1=Yes (If No, skip to Q17) Q16. If Yes: Do (did) you fill the register at the time you received the commodities? 0=No; 1=Yes Q17. Does (did) the school have a register to record the type and quantity of commodities used for the canteen every day? 0=No; 1=Yes (If No, skip to Q19) Q18. If Yes: Do (did) you fill the register at the time you used the commodities for the canteen? 0=No; 1=Yes [IF Q04 (TYPE OF SCHOOL) = 2 (LRP CONTROL), STOP]. 11 Section C. Knowledge of canteen/stock management in the LRP project (Type of school = LRP Intervention) Q19. In what year was the LRP canteen installed (or started their activities) in this school? (2017, 2018 or 2019)? /__/__/__/__/ Q20. In what month did the LRP canteen start their activities in this school? (January = 1, February = 2, March = 3, April = 4, May = 5, June = 6, July = 7, August = 8, September = 9, October = 10, November = 11, December = 12) Q21. What are the objectives/results of the LRP canteen? (Multiple responses possible) a. Use of local products for school feeding programs (01) b. Increased local capacity for food production (02) c. Improvement of students’ nutrition (regularity, quantity, quality, hygiene, etc.) (03) d. Variation of student meals (04) e. Promoting the quality of culturally accepted meals for students (05) f. Reduction of absenteeism (or delay) of students (06) g. Reduction of school drop-out (07) h. Increased academic success (08) i. Strengthening the organizational and operational capacities of associations (09) (COGES 901 ; OPA 902 ; APE 903 ; Directors 904 ; Teachers 905 ; Parents 906 ; Mothers 907 ; Students 908 ; Local authorities 909 ; Other authorities 910 ; Other 988 (specify) ………) j. other (specify) : ………………………………………………….. Q21. Has the school canteen improved student nutrition? (0 = No, 1 = Yes) (If No, skip to Q23) Q22. If yes, what has improved? (Multiple responses possible) a. The meal schedule is regular (1) b. Meals are sufficient (quantity) (2) c. Meals are varied and well prepared (quality) (3) d. The meals are hygienic (4) e. Other (specify): …………. (8) Q23. Was it necessary to implement a canteen in the school? Q23.1. If Yes; Main reason: ………………………………………………………………… Q23.2. If No; Main reason: ………………………………………………………………… Section D. Training received in canteen management (Type of school = LRP Intervention) Q24. Did the LRP project provide capacity to storekeepers? (0 = No, 1 = Yes) (If No, skip to Q26) Q25. If yes, what capacity did you acquire? What training did you receive? (Multiple responses possible) Theme Effectiveness Relevance Cost-effectiveness Sustainability Q15.1. Acquir ed capacit y (0 = No ; 1 = Yes) Q15.2. Have you receive d trainin g? Q15.3. Total numbe r of trainin gs Q15.4 . Numb er of peopl e traine d Q15.5. Did you need this training? (0 = No ; 1 = Yes) Q15.6. Do you think the training could have had other content? Q15.7. What other content would you have liked? (list them) Q15.8. Is it necessary to continuall y renew this training? Q15.9. Main reason a. Stock management b. Hygiene 12 c. Food storage d. other (specify) Section E. Lessons Learned and Sustainability of the Project (School Type = LRP Intervention) Q26. According to you, is it necessary to renew this school canteen experience? (0 = no, 1 = yes) Q26.1. If Yes, why? Main reason……………………………………………………………………… Q26.2. If No, why? Main reason …………………………………………………………………… End date and time Thank you for your collaboration 13 4. Students Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Students’ questionnaire (LRP, FFE2, PAM) Hello. My name is ___________________, and I am working with Catholic Relief Services (CRS) and their evaluation partner EVIHDAF. We are conducting a study and would like to ask you questions about your nutrition and the foods you eat at home and in your school canteen. The information collected will help CRS to understand how school canteens function in terms of quality and frequency of meals consumed. The survey will take about 30 minutes to complete. The information you provide will be kept confidential and will not be shared with other persons. Your participation in this survey is voluntary and you can choose not to answer any questions or not to participate. If you would like to know more about the survey, you can contact the following persons at CRS and EVIHDAF [OFFER THE CONTACT INFORMATION OF CRS AND EVIHDAF]. Do you have any questions for me? [ANSWER THE QUESTIONS]. Q01. Do you wish to participate in this survey? (0=No; 1=Yes) Q02. Can we start now? (0=No; 1=Yes) Section A. Sociodemographic characteristics Q03. Date: /__/__/ - /__/__/- 2019 Start time: /__/__/ H /__/__/ min Q04. Zone Type: 1 = LRP Intervention; 2 = LRP control; 3 = FFE2/PAM Comparison Q05. Commune: 1 = Parakou; 2 = N'Dali ; 3 = Pèrèrè ; 4 = Tchaourou ; 5 = Kalalé ; 6 = Gogounou; 7 = Kandi ; 8 = Malanville Q05. School Name and ID (ID with 2 digits) /__/__/ Q07. Respondent’s sex: 0=Female; 1=Male Q08. Respondent’s name………………………………………… Q09. Respondent’s age /__/__/ years Q10. Respondent’s grade (1 = CI; 2 = CP; 3 = CE1; 4 = CE2; 5 = CM1; 6 = CM2) Q11. Does your household have any of the following items? Q12. What is the primary occupation of the head of your household? 0=Unemployed, 1=Agricultural Work, 2=Farmer 3=Day labourer, 4=Trader, 5= Office worker, 6=Artisan/Craftsman, 7= Retired, 8= other (specify)………………………… [IF Q04 (SCHOOL TYPE) = 3 (FFE2/PAM COMPARISON), SKIP TO SECTION D]. Household Item Does your household have : (0=no, 1=yes) a) Electricity b) Solar energy c) Power generator d) Radio e) TV f) Phone g) Refrigerator h) Bicycle i) Motor bike j) Car or Truck K) Cart l) Tractor m) Land or farming n) Land in city for building 14 Section B. Food intake (School type = LRP Intervention, LRP Control) Q13. Do you usually eat breakfast at home? (0=No; 1=Yes) Q14. Do you usually eat lunch? (0=No; 1=Yes) Q15. Do you usually eat dinner? (0=No; 1=Yes) Q16. Did you eat at home (in the morning) before departing for school yesterday? (0=No; 1=Yes) Q17. Did you eat during school yesterday (or the last school day)? (0=No; 1=Yes) Q18. Now I would like to ask you about meals you ate yesterday. What meals did you eat yesterday? (FOR EACH MEAL, ASK SUBSEQUENT QUESTIONS ABOUT MEAL COMPOSITION AND LOCATION) Name of the meal Describe the meal composition (e.g : rice+ cowpea+ tomato soup made of palm oil with egg and cow cheese) Where did you eat the meal? 1=at home; 2= at school canteen; 3= at school vendors; 4= elsewhere Q19. Food items, (multiple responses possible) a. Porridge b. Tea or coffee c. Any food made of maize d. Any food made of rice e. Any food (bread, noodles etc.) made from cereals (sorghum, millet, wheat, etc.) f. Sweet potatoes that are yellow or orange inside, squash, carrots g. Any foods made of fruit and green vegetable h. White potatoes, white yams, cassava, or any other foods made from roots i. Any dark green leafy vegetables j. Ripe mangos, papaya or watermelon k. Any other fruits and vegetables l. Liver, kidney, heart or other organ meats m. Any meat, such as beef, pork, goat, sheep, chicken, or duck n. Eggs o. Fresh or dried fish or shellfish p. Cowpea q. Soy, peas, lentils, nuts, or other legumes r. Cheese, yogurt, or other milk products s. Shea butter t. Any other oil, fats, or butter, or foods made with any of these items u. Any sugary foods such as chocolates, sweets, candies, pastries, cakes, or biscuits etc. v. Any other solid or semi-solid food w. Grubs, snails, insects, other small protein food x. Foods made with red palm oil, palm nut, or palm nut pulp Section C. Food at school (Zone type = LRP Intervention, LRP Control) Q20. Is there a canteen in your school? (0 = No 1 = Yes) [If No, skip to Q25] Q21. If Q20=yes, do you eat at the school canteen? (0 = No 1 = Yes) Q22. If Q21 = no, why do not you eat at the canteen? (Multiple responses possible) a. I do not like canteen food b. I get hungry and eat before the canteen serves food 15 c. I get sick when I eat in the canteen d. My parents refuse that I eat at the canteen e. I bring food from home f. Other (specify) Q23. If Q21 = yes, Do you eat every school day at the school canteen? (0=No; 1=Yes) Q24. If Q23 = no, Why don’t you eat at the school canteen every school day? a. I do not attend school every school day b. Sometimes I do not have my counterpart c. Sometimes I do not like the foods d. I get hungry and eat before the school canteen serves food e. Other (specify) Q25. Do you usually buy food in school during the breaks? (10:15 AM or 12:00 or 4:00 PM or 5 PM) 0=No; 1=Yes (If No, skip to Q27) Q26. If yes, do you usually feel full after eating the food you buy during the school breaks? 0=No; 1=Yes (Skip to Q29) Q27. If Q25 = No, Why don’t you buy food in school during the breaks? 0=food provided by the canteen, 1=no money to buy food, 2=don’t like the food sold, 3=other (specify) Q28. If Q21 = No and Q25 = No, How do you eat? 0 = I do not eat anything, 1 = I go home to eat, 2 = I share friends' meals, 3 = I bring food from home Q29. Are the foods that you eat at the school similar to the foods you eat at home? 0-No, 1=Yes [IF Q04 (TYPE OF SCHOOL) = 2 (LRP CONTROL), STOP] [THE FOLLOWING QUESTIONS CONCERN THE COMPARISON SURVEY] Section D: Appreciation of the quality of the meals and the services of the canteen (type of school = LRP Intervention, FFE2/PAM Comparison) Q30. Do you like the canteen meals? (0 = No, 1 = Yes) Q31. Do you usually feel full after eating at the school canteen? (0 = No, 1 = Yes) Q32. How many meals a day are served at your school canteen? /__/__/ Q33. How many times a day do you eat meals in your school canteen? /__/__/ Q34. Are you satisfied with the number of meals offered daily by the canteen? (0 = No, 1 = Yes) (If yes, skip to Q36) Q35. If No, how many times would you like to be served each day? /__/__/ Q36. Is there often a delay in meal service hours? (0 = No, 1 = Yes) Q37. Are you paying for something to eat at your school canteen? (0 = No, 1 = Yes) (If No, skip to Q39) Q38. If Q37 = Yes, How much do you pay to eat at the school canteen? /__/__//__/ Q39. Are your parents always giving you money to eat at the school canteen? (0 = No, 1 = Yes) (If Yes, skip to end) Q40. If No, why? a. No money b. Parents consider that the canteen is free c. Other (specify): ……………………… End date and time Thank you for your collaboration 16 5. Students – Attendance Observation sheet of class attendance Investigator: Investigator Code: |___|___| Commune: School: Type: 1- LRP Intervention; 2- LRP Control N° N° Ech Name and surname Class Sex October Novembe r Decembe r January Februar y March April JO A JO A JO A JO A JO A JO A JO A 1 2 3 4 5 6 … 50 51 … 60 JO= Number of business days; A= Number of days of absence; Students 51 to 60 represent those on the reserve list 17 6. Producer Associations Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Producer Associations’ Questionnaire Hello. My name is ___________________, and I am working with Catholic Relief Services (CRS) and their evaluation partner EVIHDAF. We are conducting a study and would like to ask some questions about production and post-harvest management practices, group sales, market access and the use of market information systems. The information collected will help us to assess the level of achievement of expected results and the quality of the school canteen supply system implemented. Firstly, we will be able to learn from the different actions taken and then make recommendations to CRS and its partners for future school feeding programs based on local and regional food supply. The survey will take about 20 minutes to complete. The information you provide will be kept confidential and will not be shared with other persons. Your participation in this survey is voluntary and you can choose not to answer any questions or not to participate. If you would like to know more about the survey, you can contact the following persons at CRS and EVIHDAF [OFFER THE CONTACT INFORMATION OF CRS AND EVIHDAF]. Do you have any questions for me? [ANSWER THE QUESTIONS]. Q01. Do you wish to participate in this survey? (0=No; 1=Yes) Q02. Can we start now? (0 = no; 1 = yes) Q03. Date; |__|__|.|__|__|.2019 Start time: |__|__|h|__|__|min Q04. Zone type: 1= LRP intervention, 2= LRP Control Q05. Commune: Parakou = 1; N'Dali = 2; Pèrèrè =3; Tchaourou = 4 Q05. Association name and ID: ___________________________________ Section A. Respondent Identification Q06. Respondent’s sex: 0=Female; 1=Male Q07. Respondent’s name: ________________________ Q08. Respondent’s age: /__/__/ Years Q09. Respondent’s education level: 0= none, 1= alphabetization in local language, 2= primary, 3= secondary first cycle, 4= secondary second cycle, 5= university Q10. If Q9 = 2-5, last class completed: /_/_/_/…. (11= CI, 12= CP, 13 = CE1, 14 = CE2, 15=CM1, 16= CM2, 21= 6ème, 22= 5ème, 23= 4ème, 24= 3ème, 31= 2nde, 32= 1ère, 33= Tle, 41=Bac+1 ; 41=Bac+2 ; 43=Bac+3 ; 44=Bac+4 ; 45=Bac+5 ; 46=Bac+6 ; 47=Bac+7) Section B: General information on the functioning of the association (type of zone = intervention LRP, Control LRP) Q11. What is your main responsibility with this Producer Association? (President=1; Vice-president=2; Secretary=3; Treasurer=4; Other (Specify)….…=8) Q12. How long have you been working with this Producer Association? /_/_ / Q13. How many producer members are part of your Producer Association? How many of them are women? Q14. What is the most important crop on which the Association works? (Maize=1; Rice=2 ; Soy=3 ; Sorghum=4 ; Cowpea=5; Peanut = 6; Yam = 7; Cassava = 8; Sweet potato = 9; Other (Specify)…=88) Q15. What is the total quantity (in tons) of this crop that the association was able to mobilize during the last crop year for group sales, warrantage, etc.? /__/__/__/__ / tons Q16. Is there another crop on which this Association works? (Yes=1; No=0) If No, skip to Q19) Q17. If yes, what is the second most important crop on which your association works? (Maize=1; Rice=2 ; Soy=3; Sorghum=4; Cowpea=5; Peanut = 6; Yam = 7; Cassava = 8; Sweet potato = 9; Other (Specify)…=88) 18 Q18. What is the total quantity (in tons) of this crop that the association was able to mobilize during the last crop year for group sales, warrantage, etc.? /__/__/__/__ / tons Q19. What are the national, regional or communal organizations to which your Association belongs? (Multiple responses possible) a. Agence Béninoise de Karité (ABK), b. Union Régionale des Producteurs de Riz (URPR)/ Union Communale des Producteurs de Riz (UCPR) c. Union Régionale des Producteurs (URP)/ Union Communale des Producteurs (UCP) d. Other (specify)…………………. Q20. Have you ever been supported by ProAgri? (0=No; 1=Yes) (If No, skip to Q22) Q21. If Q20 = Yes, which support did you receive? a. Training on agricultural practices b. Seed gift or subsidy c. Material gift or subsidy d. Other input gift or subsidy e. Value chain management f. Market access g. Other (specify)……………….. Q22. Have you ever been supported by the PASDeR? (0=No; 1=Yes) (If No, skip to Q24) Q23. If Q22 = yes, which support did you receive? a. Training on agricultural practices b. Seed gift or subsidy c. Material gift or subsidy d. Other input gift or subsidy e. Value chain management f. Market access g. Other (specify)……………….. Q24. Do you have a fixed schedule for meetings of the management team? 0=No; 1=Yes, proof not presented; 2=Yes, proof seen (If No, skip to Q26) Q25. If Yes: What is the frequency? 1= Weekly, 2= Twice a month; 3=Monthly; 4=Quarterly; 5=Twice a year, 6=Yearly; 7= other (specify) (Skip to Q27) Q26. If No: How do you organize meetings of the management team? a. When/if the president or his deputies wants b. When/if any management team member wants c. When/if any member requests d. When/ if there is a project e. When/ if there is another opportunity such as product demand f. Other (specify) Q27. Do you have a fixed schedule for the activities of the producers in your Association? 0=No; 1=Yes, proof not presented; 2=Yes, proof seen (If No, skip to Q29) Q28. If Yes: What is the frequency? 1= Weekly, 2= Twice a month; 3=Monthly; 4=Quarterly; 5=Twice a year, 6=Yearly; 7= other (specify) Q29. How do you support the farmers belonging to your Association? a. Training on agricultural practices b. Seed gift or subsidy c. Material gift or subsidy d. Other input gift or subsidy (specify) e. Value chain management f. Market access 19 g. Credit access h. Other (specify) Q30. In the last 12 months, have you used the URP MIS to improve your marketing and commercialization strategies? 0=No, 1=Yes, proof not presented; 2=Yes, proof seen (If No, skip to Q32) Q31. If Yes: For what purpose? a. Producer prices b. Local market prices c. Regional market prices d. International market price Q32. Please provide me with a list of all members of the association showing their names and phone numbers. [IF POSSIBLE, TAKE A COPY OF THE LIST WITH YOUR PHONE OR COPY THE INFORMATION IN YOUR NOTEBOOKS, AND RETURN THE ORIGINAL. DON’T KEY IN THE LIST IN YOUR TABLET; IT IS TO BE USED TO DRAW A RANDOM SAMPLE OF MEMBERS TO BE INTERVIEWED] [ID Q04 (ZONE TYPE) = 2 (LRP CONTROL), STOP] Section C. Knowledge and relations with the LRP project (Zone Type = LRP Intervention) Q33. In what year did your collaboration with the LRP project begin? (2017, 2018 or 2019) Q34. In what month did your collaboration with the LRP project begin? (January = 1, February = 2, March = 3, April = 4, May = 5, June = 6, July = 7, August = 8, September = 9, October = 10, November = 11, December = 12) Q35. In your opinion, has the school canteen helped to reduce poverty in the community? (Yes = 1; No = 0) If No, skip to Q37) Q36. If Q35= yes, Why? a. Flow of agricultural food products b. Improvement of economic activities c. Increase in employment d. Revenue improvement e. Disease reduction in school f. Increase in school attendance g. Other (specify): …….. Q37. According to you, was it necessary to implement a canteen in schools of the region? (Yes=1 ; No=0) Q38. If Q37= Yes; Main reason: ………………. Q39. If Q37= No; Main reason: ………………. Section D. Training received (Zone Type = LRP Intervention) Q40. According to you, has the LRP project helped to develop capacities of members of your association? (Yes = 1, No = 0) (If No, skip to Q42) Q41. If Q40= Yes, what capacity did you acquire? What training did you receive? (multiple responses possible) Theme Acqui red capac ity Have you receive d training ? (0=no ; If yes, main them es cover ed Structure (s) a￾URP/UCP b-ProAgri c￾PASDER Numbe r of people trained Did you need this training ? Do you think the trainin g could have had other 20 1= Yes, proof not present ed; 2=Yes, proof seen) d-Other (specify) … conten t? a. Organiz ational capacity b. Improved productivity c. Harvest practices d. Post-harvest practices e. Storage practices f. Marketing strategy g. Market Information System (MIS) h. Other (specify) Section E. Implementation of learning outcomes and observed changes (Zone type = LRP intervention) Q42. Which of the trainings received could be implemented by your members? (Multiple responses possible) a. Production level Improvement b. Harvest practices c. Post-harvest practices d. Storage practices Q43. Do you have rules and regulations in your Association? (0=No; 1=Yes) (If No, skip to Q45) Q44. If Q43= Yes, was it with the intervention of the LRP project that you formalized your association (registration and possession of rules and regulations in your association)? (0 = no; 1 = yes) Q45. Does your Association have scheduled meetings? (0 = no, 1 = yes) (If No, skip to Q47) Q46. If Q45= Yes, what is the schedule (or frequency) of the Association's meetings? (1 = occasionally, 2 = regularly; specify the frequency: ……….) Q47. How do you call meetings? (Multiple responses possible) a. Regularly scheduled b. Invitation letter c. Phone call d. Social networks / SMS e. Oral communication Q48. What are the decision-making modes? a. consensus b. restricted committee c. President of the Association d. other (specify): ………… Q49. What is the average duration (in minutes) of your meetings? /__/__/__/ min 21 Q50. What are the funding sources for the association? (Contributions = 1, Loans = 2, Partner support = 3, other (specify) = 8) Q51. Do you develop a budget? (0 = no, 1 = yes) (If No, skip to Q53) Q52. Do your expenses reflect this budget? (0 = no, 1 = yes) Q53. Does the office report the expenses incurred to the members of the Association? (0 = no, 1 = yes) Q54. Had you done group sales before the LRP project? (0 = no, 1 = yes) (If No, skip to Q58) Q55. If Q54=yes, do these group sales under the LRP project seem more advantageous to you? (0=no; 1=yes) Q56. If Q55=no, why? a. Low prices b. Delay in payment c. Truncated measurement units d. Other (specify): …… Q57. If Q56= yes, how are these sales more advantageous? (Multiple responses possible) a. Easy flow of agricultural products/market guarantee b. Decrease in the sell-out of agricultural products c. Price advantage d. Short payment deadline e. Low transport costs f. Local consumption (use of local products for children's nutrition) g. Other (specify): …… Q58. What are (were) your difficulties in selling your products to the LRP project? a. Non-compliance with the delivery contract b. Administrative burden at the LRP project level for payments c. Conflicting relations with COGES d. Other (specify) Q59. Is your association able to make future commitments for the supply of agricultural products to other projects/structures? (0 = no; 1 = yes) (If No, skip to Q61) Q60. If Q59=yes, was it the LRP project intervention that gave you that capacity? (0 = no; 1 = yes) Q61. Do you think it is necessary to renew this project? (0 = no; 1 = yes) Q62. Please provide me with a list of all members of the association showing their names and phone numbers. End date and time Thank for your collaboration 22 7. Individual Producers Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Producers’ Questionnaire (LRP) INTRODUCTORY FORM Hello. My name is ___________________, and I am working with Catholic Relief Services (CRS) and their evaluation partner EVIHDAF. We are conducting a study and would like to ask some questions about production and post-harvest management practices, group sales, market access and the use of market information systems. The information collected will help us to assess the level of achievement of expected results and the quality of the school canteen supply system implemented. Firstly, we will be able to learn from the different actions taken and then make recommendations to CRS and its partners for future school feeding programs based on local and regional food supply. The survey will take about 20 minutes to complete. The information you provide will be kept confidential and will not be shared with other persons. Your participation in this survey is voluntary and you can choose not to answer any questions or not to participate. If you would like to know more about the survey, you can contact the following persons at CRS and EVIHDAF [OFFER THE CONTACT INFORMATION OF CRS AND EVIHDAF]. Do you have any questions for me? [ANSWER THE QUESTIONS]. Q01. Do you wish to participate in this survey? (0=No; 1=Yes) Q02. Can we start now? (0=No; 1=Yes) Q03. Date; Start time: |__|__|.|__|__|.2019 Start time: |__|__|h|__|__|min Q04. Zone Type: 1= LRP intervention, 2= LRP Control Q05. Commune: Parakou = 1; N'Dali = 2; Pèrèrè =3; Tchaourou = 4 Q06. Association name and ID: _________________________________ Section A: Respondent Identification Q07. Respondent’s name: _______________________________ Q08. Respondent’s age: / __/ __/ Years Q09. Respondent’s sex: 0=Female; 1=Male Q10. Respondent’s education level 0= none, 1= alphabetization in local language, 2= primary, 3= secondary first cycle, 4= secondary second cycle, 5= university Q11. If Q3 = 2 to 5, last class completed: 11= CI, 12= CP, 13 = CE1, 14 = CE2, 15=CM1, 16= CM2, 21= 6 ème, 22= 5ème, 23= 4ème, 24= 3ème, 31= 2nde, 32= 1ère, 33= Tle, 41=Bac+1 ; 41=Bac+2 ; 43=Bac+3 ; 44=Bac+4 ; 45=Bac+5 ; 46=Bac+6 ; 47=Bac+7 Section B: Producer activities within this Association (Zone type = LRP intervention, LRP control) Q12. Which Producer Association do you belong to? Q13. Last season, did you produce any of the following crops? (LIST CROPS ONE BY ONE. IF YES, ASK SUBSEQUENT QUESTIONS). Crops Did you produce (0=no, 1= yes) Area sown (hectare) Quantity produced last season Unit of measurement 1= Basin, 2= Bag, 3= cart, 4=other (specify) Weight that contain the unit (kg) Quantity produced (unit of measure) Maize Rice Cowpea 23 Shea collection Roots and tubers Other cereals Other leguminous Vegetables & fruits Q14. Did you receive any training from your Producer Association in the following areas? (LIST TRAINING AREAS ONE BY ONE. IF YES, ASK SUBSEQUENT QUESTIONS) Training thematic Did you receive (0=no, 1=Yes, proof not presented; 2=Yes, proof seen) If yes, main topics covered If Yes, Did you ever use the techniques the Association taught you? 0=No; 1=Yes If No: why 1=Difficult to implement, 2=Costly to implement, 3=Access to equipment, 4= other (specify) If Yes: were you satisfied with these techniques? 0=No; 1=Yes If No why? 1= the promised results were not obtained 2= other (specify) Production level improvement Harvest practices Post-harvest practices Storage practices Marketing and commercialization strategies Market Information System (MIS) Other (specify) Q15. In the last 12 months, did you sell your product through a group sale organized by your Association? 0=No; 1=Yes (If No, skip to Q18) Q16. If Yes: Were you satisfied with the group sale? 0=No; 1=Yes (If Yes, skip to Q18) Q17. If Q16=No: Why were you not satisfied with the group sale? a. Price was low b. I was not paid on time c. Did not trust the scale used d. Did not trust management team e. Other (specify) [IF Q04 (ZONE TYPE) = 2 (LRP CONTROL), STOP] 24 Section C: Knowledge of the LRP project (Zone Type = LRP Intervention) Q18. In your opinion, was it necessary to set up this school canteen based on the purchase of local products? Yes=1; Main reason: ………………. No=0; Main reason: ………………. Q19. In your opinion, has the school canteen helped to reduce poverty in the community? (Yes = 1; No = 0) If yes, yhy? a. Flow of agricultural food products b. Improvement of economic activities c. Increase in employment d. Revenue improvement e. Disease reduction in school f. Increase in school attendance g. Other (specify): …….. 25 Section D: Training received (Zone Type = LRP Intervention) Q20. Do you think that the LRP project has enabled you to acquire technical and management skills for your farm? (Yes = 1, No = 0) (If No, skip to Q22) Q21. If yes, what capacity did you acquire? What training did you receive? (Multiple responses possible) Effectiveness Relevance Cost-effectiveness Sustainability Theme Have you received training? (0=No ; 1= Yes, proof not presented; 2=Yes, proof seen) If yes, main themes covered Structure a￾URP/UCP b-ProAgri c-PASDER d- Other (specify)… Number of people trained Did you need this training? Do you think the training could have had other content? What other content would you like? (list them) Is it necessary to continually renew this training? (1=yes ; 0=no) Main reason b. Production level improvement c. Harvest practices d. Post-harvest practices e. Storage practices f. Marketing and commercialization strategies g. Market Information System (MIS) h. Other (specify) 26 Section E: Implementation of learning outcomes and observed changes (Zone type = LRP intervention) Q22. Which of the trainings received did you put into practice? (Multiple responses possible) a. Good production practices b. Harvest practices c. Post-harvest practices d. Storage practices e. Climate-smart production practices Q23. Have your yields increased using the new techniques learned? (0 = No, 1 = Yes) Q24. Have you participated in the group sales of your Association as part of the LRP Project? (0 = No, 1 = Yes) (If No, skip to Q30) Q25. If Q24= Yes, what product and quantity did you provide to your association? Q26. Have you been satisfied with group sales as part of the LRP project? (0 = No, 1 = Yes) (If yes, skip to Q28) Q27. If Q26= no, why? a. Low prices b. Delay in payment c. Truncated measurement units Q28. If Q26=Yes, do these group sales under the LRP project seem more advantageous to you? (0=no; 1=yes) Q29. If Q28= Yes, why? (Multiple responses possible) a. Easy flow of agricultural products/market guarantee b. Decrease in the sell-out of agricultural products c. Price advantage d. Short payment deadline e. Low transport costs f. Local consumption (use of local products for children's nutrition) g. Other (specify): …… Q30. Do you think it is necessary to renew this project? (0 = No; 1 = Yes) End date and time Thank for your collaboration 27 II. Individual Interviews 1. School Principals Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Interview Guide with School Principals INRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. 1. Do you wish to participate in this survey? (0 = No; 1 = Yes) 2. Can we start now? (0 = No; 1 = Yes) Section A: General Information and Respondent Identification 3. Commune (Parakou=1; N'Dali=2; Pèrèrè=3; Tchaourou=4; Kalalé=5; Gogounou = 6; Kandi=7; Malanville=8) 4. School Identification Number: /___ //___ / 5. Information on the number of students in the school Registered Present Absent Abandonment Boys Girls Total 6. Respondent’s sex: 0=Female; 1=Male 7. Respondent’s name_______________________________ Section B: LRP project evaluation Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: According to you, was it necessary to implement a canteen in the school? Why or why not? Question 2: Do you think the LRP project was necessary? Why or why not? In the implementation, a) What did you like as an approach? Why? (Quantity and quality of meals served, cultural acceptability of meals, financial participation, local supply of products, etc.) b) What didn’t you like in the implementation of the canteen at your school? (Quantity and quality of meals served, cultural acceptability of meals, financial participation, local supply of products, etc.) Question 3: Tell us about COGES a) composition (different positions with their responsibilities, assignment of positions) b) Mission and contribution of the COGES c) COGES operating rules d) Methods of communication (meetings, various notes, evaluation sheets or pupil satisfaction surveys, activity reports, telephone communications, use of ICTs, etc.) 28 e) Decision-making methods Question 4: What are your relationships with other actors in the school, as part of this LRP project? (COGES, chef, APE, ME, teachers, nurse/responsible for medical care, students/student leaders, OPA, CRS-LRP, other schools, etc.). What is the nature or form of the relationship with each of them? What do you think of this relationship: a good thing, why? A wrong approach, why? What do you suggest? Question 5: Is there a mechanism to monitor and evaluate the objectives and results of the canteen (or project)? Who ensures the organization of servicesto students? Are there rules relating to: meal times, volume of food, menus, etc.? How do you ensure compliance with these rules? Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 6: What are the objectives of the LRP canteen? And what are the expected results? Question 7: Have you received training as part of this project? If yes, what are the topics on which you were trained? Question 8: Has the LRP project facilitated the supply and availability of nutritious foods in your school? Question 9: Has the implementation of this project improved school performance and reduced absenteeism of some students at school? The third component relates to the cost-effectiveness of the project. Item 3: Cost-effectiveness of the project Question 10: What are the expected results that seem to be achieved? Why? Question 11: What are the results that do not seem to be achieved? Why? Question 12: What are the internal or external factors that have prevented the achievement of these results? Thank you for your answers. Now let's talk about the impact of the project. Item 4: Project Impact Question 13: What are the aspects of the training that you could implement quite easily? What impact has the training had? What were the contributing factors? Question 14: Which of the training components received have not been implemented? Why? What changes could not be made according to the expectations of the project? What are (have been) your difficulties? Question 15: What other changes do you think the project has promoted in the school? Question 16: What other changes did you notice? To whom do you attribute them? Question 17: According to you, what are the aspects that have not changed in your school since the establishment of the canteen (Good nutrition, delay, attendance, absence from school, school performance, hygiene, diseases, etc.)? Can the school canteen change them? If yes, why was there no change? (Short project duration, insufficient resources, no training, etc.) Question 18: According to you, have there been other changes in your environment in terms of the dynamics of socio-economic life that you can relate to the intervention of the LRP Project? The next part we will focus on is the sustainability of the project Item 5: Project sustainability Question 19: Do you think it is necessary to renew this school canteen experience? If yes, under what conditions and in what forms? Question 20: What actions would you have planned to ensure school canteen continuity in terms of organization at the local level or with the support of a partner? Question 21: Have there been any sharing of experiences (constraints, lessons learned) with other schools with or without school canteens? 29 The sixth part of our discussion is about learning Item 6: Satisfaction and lessons learned Question 22a: What were your expectations at the start of the LRP project? Question 22b: Have your expectations for this project been met? Why or why not? Question 22c: Through this school canteen experience, what has caught your attention that you want to share with others now? Let's move on to the last part of our discussion about comparing the LRP project to other projects with similar goals. Item 7: Comparative questions with other projects Question 23: Before the establishment of this canteen, had you known or experimenced other school canteens? (If No, end survey) Question 24: If yes, compared to school canteens that you had known or experienced in the past: • What are the benefits of your school canteen (LRP)? Why? • What are the shortcomings of this canteen in your school (LRP)? Why? End date and time Thank for your collaboration 30 2. Parent Associations Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Interview Guide with Parent Associations (APE)-a COGES representative or a key informant INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: What are the problems (or questions) in your school that the canteen is supposed to help solve? (Daily food, delay, attendance in class, academic performance, hygiene, illness, etc.) Question 2: Did you participate in the information meetings on the prospects of implementing the LRP project? (If yes, number of meetings). What was said in those meetings? Has your opinion been taken into account? (Have your proposals been accepted and used?) Question 3: Did you participate in the decision-making meeting on the composition of COGES? Was your opinion taken into account during this meeting? Question 4: Have you been associated (did you give your opinion) in the designation of COGES members? Question 5: Are you a member of COGES? How many APE members belong to COGES? What are their roles on this committee? Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 6: Before the implementation of this canteen, how did your children get food when they were at school? Question 7: What are the advantages that you perceive since the implementation of this canteen in your child's school (ease of access, good school feeding, and lower financial cost)? Question 8: Are there any negative aspects related to the presence of this canteen in your child's school? Thank for your responses. Let’s talk about the impact of the project Item 3: Impact of the project Question 9: What changes have you noticed since the implementation of this school canteen? (good nutrition at school, marketing of local products, organization of producers, presence and attendance in class, academic performance, hygiene, diseases, etc.). Which of these changes can you attribute to the arrival of the school canteen? To whom do you attribute the others? Question 10: What has not changed in the school since the arrival of this school canteen? (Good nutrition, marketing of local products, organization of producers, presence and attendance in class, academic performance, hygiene, diseases, etc.). Can the school canteen change them? If yes, why was there no change? Question 11: Tell us about what you liked in the implementation of the school canteen. Question 12: What did not you like in the implementation of the canteen at your school? 31 The next part we will focus on is the sustainability of the project Item 4: Sustainability of the project Question 13: Do you think that it is necessary to continue this school canteen? (If yes, under what conditions and in what forms?) The fifth part of our discussion concerns lessons learned. Item 5: Satisfaction and lessons learned Question 14a: What were your expectations at the start of the LRP project? Question 14b. Have your expectations for this project been met? Why or why not? Question 14c: Through this school canteen experience, what has caught your attention that you want to share with others? The sixth and final part of our discussion focuses on the comparison of the LRP project with other projects with similar objectives. Item 6: Comparative Questions with other projects Question 15: Before the establishment of this canteen, have you known or experienced other school canteens? (If No, end survey) Question 16: If yes, compared to school canteens that you have known or experienced in the past: a) What are the benefits of your school canteen (LRP)? Why? b) What are the shortcomings of this canteen in your school (LRP)? Why? End date and time Thank for your collaboration 32 3. Government authorities Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Interview Guide with Government authorities (SAS/DDEMP, DDAEP/MAEP) INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Localization of the community Department : Commune : Village : Phone : Email : Zone Type: LRP intervention = 1; FFE2 / PAM comparison = 2 [IF ZONE TYPE = FFE2/PAM COMPARISON, GO DIRECTLY TO SECTION B] Section A: Assessment of the LRP project Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: What can you tell us about the school canteens of the LRP project? Question 2: In the implementation, a) What did you like as an approach? Why? (Quantity and quality of meals served, cultural acceptability of meals, financial participation, local supply of products, etc.) b) What didn’t you like in the implementation of the canteen at your school? Question 3: Do national policies consider and take into account support for producers and their access to local markets and school canteens, and the food security of schoolchildren? If yes, which ones? What is the degree of consideration? Question 4: . What is the nature of your relationship with each of the actors involved in the project (COGES, APE, Directors and teachers, CRS-LRP, OP Faitière, producer associations, etc.)? What do you think of each relationship? Are these relationships good or bad? Why? What do you suggest to maintain or improve the relationship? Question 5: Are there any information systems at your level to follow the activities of this project so that you can capitalize on these experiences if necessary? If yes, name them. Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 6: What are the objectives of the LRP canteen? What are the expected results? Question 7: What are the expected results that seem to be achieved? Why? Question 8: What are the results that do not seem to be achieved? Why? 33 Thank you for your answers. Now let's talk about the impact of the project. Item 3: Project Impact Question 9: Have the LRP project contributed to the government's objectives of supporting producers in their access to local markets and school canteens, and in terms of the food security of school children? Question 10: What did you like in the implementation of school canteens through the LRP project? What didn’t you like in the implementation of school canteens through the LRP project? The next part we will focus on is the sustainability of the project Item 4: Project sustainability Question 11: Do you think it is necessary to renew this school canteen experience? If yes, under what conditions and in what forms? Question 12: Which of the expected or unexpected results of the project are likely to be maintained beyond the implementation of the project? Question 13: What are the main internal or external factors that could influence the success or failure to achieve the sustainability of the results achieved? Question 14: Are there government policies, procedures or priorities that contribute to the sustainability of the results of this project? If yes, which ones and how? Question 15: In view of the results achieved, is the government ready to continue local purchases and local management of school feeding? (If No, skip to Q17) Question 16: If yes, what measures have been taken to integrate it into national and municipal plans? The fifth part of our discussion is about learning. Item 5: Satisfaction and lessons learned Question 17a: What were your expectations at the start of the LRP project? Question 17b: Have your expectations for this project been met? Why or why not? Question 17c: Through this school canteen experience, what has caught your attention that you want to share with others now? Are there any strengths in this project that you are planning to or have already capitalized? Question 18: Do you think that the stakeholders have acquired the application and replicability capacities of the local procurement approach? Section B: Comparison of the LRP project with FFE2 and PAM projects N. B: Please consider the questions in this section with the eyes of LRP or FFE2 or PAM according to the reference project for the purpose of comparison Let's move on to the last part of our discussion about comparing the LRP project to other projects with similar goals. Item 6: Comparative questions with other projects Question 19: Before the establishment of this canteen, had you known or experienced other school canteens? (If No, skip to Q21) Question 20: If yes, compared to school canteens that you had known or experienced in the past: a) What are the benefits of your school canteen of LRP/FFE2/WFP project? Why? b) What are the shortcomings of this canteen of LRP/FFE2/WFP project? Why? Question 21: To what extent is the procurement approach developed by the LRP/FFE2/WFP project more cost-effective and timely than the approaches taken by other projects? Question 22: What are the positive or negative externalities resulting from the LRP/FFE/WFP project procurement approach to the local economy compared to the approaches adopted by the other projects? Question 23: Do the beneficiaries of this project (producers, population, students, COGES) seem satisfied to you? What appreciation do they show of this project? End date and time Thank for your collaboration 34 4. Municipal authorities (Mayor / Representative in charge of economic/social affairs) Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Interview Guide with Municipal authorities (LRP, FFE, PAM) Date of the interview : Investigator Name : INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Localization of the community Department : Commune : Village : Phone : Email : Zone Type: LRP intervention = 1; FFE2 / PAM comparison = 2 [IF ZONE TYPE = FFE2/PAM COMPARISON, GO DIRECTLY TO SECTION B] Section A: Assessment of the LRP project Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: What can you tell us about the school canteens of the LRP project? Question 2: What are the problems (or questions) in your school that the canteen is supposed to help solve? (daily food, delay, attendance in class, academic performance, hygiene, illness, etc.) Question 3: Was the selection of your municipality as part of the LRP project necessary? Why? Question 4: Did you participate in the information meetings on the prospects of implementing the LRP project? (If yes, number of meetings). What was said in those meetings? Has your opinion been taken into account? (Have your proposals been accepted and used?) Question 5: In the implementation, a) What did you like as an approach? Why? (Quantity and quality of meals served, cultural acceptability of meals, financial participation, local supply of products, etc.) b) What didn’t you like in the implementation of the canteen in your municipality? Question 6: Do municipal policies consider and take into account support for producers and their access to local markets and school canteens, and the food security of schoolchildren? If yes, what is the degree of consideration in the PDC, and allocated budgets, etc.? Question 7: Are the LRP project responses to these concerns well-suited? Question 8: What is the nature of your relationship with each of the actors involved in the project (COGES, APE, Directors and teachers, CRS-LRP, OP Faitière, producer associations, etc.)? What do you think of each 35 relationship? Are these relationships good or bad? Why? What do you suggest to maintain or improve the relationship? Probe: What are the communal committees for the supply of local products? What are the municipal supervision committees? What role did they play in your commune in relation to the LRP project? According to these committees, did they fulfil their roles as planned? Are you satisfied? Why or why not? Question 9: Are there any information systems at your level to follow the activities of this project so that you can capitalize on these experiences if necessary? If yes, name them. Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 10: What are the objectives of the LRP canteen? What are the expected results? Question 11: What are the expected results that seem to be achieved? Why? Question 12: What are the results that do not seem to be achieved? Why? Question 13: Have you received training as part of this project? If yes, what are the topics on which you were trained? In which concepts did you improve your knowledge? Question 14: Did purchases of small lots work at the commune level? If yes, what strategies did you adopt? The third component relates to the cost-effectiveness of the project. Item 3: Cost-effectiveness of the project Question 15: Has the project been implemented in the most efficient way compared to alternatives? Question 16: What are the internal or external factors that have hindered the effective implementation of the project? Thank you for your answers. Now let's talk about the impact of the project. Item 4: Project Impact Question 17: What changes have you noticed since the arrival of this school canteen? (Sale of agricultural food products, improvement of development tax payment capacity, reduction of food poisoning, improvement of economic activities, improvement of income, etc.). Which of these changes do you attribute to the arrival of the school canteen? To what do you attribute others? Question 18: What are the aspects that have not changed in your school since the establishment of the canteen (sale of agricultural food products, improvement of development tax payment capacity, reduction of food poisoning, improvement of economic activities, improvement of income, etc.)? Can the school canteen change them? If yes, why was there no change? Question 19: What did you like in the implementation of the LRP canteen in your municipality? What did not you like in the implementation of the canteen in your municipality? The next part we will focus on is the sustainability of the project Item 5: Sustainability of the project Question 20: Have institutional relations been formalized between the project actors at the local level? To what extent do such relationships guarantee the continuity of the project or the emergence of a school canteen project based on local products with or without external support? Question 21: Do you think that it is necessary to renew this school canteen? (If yes, under what conditions and in what forms?) Question 22: Which of the expected or unexpected results of the project are likely to be maintained beyond the implementation of the project? Question 23: What are the main internal or external factors that could influence the success or failure to achieve the sustainability of the results achieved? Question 24: Are there government policies, procedures or priorities that contribute to the sustainability of the results of this project? If yes, which ones and how? 36 Question 25: In view of the results achieved, is the government ready to continue local purchases and local management of school feeding? Question 26: If yes, what measures have been taken to integrate it into national and municipal plans? The fifth part of our discussion is about satisfaction and learning. Item 6: Satisfaction and lessons learned Question 27a: What were your expectations at the start of the LRP project? Question 27b: Have your expectations for this project been met? Why or why not? Question 27c: Through this school canteen experience, what has caught your attention that you want to share with others now? Question 28: Do you think that the stakeholders have acquired the application and replicability capacities of the local procurement approach? Section B: Comparison of the LRP project with FFE2 and PAM projects N. B: Please consider the questions in this section with the eyes of LRP or FFE2 or PAM according to the reference project for the purpose of comparison Let's move on to the last part of our discussion about comparing the LRP project to other projects with similar goals. Item 6: Comparative questions with other projects Question 29: Before the establishment of this canteen, had you known or experienced other school canteens? (If No, skip to Q31) Question 30: If yes, compared to school canteens that you had known or experienced in the past: a) What are the benefits of your school canteen of LRP/FFE2/WFP project? Why? b) What are the shortcomings of this canteen of LRP/FFE2/WFP project? Why? Question 31: To what extent is the procurement approach developed by the LRP/FFE2/WFP project more cost-effective and timely than the approaches taken by other projects? Question 32: What are the positive or negative externalities resulting from the LRP/FFE/WFP project procurement approach to the local economy compared to the approaches adopted by the other projects? Question 33: Do the beneficiaries of this project (producers, population, students, COGES) seem satisfied to you? What appreciation of this project do they show? End date and time Thank for your collaboration 37 5. Local Supply Committees (CPC) and Local supervision committees (CLOC) Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Interview Guide with CPC and CLOC Date of the interview : Investigator Name : INTRODUCTORY FORMULA Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Localization of the community Department : Commune : Village : Phone : Email : Zone Type: LRP intervention = 1; FFE2 / PAM comparison = 2 Q1. Do you have statutes and regulations for the committee? If, yes, was it with the intervention of the LRP project that you created this committee? Q2. Does the regulation schedule meetings? Q3. If yes, what is the calendar (or the frequency) of the association’s meetings? How do you convene meetings? Q4. How are decisions made in the committee? Q5. What are the sources of funding for your committee? Q6. Do you develop a budget? If yes, do your expenses reflect this budget? Q7. Does the office report to other members/LRP Project on the expenses incurred? (0=no; 1=yes) Q8. What are your roles in supply, monitoring of the procurement process, and distributing products as part of the LRP project? What is your appreciation of your collaborations as part of this project? Q9. What are (were) your difficulties in managing your roles in the LRP project? (Easy relationships, conflicting relationships with other actors (COGES, CRS-LRP) Q10. Through this school canteen experience, what has caught your attention and you want to share with others, or take advantage now? Are there any strengths in this project that you are planning or have already capitalized? Q11. Do you think it is necessary to renew this project? If yes, under what conditions and in what forms? End date and time Thank for your collaboration 38 6. Producer Associations Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Interview Guide with members of Producer Associations Date of the interview: Investigator Name: INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Location of the association Department : Commune : Village : Denomination : Initials : Phone : Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: What can you tell us about the school canteens of the LRP project? Question 2: Did you participate in the information meetings on the prospects of implementing the LRP project? (If yes, number of meetings). What was said in those meetings? Has your opinion been taken into account? (Have your proposals been accepted and used?) Question 3: Did you participate in the decision-making meeting on the composition of COGES? Was your opinion been taken into account during this meeting? Question 4: Have you participated (did you give your opinion) in the designation of COGES members? Question 5: Are you a member of COGES? How many OPA members belong to COGES? What are their roles on this committee? Question 6: In the implementation, a) What did you like as an approach (local procurement)? Why? b) What didn’t you like in the implementation of the canteen? Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 7: What are the objectives of the LRP canteen? What are the expected results? Question 8: What are the expected results that seem to be achieved? Why? Question 9: What are the results that do not seem to be achieved? Why? Question 10: Have you received training as part of LRP project? If yes, what are the topics on which you were trained? 39 Probe: Have you been trained on: improvement production practices and post-harvest management, group selling practices, market access and the use of market information systems? (If No, skip to Q12) Question 11: Did you really need these trainings? Is there a difference between these trainings and those you have received in other projects in terms of the specificity of training approaches or content? The third component relates to the cost-effectiveness of the project. Item 3: Cost-effectiveness of the project Question 12: Has the project allowed your association to have better financial benefits? If yes, what were the most profitable activities? Question 13: What are the internal or external factors that have hindered the effective implementation of other project activities? Thank you for your answers. Now let's talk about the impact of the project. Item 4: Project Impact Question 14: What are the aspects of the training that you could implement quite easily? What effects have they had? (improvement of production practices and post-harvest management, [new farming practices including climate smart practices, access to appropriate equipment, infrastructure and techniques for storage and conservation], improvement of group selling practices, use of market information systems and market access[guaranteed access to markets, warrantsupport, decrease in the price of agricultural products, position improvement over traders])? What were the contributing factors to these effects? Question 15: Which of the training components received have not been implemented? Why? What changes could not be made according to the expectations of the project? What are (have been) your difficulties? Question 15a: What are the benefits of the LRP project for your socio-economic life? The next part we will focus on is the sustainability of the project Item 5: Project sustainability Question 16: Do you think it is necessary to renew this school canteen experience? If yes, under what conditions and in what forms? The sixth part of our discussion is about learning Item 6: Satisfaction and lessons learned Question 17a: What were your expectations at the start of the LRP project? Question 17b: Have your expectations for this project been met? Why or why not? Question 17c: Through this school canteen experience, what has caught your attention that you want to share with others now? Question 18: Does your participation in this project make your association able to take future commitments for the supply of agricultural products to other school canteen projects/structures or other sales to institutions such as WFP? If yes, how? Let's move on to the last part of our discussion about comparing the LRP project to other projects with similar goals. Item 7: Comparative questions with other projects Question 19: Before the establishment of this canteen, had you known or experienced other school canteens? (If No, end survey) Question 20: If yes, compared to school canteens that you had known or experienced in the past: • What are the benefits of your school canteen (LRP)? Why? • What are the shortcomings of this canteen in your school (LRP)? Why? End date and time Thank for your collaboration 40 7. OP Faitières Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Interview Guide with OP faitières Date of the interview : Investigator Name : INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Location of the association Department : Commune : Village : Denomination : Initials : Phone : Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: Did you participate in the information meetings on the prospects of implementing the LRP project? (If yes, number of meetings). What was said in those meetings? Has your opinion been taken into account? (Have your proposals been accepted and used?) Question 2: Did you participate in the decision-making meeting on the composition of COGES? Has your opinion been taken into account during this meeting? Question 3: Have you participated (did you give your opinion) in the designation of COGES members? Question 4: Are you a member of COGES? How many OP Faitières members belong to COGES? What are their roles on this committee? Question 5: In the implementation, a) What did you like as an approach (local procurement)? Why? b) What didn’t you like in the implementation of the canteen of your school? Question 6: Do you think that your participation in this project has had a positive impact on the local school feeding model? Why or why not? Question 7 What support or facilitation did you give to Producer Associations to ensure compliance with LRP contracts in terms of: capacity building on climate-smart farming practices, organizational management, and use of information systems on agricultural markets? Question 8: What was your role in the management of Producer Associations' supply contracts for agricultural products for the LRP project? Was it a good or bad approach? Why? Question 9: What are the challenges your association had to respond to? How did you deal with these challenges? Have there been conflicts related to non-compliance with commitments? What were the resolution mechanisms? 41 Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 10: What can you tell us about the school canteens of the LRP project? Question 11: What are the objectives of the LRP canteen? What are the expected results? a) What are the expected results that seem to be achieved? Why? What factors have contributed to the achievement of these results? b) What are the results that do not seem to be achieved? Why? What factors have hindered the achievement of these results? Question 12: Have you received training as part of LRP project? If yes, what are the topics on which you were trained? Probe: Have you been trained on: organizational capacity building (formalization, application of regulatory documents, maintenance of administrative and management documents), improvement production practices and post-harvest management, group selling practices, market access and the use of market information systems The third component relates to the cost-effectiveness of the project. Item 3: Cost-effectiveness of the project Question 13: Has the project provided financial benefits to your Association? If yes, what were the most profitable activities? Question 14: What are the internal or external factors that have hindered the effective implementation of other project activities? Thank you for your answers. Now let's talk about the impact of the project. Item 4: Project Impact Question 15: What are the aspects of the training that you could implement quite easily? What changes have they made (organizational reinforcement [holding regular meetings, collective decision making, group agreement], improvement of production practices and post-harvest management, [new farming practices including climate smart practices, access to appropriate equipment, infrastructure and techniques for storage and conservation], improvement of group selling practices, use of market information systems and market access [guaranteed access to markets, decrease in the price of agricultural products, position improvement over traders])? What were the contributing factors? Question 16: What are the positive/negative effects of the training on climate change on your productivity? Question 17: What changes could not be made according to the expectations of the project? What are (were) your difficulties? Question 18: Does your participation in this project make your association able to facilitate future commitments of producer associations for the supply of agricultural products to other school canteen projects/structures or other sales to institutions such as WFP? If yes, how? The next part we will focus on is the sustainability of the project Item 5: Project sustainability Question 19: Do you think it is necessary to renew this school canteen experience? If yes, under what conditions and in what forms? The sixth part of our discussion is about satisfaction and learning Item 6: Satisfaction and lessons learned Question 20a: What were your expectations at the start of the LRP project? Question 20b: Have your expectations for this project been met? Why or why not? 42 Question 20: Through this school canteen experience, what has caught your attention that you want to share with others now? Let's move on to the last part of our discussion about comparing the LRP project to other projects with similar goals. Item 7: Comparative questions with other projects Question 21: Before the establishment of this canteen, had you known or experienced other school canteens? If yes, compared to school canteens that you had known or experienced in the past: a. What are the benefits of the LRP school canteen? Why? b. What are the shortcomings of the LRP school canteen? Why? End date and time Thank for your collaboration 43 8. PASDeR/ProAgri Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Interview Guide with other project managers (PASDeR, ProAgri) Date of the interview : Investigator Name : INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Localization of the community Department : Commune : Village : Phone : Email : Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: What can you tell us about the school canteens of the LRP project? Question 2: Did you participate in the information meetings on the prospects of implementing the LRP project? (If yes, number of meetings). What was said in those meetings? Has your opinion been taken into account? (Have your proposals been accepted and used?) Question 3: In the implementation, a) What did you like as an approach (local procurement)? Why? b) What didn’t you like in the implementation of the LRP canteen? Question 4: What is the nature of your relationship with each of the actors involved in the project (COGES, APE, Directors and teachers, CRS-LRP, OP Faitière, producer associations, etc.)? What do you think of each relationship? Are these relationships good or bad? Why? What do you suggest to maintain or improve the relationship? Question 5 Was it necessary for the LRP Project to train producers? Was it a duplication of the trainings you already gave? Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 6: What are the objectives of the LRP canteen? What are the expected results? Question 7: What are the expected results that seem to be achieved? Why? Question 8: What are the results that do not seem to be achieved? Why? Question 9: Is there complementarity in your interventions? 44 Thank you for your answers. Now let's talk about the impact of the project. Item 3: Project Impact Question 10: Have the project interventions improved some of your practices in terms of: strengthening organizational capacities of Producer Associations and individual producers, improvement of production practices and post-harvest management, group selling practices, use of market information systems and market access? The next part we will focus on is the sustainability of the project Item 4: Project sustainability Question 11: Do you think it is necessary to renew this school canteen experience? If yes, under what conditions and in what forms? Question 12: What are the main internal or external factors that could influence the success or failure of the sustainability of the results achieved? The fifth part of our discussion is about satisfaction and learning. Item 5: Satisfaction and lessons learned Question 13a: What were your expectations at the start of the LRP project? Question 13b: Have your expectations for this project been met? Why or why not? Question 13c: Through this school canteen experience, what has caught your attention that you want to share with others now? The sixth and final part of our discussion focuses on the comparison of the LRP project with other projects with similar objectives. Item 6: Comparative Questions with other projects Question 14: Before the establishment of this canteen, had you known or experienced other school canteens? Question 15: If yes, compared to school canteens that you have known or experienced in the past: a) What are the benefits of the LRP school canteen? Why? b) What are the shortcomings of the LRP school canteen? Why? End date and time Thank for your collaboration 45 9. LRP Staff in charge of program Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Individual Interview Guide with LRP Staff in charge of program INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: Have your expectations been met with regards to different actors facilitating the project implementation? Why or why not? Question 2: Have your expectations of the project results been met? Why or why not? Question 3: What are the results that do not seem to be achieved? Why? Question 3a: Are you satisfied with the overall progress of the project? Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 4: Were the planned activities to achieve the transformations described in the project's theory of change completed? Question 5: What specific events attracted your attention during the project implementation process and which are important to the project or its results? Why? The third component relates to the cost-effectiveness of the project. Item 3: Cost-effectiveness of the project Question 6: a- What are the communal supply committees? What are the communal supervision committees? What is their role in the LRP project? b- How is the supply of local products organized? Please describe c- What are the different advantages observed with regards to the supply approach according to the target zones or communes? The next part we will focus on is the sustainability of the project Item 4: Sustainability of the project Question 7: In the event of continuing project activities, what may be necessary to improve in your opinion? Question 8: What contribution do you expect from the community or Producer Associations (free supply of school feeding products for example)? The fifth part of our discussion is about learning. Item 5: Lessons learned Question 9: What are the strengths of the project approach? (In relation to the participation of different actors) Question 10: What are the challenges of the project? Why? 46 III. Focus group discussions 1. Parents Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Focus group discussions with Parents INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: What are the problems (or questions) in your school that the canteen is supposed to help solve? (Daily food, delay, attendance in class, academic performance, hygiene, illness, etc.) Question 2: Did you participate in the information meetings on the prospects of implementing the LRP project? (If yes, number of meetings). What was said in those meetings? Has your opinion been taken into account? (Have your proposals been accepted and used?) Question 3: Did you participate in the decision-making meeting on the composition of COGES? Was your opinion been taken into account during this meeting? Question 4: Did you participate (did you give your opinion) in the designation of COGES members? Question 5: Are you a member of COGES? How many APE members belong to COGES? What are their roles on this committee? Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 6: Before the implementation of this canteen, how did your children get food when they were at school? Question 7: What are the advantages that you perceive since the implementation of this canteen in your child's school (ease of access, good school feeding, and lower financial cost)? Question 8: Are there any negative aspects related to the presence of this canteen in your child's school? Thank for your responses. Let’s talk about the impact of the project Item 3: Impact of the project Question 9: What changes have you noticed since the implementation of this school canteen? (good nutrition at school, marketing of local products, organization of producers, presence and attendance in class, academic performance, hygiene, diseases, etc.). Which of these changes can you attribute to the arrival of the school canteen? To what do you attribute the others? 47 Question 10: What has not changed in the school since the arrival of this school canteen? (Good nutrition, marketing of local products, organization of producers, presence and attendance in class, academic performance, hygiene, diseases, etc.). Can the school canteen change them? If yes, why was there no change? Question 11: What did you like in the implementation of the school canteen? Question 12: What did not you liked in the implementation of the school canteen? The next part we will focus on is the sustainability of the project Item 4: Sustainability of the project Question 13: Do you think that it is necessary to continue this school canteen? (If yes, under what conditions and in what forms?) The fifth part of our discussion concerns lessons learned. Item 5: Satisfaction and lessons learned Question 14a: What were your expectations at the start of the LRP project? Question 14b: Have your expectations for this project been met? Why or why not? Question 14c: Through this school canteen experience, what has caught your attention that you want to share with others now? The sixth and final part of our discussion focuses on the comparison of the LRP project with other projects with similar objectives. Item 6: Comparative Questions with other projects Question 15: Before the establishment of this canteen, had you known or experienced other school canteens? Question 16: If yes, compared to school canteens that you had known or experienced in the past: a) What are the benefits of the LRP school canteen? Why? b) What are the shortcomings of the LRP school canteen? Why? End date and time Thank for your collaboration 48 2. Mothers Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Focus group discussions with mothers INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: What are the problems (or questions) in your school that the canteen is supposed to help solve? (Daily food, delay, attendance in class, academic performance, hygiene, illness, etc.) Question 2: Have there been any information meetings for mothers (or their representatives) about the prospects of implementing the LRP project? If yes, number of meetings. What was said during those meetings? Question 3: Were the opinions of mothers (or their representatives) solicited and/or taken into account on aspects of conception or implementation of this project? If yes, what aspects? If not, why? How was this helpful (or unhelpful)?? Question 4: Have you been asked to give your opinion on the composition of the COGES? a) If yes, was your opinion taken into account? How was this helpful (or unhelpful)? b) If not, why was your opinion not solicited or taken into account? How was this helpful (or unhelpful)?? Question 5: Have you been involved (or given your opinion) in the designation of COGES members? a) If yes, was your opinion taken into account? How was this (helpful (or unhelpful)?? b) If not, why was your opinion not solicited or taken into account? How was this (helpful (or unhelpful)?? Question 6: Do mothers have a place (or a position) in the COGES? a) If yes, how many positions? What are these positions? How was this helpful (or unhelpful)?? b) If No, why do they not have positions in the COGES? How was this helpful (or unhelpful)?? Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 7: Before the implementation of this canteen, how did your children get food when they were at school? Question 8: What are the advantages that you perceive since the implementation of this canteen in your child's school (ease of access, good school feeding, and lower financial cost)? Question 9: Are there any negative aspects related to the presence of this canteen in your child's school? 49 Thank for your responses. Let’s talk about the impact of the project Item 3: Impact of the project Question 10: What changes have you noticed since the implementation of this school canteen? (good nutrition at school, marketing of local products, organization of producers, presence and attendance in class, academic performance, hygiene, diseases, etc.). Question 11: Which of these changes can you attribute to the arrival of the school canteen? To what do you attribute the others? Question 12: What has not changed in the school since the arrival of this school canteen? (Good nutrition, marketing of local products, organization of producers, presence and attendance in class, academic performance, hygiene, diseases, etc.). Can the school canteen change them? If yes, why was there no change? Question 13: What did you liked in the implementation of the school canteen? Question 14: What did not you liked in the implementation of the school canteen? The next part we will focus on is the sustainability of the project Item 4: Sustainability of the project Question 13: Do you think that it is necessary to continue this school canteen? (If yes, under what conditions and in what forms?) The fifth part of our discussion concerns lessons learned. Item 5: Satisfaction and lessons learned Question 16a: What were your expectations at the start of the LRP project? Question 16b: Have your expectations for this project been met? Why or why not? Question 16c: Through this school canteen experience, what has caught your attention that you want to share with others now? The sixth and final part of our discussion focuses on the comparison of the LRP project with other projects with similar objectives. Item 6: Comparative Questions with other projects Question 15: Before the establishment of this canteen, had you known or experimeced other school canteens? Question 16: If yes, compared to school canteens that you had known or experienced in the past: a) What are the benefits of the LRP school canteen? Why? b) What are the shortcomings of the LRP school canteen? Why? End date and time Thank for your collaboration 50 3. Teachers Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Focus Group Discussion with Teachers INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: According to you, was it necessary to implement a canteen in the school? Why or why not? Question 2: Do you think the LRP project was necessary? Why or why not? In the implementation, a) What did you like as an approach? Why? (Quantity and quality of meals served, cultural acceptability of meals, financial participation, local supply of products, etc.) b) What didn’t you like in the implementation of the canteen at your school? (Quantity and quality of meals served, cultural acceptability of meals, financial participation, local supply of products, etc.)? Question 3: What are the problems (or questions) in your school that the canteen is supposed to help solve? (Daily food, delay, presence and attendance in class, academic performance, hygiene, illness, etc.). Question 4: Have teachers or teacher representatives participated in information meetings on the prospects of implementing the LRP project? (If yes, number of meetings). What was said during these meetings? Was their opinion been taken into account? (Have your proposals been accepted and used?) Question 5: Have teachers or teacher representatives taken part in decision-making meetings on the composition of COGES? Was their opinion been taken into account? Question 6: Have teachers or teacher representatives been associated (or given their opinion) in the designation of COGES members? Are teachers members of COGES? How many teachers participate in COGES? What are their roles on this committee? Question 7: Tell us about: a) The COGES composition (different positions and their responsibilities, assignment of positions) b) Missions and objectives of the COGES c) COGES operating rules d) Methods of communication (meetings, various notes, evaluation sheets or pupil satisfaction surveys, activity reports, telephone communications, use of ICTs, etc.) e) Decision-making methods Question 8: What are your relationships with other actors in the school, as part of this LRP project? (COGES, chef, APE, ME, teachers, nurse/responsible for medical care, students/students leaders, OPA, CRS-LRP, other schools, etc.). What is the nature of your relationship with each of them? Do you think this relationship is good or bad? Why? What do you suggest to improve the relationship? 51 Question 9: Who ensures the meal service for students is organized? Are there rules relating to: meal times, food volumes, menus, etc.? How do you ensure compliance with these rules? Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 10: What are the objectives of the LRP canteen? What are the expected results? Question 11: What are the expected results that seem to be achieved? Why? Question 12: What are the results that do not seem to be achieved? Why? Question 7: Have you received training as part of this project? If yes, what are the topics on which you were trained? Thank you for your answers. Now let's talk about the impact of the project. Item 3: Project Impact Question 15: What are the aspects of the training that you could implement quite easily? What effects have they had? What were the contributing factors to these effects? Question 16: Which of the training components received have not been implemented? Why? What changes could not be made according to the expectations of the project? What are (have been) your difficulties? Question 17: What other changes do you think the project has promoted in the school? Question 18: What other changes did you notice? To what do you attribute them? Question 15: According to you, what are the aspects that have not changed in your school since the establishment of the canteen (Good nutrition, delay, attendance, absence from school, school performance, hygiene, diseases, etc.)? Can the school canteen change them? If yes, why was there no change? (Short project duration, insufficient resources, no training, etc.) The next part we will focus on is the sustainability of the project Item 4: Project sustainability Question 20: Do you think it is necessary to renew this school canteen experience? If yes, under what conditions and in what forms? Question 21: What actions would you have planned to ensure school canteen continuity in terms of organization at the local level or partnerships? Question 22: Have there been any sharing of experiences (constraints, lessons learned) with other schools with or without school canteens? The fifth part of our discussion is about satisfaction and learning Item 5: Satisfaction and lessons learned Question 23a: What were your expectations at the start of the LRP project? Question 23b: Have your expectations for this project been met? Why or why not Question 23c: Through this school canteen experience, what has caught your attention that you want to share with others now? Let's move on to the last part of our discussion about comparing the LRP project to other projects with similar goals. Item 6: Comparative questions with other projects Question 24: Before the establishment of this canteen, had you known or experienced other school canteens? Question 25: If yes, compared to school canteens that you had known or experienced in the past: • What are the benefits of the LRP school canteen? Why? • What are the shortcomings of the LRP school canteen? Why? End date and time Thank for your collaboration 52 4. COGES Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Focus Group Discussion with COGES members Date of interview : Investigator name : INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Number of participants Male : Female : Total : Type of school : LRP Intervention = 1 ; FFE2/PAM Comparison = 2 [IF ZONE TYPE = FFE2/PAM COMPARISON, GO DIRECTLY TO SECTION B] Section A: Assessment of the LRP project Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: According to you, was it necessary to implement a canteen in the school? Why or why not? Question 2: Do you think it was necessary to implement the LRP canteen that you manage? Why or why not? Tell us about the different COGES you represent. Question 3: Tell us about: - The COGES composition (different positions and their responsibilities, assignment of positions) - Missions and objectives of the COGES - COGES operating rules - Methods of communication (meetings, various notes, evaluation sheets or pupil satisfaction surveys, activity reports, telephone communications, use of ICTs, etc.) - Decision-making methods - What is your relationships with other actors? (Chef [Provision of Products, Cooking Utensils and Other, Storage, Inventory Management], APE, ME, Principals and Teachers, Nurse /responsible for medical care, Class Managers, OPA [Identification, Procurement],], CRS-LRP, other COGES, etc.). What is the nature of your relationship with each of them? Do you think this relationship is good or bad? Why? What do you suggest to improve the relationship? 53 - Who ensures the meal service for students is organized? Are there rules relating to: meal times, food volumes, menus, etc.? How do you ensure compliance with these rules? Let's look at the effectiveness aspect of the LRP project Item 2: Project effectiveness Question 4: What are the objectives of the LRP canteen? What are the expected results? Question 6: What are the expected results that seem to be achieved? Why? Question 6: What are the results that do not seem to be achieved? Why? Question 7: In the Project implementation: - What did you like as an approach? Why? - What didn’t you like in the implementation of the school canteen? The part we are now discussing is the cost-effectiveness of the project Item 3: Cost-effectiveness of the project Question 8: Have you received training as part of this project? If yes, what are the topics on which you were trained? Probe: Have you been trained on inventory management, quantities to be served, hygiene rules, schedules, different forms of communication, etc.? Tell us about it. Question 9: What are the aspects of the training that you could implement quite easily? What changes have they made? What were the contributing factors? Question 10: Which of the training components received have not been implemented? Why? What changes could not be made according to the expectations of the project? Question 11: What are your difficulties in the functioning of COGES? Let's talk about the impact of the project within the school. Item 4: Project impact Question12: What changes have you noticed since the implementation of this school canteen? Question 13: What other changes have there been in the school and not related to the arrival of school canteens? To what do you attribute these other changes? Question 14: What has not changed in the school since the arrival of this school canteen? (Good nutrition, marketing of local products, organization of producers, presence and attendance in class, academic performance, hygiene, diseases, etc.). Can the school canteen change them? If yes, why was there no change? Let's take a look at the sustainability of the project. Item 5: Sustainability of the project Question 15: Do you think it is necessary to renew this school canteen experience? If yes, under what conditions and in what forms? Question 16: What actions would you have planned to ensure school canteen continuity in terms of organization at the local level or partnerships? Thank for your responses. Let’s talk about satisfaction and lessons learned Item 6: Satisfaction and lessons learned Question 17a: What were your expectations at the start of the LRP project? Question 17b: Have your expectations for this project been met? Why or why not? Question 17c: Through this school canteen experience, what has caught your attention that you want to share with others now? Question 18: Have there been any sharing of experiences (constraints, lessons learned) with other schools not benefiting from school canteens? 54 Section B: Comparison of the LRP project with FFE2 and PAM projects N. B: Please consider the questions in this section with the eyes of LRP or FFE2 or PAM according to the reference project for the purpose of comparison Let's move on to the last part of our discussion about comparing the LRP project to other projects with similar goals. Item 7: Comparative questions with other projects (specify the questions in relation to the school canteen project LRP or FFE or PAM in the presence) Question 19: • Describe your food supply mechanism for your school canteen. • Before the establishment of this canteen, had you known or experienced other school canteens? If yes, compared to school canteens that you had known or experienced: - What are the benefits of your school canteen? Why? - What are the shortcomings of your school canteen? Why? Question 20: What are the criteria on which the cost of a meal in the school canteen is based on your project? Question 21: Do you think that the cost of a canteen meal is affordable for all students in the school? Do you think students are satisfied with this cost? End date and time Thank for your collaboration 55 5. Producer Associations Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Focus Group Discussion with Producer Associations Date of interview : Investigator Name : INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the second hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. Location of association Department : Commune : Village : Denomination : Initials : Phone : Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: Did you participate in the information meetings on the prospects of implementing the LRP project? (If yes, number of meetings). What was said in those meetings? Has your opinion been taken into account? (Have your proposals been accepted and used?) Question 2: Did you participate in the decision-making meeting on the composition of COGES? Was your opinion taken into account during this meeting? Question 3: Did you participate (did you give your opinion) in the designation of COGES members? Question 4: Are you a member of COGES? How many OPA members belong to COGES? What are their roles on this committee? Question 5: In the implementation, a) What did you like as an approach (local procurement)? Why? b) What didn’t you like in the implementation of the LRP canteen? Question 6: Do you think that your participation in this project has had a positive impact on the local school feeding model? Why or why not? Question 7: Tell us about: a) Your Association composition (different positions and their responsibilities, assignment of positions) b) Missions and objectives of the Association c) Association operating rules d) Methods of communication (meetings, various notes, evaluation sheets or pupil satisfaction surveys, activity reports, telephone communications, use of ICTs, etc.) e) Decision-making methods 56 - Question 8: What is your relationship with other actors? (COGES [Contract management, readiness to respond, quality of the products provided, constraints to meet the demand, management of conflicts related to failures in compliance with the terms of contract, satisfaction with product prices and payment methods], APE, Directors and teachers, CRS-LRP [identification and selection criteria in the project], OP faîtières [intermediation for the contracting with the COGES, contribution to facilitate the realization of the contracts], other producer associations, etc.). What is the nature of your relationship with each of them? Do you think this relationship is good or bad? Why? What do you suggest to improve the relationship? Question 9: What were the constraints you faced in managing your relationships with other actors? What solutions have you developed? Which of these constraints have been discussed within the association? Which of these constraints have been discussed with other actors (CRS-LRP and COGES)? Why? Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 10: What can you tell us about the school canteens of the LRP project? Question 11: What are the objectives of the LRP canteen? What are the expected results? a) What are the expected results that seem to be achieved? Why? What factors have contributed to the achievement of these results? b) What are the results that do not seem to be achieved? Why? What factors have hindered the achievement of these results? Question 12: Have you received training as part of LRP project? If yes, what are the topics on which you were trained? Probe: Have you been trained on: strengthening organizational capacities (formalization, application of statutory regulatory documents, maintenance of administrative and management documents, admissibility to members), improvement production practices and post-harvest management, group selling practices, market access and the use of market information systems? Question 13: Did you really need these trainings? Is there a difference between these trainings and those you have received in other projects in terms of training approaches or content? The third component relates to the cost-effectiveness of the project. Item 3: Cost-effectiveness of the project Question 14: Has the project allowed your association to have better financial benefits? If yes, what were the most profitable activities? Question 13: What are the internal or external factors that have hindered the effective implementation of other project activities? Thank you for your answers. Now let's talk about the impact of the project. Item 4: Project Impact Question 16: What are the aspects of the training that you could implement quite easily? What effects have they had?: strengthening organizational capacities (formalization, application of regulatory documents, maintenance of administrative and management documents), interest for the production of crops targeted by the LRP, improvement of production practices and post-harvest management [new farming practices including climate smart practices, access to appropriate equipment, infrastructure and techniques for storage and conservation], improvement of group selling practices, use of market information systems and market access [guaranteed access to markets, warrantsupport, decrease in the price of agricultural products, position improvement over traders])? What were the contributing factors? Question 17: What are the positive/negative effects of strengthening your capacity on climate change on your productivity? Question 18: Which of the training components received have not been implemented? Why? What changes could not be made according to the expectations of the project? What are (have been) your difficulties? 57 Question 19: Does your participation in this project make your association able to take future commitments for the supply of agricultural products to other school canteen projects/structures or other sales to institutions such as WFP? If yes, how? The next part we will focus on is the sustainability of the project Item 5: Project sustainability Question 16: Do you think it is necessary to renew this school canteen experience? If yes, under what conditions and in what forms? The sixth part of our discussion is about learning Item 6: Satisfaction and lessons learned Question 21a: What were your expectations at the start of the LRP project? Question 21b: Have your expectations for this project been met? Why or why not? Question 21c: Through this school canteen experience, what has caught your attention that you want to share with others now? Let's move on to the last part of our discussion about comparing the LRP project to other projects with similar goals. Item 7: Comparative questions with other projects Question 22: Before the establishment of this canteen, had you known or experienced other school canteens? If yes, compared to school canteens that you had known or experienced in the past: • What are the benefits of the LRP school canteen? Why? • What are the shortcomings of the LRP school canteen)? Why? End date and time Thank for your collaboration 58 6. Students Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP)) Focus Group Discussion with Students INTRODUCTORY FORM Hello. My name is ........................, and I work for the firm EVIHDAF, a partner of Catholic Relief Services (CRS), which evaluates the LRP project "Bèsèn Diannou", which has been in implementation since 2018. In order to appreciate the level of achievement of the expected results and the quality of the school canteen supply system in place, we would like to have a discussion with you. This discussion will take about 30 minutes to complete. On the one hand, this discussion will allow us to learn from the various actions carried out and, on the other hand, to make recommendations to CRS and its various partners for future school feeding programs based on a local and regional food supply. If you approve, we would like to record the discussion to make sure your thoughts, opinions and ideas are accurately translated. Your identity will be kept confidential and we will not associate your name with what you say. You may decline to answer any questions. Similarly, if necessary, you can withdraw momentarily or permanently. SHEET IDENTIFICATION Sheet’s number (Use a 2-digit number. e.g. 01, 02, 10, 30 ...): /________/ Investigator’s name: ………………………………………/ Date of investigation: J /___/ M/___/2019 Start time: /___/ H/___/Mn End time: /___/ H/___/Mn Supervisor's name: ……………………………………………/ Date of supervision: J /___/ M/___/2019 Questions Tick the right answers Commune 1_ Parakou ; 2_ N’Dali ; 3_ Pèrèrè ; 4_ Tchaourou Number of schoolchildren Boys Girls Total Our first round of discussion will focus on the relevance of the LRP project Item 1: Relevance of the project Question 1: Do you think it was necessary to implement a canteen in the school? Why or why not? Question 2: Are schoolchildren or representatives of schoolchildren members of COGES? If yes, what is their role within COGES? Question 3: Is the opinion of the schoolchildren taken into account when planning the meals? If yes, who considers their opinion? Question 4: Is the opinion of schoolchildren on the quantity and quality of the meals served taken into account? If yes, who considers their opinion? Question 5: If a meal is not enough for a school child, can he or she ask for additional meal service? Tell us about your experiences. Question 6: Are parents of students involved in canteen management? If yes, how? If not, why? Question 7: Are students’ mothers involved in canteen management? If yes, how? If not, why? Question 8: Are school principals involved in canteen management? If yes, how? If not, why? Question 9: Are teachers involved in canteen management? If yes, how? If not, why? 59 Let's move on to the second part of the discussion about the effectiveness of the LRP project Item 2: Project effectiveness Question 10: In the functioning of the school canteen, what did you like? Why? a) Composition of dishes: Why? b) Amount of meals served: Why? c) Quality of meals served: Why? d) Local meals: Why? e) Number of meals served per school day: How much and why? f) Hours of meals : Why? g) Meal Costs: Why? h) Fewer diseases at school: Why? i) Less absence or delay of schoolchildren in class: Why? Question 11: What did not you like in the implementation of the school canteen? a) Composition of dishes: Why? b) Amount of meals served: Why? c) Quality of meals served: Why? d) Local meals: Why? e) Number of meal served per school day: How much and why? f) Hours of meals : Why? g) Meal Costs: Why? Thank for your responses. Let’s talk about the impact of the project Item 3: Impact of the project Question 12: Before the implementation of this canteen, how did you get meals at school? Question 13: What are the advantages that you perceive since the implementation of this canteen in your school (ease of access, good school feeding, and lower financial cost)? Question 14: Are there any negative aspects related to the presence of this canteen in your school? The next part we will focus on is the sustainability of the project Item 4: Sustainability of the project Question 15: What can disturb the functioning of your school canteen? The last part we will focus on is the lessons learned Item 5: Lessons learned Question 16: Through this school canteen experience, what has caught your attention that you want to share with others? Question 17: Do you think that it is necessary to continue this school canteen? Why? End date and time Thank for your collaboration 60 IV. Direct Observations 1. COGES Observation (Cooks and Storekeepers) Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Direct observation sheet COGES (Cooks and Storekeepers) Commune: Parakou=1; N'Dali=2; Pèrèrè=3; Tchaourou=4; Kalalé=5; Gogounou=6; Kandi=7; Malanville=8 School ID /___ //___ / Body and dress hygiene 1 Do the cooks wash their hands before preparing the kitchen? (Yes=1 ; No=0) 2 Do the cooks protect their heads while cooking? (Yes=1 ; No=0) 3 Do the cooks wash their hands before handling food (use of starch, crushing or cutting of condiments, etc...) and after a break or rest? (No=0 ; With water only =1 ; With water and soap or ashes =2) 4 Do cooks have nails well cut? (Yes=1 ; No=0) 5 Do cooks carry beauty materials that can rust? (Rings=1 ; Earrings =2 ; Bracelets=3 ; Necklace =4 ; No=0) [multiple responses possible] 6 Do the cooks wear kitchen uniforms? (Aprons, vests, blouses, etc.) ? (Yes=1 ; No=0) 7 Do the cooks wear clean clothes? (Yes=1 ; No=0) Food hygiene 8 Are the foods used for the preparation of meals of good quality (absence of weevils, cereals without holes, undamaged, etc.)? (Yes=1 ; No=0) 9 Are the foods washed before preparation? (Yes=1 ; No=0 ; Not applicable=2) 10 Are the meals placed on a tray before and during the service? (Yes=1 ; No=0) 11 Is the food is sorted before use? (Yes=1 ; No=0 ; Not observed) 12 Is the water used for the preparation clear (colourless, odourless)? (Yes=1 ; No=0) Is the water used for the dishes clear (colourless, odourless)? (Yes=1 ; No=0) Dishes 13 Are cooking tools washed well with soap and water? (Yes=1 ; No=0) These are pots, pans, basins, buckets, knives, bowls etc ... They must be washed with soap and water before and after each preparation. 14 Are the utensils (bowls, ladles, spoons, dishes) not dropped on the ground? (Yes=1 ; No=0) 15 Are the utensils rinsed with potable water at each step (change of food)? (Yes=1 ; No=0) 16 Are the utensils that are in contact with raw food immediately washed with clean water and soap? (Yes=1 ; No=0) 17 Are the dished placed far from where the meals are prepared? (Yes=1 ; No=0) The investigator may consider that the distance is far if it is more than 100 m from the place where the meals are prepared. Cooking and preparation 18 Are the foods well cooked? (Yes=1 ; No=0) 19 Are canned foods used in food preparation? (No=0 ; Yes=1 ; Not applicable=2) 20 Are expiry dates correctly respected? (Yes=1 ; No=0) 21 Are thepots covered before, during and after cooking? (Yes=1 ; No=0) 61 22 Are dishes from previous days recycled? (Yes=1 ; No=0 ; Not applicable=2) 23 If yes, are they hygienic (non-sticky, without mold or smell of fermentation, etc.)? (Yes=1 ; No=0) Conservation 24 After cooking, are the foods well preserved (kept hot in a well-covered container)? (Yes=1 ; No=0) 25 Are cooked foods that are not consumed well preserved (kept refrigerated)?Yes=1 ; No=0) Service 26 Do cooks ensure that meals are hot when they are distributed (reheating as needed)? (Yes=1 ; No=0) 27 Do cooks serve meals in clean bowls/plates? (Yes=1 ; No=0) Environmental hygiene 28 Are the surroundings of the kitchen well cleaned? (Yes=1 ; No=0) 29 Is the kitchen cleaned before and after the preparation and cooking of meals? (Yes=1 ; No=0) 30 Is wastewater discharged away from the meal preparation area? (Yes=1 ; No=0) 31 Is kitchen waste cleared every day? (Yes=1 ; No=0) 32 Is the kitched far from unsanitary areas (WC, channels, sumps, pile of garbage)? (Yes=1 ; No=0) Hygiene in the store 33 Is the store well swept? (Yes=1 ; No=0) 34 Number of times swept per day /__/__/ 35 Storage mode (In bags =1 ; In bulk =2 ; In basins =3 ; other (specify)=8) 36 Are the products on pallets? (Yes=1 ; No=0) 62 2. COGES Observation (community involvement in the buying process) Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Direct observation sheet COGES (community involvement in the buying process) Commune: Parakou=1; N'Dali=2; Pèrèrè=3; Tchaourou=4; Kalalé=5; Gogounou=6; Kandi=7; Malanville=8 School ID /___ //___ / Q01. Are parents of students/community members present during inventories? (0=No, 1=Yes) Q02. Is the opinion of the parents of students/community members considered when the orders are made? (0=No, 1=Yes) Q03. Are there parents of students/community members when receiving delivered products? (0=No, 1=Yes) 63 3. Inventory of infrastructures Project evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Infrastructure Inventory Sheet Commune: Parakou=1; N'Dali=2; Pèrèrè=3; Tchaourou=4; Kalalé=5; Gogounou=6; Kandi=7 ; Malanville=8 School ID /___ //___ / Section A. Inventory in kitchens Location of the kitchen (Within the school =1 ; Out of school=2) Is there a cafeteria for students? (Oui=1 ; Non=0) If not, where do they eat? (In the classrooms =1 ; Under a tree =2 ; in the school yard =3 ; other (specify)=8) Is there a hand washing device for children? (0=No, 1=Yes) If yes, which ones: 1=Tippy Tap, 2=DLM, 3= other (specify)____________________ Structure used for cooking (Outdoors =1 ; Tent/Hangar=2 ; Well-built =3 ; other (specify)=8) If <>1, nature of the walls of the kitchen (Clay =1 ; Brick =2 ; Straw =3 ; Attic =4 ; sheet metal =5 ; Woods=6 ; other (specify)=8) If <>1, nature of the kitchen roof (Sheet metal/Tile=1 ; Straw=2 ; Tile=3 ; other (specify)=8) IF <>1, nature of the kitchen floor (Groupnd=1 ; Cement=2 ; Tile=3 ; other (specify)=8) Type of cooking firebox used Number of fireboxes for cooking meals Presence of a fridge Section B. Inventory in Stores Nature of store walls (Clay =1 ; Brick =2 ; Straw =3 ; Attic =4 ; Sheet metal =5 ; Woods=6 ; other (specify)=8) Nature of the store roof (Sheet metal/Tile=1 ; Straw=2 ; Tile=3 ; other (specify)=8) Nature of the store floor (Ground=1 ; Cement=2 ; Tile=3 ; other (specify)=8) Location (Within the school in a location of its own =1 ; Within the school in a classroom =2 ; Within the school in the Director's office =3 ; Out of school at home =4 ; Out of school at a community store =5) What are the stored foods? Are they in sufficient to keep the stock filled? 64 Second Part: Comparative study on costs and times I. Quantitative Surveys 1. Sheet for LRP/FFE/PAM Staff in charge of procurement and logistics Project Evaluation Bèsèn Diannou Local and Regional Food Aid Procurement (LRP) Data collection sheet with LRP, FFE and PAM Staff in charge of purchases and finances Date of interview : Investigator Name : INTRODUCTORY FORM Location of association Department : Commune : Village : Denomination : Initials : Phone : Section A: Collection of quantitative data Project (LRP=1; FFE2=2; PAM=3) Q1. Take stock of the food supply operations you have done in the past 12 months by completing the following table: (See next page) 65 Sheet of food supply operations for school canteensa Product b Wave of purchase Quantity (in tons) Amount Cost of transport (until delivery) Duration (in days) of the purchase process) c Number of days from order to delivery Duration (days) of the distribution of products to beneficiary schoolsd Cost of transportation of product distribution to beneficiary schoolsd Number of beneficiary schools consideredd Number of Km counted for distribution in the beneficiary schoolsd Number of target students considered or estimatedd Maize 1 2 3 4 Other wave (to be completed) Soy 1 2 3 4 Other wave (to be completed)) Rice 1 2 3 4 Other wave (to be completed) Shea butter 1 2 66 3 4 Other wave (to be completed) Other products (to be specified and completed) 1 2 3 4 Other wave (to be completed) a: Please prepare for us documents on these procurement operations such as registers, slips, purchase/orders contract with suppliers, carriers, etc. b: You can add lines to supplement the food products bought for the benefit of the canteen as part of your project whose name is not on the form or also add lines to inform the waves of purchase made when there is an amountmore than what is mentioned. c: The duration (in days) of the purchase process includes the time from the launch of the tender (submission, selection of suppliers and signature of contract) or the time of launching or negotiation of the order until the signing of the order contract. d: Please provide data by target commune if possible. Q2. How much do you consider a dish offered by the canteen to Students? /__ /__ /__ /__ / Section B: Collection of qualitative data Q3. Describe your food supply mechanism for your school canteen. Q4. Do you think that the method of supplying your project is faster and cheaper than the methods of supplying other projects? Why? Q5. Which products are the least expensive and easily accessible on the market? Q6. What strategy do you use to reduce delivery times in schools to avoid selling out? 67 2. COGES - Representative See Section G of the corresponding questionnaire in the Endline study 3. Students See Section D of the corresponding questionnaire in the Endline study II. Individual Interviews 1. LRP/FFE/PAM staff in charge of procurement and logistics Following the sheet of LRP/FFE/PAM Staff 2. Communes See Section B of the corresponding Interview Guide in the Endline Study 3. Government authorities See Section B of the corresponding Interview Guide in the Endline Study III. Focus Group Discussions 1. COGES See Section B of the corresponding Interview Guide in the Endline Study 68 dissemination. 69 70 71 72 Annex 4. Highlights of key review meetings September 19th Webinar A webinar was organized on September 19, 2019 to review and discuss the evaluation finding and identify areas of further improvements. In attendance were CRS staff including: Blandine Ekpodile (LRP Director), Marie Rongeard, Stacy Prieto, Megan, Caitlin, Thierry Yabi (MEAL Coordinator for Benin-Togo), and Brice Denakpo (MEAL LRP). From the evaluation team, Jean Christophe Fotso and Paul Kouete Jimmy were in attendance. The meeting began with a presentation by Jean Christophe, with focus on the methodology, key findings and the recommendations. Followed a session for questions and answers during which the CRS team made comments and suggestions, and sought clarifications from the evaluation team. Major findings were discussed, their interpretation revisited, and suggestions made to improve on the report. Discussion with FFE and PAM staff After reviewing the preliminary findings from the comparative study, a review meeting was organized on September 9, 2019 to harmonize understanding of the different cost and time variables, and ensure that data is comparable across the three programs. Participants included: Blandine Ekpodile, Salimanou Issa and Brice Denakpo from LRP; Simplice Gangbadjo from MGD/FFE; Wilfried Chigblo and Armelle Korogone from WFP; Thierry Yabi from CRS; and Jean Christophe Fotso from the evaluation team. During the meeting, a number of adjustments were proposed, such as the inclusion of a description of meals served weekly to students in the datasets. Another key outcome of the meeting was the realization that the indicator on cost per meal will not apply to WFP, given the way their information system is structured. Following the meeting, the analysis was further refined. 73