Decentralised Evaluation Decentralised evaluation for evidence-based decision ki Prepared by Ian Pinault, team leader Friederike Bellin-Sesay, nutritionist Covadonga Canteli, statistician Jean-Pépin Pouckoua, national consultant Jean-Pierre Silvéréano, education expert October 2018 Agreement Number: FFE-679-2017/020-00 Commissionner: WFP Republic of Congo Country Office Evaluation Manager: William Vu Baseline Report of the WFP McGovern Dole Funded School Feeding Programme in the Republic of Congo 2018 - 2022 Evaluation Report Baseline report – Second draft – July 2018 i | Page Acknowledgements The KonTerra team would like to thank the Institute for National Statistics (INS) and the Direction des Études et de la Planification within the Ministry of Education, and the enumeration teams for their dedication to the survey, literacy testing and reporting. The evaluation team also gratefully acknowledges the community participants in baseline survey process (pupils, teachers, school directors and parents). Disclaimer The opinions expressed in this report are those of the Evaluation Team, and do not necessarily reflect those of the World Food Programme or the United States Department of Agriculture. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP or USDA of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. Accessibility Note An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov Baseline report – Second draft – July 2018 ii | Page Table of Contents Acknowledgements........................................................................................i Disclaimer.....................................................................................................i List of Figures..............................................................................................iv List of Tables ...............................................................................................iv List of Acronyms........................................................................................... v Executive Summary.....................................................................................vi Methodology for the baseline report and baseline data collection................................................vi Key Findings of the Baseline data collection and analysis........................................................... vii Conclusions and recommendations............................................................................................. viii 1.Introduction............................................................................................... 1 2.Country and Development Context.............................................................2 3.Subject of the Baseline Report, Theory of Change and baseline questions ..6 3.1. Subject of the Baseline Report, Theory of Change, Activities and indictors......................... 6 3.2. Baseline Evaluation Questions and Evaluation Criteria........................................................ 8 4.Baseline Report approach and methodology for baseline data collection ...8 4.1. Baseline report and methodology........................................................................................... 8 4.2. Baseline Data Collection Methods and Tools........................................................................ 11 4.3. Limitations and risks .............................................................................................................15 4.4. Quality assurance of the baseline data collection ................................................................. 16 4.5. Ethics ......................................................................................................................................16 5.Baseline Findings and Discussion ............................................................ 16 5.1. Baseline Values and validation of programme yearly targets (Q. 1 & 2).............................. 17 5.1.1. MGD SO1: Improved Literacy of School-Age Children .................................................. 17 5.1.2. MGD SO2: Increased Use of Health and Dietary Practices........................................... 30 5.2. Success factors for efficient and effective M&E of the MGD programme (Q. 3) ................ 40 Baseline report – Second draft – July 2018 iii | Page 5.3. Feasibility of the evaluation design and evaluation questions (Question 3) .......................41 5.4. Gender equality and women empowerment throughout the programme.......................... 42 6.Conclusions and Recommendations.........................................................44 6.1. Overall Assessment and Conclusions ................................................................................... 44 6.2. Recommendations ................................................................................................................ 47 Annexes......................................................................................................52 Annex 1. Terms of reference .............................................................................................. 53 Annex 2. Map of intervention............................................................................................ 30 Annex 3. Resource requirements and funding situation...................................................31 Annex 4. Detailed description of all MGD activities according to the project agreement between WFP and USDA.................................................................................... 32 Annex 5. Literacy Result Framework (RF 1)..................................................................... 35 Annex 6. Health, Nutrition and Dietary Practices Results Framework (RF 2) ............... 36 Annex 7. Foundational Results...........................................................................................37 Annex 8. Outputs according to actual PMP ...................................................................... 38 Annex 9. Outcomes according to actual PMP ................................................................... 42 Annex 10. Analytical Framework ........................................................................................ 44 Annex 11. Baseline interview guides ................................................................................... 45 Annex 12. Evaluation matrix ............................................................................................... 46 Annex 13. Team schedule during field mission .................................................................. 48 Annex 14. Teacher questionnaire ........................................................................................ 50 Annex 15. School administrator questionnaire .................................................................. 52 Annex 16. School feeding committee questionnaire........................................................... 56 Annex 17. Pupil questionnaire............................................................................................. 59 Annex 18. National Institute of Statistics (INS) overview.................................................. 62 Annex 19. Additional note to the quantitative survey strategy and approach................... 63 Annex 20. Training outline .................................................................................................. 65 Annex 21. Survey data analysis report ................................................................................ 70 Annex 22. Table of all Standard and Custom indicators (WFP PMP vs. ET findings)....... 71 Annex 23. Rationale on the indicator measuring improved literacy.................................. 74 Annex 24. Report from the PASEC literacy test...................................................................77 Annex 25. Comparison of results to literacy test at the end of two grades of primary schooling ............................................................................................................. 94 Annex 26. Various concepts of dietary diversity in Congo ................................................. 96 Bibliography / Desk Review ...................................................................... 100 Baseline report – Second draft – July 2018 iv | Page List of Figures Figure 1: Overview of the degree of relevance of the M&E system using the PMP indicators ....... 45 Figure 2: Details of the project operating costs................................................................................. 31 List of Tables Table 1: Case and comparison groups considered in the evaluation design ................................... 10 Table 2: Methods used for qualitative data collection ...................................................................... 11 Table 3: Quantitative survey sample ................................................................................................. 12 Table 4: Planned sample sizes, error and effect size estimations.....................................................13 Table 5: Stratification of samples by department (sample size of 50 schools in each main group)14 Table 6: Children who report missing less than 20% of school days in prior month..................... 18 Table 7: Average perc. of pupils reported to be inattentive (every day or sometimes) .................. 27 Table 8: Percentage of days missed due to sickness (according to pupils).....................................28 Table 9: Average perc. of days that teachers are absent (over the last month) .............................. 29 Table 10: Highest educational diploma............................................................................................ 29 Table 11: Highest professional diploma............................................................................................ 29 Table 12: Water source characteristics ............................................................................................. 32 Table 13: Latrines' characteristics .................................................................................................... 33 Table 14: Food before school. Do you usually eat before school?.................................................... 35 Table 15: Overall dietary diversity*, 0-7 scale (7 food groups)........................................................ 36 Table 16: Hand washing after toilet.................................................................................................. 36 Table 17: Hand washing before eating.............................................................................................. 37 Table 18: Hand washing after eating ................................................................................................ 37 Table 19: Reasons for absence .......................................................................................................... 37 Table 20: FSC size and female presence........................................................................................... 38 Table 21: Position of female in SFCs................................................................................................. 38 Table 22: Estimation of error in the output achievement indicator by number of schools visited 63 Table 23: Main characteristics of enrolled children by region/zone (beginning on school life).... 64 Table 24: Percentage of enrolled children by literacy level (as issued from the literacy test) and region/zone........................................................................................................................................64 Baseline report – Second draft – July 2018 v | Page List of Acronyms ACTED Agence d'aide à la coopération technique et au développement ASPC Association des pères spiritains du Congo BR Baseline Report CAR Central African Republic CP Cours préparatoire CONFEMEN Conférence des ministres de l'Éducation des États et gouvernements de la Francophonie (Conference of Ministers of Education of French-Speaking Countries) CSFVA Comprehensive Food Security and Vulnerability Analysis DAC Development Assistance Committee DDEPSA Direction départementale de l’éducation primaire, secondaire et de l’alphabétisation DEP Direction des Etudes et de la Planification DRC Democratic Republic of Congo EM Evaluation Manager EMOP Emergency operation ET Evaluation Team FAS Foreign Agricultural Service FGD Focus group discussions GDP Gross domestic product GEEW Gender Equality and Empowerment of Women INS Institut National de la Statistique (National Statistics Institute) IR Inception Report ITC Information and Communication Technologies KAP Knowledge, attitudes and practices MEPSA Ministère de l'Enseignement Primaire, Secondaire et de l'Alphabétisation (Ministry of Primary and Secondary Education and of Alphabetisation) MGD McGovern-Dole NSFP National School Feeding Policy OECD Organisation for Economic Co-operation and Development OEV Office of Evaluation PASEC Programme d’analyse des systèmes éducatifs de la CONFEMEN (CONFEMEN programme for the Analysis of Education Systems) PMP Performance Monitoring Plan PRAASED Projet d'Appui à l'Amélioration du Système Éducatif PSU Primary sampling units RBJ Regional Bureau Johannesburg RF Result Framework RoC Republic of Congo SABER Systems Approach for Better Education Results SFC School feeding committees SGBV Sex and Gender Based Violence SOP Standard Operating Procedure ToR Terms of Reference UNDP United Nation Development Programme UNESCO United Nations Educational, Scientific and Cultural Organisation UNICEF United Nations Children’s Fund USDA United States Department of Agriculture WFP World Food Programme Baseline report – Second draft – July 2018 vi | Page Executive Summary ES.1. This document is the baseline report for the World Food Programme’s (WFP) McGovern-Dole International Food for Education and Child Nutrition Programme (McGovern-Dole) in the Republic of Congo (RoC). The McGovern-Dole (MGD) programme consists of a $30,022,053 grant over a five-year period (2017-2022)1. Through a broad panel of activities (school feeding, health and dietary sensitisation, capacity building, infrastructures rehabilitation, etc.), WFP and its three sub-recipient partners2 will work towards achieving the overall objectives of the programme which are “Improved Literacy of School-Age Children” and “Increased Use of Health and Dietary Practices”. ES.2. The main purpose of this Baseline Report (BR) is to assess the situation before the programme takes place by establishing baseline figures and assessing feasibility/relevance of the yearly targets as outlined in WFP’s Performance Monitoring Plan (PMP). This baseline will serve as a comparison point to assess changes during mid-term review and final evaluation. The baseline report also serves the objectives of accountability of the actions and results planned as well as learning and adjusting based on the findings of the report. ES.3. The Republic of Congo is classified as a medium human development country with a human development index of 0.5923. The Subjective Well-Being (SWB) of the population in 2015 determines that 82.6% of the population are generally satisfied with their lives. Congo’s gross domestic product (GDP) dropped from US$14,000 million in 2014 to US$7,800 million in 2016, which is a direct consequence of the oil price shock in 2014.4 Congo ‘s first school feeding activities started in 2001. The government wishes to expand school feeding to the whole country by 2025. In 2016, Congo adopted a National School Feeding Policy (NSFP). This policy includes the establishment of the School Feeding Directorate that will oversee school feeding activities around the country. Its director was appointed in May 2018. The MGD programme is implemented in a time where the Republic of Congo has been suffering an economic crisis since 2016. The crisis has had negative repercussions on WFP’s budget5 and consequentially on the implementation of a large amount of their programme activities. The expected users for this BR are the WFP (country office, regional bureau, headquarters, office of evaluation), the Ministries associated with the MGD programme, sub-recipient partners, implementing partners and the US Foreign Agricultural Service (FAS). Methodology for the baseline report and baseline data collection ES.4. The main evaluation questions, as indicated in the Terms of Reference, are:  Q1 - What are the baseline values for each indicator in the Performance Monitoring Plan (PMP)?  Q2 - Given these baseline values, the objectives of the MGD programme and within the context of Congo, are the targets set for each indicator realistic? Are any of them too low or too high? 1 Due to late implementation of the programme, the dates of the MGD may shift to 2018-2023. 2 Agence d'aide à la coopération technique et au développement (ACTED), the United Nations Educational, Scientific and Cultural Organisation (UNESCO) and the United Nations Children’s Fund (UNICEF). 3 United Nation Development Programme (UNDP); Human Development Report 2016- Human Development for Everyone; 2016 4 World Bank, http://donnees.banquemondiale.org/pays/congo-republique-du, visited on the15th of August 2017 5 Less than 17% of WFP’s country programme 200648 (2015-2018) had been financed before the attribution of MGD. Most of the planned activities of the programme have been reduced and some have been cancelled. Baseline report – Second draft – July 2018 vii | Page  Q3 - Given the objectives and activities of MGD and the context of Congo, what are the key success factors for efficient and effective M&E of the programme? How feasible is the evaluation design and evaluation questions? ES.5. In order to respond to these questions, an inception report was elaborated and validated by WFP in March 2018 before the field work for this baseline study. The baseline applied a mixed-method approach, using three main collection methods: (1) secondary data review (reports, former evaluations, MGD proposal, etc.), collection of (2) qualitative (semi-structured interviews and observation) and (3) quantitative data (large scale survey) to derive evaluation findings. The quantitative component of the evaluation applied a quasi-experimental approach which included a case and a control group of schools so that at endline, the impact of the intervention can be measured using pre-and post￾comparisons. ES.6. Data collection took place from the 7th of May until the 15th of June 2018. A total of 1,590 children have been sampled from 100 schools in 5 departments to inform this design.6 This satisfies the targets set during the inception phase (below 5% sampling error and 8% effect size). Additionally, 219 children from 14 ORA schools 7 of the departments of Likouala and Sangha were surveyed using mostly the same tools.8 One hundred and fourteen administrators and 228 teachers of those all schools were also interviewed. Around 64 school feeding committees were also interviewed when existing in the schools visited. In addition to quantitative data, qualitative data was collected by the evaluation team through desk review and semi-structured interviews with the main stakeholders (sub-recipient partner, relevant ministries, school administrators, etc.). ES.7. The main limitation of the study is the absence of available data on school figures (absence of education statistical yearbook). Hence, certain indicators rely only on the availability and accuracy of data collected in schools during the baseline study. During the mid-term evaluation, the education statistical yearbook 2017 should be available. This will allow the ET to triangulate certain findings regarding some of the baseline indicators. No other major limitations having significant impact on the study have been encountered. Key Findings of the Baseline data collection and analysis ES.8. Evaluation question 1: WFP’s PMP presents a total of 33 indicators that support M&E activities. Twenty-five of the performance indicators are standard MGD indicators, while eight have been added by the CO to better fit the programme’s context. Apart from when baseline figures are supposed to equal zero (26 out of 33), all baseline figures need to be revised. New custom indicators should also be added to either follow activities that are not reflected in the standard indicators (training for teachers in nutritional education; teacher’s attendance; teacher’s level of education; change over time of the use of health and dietary practices; etc.) or disaggregated indicators that are not specific enough (indicators related to training and sensitisation need to be specific on the type of training and the type of beneficiaries; how will the programme specifically help implement the new National School Feeding Policy; etc. ). Additional baseline indicators have been considered and collected through the INS survey to better monitor expected effects or unexpected effects over the duration of the programme. Gender disaggregated indicators are operational, but 6 The quasi-experimental design for impact quantification is applied in the departments of Bouenza, Cuvette, Lekoumou, Plateaux, and Pool. 7 Created in 1980 for Pygmy children in eastern Cameroon, the ORA schools (Observer, Think, Act) are refresher schools: they allow children who are behind in their schooling to reintegrate the normal education system after three years. 8 Schools in these two departments operate in a very different cultural context using a different curriculum. They are therefore studied separately and don’t inform the impact assessment Baseline report – Second draft – July 2018 viii | Page the data collected and interviews conducted suggest there is room for further integration of additional indicators in the PMP intended to capture gender transformation results. ES.9. Evaluation question 2: As the number of schools and pupils has changed between WFP’s initial proposal and the current situation, all indicators related to the number of schools and pupils need to be revised. In addition, changes over time of yearly targets sometimes contradict each other (e.g. number of pupils increase each school year while the number of meals distributed does not). Fundamental activities linked to the Improved Quality of Literacy Instruction (such as better access to school supplies, improved literacy instructional materials or increased skills and knowledge of teachers) will actually not be conducted by MGD implementing partners but rather by the World Bank. It will therefore be impossible to attribute these indicator outcomes “as a result of USDA assistance”. Yearly targets need to consider the past targets achieved and should not only focus on the activities done during the current year. This undermines the sustainability of the programme. MoUs between WFP and sub-recipient partners are based on one year which prevents the forecasting of targets over the life of the programme. The baseline found that at several indicators need to be disaggregated by type of recipients or activity in order to be specific and measurable across the life of the MGD programme (Standard ind. # 4, 6, 7, 9, 12 and 18 and Custom ind. # 1, 2, 3 and 6). Indicators on improved water source and improved sanitation facilities need to be collected as quantities rather than percentages. This means that it would be preferable if these indicators were tracked through an individual follow-up of each school at baseline, as well as during the monitoring of the programme. ES.10. Evaluation question 3: Based on evidence obtained during the data collection of the baseline, MGD programme partners do not seem to have fully integrated the PMP indicators to which they and WFP will be held accountable for programme implementation. A strong M&E system which coordinates the diversity of activities and partners involved in the MGD programme is not yet in place.9 Fundamental activities linked to educational training and elaboration of new school books based on a revised curriculum are not being conducted by WFP or MGD sub-recipient partners. Hence coordination with external MGD actors, namely the World Bank, is essential to ensure that those activities will directly affect MGD beneficiaries. The approach on dietary practices does not seem to be coordinated between implementing partners and other actors in the country. If all implementing partners do not use/have the same approach on dietary practices (same food groups, same key messages, etc.), a common M&E system will not be easily applicable. ES.11. Gender. The MGD M&E plan intends to track equal participation-related objectives but it does not foresee indicators that relate to other WFP Gender Policy objectives, particularly related to empowerment, decision-making and protection. Conclusions and recommendations ES.12. Conclusions Q1 & Q2: Eight of the thirty- three PMP indicators only need minor changes to be adequately used in the M&E of the MGD programme. Sixteen indicators out of thirty-three require changes in order to provide proper information during the whole M&E process of the programme. Those changes consist in recalculating yearly objectives according to MoUs, ensure that target numbers account for the latest number of schools and pupils to be assisted, etc. Four indicators need major change in order to determine more specifically the beneficiaries and/or the activities they intend to measure. If those indicators stay too vague, it will be difficult to monitor the programme over time. Five 9 A senior M&E office is supposed to be recruited by WFP for this task. Baseline report – Second draft – July 2018 ix | Page indicators are not linked to the activities that are actually planned by the MGD programme (Standard ind. # 2, 5, 6 and 25 and Custom ind. # 4). ES.13. Conclusions Q3: The key success factors for efficient and effective M&E of the programme relies on a closer collaboration with sub-recipients as well as with implementing partners. A global and consistent M&E still needs to be put in place as planned by the WFP CO.10 The overall evaluation of the MGD programme design (Baseline Report, Mid-term Review and Final Evaluation) envisioned as part of the Inception Phase is still appropriate. However, as ET will only be able to collect data on a representative sample of the schools, WFP and its partners should conduct specific monitoring (including baseline) for each school. This will enable accurate decision making based on specific assessment instead of basing decisions on general findings. ES.14. Recommendations: The findings and conclusions of this baseline report led to the evaluation team making the following recommendations that will need to be addressed before the actual start of the programme in September/October 2018:11 ES.15. Recommendation 1: The ET recommends12 that WFP, the sub-recipient partners and other main stakeholders in the education and school feeding sector (such as the Directorate of School Feeding, the World Bank and the ASPC - Association des pères spiritains du Congo) enhance coordination before the start of the MGD programme in September/ October 2018 in order to respond to the weaknesses in the planning of the programme as identified in this baseline report. These refer mainly to insufficient common understanding of programme outcomes between WFP and cooperating partners, lack of appropriate activities planned to reach the “Improved Quality of Literacy Instruction” Result Stream, insufficient details and disaggregation of the implementation plan, and nonalignment between sub-recipient partners’ MoUs and the initial WFP/USDA agreement. ES.16. Recommendation 2: Following the baseline findings, several indicators need to be modified either through further disaggregation or through re-definition in order to better match the activity they intend to measure. Moreover, new indicators should be incorporated to ensure that adequate information is tracked to properly measure planned activities. Yearly targets need better definition and as much as possible they should be articulated throughout the programme’s duration. Those indicators should be directly linked to activities as per agreed MoUs with WFP. ES.17. Recommendation 3: Observing that the fundamental activities linked to the Improved Quality of Literacy Instruction will not be conducted by MGD sub-recipients but rather fall under the PRAASED13 World Bank programme, the ET recommends that MGD works in tight collaboration with PRAASED by elaborating a common framework in order to enhance programme complementarities, avoid overlap and allow MGD’s monitoring system to establish the necessary connections with PRAASED for an effective monitoring of the MGD programme. This is particularly important regarding the “Improved Quality of Literacy Instruction” result stream. ES.18. Recommendation 4: WFP and its sub-recipient partners should apply the Congolese Food Based Dietary Guidelines to overcome the inconsistencies found across various documents on the use of food groups, allowing for a unified understanding and measurement of activities and indicators linked to nutrition. In addition, the key messages 10 WFP CO was recruiting a senior M&E officer during the ET’s field mission. 11 This is a resumé of the recommendations, see full recommendations at the end of the report. 12 This recommendation can be considered as beyond the mandate of the baseline report, but these foundation steps needs to be addressed in order for the MGD programme to start on a strong basis. 13 Projet d'Appui à l'Amélioration du Système Éducatif Baseline report – Second draft – July 2018 x | Page on the improved use of health and dietary practices that will be used in schools, health centres and agricultural extension should be harmonised among all implementing partners. ES.19. Recommendation 5: WFP’s M&E system needs to increase its capacity through the recruitment of staff and more detailed tracking of activities in order to be able to monitor MGD programme implementation performance. According to the baseline findings, monitoring the MGD programme should include:  close and detailed cooperation between all involved partners,14 including a coordinated and aligned M&E system;  an individual follow-up of each MGD-assisted school which will allow reporting on specific needs rather than relying on broad indicator values as collected by the INS survey. ES.20.Recommendation 6: In order to meet Gender Policy commitments, the WFP CO should develop specific indicators that monitor the extent to which the programme promotes participation of women in School Feeding Committees (SFCs) in decision￾making positions and others to prevent unforeseen protection issues. Attention should be given on how MGD activities can promote women’s empowerment, and how to promote gender equality in schools. 14 Not only sub-recipients, but also implementing partners such as the World Bank and the ASPC. Baseline report – Second draft – July 2018 1 | Page 1. Introduction 1. This Baseline Report (BR) is for the evaluation of the World Food Programme’s (WFP) McGovern-Dole International Food for Education and Child Nutrition Programme in the Republic of Congo (RoC) 2017-2022. The McGovern-Dole (MGD) is funded by the United States Department of Agriculture’s (USDA) Foreign Assistance Service (FAS). The MGD’s project number is FFE-679-2017/020-00 and it is run under WFP’s Country Programme (CP) 200648. This BR is commissioned by the WFP Congo Country Office (CO). 2. The BR will help establish the baseline values of the programme indicators as outlined in WFP’s Performance Monitoring Plan (PMP). The BR will enable programme performance measurement at the mid-term review (2020) and final evaluation (2022). The BR will inform the update of the Performance Monitoring Plan (PMP) and allow WFP’s CO and the Regional Bureau of Johannesburg (RBJ) as well as its partners to incorporate modifications to the programme as considered appropriate before the start of the implementation. 3. Based on the terms of reference (see Annex 1 ), the main objectives of the BR are:  Accountability for actions and results: The objective of the baseline is to assess the situation before the programme takes place as outlined in WFP’s Performance Monitoring Plan (PMP).  Learning and adjusting based on lessons: The baseline will provide evidence on whether the targets set in the PMP are realistic, and hence to decide whether they need adjusting. The ET will make recommendations on the most efficient approach to monitor the programme based on PMP indicators. 4. The objective of the baseline is not to redefine indicators or to assess the relevance of the activities planned to achieve the overall objectives of the programme. These will come under review during the mid-term review and final evaluation. 5. This BR was carried out by a team of five independent consultants of the KonTerra Group. The survey was done in collaboration with the National Institute of Statistics, and the Congolese PASEC15 group. The BR was conducted in May/June 2018 and covered all seven departments16 that will benefit from the MGD programme. The purpose of this BR is to establish a set of benchmarks that will be used to track the effects over the life span of the programme, and particularly during the mid-term and final evaluations. They will also provide valuable information on key success factors for efficient and effective monitoring and evaluation (M&E). 6. The expected users of this BR are WFP (CO, RBJ, headquarters, Office of Evaluation), the Ministries associated with the MGD programme, sub-recipient partners, implementing partners and the FAS. The BR will allow all users to properly understand where the programme begins (baseline figures), where it is going (annual objectives), and how progress will be explicitly measured (indicators). Furthermore, the BR is the opportunity for WFP and its implementing partners to identify gaps revealed by the ET and adopt preventive and corrective measures. 15 Programme for the Analysis of Education Systems (Programme d’Analyse des Systèmes Educatifs de la CONFEMEN - PASEC) 16 Bouenza, Lekoumou, Pool, Plateaux, Cuvette, Likouala and Sangha. Baseline report – Second draft – July 2018 2 | Page 2. Country and Development Context 7. The Republic of Congo (Congo), also known as Congo-Brazzaville, has almost four million inhabitants17 and life expectancy is 62.5 years.18 The Human Development Report 201619 classifies the Republic of Congo as a medium human development country with a human development index of 0.592 (above the average of sub Saharan countries of 0.523). The Subjective Well-Being (SWB) of the population in 2015 determines that 82.6 percent of the population are generally satisfied with their lives, 84 percent are generally happy, and 45.75 percent think that their lives have improved during the last year and that it will continue so over the next year.20 Since 2006, Congo is a Lower Middle-Income Country (LMIC) with a gross domestic product (GDP) that dropped from US$14,000 million in 2014 to US$7,800 million in 2016.21 The Congolese economy is suffering from consequences of the oil price shock since 2014. GDP growth continues to decline for the fourth consecutive year, contracting at 2.8 percent in 2016, and at least 1.1 percent in 2017.22 8. Poverty and food security: Roughly 48 percent of Congolese live on less than US$1.25 per day while up to 77 percent live below US$1.90 per day.23 A Gini index of 47.3224 indicates strong inequalities. From 2005 until 2011, poverty rates went from 65.2 percent to 74.8 percent.25 The ET has not found available department-level data of poverty and food security. The native population (Pygmies) living mostly in the Likouala and Sangha departments, represent 8 percent of the population26 and are the poorest and most vulnerable. According to the Food and Agriculture Organisation (FAO), around 1.3 million people were undernourished between 2014-17.27 Congo has 10 million hectares of arable land of which only 3 to 4 percent are exploited.28 The country imports 70 percent of its food which exposes the population to a quasi-total food dependence vis-à-vis food imports. 9. Education: Though Congo achieved gender parity in primary education, it did not meet the Millennium Development Goals (MDGs) related to universal primary completion.29 The net rate of primary school enrolment is slightly higher in urban areas than in rural areas (91% against 87%) and is practically the same for girls and boys. Twenty-four percent of pupils do not finish primary school, mostly due to the high cost of school attendance.30 Only 60 percent of children enrolled in primary education reach secondary school.31 Native children are the most disadvantaged in terms of access to education and health.32 Most pupils leaving primary school do not have sufficient foundational literacy and numeracy skills.33 In an effort to address these gaps, the 17 National Food Security and Nutrition Policy 2017 18 According to the United Nation Department of Economic and Social Affairs; https://esa.un.org/unpd/wpp/Download/Standard/Interpolated; visited on the 15th of August 2017. 19 United Nation Development Programme (UNDP); Human Development Report 2016- Human Development for Everyone; 2016 20 Ministry of Planning and Integration & UNICEF; Enquête par grappes à indicateurs multiples MICS Congo 2014-2015; November 2015. 21 World Bank, http://donnees.banquemondiale.org/pays/congo-republique-du, visited on the15th of August 2017 22 World Bank, http://www.worldbank.org/en/country/congo/overview, visited on the 28th of February 2018 23 According to the ToR. 24 According to figures from 2005 http://www.nationmaster.com/country-info/stats/Economy/Inequality/GINI-index visited on the 20th of August 2017. 25 FMI, République du Congo, Questions générales, juillet 2015. 26 Official figures estimate the Pygmie population at less than 2% of the entire population. However, many sources believe that the figure is much higher (up to 10%). 27 FOA ; http://www.fao.org/faostat/en/#country/46; visited on the 28th of March 2018. 28 InfoCongoPresse, http://www.infocongopresse.com/secteuragriculture.html, visited on the 15th of August 2017 29 Multi-indicator Cluster Survey 5 (MICS 5) 2014-2015. 30 Enquête congolaise auprès des ménages, 2012. 31 Partenariat mondial pour l’éducation, http://www.globalpartnership.org/fr/country/republic-congo, visited on the 15th of August 2017. 32 Libre Afrique, http://www.libreafrique.org/Kodia_pygmees_160112, visited on the 16th of August 2017. 33 See the World Bank’s Education Sector Support Project (Projet d'Appui et d'Amélioration du Système Educatif – PRAASED Baseline report – Second draft – July 2018 3 | Page Government conducted a substantial revision of its Education Sector Strategy (2015- 2025).34 10. School Feeding / Social Protection: In 2016, Congo adopted a National School Feeding Policy (NSFP - Politique Nationale d'Alimentation scolaire - PNAS). This policy includes the establishment of the School Feeding Directorate that will oversee school feeding activities around the country. Though its director was appointed in May 2018, the operationalisation of the Directorate remains ineffective to date.35 Operating in school feeding since 2001, the two major actors in the country, WFP and the Institute for Philanthropy and Humanitarian Development (IPDH), covered 41.8 percent of primary school needs during the 2014-2015 school year.36 The government wishes to extend school feeding to the whole country by 2025. In 2012, the government of Congo developed its national policy on social action37 that has the ambition to extend social protection to social categories excluded from the traditional system38 which covers less than 20 percent of the population. As stipulated in National School Feeding Policy (NSFP), school feeding meals are regarded as transfers of the cost of meals to the families, contributing indirectly to household revenue. In that sense, school feeding fits into the government’s broader social safety net intentions. 11. Health and nutrition: According to data from UNICEF (2014/2015),39 stunting affects 21.2 percent of children under five. It must be noted that stunted children will never reach full physical and cognitive potential, which will be an explanatory factor for some of the performance results of pupils in school. Wasting (or acute malnutrition) is estimated at 8.2 percent for children under five including 2.6 percent under a severe form. According to the strategic framework against malnutrition in Congo,40 micronutrient deficiency is still a real challenge in the country. The Congolese continue facing many challenges regarding health. In 2012, around 62 percent of deaths were due to communicable diseases, maternal or prenatal, as well as by nutrition conditions.41 Water, sanitation and hygiene are key areas to control the outbreak of some communicable diseases. To fight against nutritional deficiencies, the Ministry of Industrial Development and promotion of the Private Sector,42 created in May 2012 the National Commission for the Fortification of Food.43 In 2013, Congo committed to the Scaling up Nutrition (SUN) movement. In 2015, the government put in place the Strategic Framework Against Malnutrition in Congo – Horizon 2025.44 Its objective is to reduce the prevalence of all forms of malnutrition regarding vulnerable populations by at least 50 percent by 2025. 12. Gender: The Gender Inequality Index is 0.592 in 2015, which ranks Congo at 141 out of the 188 countries evaluated.45 The Republic of the Congo is currently not among the 34 Strategy themes include (a) providing quality education for all, with a focus on a basic education cycle of 10 years and on moving toward universal access to pre-schooling, especially in rural areas; enhancing access to informal education and literacy services for youth and adults, including second chance education; (b) responding to the need for quality human resources in an emerging economy, with a focus on creating quality schools, establishing a newly reformed technical and vocational education and training (TVET) system, and improving the relevance of higher education; and (c) creating a well-performing education system, with a focus on improved institutional management, enhanced human resource management, and better management of financial resources and statistics, planning and strategy, and crises and emergencies. 35 This is one of the recommendations made in the mid-term evaluation of WFP’s country programme 201486. 36 According to the NSFP, page 19. WFP was covering 15.3% whilst IPHD was covering 26.5% of needs. IPHD is no longer implementing school feeding due to lack of funding by USDA. IPHD had been implementing MGD programmes in Congo since 2002. 37 Politique nationale d’action sociale. 38 Law n° 31-2011 of the 15th of July 2011. 39 UNICEF, https://www.unicef.org/congo/situation_10654.html, visited on the 15th of August 2017. 40 Republic of Congo, Cadre stratégique de lutte contre la malnutrition au Congo – Horizon 20125, Avril 2015. 41 SABER rapport pays 2015, République du Congo – Alimentation scolaire, p. 2 42 Ministère du Développement industriel et de la Promotion du secteur privé 43 Commission nationale pour la fortification alimentaire (CNFA) 44 Cadre stratégique de lutte contre la malnutrition au Congo - Horizon 2025 45 UNDP; Human Development Report, 2016; http://hdr.undp.org/en/composite/GII; visited on the 28th of March 2018 Baseline report – Second draft – July 2018 4 | Page 144 countries ranked in the Global Gender Gap Index (GGI) in 2017.46 Even if gender equality is part of the National Constitution, legal and societal discrimination towards women persists. Traditionally, women perform most domestic work and are involved in agricultural activities as well. However, according to the latest DHS (2011/12), 84.9 percent do not own land and have little power in decision making. 13. Congo’s 2008 National Policy on Gender stipulates guiding principles of gender equality in all sectors of development, and respect for the commitments taken in front of the African and international community on the importance attached to the promotion of gender equality and the reduction of gender inequality.47 Five critical issues are specifically mentioned: (1)improvement of women’s productivity and hence their income; (2) improved access to services supporting productivity; (3) improved access to social services; (4) promotion of equal participation, rights; and finally (5) the reduction of gender based violence. As to education, the prevalence of women with secondary or higher education is of 70.4 percent in 2011/12 against 81.4 percent for men. With respect to a school feeding programme it is as well important to see, how many children work up to 28 hours per week at home. In the group of children aged 5 to 11 years the rate is 43.2 percent for boys and 65.0 percent for girls and the rates for children aged 12 to 14 years rises to 68.1 percent for girls and 82.6 percent for boys. This compromises school attendance rates. 14. There is no official statistic on violence against women, but the Demographic and Health Survey (DHS - 2011/12; chapter 16) highlights important aspects of women’s status in the society. 73.1 percent of women believe that men have the right to beat them for reasons such as going out of the house without telling the husband, denial of sex, neglecting child care or spending money without notification. On the other side, 59.8 percent of men believe that they have the right to beat their wives. It is disturbing that this percentage rises with younger men (50.3% for men aged 45 to 49; 75.8 % for boys aged 15 to 19). 15. The humanitarian/development situation and WFP: WFP has been present in Congo since 1968 providing development and humanitarian support. Currently, WFP implements two programmes in Congo: the Country Programme (CP) 200648 (2017- 2018) and the Emergency Operation (EMOP) 201066 (2017-2018). See section 3 for further details on WFP’s activities. 16. National policies and strategic frameworks: The National Development Plan (NDP 2012 - 2016)48 lays out the president’s vision and gives a strategic and multisectoral pacification49 framework. Its main objective is to “accelerate the modernisation of the society and the industrialisation of Congo in the perspective of generating increased and shared prosperity in order to prepare for the emergence of Congo into the worldwide economy”. The 2018-2022 NDP is still under validation process. The Strategic Document for Growth, Employment and Poverty Reduction (2012- 2016)50 is the main document of the NDP. This document is “the framework for political and governmental action […] regarding cooperation with technical and financial partners”. 17. WFP started its first school feeding programmes in Congo in 2001. Since then, policies and frameworks have developed. In 2012 the inter-ministerial “Congolese Feed Congolese”51 initiative was put in place by WFP and the Centre of Excellence against 46 Gender Gap Index ranks countries according to calculated gender gap between women and men in four key areas: health, education, economy and politics to gauge the state of gender equality in a country. http://www.peacewomen.org/profile/country-region-profile￾congo-brazzaville; visited June 13th, 2018 47 Ministry of the Promotion of Women and her Integration into the Development Process / Ministère de la Promotion de la femme et de l’intégration de la femme au développement, National Gender Policy, April 2008 (page 22). 48 Plan national de dévelopement (PND-2012). 49 Transalated from the French: “cadre de pacification stratégique et multisectoriel”. 50 Document de stratégie pour la croissance, l’emploi et la réduction de la pauvreté (DSCERP 2012-2016). 51 Le congolais nourrit le congolais. Baseline report – Second draft – July 2018 5 | Page Hunger in Brazil. Its objective is to implement programmes which use food produced locally to promote the local economy. In 2014, a national assessment and planning workshop on school feeding allowed for a diagnosis of national capacity in school nutrition. In 2015, Congo conducted a Systems Approach for Better Education Results (SABER) which was developed with the World Bank and WFP’s contributions (Figure 1). 18. WFP’s School Feeding Policy was revised in 201352 and the nutrition policy in 2017.53 According to WFP,54 school feeding contributes directly to the Sustainable Development Goal (SDG) 2: zero hunger, SDG 4: Quality Education and SDG 5: Gender Equality. It indirectly contributes to SDG 1: No Poverty, SDG 8: Decent Work and Economic Growth, and SDG 10: Reduce Inequalities. 19. WFP’s Gender Policy (2015-2020) considers school feeding as an opportunity to promote gender equality in the access to education on one side and on the functioning of school feeding management on the other. WFP’s Guidance for the Gender Marker suggests that all WFP programmes should collect data in a disaggregated way in order to get a proper understanding of gender equality regarding access to primary education, nutrition, etc. Furthermore, programmes on school feeding should discuss the domestic roles of women, men, girls and boys, and the implications of these chores, such as time burden, the risks of violence and dropping out from school. More broadly, the programme should also discuss the risks of women, men, boys and girls to violence in the national context as well as the contexts of assistance. Those pre-programme discussions / needs assessments can help orient a more gender transformative approach of activities and programme. 20. Activities of the three sub-recipient partners: ACTED works in Congo since 1997 with an initial focus on reconstruction projects focused on access to heath, hygiene and sanitation. Since the end of the 1990s, ACTED has refocused its activities on a multisectoral approach to fight poverty and inequalities, including local authorities and communities. ACTED supports farmers to grow, stock, transport and sell their production through training, support of farmer associations, microfinance, etc.55 21. UNICEF has been in Congo since the 1960s. As all UN agencies, UNICEF’s engagement in the country follows the United Nations Development Assistance Framework (UNDAF) which is written together with the host government.56 In parallel to the UNDAF, UNICEF has put in place a Cooperation Plan UNICEF – Congo (2014-2018) that focuses on children’s rights (child development, education, protection, discrimination, and collection of factual data to use for advocacy). 22. Congo was one of eight countries which benefited from the first phase of UNESCO￾China Funds-in-Trust (CFIT) project "Harnessing Technology for Quality Teacher Training in Africa" that was launched in 2012 with the support of the Chinese government who donated a budget of US$8 million for a duration of four years (2013-2016). Phase II of CFIT project (2017-2018) aimed at providing quality teacher education and training, to contribute to Sustainable Development Goals (SDGs 4 and 9). Under CFIT Phase II, in July 2017, ten educational tutors and thirty distance learning room managers were trained in two sessions in Information and Communication Technologies (ICT) and ICT in Education. After the training, they were able to take charge of operating the interactive platform and had gained skills in ICT tutoring techniques and management of distance 52 WFP; Reveised School Feeding Policy, Promoting innovation to achieve national ownership; 2013 53 WFP; Nutrition Policy ; 2017 54 WFP; How School Meals Contribute to the Sustainable Development Goals – A Collection of Evidence; 2017 55 ACTED; http://www.acted.org/fr/acted-congo-f-te-ses-20-ans; visited on the 24th of May 2018. 56 The latest UNDAF goes from 2014 to 2018. Baseline report – Second draft – July 2018 6 | Page learning rooms. In August and September 2017, 249 teachers were trained in ICT and ICT for education in two stages. 3. Subject of the Baseline Report, Theory of Change and baseline questions 3.1. Subject of the Baseline Report, Theory of Change, Activities and indicators 23. The MGD programme is a five-year programme with two Strategic Objectives (SO): “Improved Literacy of School-Age Children” (SO1) and “Increased Use of Improved Health, Nutrition and Dietary Practices” (SO2).57 The project agreement between the FAS and WFP was signed on 29 September 2017. The official start of the MGD programme took place in Brazzaville on the 1st of March 2018 with a signed agreement between the Minister of Primary, Secondary Education and Literacy, the ambassador of the United States of America and WFP’s country director. At the time of writing this BR, it is expected that provision of school meals will start in September 2018 and should end in June 2022. However, due to the late start58, it can be expected that the programme extends until June 2023. 24. According to the agreed list of schools integrating the MGD programme signed by the Ministry of Education on the 10th of April 2018, the programme will reach 54,000 primary school children (25,856 girls and 28,144 boys) in 318 schools situated in the 7 departments of Bouenza, Lekoumou, Pool, Plateaux, Cuvette, Likouala and Sangha (see map in Annex 2).59 However, according to the agreement signed between FAS and WFP, the total number of school aged students receiving daily school meals should be 53,750 whereas he number of school is not specified. The ET has used the latter number of beneficiaries in the report in order to go by the agreement between the donor and WFP. Those schools/beneficiaries have been selected from the list of schools/beneficiaries already assisted through the CP 200648 in close collaboration with the MoE on the basis of easiest accessibility and greater vulnerability. MGD’s funding allows a large number of these former schools to continue being assisted, otherwise the Country Office’s budget constraints would have curtailed future school feeding at these sites. The programme will distribute approximately 9,950 metric tonnes of school meals such as vegetable oil, split yellow peas and fortified rice. The meals will be supplemented by iodised salt provided by the government of Congo and canned fish provided by Japan60. Government staff will be trained and supported, beneficiaries will be sensitised in health and dietary practices, school feeding committees (SFC) will receive regular training, latrines and water sources will be restored and/or built, etc.61 25. The FAS has allocated up to $30,022,053, to be used over the duration of the programme for commodities and funding of ocean transportation.  $5,620,250 for commodity cost (food provided by USDA)  $4,776,000 for international freight cost  $19,625,803 for project operations (see Annex 3 for further details on budget) 57 Fiscal Year (FY) 2017 Food Assistance Proposal Guidance and Notice of Funding Opportunity, page 4. 58 According to WFP staff in the CO, the late start of the programme was due to the late arrival of the food to be distributed in school. During the field mission (5th to the 19th of May), some food was still in customs in Pointe Noire. 59 Pool, Bouenza, Cuvette, Plateaux, Lekoumou, Likouala and Sangha. 60 Nothing has been agreed for the moment with those partners. 61 Precise numbers of government staff trained, SFC assisted, latrines built, etc. can be found in section 4 under each indicator. However, the ET does not present them here as they are numerous and will need to go through major updates and changes before the programme implementation begins. Baseline report – Second draft – July 2018 7 | Page 26. The ongoing CP 200648 focuses on school feeding, social safety nets, nutrition and capacity building. Due to the economic crises and lack of funding, only components of school feeding and to a certain extent of social safety nets have been implemented until 2017. As per WFP’s Integrated Road Map (IRM) strategy 2017-202162, the MGD programme will integrate the current CP. Regarding school feeding, the mid-term evaluation of the CP carried out in 2017, mainly recommended (1) continuing advocacy towards government to prioritise school feeding programmes and enhance sustainability of WFP’s school feeding programmes, (2) enhance synergies between school feeding and local food production to promote the local economy, (3) strengthen aspects of the programme linked to gender equality and empowerment of women (GEEW) by better training national staff and integrating national counterparts such as the Ministry of Women, and (4) reinforcing collaboration with other partners for increased impact of operations in communities. Regarding local food production, WFP, through its CP, is supporting 200 small farmers with the production of beans that are intended to be supplied to the school canteens (performance remains to be measured). 27. In the implementation of the MGD programme, WFP CO works in close collaboration with the government. The four government partners are the Ministry of Primary, Secondary Education and Literacy (MoE), the Ministry of Agriculture, Livestock and Fisheries (MoA), the Ministry of Health and Population (MoH), and the Ministry of Social Affairs, Humanitarian Action and Solidarity (MoSA). WFP also works with other United Nations (UN) agencies. The implementation of the activities will be done in partnership with key sub-recipients of the MGD funds: the United Nations Children’s Fund (UNICEF), the United Nations Educational, Scientific and Cultural Organisation (UNESCO) and the Agency for Technical Cooperation and Development ACTED. Other national and international NGOs are also direct partners of the MGD programme. These include: l’Association des pères spiritains du Congo (ASPC), RENAPAC63, Pioneer Hospital, Partnership for Child Development (PCD), Initiative Development and the World Bank. Private sector entities such as NG Enterprise and Minoco are also implementing partners. 28. Gender dimension: During the data collection process, the evaluation team put special emphasis on gathering data on women’s and girls’ roles and responsibilities, opportunities and obstacles concerning education. The ET provided an adapted environment to promote free discussion, considering local practices and cultural habits. It ensured that confidentiality was well understood by all interviewees and that gender sensitive issues were addressed during these discussions. During baseline, the intention of the ET was to identify eventual gaps in the PMP and provide guidance accordingly. 29. Theory of Change: To reach the programme objectives, a total of eight groups of activities have been planned64 by WFP and its sub-recipient implementing partners: 1) Build capacity; 2) Improve pupil enrolment; 3) Establish/Strengthen local agriculture and school communities to support graduation; 4) Distribute food; 5) Promote improved health; 6) Support improved literacy; 7) Promote improved nutrition; 8) Support improved safe food preparation and storage (see Annex 4 for details about each activity). Each activity is reflected in the two results frameworks (RFs) developed by the FAS against which each SO can be implemented. Those RFs depict a Theory of Change for how the SOs can be achieved through a set of activities (see Annex 5 & Annex 6). To increase the 62 WFP’s future work in each country will be conceived under a unique Country Strategic Plan allowing for further internal synergies and coherence of approaches 63 RENAPAC is a platform of NGOs and associations working on the promotion and protection of the rights of indigenous peoples in Congo. 64 From the ToR, pages 5 and 6 and Project Agreement Between FAS and WFP, pages 21-24 Baseline report – Second draft – July 2018 8 | Page likelihood of achieving the SOs and sustainability of results once FAS assistance ends, a set of standardised Foundational Results common to both RFs have been identified by USDA (see Annex 7). 30. WFP’s PMP presents a total of 33 indicators that support M&E activities. Twenty￾five of the performance indicators are standard MGD indicators, while eight have been added by the CO to better fit the programme’s context. Some indicators are linked to the planned activities (output indicators) and others to the expected results (outcome indicators). WFP has summarised all indicators in the PMP. For the reader’s ease, the PMP has been simplified and annexed to the BR (see Annex 8 for programme’s outputs and Annex 9 for programme’s outcomes). 3.2. Baseline Evaluation Questions and Evaluation Criteria 31. The baseline was designed to address the following questions which still remain valid:  Q1 - What are the baseline values for each indicator in the Performance Monitoring Plan (PMP)?  Q2 - Given these baseline values, the objectives of the MGD programme, and within the context of Congo, are the targets set for each indicator realistic? Are any of them too low or too high?  Q3 - Given the objectives and activities of MGD and the context of Congo, what are the key success factors for efficient and effective programme M&E? How feasible is the evaluation design and evaluation questions? 32. The overall evaluation of the MGD programme design (Baseline Report, Mid-term Review and Final Evaluation) as envisioned as part of the inception phase is still appropriate. However, as ET will only be able to collect data on a representative sample of the schools, WFP and its partner should conduct specific M&E (including baseline) for each single school. This will enable accurate decision-making based on specific assessment instead of basing decisions on general findings. 4. Baseline Report approach and methodology for baseline data collection 4.1. Baseline report and methodology 33. This multi-year evaluation has been conceived to systematically establish benchmarks against which to measure progress and long-term effects of the programme from its start in 2018 through to the end of the programme in 2022. This will be done through three inter-connected exercises that will use the MGD results frameworks and performance indicators as the basic pillars for performance measurement. The three exercises are the following:  Conduct a baseline in 2018 (current exercise): to establish benchmarks in relation with each one of the performance indicators. Based on these, the team will critically review the programme targets and the existing Performance Monitoring Plan (PMP) and determine whether programme targets are appropriate and realistic. If the indicators presented in the PMP are considered insufficient to assess the programme’s contribution to these results, then other indictors will be proposed and presented in the baseline report for consideration by WFP and USDA. Baseline report – Second draft – July 2018 9 | Page  Conduct a mid- term evaluation in 2020-21: to take stock of the programme’s progress and make recommendations to improve programme implementation and introduce corrective measures if necessary.  Conduct a final evaluation in 2022-23: to assess the programme’s results and make recommendations aimed at future WFP interventions. 34. The analytical framework of the overall evaluation (see in Annex 10) depicts the logic of analysis throughout the evaluation plan to ultimately measure, analyse and understand the performance of the MGD programme. 35. The evaluation and also the baseline applies a mixed-method approach, using three main collection methods: (1) secondary data review, collection of (2) qualitative and (3) quantitative data to derive evaluation findings. The quantitative component of the evaluation applies a quasi-experimental prospective approach which includes a case and a control group of schools so that at endline, the impact of the intervention can be measured using difference in differences tests. For further details see section below. 36. The population of interestfor the evaluation is the (slightly less than65) 54,00066 children attending primary rural schools where the MGD programme is being implemented as of 2018. These rural schools have all received WFP’s support (or discontinuing support according to budget availability) over the last ten years and are located in seven different departments, namely Pool, Plateaux, Cuvette, Bouenza, Lekoumou, Likouala and Sangha. In the latter two, schools are adapted to the local cultural background of the Pygmy population. In those departments, the schools targeted by MGD are known as ORA schools.67 Those schools give the opportunity to Pygmy children to follow a three-year school programme that eventually enables them to integrate into the public system at the beginning of the third year of primary school. The evaluation considered children attending MGD schools in the former five departments as the main case population group of the evaluation (CaseG1, 50,209 children) to be compared to the comparison group. Children attending MGD ORA schools have been studied as a second case population group (CaseG2, 3,791 children) and analysed separately. This allows the isolation of MGD programme effects, preventing them from being confounded with important and unique cultural factors in the Likouala and Sangha departments. 37. Thus, the evaluation design has considered the following case and comparison groups, their size is specified in the table that follows:  CaseG1. Main case or treatment group in the quasi-experimental design. Children attending MGD rural primary schools located in the departments of Pool, Plateaux, Cuvette, Bouenza and Lekoumou.  CaseG2. Secondary case group. Children attending MGD rural primary schools located in the departments of Likouala and Sangha. These schools follow a different curriculum, adapted to cultural characteristics of the population in these two departments. Their uniqueness makes them not appropriate to be included in the main case group. Their differences with the latter, and the differences of MGD programme design and performance in both settings will be analysed qualitatively.  CompG1. Main comparison or control group in the quasi-experimental design. Children attending public rural primary schools that have never received any WFP 65 As children in Prescolaire only schools are not included 66 Originally, the ET considered the total amount of pupils being assisted to be 54,000 according to the figures given by the MoE. After discussion with the CO during draft revision, the ET revised this number in order to reflect the agreement signed between the FAS and WFP. However, the figure used for the survey is 54,000 pupils. 67 ORA (Observer, Réfléchir, Agir). English: Observe, Think, Act. Baseline report – Second draft – July 2018 10 | Page support or any other school feeding programme from another NGO/government/UN agency. They are located in the departments of interest (Pool, Plateaux, Cuvette, Bouenza and Lekoumou). Within these departments, they are in districts where MGD schools are found. None of these schools are implementing the MGD programme as of 2018 and will not be implementing it in the years that follow.  No comparison group can be attributed to the ORA schools (CaseG2) as all the ORA schools have been integrated into the MGD programme. Table 1: Case and comparison groups considered in the evaluation design 1 2 3 4 5 6 MGD coverage Number of depts. Total number of public rural primary schools68 Total number of MGD supported primary schools69 Total number of public rural primary schools not receiving any support 70 Total number of children enrolled in rural public primary schools (estimated) Total number of children in MGD schools71 CaseG1 5 854 236 0 < 180,000 approx. < 52,107 CompG1 5 0 494 (142 in MGD target districts) 0 CaseG2 2 60 ORA schools (123 non-ORA) 65 0 11,707 < 4,893 38. To guide the evaluation team during this evaluation, three inter-connected evaluation matrices have been produced (one evaluation matrix per exercise), including the questions that have been answered, how the judgement was formed, expected information sources, opportunities for triangulation, and the data collection methods. The evaluation matrix for the baseline assessment is currently the most detailed and addresses the three baseline questions as per the ToR. The evaluation matrices for the mid-term and final evaluations are currently broad but will be revised during the inception phases of those evaluations. 39. The evaluation design seeks a balanced participation of males and females as much as this is possible in all groups of stakeholders. It puts special emphasis on gathering data on women’s and girls’ roles and responsibilities, opportunities and obstacles concerning education. During the data collection process, the ET and the INS provided an adapted environment to promote free discussion, considering local practices and cultural habits. They ensured that gender sensitive issues were addressed during these discussions. All data collection tools were put together with a gender sensitive approach. As much as possible, girls have been interviewed by female enumerators, and boys by male enumerators. 40. The ET carried out a gender review during the field mission. To do so, the ET expanded consultation of secondary sources and conduct KII with key stakeholders such as the Ministry of Gender, teachers, NGO staff encountered and UN staff. This enabled the establishment of relevant gender indicators that should, in the opinion of the ET, be 68 Source: Excel file: “Liste des écoles primaires prescolaires ORA du Congo VF”, prescolaire and ORA schools not included in primary schools. 69 Source: PDF file: “Ecoles MGD- 2017-2018 Version Finale” 70 Source: Excel file: “Liste des écoles primaires prescolaires ORA du Congo VF”, prescolaire and ORA schools not included in primary schools. 71 Numbers are smaller than presented, because this number includes “prescolaire” schools and they are out of the population of interest Baseline report – Second draft – July 2018 11 | Page tracked across time to capture contributions of the MGD programme to GEEW objectives, including protection and mitigation/reduction of SGBV in the school environment. 4.2. Baseline Data Collection Methods and Tools 41. To address the baseline questions, and triangulate findings, the following evaluation tools have been used: a general desk review (see Bibliography / Desk Review page 100), collection of quantitative data through a baseline survey using five different questionnaires and qualitative data collection through semi-structured interviews (Annex 11). All methods are directly linked to the evaluation matrix (see Annex 12). Using different methods and tools to collect data on the same questions from multiple sources will allow triangulation of information so that conclusions do not reflect the view of a single informant or group of informants. Annex 13 shows the schedule of the ET during the field mission and recapitulates the key informants encountered. 42. Desk Review: Through the review of past activities, national policies and other documentation related to school feeding, the ET got a global understanding of the context of school feeding in Congo. It allowed validation of certain PMP indicators against existing data and allowed to identify gaps. It also helped to design the scope of the survey and identify the comparison group. 43. Qualitative tools: The qualitative assessment examined the coherence of all programme activities implemented by WFP Congo and its partners under the MGD agreement. It elicited stakeholder perceptions with a special focus on relevance. During the field phase, qualitative data was obtained through key informant interviews and desk review as per table below. Table 2: Methods used for qualitative data collection Method Elements Purpose Key Informant Interviews 1. Briefings with relevant WFP staff both in the Brazzaville office and the Nkayi sub-office (programme officers, field staff, M&E officer, VAM officer, Gender officer, nutrition officer). 2. Interviews with all sub-recipient partner. 3. Interviews with implementing partners: ASPC and World Bank. 4. Interviews with MoE and Ministry of Women. Informal discussion with Ministry of Planning and Ministry of Agriculture. 5. Interviews with for Departmental Directors of Primary and Secondary Education (DDEPSA) 6. Interview with two school inspectors. 7. Interview with 4 school directors Key informant interviews have been used to determine the following: Assess relevance of the MGD programme approach. Determine the coherence of the programme to government, WFP and other development partner’s strategies and direction. Identify key success factors for efficient M&E (Q3). Identify any potential risks to the programme’s success and identify any potential changes for more effective programme implementation Document Review 1. WFP project proposals and annual reports (SPRs); 2. Previous evaluation reports; 3. M&E department reports (WFP and partners); Determine the general context of the evaluation (school feeding, MGD, RF, etc.) Validates PMP against Baseline report – Second draft – July 2018 12 | Page 4. Organisational policies on school feeding and gender; 5. Country Strategic Plan; and 6. Other literature related to the assessment including gender related documentation. existing data Design baseline survey and scope Identify case and control groups 44. Quantitative tools: The tools for collecting the quantitative data consisted of four surveys (teachers, administrative personnel, school feeding committees and pupils – (see Annex 14, Annex 15, Annex 16 & Annex 17) and one literacy test for grade 2 pupils. The four surveys have been sub-contracted to the National Institute of Statistics (INS) (see profile of INS in Annex 18). The literacy test was conducted by the PASEC team under the leadership of the Director of the DEP within MEPSA. 45. Quantitative survey sampling approach: The baseline report, and subsequently the mid-term review and endline evaluation, utilises a quasi-experimental case-control methodology to enable determination of impact and attribution at endline. 46. A total of 1,590 children were sampled from 100 schools in five departments to inform this design.72 This satisfies the targets set during the inception phase (1,600 children, leading to a below 5% sampling error and 8% effect size). Additionally, 219 children from 14 schools of Likouala and Sangha departments were surveyed using mostly the same tools73 (see Annex 19 for further details on the quantitative survey strategy and approach). Qualitative data was collected in all departments except Likouala and Pool.74 The table below shows the number of schools and children sampled. Table 3: Quantitative survey sample MGD Supported schools (cases) Non-supported schools (comparison) Total #Schools #Students #Schools #Students #Schools #Students Quasi￾experimental design sample in 5 depts. 50 791 50 799 100 1590 Additional sample in 2 depts. 14 219 14 219 TOTAL 64 1010 50 779 114 1809 47. The sampling strategy adopted is suitable for quantifying the impact of the MGD programme on enrolled children in terms of changes in their literacy level and improvement of their use of new health and dietary practices. 72 The quasi-experimental design for impact quantification is applied in the departments of Bouenza, Cuvette, Lekoumou, Plateaux, and Pool. 73 Schools in these two departments operate in a very different cultural context using a different curriculum. They are therefore studied separately and don’t inform the impact assessment 74 The ET, in discussion with the CO, did not feel it was absolutely necessary to visit all 7 departments in order to get the qualitative information needed to conduct the baseline survey. Baseline report – Second draft – July 2018 13 | Page 48. An assessment of the MGD programme’s effect in terms of literacy (through the literacy test) has guided the choices in sample sizes of case and comparison groups (CaseG1 and CompG1 respectively). The sampling strategy that has been adopted is a two-level cluster random sample, where schools are the primary sampling units (PSUs) and children are selected randomly among those in 2nd grade.75 Cluster sampling allows for better efficiency in geographically spread survey contexts. 49. Schools (PSUs) were proportionately stratified by department. Departments present different socioeconomic contexts to one another, and this influences literacy level of children (see data extracted from the PASEC2014 report in Annex 19). Therefore, stratification by department reduces the variance of estimations of pupils’ literacy levels (i.e. reduces sampling error). PSUs were sampled through Probability Proportionate to Size sampling, according more weight to departments where MGD programme is more present. 50. The accepted error presented in the table below is calculated for a sample size per school of 16 children, 8 girls and 8 boys being interviewed/assessed in each school76. Table 4: Planned sample sizes, error and effect size estimations Populatio n groups Numbe r of depts. Number of schools (PSUs) Number of sampled children Number of sampled children by sex (gender parity) Sampling error (+/- ) Samplin g error by sex group (+/-) Effect size capture d Effect size captured by sex group CaseG1 5 50 800 400 4.6% 5.6% 8.0% 9.9% CompG1 5 50 800 400 4.6% 5.6% CaseG2 2 14 224 112 8.6% 10.6% TOTAL 114 1,824 912 51. Estimations above are based on the following assumptions:  Estimation of sampling error is based on the estimation of the variance of the estimate of the proportion of the population that verifies a particular characteristic. This is done assuming a 95 percent confidence interval, and a probability of 50 percent, which is the most conservative value for the true proportion of the population. We could have used a probability of 38 percent, as that is the average percentage of children passing the literacy test according to data from PASEC2014. However, this percentage varies significantly from one region to another (see Table 24 below), so the most conservative option is preferred.  Sampling error and effect size estimations take cluster design effect into account. Within cluster correlation coefficient is assumed to be 0.05. The formula used to estimate design effect is the standard 𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷 = 1 + 𝜌𝜌(𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 − 1). Cluster size is 16 across groups, and 8 by sex group.  Effect size is calculated for each proposed sampling strategy option as P1-P0 assuming the following: P0=0.38 (as per PASEC2014, excluding Pointe Noire 75 Second grade being the age at which they take their literacy test. Children among those in 4th grade are selected randomly for the assessment of their knowledge of health and dietary practices. 76 This was respected as much as field work realities allowed Baseline report – Second draft – July 2018 14 | Page region), P1>P0, power is 80 percent and confidence interval is 90 percent. P1 is the proportion of children passing the literacy test in the case group, while P0 is that of the comparison group. Table 5: Stratification of samples by department (sample size of 50 schools in each main group) CaseG1 universe: MGD supported rural primary schools CaseG2 universe: ORA MGD schools TOTAL Department Pool Plateaux Cuvette Bouenza Lekoumou Subtotal Likouala Sangha Subtotal Schools 63 43 20 61 46 233 48 17 65 298 Pupils (approx.) <18,879 <7,909 2,167 16,414 6,738 <52,107 3,071 1,822 4,893 <54,00077 CaseG1 sample CaseG2 sample Number of sample schools 14 9 5 13 9 50 12 4 16 66 Number of sample children 224 144 64 208 160 800 192 64 256 CompG1 universe: public rural primary non-supported schools by any school feeding programme Schools in the department 150 113 112 110 9 494 Schools in target districts within each department 81 23 5 24 9 142 CompG1 sample Number of sample schools 14 9 5 13 9 50 Number of sample children 224 144 64 208 160 800 52. Sampling frames used were official and exhaustive, provided by WFP. For MGD supported schools, an exhaustive list of targeted schools was used. For non-supported MGD schools, a full dataset of public non-supported schools was utilised. 53. Descriptive analysis of baseline data for the pupils’ survey and the comparison of means by population group (taking into account the presence or absence of intersections of their confidence intervals) show that CaseG1 and CompG1 are very similar. Children in both groups have similar food consumption patterns, while the only significant difference among them seems to be the percentage of children reporting absences within the last month, higher among children not attending MGD schools (46.3% against 40.1%). This is not the case for CaseG2, as children from Likouala and Sangha attending ORA schools 77 See footnote 66. Baseline report – Second draft – July 2018 15 | Page show significantly different response patterns across the entire survey. Within CaseG2, differences are also significant by sex. 54. Data collection took place during the last two weeks of the school year. Deviations from the initial randomly selected sample took place at two stages:  An accessibility assessment was conducted in collaboration with the Country Office before the survey launch. A reduced number of sampled schools (9 out of 114) had to be replaced by other randomly selected ones in the departments of Bouenza (1) and Pool (8), due to access issues caused by flood or conflict.  Further adjustments had to be done during survey implementation. Again, 9 schools were replaced for feasibility reasons. Replacements were randomly selected in Bouenza (3) and Pool (6). 55. Internal reliability of the pupils’ survey (informing impact indicators) has been tested and found acceptable.78 Validity was preserved by conducting the pre-test in a different school outside the sample (not to influence or pre-condition actual survey responses), by randomly sampling children within each school and, as much as logistics allowed for it, by following the initial plan of randomly sampled schools (in this sense, the departments of Bouenza and Pool were the only ones affected by logistic limitations, as noted above). An important aspect influencing validity of some survey questions is the fact that pupils do know that handwashing is a positive healthy practice and therefore tend to respond that they always do it before or after meals and after using the toilet. This needs to be considered when analysing responses to handwashing related questions.79 4.3. Limitations and risks 56. Due to budget constraints80, the PASEC literacy test was conducted in 80 schools to 800 pupils instead of 100 schools to 1,600 pupils as planned in the Inception Report. This will lead to an effect size captured of 11 percent. In other words, differences in improvements of literacy level between children in CaseG1 and CompG1 will have to exceed 11 percent to be captured by the quantitative component of this evaluation design during mid-term review and final evaluation. The ET is aware of how high this is as a target and will seek complementary evidence of results and factors that may contribute to literacy improvements so that potentially smaller positive effects of the programme are captured (analysing differences by department, assessing differences in attentiveness and decreases in absence rates). 57. Similarly, differences in improvements in reported hygiene practices will have to exceed 8 percent to be captured81. The evaluation will detect improvements in health practices as an effect of the MGD program only if a very high level of adherence to these practices (well above 90%) is reached in case schools (or if children in control group abandon such practices). 58. To overcome this limitation, the qualitative component of the research will explore how the MGD intervention addresses and promotes hygiene practices more in-depth, to 78 The Cronbach Alpha coefficient was used on theoretically correlated question and found to be above 0.7 (for example question “Do you usually eat something before coming to school?” and dietary diversity score before school calculated from question “What did you eat this morning before school?” present a standard Cronbach Alpha coefficient of 0.78) 79 There is an important contrast, for instance, in the high percentage of children reporting that they wash their hands (above 85%) and the low availability of handwashing stations in schools (only 16% of MGD supported schools equipped with them). 80 KonTerra proposed different approaches implying different cost. WFP selected the option that better fitted with their available budget. 81 Note that the high percentage of children reporting adherence to hygiene practices (above 85%) leads to the re-estimation of size effect to 6 percent. However, the more conservative estimation of 8 percent elaborated in the inception phase is kept as a reference. Baseline report – Second draft – July 2018 16 | Page find whether it has led to a qualitative improvement in how children adhere to hygiene practices, beyond the already high quantitative reported adherence to these practices. 59. As anticipated in the Inception Report, due to the absence of available data on school figures (absence of education statistical yearbook), certain indicators rely only on the availability and accuracy of data in schools collected by the INS. During the mid-term evaluation, the education statistical yearbook 2017 should be available. This will allow the ET to triangulate certain findings regarding some of the baseline indicators. 4.4. Quality assurance of the baseline data collection 60. Evaluation standards have been measured against the OEV’s Decentralised Evaluation Quality Assurance System (DEQAS), which defines the internal quality standards expected of the evaluation and the processes for accomplishing quality assurance. These steps are based on the UNEG norms and standards and good practice of the international evaluation community (OECD-DAC and ALNAP). KonTerra’s Quality Advisor conducted a quality control of the deliverables prior to submission to WFP. In addition, a post-hoc quality assessment of the draft inception and evaluation reports was conducted. 61. Quantitative data collected through different surveys has been subject to different analyses allowing for data cleaning: such as analysis of inconsistencies among specific questions, analysis of missing values and double entry cleaning. 4.5. Ethics 62. Ethical protocols were implemented during data collection and enumerators received training ahead of the survey (see Annex 20 for training outline). Standards ensured that participation was voluntary and informed all participants of the implications of participating in the research. Analysis of responses was treated anonymously so that findings could not be attributed to specific respondents. The enumerators and ET obtained consent from all respondents. As specified above (in paragraph 39), the overall approach ensured that methods and tools were GEEW-responsive. 5. Baseline Findings and Discussion 63. The baseline findings and the evidence to substantiate them are presented below. The 25 standard indicators and eight custom indicators in the PMP are presented by strategic objective (SO1 and SO2) and are reviewed individually. In addition, the ET presents several indicators for WFP’s consideration to improve programme monitoring and during mid-term review and final evaluation82. Quantitative data drawn from the INS surveys can be found in detail in the survey data analysis report in Annex 21. 64. For better clarity and ease of reading, evaluation questions 1 and 2 have been treated at the same time in the section below. The report goes through each standard and custom indicator as per the WFP PMP which is the document that summarizes all activities and objectives for the MGD programme over the five next years. Standard indicators 10, 13, 14 have been removed by WFP as they are not relevant to the programme.83 82 To measure the potential impacts of the MGD programme, the ET has collected eight additional indicators through the INS survey and proposes that these are incorporated into the PMP. Data has not been explicitly analysed for the baseline report. This will be done during the mid-term review and final evaluation. The data from which baselines are issued have extracted from Annex 21 and represented under each indicator. 83 Indicator 10 is the number of public-private partnerships formed as a result of USDA assistance. It is similar to indicator 11. Indicators 13 and 14 are linked to take-home-ration distributed at school which is not relevant for the MGD programme in Congo. Baseline report – Second draft – July 2018 17 | Page 65. Each indicator has been extracted from the PMP and individually analysed. As in the example below, a second line of values represents the ET’s findings from the field (line with a background colour). Standard ind. # x Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project xx,xxx xx,xxx xx,xxx xx,xxx xx,xxx xx,xxx xxx,xxx Value from PMP xx,xxx xx,xxx xx,xxx xx,xxx xx,xxx xx,xxx xxx,xxx Value from field findings Baseline for ORA MGD schools X Baseline for non-MGD￾supported schools x Baseline for ORA schools and comparison group when relevant 66. As further developed throughout the analysis of the indicators, conclusions and recommendations sections, this baseline report will advise the review of some yearly targets currently set in the PMP. In these cases, a rationale is provided for the relevant indicator, based on findings and the acronym TBR will appear in the “field-findings” cells meaning “to be recalculated/redefined”. 67. A colour code has been put in place in order to visualise the extent to which, according to the ET, the indicator requires modifications. Indicator may need minor changes but is considered appropriate. Indicator definitely needs to be changed, but not major changes. Indicator needs major change. Indicator not linked to any MGD activity. 68. All the following indicators (standard and custom) have been compiled in Annex 22 so reference to them can be made more easily. 5.1. Baseline Values and validation of programme yearly targets (Q. 1 & 2) 5.1.1. MGD SO1: Improved Literacy of School-Age Children 5.1.1.1. Standard indicators a) Number of students regularly (80%) attending USDA supported classrooms/schools Standard ind. # 1 Number of students regularly (80%) attending USDA supported classrooms/schools Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 40,000 43,000 45,150 47,408 49,778 52,267 92,92084 91.5% (see Table 6) TBR TBR TBR TBR TBR TBR Female / Male 20,000/20,000 21,500/21,500 22,757/22,757 23,704/23,704 24,889/24,889 26,133/26,133 46,460/46,460 94.1% girls / 89.1% boys TBR TBR TBR TBR TBR TBR Baseline for ORA MGD schools 98.2% of total number of ORA school pupils Baseline for non-MGD￾supported schools 91.5% of total num. of non￾supported schools pupils 69. Question 1: USDA’s indicator handbook notes that attendance should be measured over the years and should not reflect a single point in time. However, this kind of data is not available in Congo. Even if school attendance books were available in a sufficient number of schools, it was not possible for INS enumerators to extract attendance data from those. The INS has estimated attendance over the prior month by asking pupils 84 This figure considers that 20% of the beneficiaries are renewed every year (year 1 + (0.2 x year 2) + (0.2 x year 3), etc.). Baseline report – Second draft – July 2018 18 | Page how often they missed school over the last month. During mid-term review, the ET will also triangulate data with the education statistical yearbook that should be in place according to the INS and the MoE and provide more information on school attendance. According to ET findings, gender balance is not 50-50 as stated in the PMP. Table 6: Children who report missing less than 20% of school days in prior month85 n TOTAL BOYS GIRLS Non-ORA schools 1590 91.5% 90.9% 92.1% MGD supported schools 791 91.5% 89.1% 94.1% Non-supported schools 799 91.5% 92.7% 90.2% ORA schools 219 98.2% 97.2% 99.1% 70. Question 2: The figures provided for the yearly targets by the PMP start with 80% of 53,750 pupils for year 1 and then increase by 5% per year. Considering the baseline figure given by the ET, yearly target should be revised. Even if the FAS considers that regular attendance is reached when pupils attend school at least 80% of the time, WFP should still fix yearly objectives considering the actual baseline figure and aim for at least maintaining it or increasing it a little. In fixing yearly targets, gender balance should take into account the figures given by the MoE86. b) Number of textbooks and other teaching and learning materials provided as a result of USDA assistance Standard ind. # 2 Number of textbooks and other teaching and learning materials provided as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 50,000 10,000 12,000 14,000 17,000 103,000 0 0 – The activities planned do not reflect the expect activities related to this indictor (see comments below) 71. Question 1: This is an output indicator. The baseline value is zero. 72. Question 2: When considering the precise definition of this MGD standard indicator,87 it appears that none of the four main partners (WFP, ACTED, UNESCO, UNICEF) implementing the programme is in charge of providing teaching and learning materials in such quantity. The large number of materials (50,000) targeted for year 1 does not appear in the production/distribution plans of any partner. 73. In 2018, UNICEF intends to issue 120 posters and 2,000 leaflets on topics related to hygiene rules (hand washing with soap), food and nutritional hygiene, and health. In addition, UNICEF will broadcast messages on similar topics and on the importance of education on local radio station (1 message each week, 4 broadcasts per week). The extent to which these 'products' can be considered as teaching and learning materials in support of improved literacy of school-age children is questionable. The ET suggest new custom indicator for those activities. According to field mission findings, WFP should work together with the World Bank’s PRAASED programme to support the design of new school curriculum and the production of new teaching material and books for all schools in the country. 85 i.e. who report to have assisted school at least 80% of days in the last month 86 See list of beneficiary schools signed in the 10th of April 2018 by the MoE. 87 "Food for Progress and McGovern-Dole Indicators and Definitions", United States Department of Agriculture, Foreign Agricultural Service, Food Assistance Division, Office of Capacity Building and Development, 08/16/2016. Baseline report – Second draft – July 2018 19 | Page c) Number of school administrators and officials in target schools who are trained and who demonstrate use of new techniques or tools as a result of USDA assistance Standard ind. # 3 Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 520 560 600 640 680 3,000 0 TBD – Need to consider previous annual target when defining actual yearly target (see comments below) 74. Question 1: This outcome indicator is a direct result of the following indicator. Its baseline value is zero. 75. Question 2: The number of school administrators and officials in target schools who demonstrate use of new techniques is a percentage of people trained (relating to standard indicator 4). It starts in year one with 65 percent and ends in year 5 with 85 percent. However, the ET considers that each annual objective should consider the objective of the past year(s) as in table below. Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 65% of standard ind. #3 objective 70% of standard ind. #3 objective + previous year objective 75% of standard ind. #3 objective + previous years objectives 80% of standard ind. #3 objective + previous years objectives 85% of standard ind. #3 objective + previous years objectives To be calculated 76. As the standard indicator 4 should to be revised, standard indicator 3 should be revised accordingly. Standard ind. # 4 Number of school administrators and officials trained or certified as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 800 800 800 800 800 4,000 0 960 800 800 800 800 4160 Indicators needs to b disaggregated according to the topic of training 77. Question 1: This indicator is an output indicator. The baseline value is zero. 78. Question 2: School administrators and officials in all schools benefiting from MGD assistance will be trained by WFP in school feeding management and monitoring each year. Based on previous experience, WFP estimates that due to both the need for refresher sessions and the high turnover of school directors and officials, this training will have to be replicated each year. According to the MoU with WFP, UNICEF will train 160 additional ORA school administrators “in the management of ORA school in order to attract teachers who can successfully prepare pupils to integrate the public-school system after ORA3”. The ET recalls that all training can be only claimed as such if it is at least two working days (16 hours) in duration.88 It will be important to disaggregate this indicator according to the topic of training. d) Number of teachers/educators/teaching assistants in target schools who are trained and who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Standard ind. # 5 Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 88 According to the FAS handbook on MGD indicators and definition. Baseline report – Second draft – July 2018 20 | Page 0 120 0 0 0 0 120 TBR TBR according to standard ind. # 6 79. Question 1: N/A, see indicator #6. 80. Question 2: N/A, see indicator #6. Standard ind. # 6 Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 160 0 0 0 0 160 0 0 - The activities planned do not reflect the expect activities related to this indictor (see comments below) 81. Question 1: This is an output indicator. The baseline value is zero. 82. Question 2: The figure indicated in the PMP corresponds to 160 teachers who will benefit from a Training of Trainers (ToT) provided at national level on nutrition education under the umbrella of UNESCO. According to the MoU between WFP and UNESCO,89 UNESCO’s contribution to the MGD programme will be limited to one year. The ET noted that this activity has not been specifically designed for the MGD programme but was part of an already existing UNESCO three-component ToT program covering a range of topics including an unfunded nutrition component initially planned for 500 teachers.90 Out of the actual 163 teachers that will benefit from the ToT, 103 will be trainers from primary schools, whilst 34 will be from pre-schools and 26 from adult literacy courses. As this ToT will be targeted to all departments throughout the country, only 52 teachers out of the 103 primary schools' teachers trained will be linked to MGD-supported departments. While the rationale of this standard MGD indicator is “training teachers to effectively teach literacy to children of different skill levels […] to improving the overall quality of instruction”91 this ToT does not fit the exact indicator definition. As nutrition education cannot be equated with French didactics the ET maintains that the programme does not address this indicator. e) Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance Standard ind. # 7 Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 18 18 7 7 7 57 0 11 5 0 0 0 16 83. Question 1: This is an output indicator. The baseline value is zero. 84. Question 2: According to project documents and interviews conducted during the field mission, UNICEF and ACTED are responsible for these objectives. ACTED will build ten latrines and rehabilitate one in the Bouenza department during the first two years, whilst UNICEF will construct five in the first year in the Likouala department. It appears that the MoUs between WFP and each of those partners are based not on the whole life cycle of the project. Hence, no objectives have been set for years 3, 4 and 5. 89 WFP and UNESCO contribution agreement, 13 December 2017. 90 WFP, Republic of Congo, FY2017 McGovern-Dole Proposal, 2017; page 20 91 See Food for Progress and McGovern-Dole Indicators and Definitions page 58. Baseline report – Second draft – July 2018 21 | Page 85. No school buildings/classrooms have been taken into consideration by the programme, but only latrine construction/rehabilitation. The ET points out that it has not seen a needs assessment to substantiate the prioritisation of latrine construction/rehabilitation rather than school buildings or classrooms. f) Number of students enrolled in school receiving USDA assistance Standard ind. # 8 Number of students enrolled in school receiving USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 51,190 53,750 56,438 59,259 62,222 65,333 102,651 See comment below 53,750 56,438 59,259 62,222 65,333 102,651 Female / Male 25,595/25,595 26,875/26,875 28,219/28,219 29,630/29,630 31,111/31,111 32,667/32,667 51,325/51,325 See comment below TBR TBR TBR TBR TBR TBR 86. Question 1: The ET did not verify the baseline value of this indicator since it would have involved visiting each of the 318 schools covered by MGD assistance. However according to the list of schools given by the MoE on the 10th of April 2018, girls represent 47.8 percent of pupils. This same trend can be found in a quite consistent manner across the surveys where girls represent 45 percent of children in non-ORA schools and 47-48 percent of children in ORA schools. This does not reflect the baseline figures but no conclusion can be taken about this at baseline. 87. Question 2: WFP estimates a 5 percent increase per year. The yearly targets for disaggregation by sex needs to be revised by the CO. g) Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance Standard ind. # 9 Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 470 470 470 470 470 470 0 318 318 318 318 318 318 Need to disaggregate by type of support 88. Question 1: This is an output indicator. The baseline value is zero. However, the ET notes that WFP has been operating in those schools at least over the last five years92, and therefore PTA’s have already been trained. 89. Question 2: Under this indicator, WFP is planning to assist all School Feeding Committees (SFC) in schools benefiting from the MGD programme. The target objective needs to be changed from 470 to 318 (according to list of schools readjusted on the 10th of April 2018) to match with the readjusted objectives of the programme. The ET notes that WFP has not provided a detailed record of the type of support it will provide to SFCs (e.g. training on cooking, on storage, on nutrition, on dietary practices, etc.). h) Value of new public and private sector investments leveraged as a result of USDA assistance Standard ind. # 11 Value of new public and private sector investments leveraged as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 92 Confirmed with WFP staff Baseline report – Second draft – July 2018 22 | Page 0 0 0 100,000 0 0 100,000 TBR TBR – Needs to be based on anticipated agreements or negotiations 90. Question 1: This is an output indicator. The baseline value is zero. 91. Question 2: The ET could not find a rationale for setting this indicator’s value. However, it seems to the ET that US$100,000 is a relatively small amount of new expected investment considering the overall cost of the programme. US$100,000 represents 0.33 percent of the overall cost. The value of this indicator should be reconsidered based on negotiations and anticipated agreements with potential partners for the next five years. i) Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance Standard ind. # 12 Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance: Stage 5: Passed for which implementation has begun Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 1 0 0 0 0 1 0 TBR – Needs to be more specific/disaggregated (see comments below) 92. Question 1: This is an output indicator. The baseline value is zero. 93. Question 2: This indicator refers to the National School Feeding Policy (NSFP) adopted in 2016. As the NSFP is based on six specific objectives93 and as four different institutional bodies94 are in charge of the coordination of school feeding policy, WFP should be more specific on the indicator related to what regulation and/or administrative procedure it intends to assist. 94. According to the information gathered during the field mission, the administrative procedure/regulation that the programme will support is the implementation of the School Feeding Directorate (SFD). The SFD is the entity tasked with ensuring the coordination of the school feeding programme in the medium and long run; it makes sense for the Directorate to be operational as soon as possible. Hence, the decentralisation of the Directorate as planed in the NSFP (page 37) could be an explicit target of this indicator. The assistance in the implementation of other administrative procedures or regulations can also be added and specifically addressed. 95. It is important that this indicator focuses on a measurable and achievable target; the MGD programme cannot claim to assist the NSFP’s overall implementation. For example, the MGD programme is not implementing the specific objective linked to local production even if WFP has other programmes linked to local production in the Bouenza department and have undertaken a study on manioc flour fortification in Congo in 2016.95 To an extent, the MGD programme could be viewed as contradicting the NSFP which states that specific norms and standards need to be defined for diverse food consumption considering departmental specific production and consumption patterns in order to promote behavioural change towards local food consumption in school meals (NSFP, page 93 Education / Health, nutrition and sanitation / Social development / Local production / Trade and industry / Poverty reduction (NSFP – page 10). 94 Ministry of basic education / the Multisectorial National Council for School Feeding / the Inter-Ministry Technical Comity / the School Feeding Directorate (NSFP – pages 34-35) 95 A Fortified Rapid Assessment Tool (FRAT) on manioc consumption was undertaken in 2016 in collaboration with the Ministry of Industrial Development, the Ministry of health, the National Committee for Fortification, UNICEF, FAO, FIDA and WFP. Due to a lack of funding, the implementation of the activities did not take place. Baseline report – Second draft – July 2018 23 | Page 20). In this sense, the rice-import aspect of the MGD programme should be used as a transition tool whilst local food production and purchase is put in place. j) Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance Standard ind. # 15 Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 9,675,000 9,675,000 9,675,000 9,675,000 9,675,000 48,375,000 0 9,675,000 9,675,000 9,675,000 9,675,000 9,675,000 48,375,000 96. Question 1: This is an output indicator. The baseline value is zero. However, the ET notes that WFP has been operating in those schools at least over the last five years, and therefore daily school meals have already been distributed in those school conditioned to the available resources. 97. Question 2: The yearly targets for the school meals are the number of pupils multiplied by 180 days, resulting in the number of school days in one year. k) Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance Standard ind. # 16 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 53,750 53,750 53,750 53,750 53,750 96,750 0 53,750 TBR TBR TBR TBR TBR Female / Male 0/0 26,875/26,875 26,875/26,875 26,875/26,875 26,875/26,875 26,875/26,875 48,375/48,375 New / Continuing 0 53,750/0 10,750/56,000 10,750/66,750 10,750/77,500 11,200/88,250 97,200/99,450 98. Question 1: This is an output indicator. The baseline value is zero. However, the ET notes that WFP has been operating in those schools at least over the last five years. 99. Question 2: Standard ind. # 8 anticipates addition pupils enrolled in the school assisted over the years. However, the programme has not provided additional meal for those new pupils. According to the list of school produced by the MoE on the 10th of April 2018, girls represent 47.8 percent of the pupils and boys 52.2 percent. Yearly targets would need to be revised taking into consideration those comments. l) Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance Standard ind. # 17 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 53,750 53,750 53,750 53,750 53,750 96,750 0 53,750 53,750 53,750 53,750 53,750 96,750 Female / Male 0/0 26,875/26,8 75 26,875/26,87 5 26,875/26,87 5 26,875/26,87 5 26,875/26,87 5 48,375/48,375 New / Continuing 0 53,750/0 10,750/56,00 0 10,750/66,75 0 10,750/77,50 0 11,200/88,25 0 97,200/99,450 Disaggregated figures need to be adapted. ET suggests removing 100. According to the USDA indicator handbook, three kinds of activities can provide the foundation for a “productive safety net” program. These are: Baseline report – Second draft – July 2018 24 | Page  Activities which strengthen community assets (e.g. public works);  Activities which strengthen human assets (e.g. school feeding, maternal and child health visits such as prenatal and well-baby visits); and/or  Activities which strengthen household assets (e.g. take-home rations). 101. Due to the nature of the programme (i.e. not strengthening community assets other than in schools, nor implementing take-home rations), the ET understands that this standard indicator is exactly the same as the previous indicator. Hence, the ET suggests removing it as it does not bring any new information. m)Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text Standard ind. # 26 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 43% 50% 60% 70% 80% 80% 80% 9.9% TBR – Yearly targets to ambitious (see comments below) Female / Male 43%/43% 50%/50% 60%/60% 70%/70% 80%/80% 80%/80% 80%/80% 3.2% / 5.7% TBR - Yearly targets to ambitious (see comments below) Baseline for ORA MGD schools 2.8% Baseline for non-MGD￾supported schools 4.2% 102. Question 1: The figure (43%) indicated at baseline within the approved PMP is not appropriate as it refers to the result of PASEC2014 relative to late primary reading competencies (see Annex 23). In fact, 38 percent of pupils were above the "sufficient" competency threshold in French language at early primary.96 103. Through an agreement with The KonTerra Group, an evaluation team from the DEP97 within MEPSA, with expertise in PASEC98 from the CONFEMEN99, conducted a specific literacy test as a baseline. Detailed results from this exercise have been presented in a report (Annex 24). The outstanding difference between the results from the PASEC2014 and those from this 2018 literacy test stems from the difference between the objectives of both tests and their related score scales. The purpose of the PASEC was to determine the percentage of pupils whose level is above a qualified "sufficient" threshold showing that they have acquired the knowledge and competencies required to pursue their education in good conditions. The purpose of the literacy test conducted on request from the ET was to specifically inform the MGD literacy indicator with reference to the expected performances of pupils at the end of CP2, according to the primary school curriculum ("All learner readers should be able to read and understand a short, simple and familiar message"). A detailed comparison of scales and scores has been put in Annex 25. 104. An oral test of reading capacity of students attending grade 2 takes place every quarter and at the end of the school year. This individual reading test is taken in the classroom, in front of all students, where pupils have to read a text written on the blackboard. The teacher grades each student according to his/her performance. The results of schoolchildren at the final exam collected by INS on request from the ET in 47 primary 96 PASEC (2017). PASEC2014 – Performances du système éducatif congolais : Compétences et facteurs de réussite au primaire. PASEC, CONFEMEN, Dakar. 97 Département des Etudes et de la Planification / Department for Studies and Planification, 98 Programme d'Analyse des Systèmes Educatifs de la CONFEMEN / CONFEMEN Programme for the Analysis of Education Systems 99 Conférence des ministres de l'Education des Etats et gouvernements de la Francophonie / Conference of Ministers of Education of French-Speaking Countries Baseline report – Second draft – July 2018 25 | Page schools in June 2018 show that 75.80 percent of pupils having taken the exam have obtained a score above average (80.65% for boys, 70.67% for girls). Nevertheless, the weak reliability of the data collected prevents from taking these results into consideration: in 3 schools, the number of pupils scoring above average is higher than the number of pupils having taken the exam; a surprising score of one hundred percent success is reported in 6 schools. 105. Question 2: Over the 2018-2022 phase of MGD implementation, the target is to increase the percentage to 80 percent. This increase is a random estimate as it depends on many external factors out of WFP’s control: teaching quality, access to educational material, attendance rate, attentiveness of pupils, and so on. Considering the baseline defined by the PASEC test, the yearly targets do not seem to be realistic. n) Number of individuals benefiting directly from USDA-funded interventions Standard ind. # 27 Number of individuals benefiting directly from USDA-funded interventions Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 56,261 55,950 55,800 55,800 55,800 108,061 0 TBR – This “aggregate indicator” needs to be update Number of individuals benefiting directly from USDA-funded interventions (female) Female / Male 0/0 27,879/28,382 27,755/28,195 27,695/28,105 27,695/28,105 27,695/28,105 52,944/55,117 New / Continuing 0/0 56,261/0 12,950/56,261 12,800/69,061 12,800/81,861 12,800/95,111 108,061/108,361 106. Question 1: This is an output indicator. The baseline value is zero. 107. Question 2: This is an aggregate indicator that considers indicators 4, 6, 17, 18 and 20. Targets need to be updated based on revised values of those indicators as per comments above. o) Number of individuals benefiting indirectly from USDA-funded interventions Standard ind. # 28 Number of individuals benefiting indirectly from USDA-funded interventions Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 268,750 268,750 268,750 268,750 268,750 483,750 0 TBR – See comments below 108. Question 1: This is an output indicator. The baseline value is zero. 109. Question 2: WFP considers indirect beneficiaries the entire family of each pupil receiving school meals. WFP estimates the average family size at five members in Congo, hence the figures of the yearly targets are the number of school children benefiting from school meals multiplied by five. However, this calculation presumes that each family has only one child receiving a school meal, which in the opinion of the ET is not accurate. The targets should be recalculated considering the average number of primary school aged children per family100 attending school. 5.1.1.2. Custom indicators 100 ET was not able to obtain that figure. If possible and available, this figure will be given in the final version of the report. Baseline report – Second draft – July 2018 26 | Page a) Number of PTAs, communities members; farmers organization trained or sensitized about the importance of education Custom ind. # 2 Number of PTAs, communities members; farmers organization trained or sensitized about the importance of education Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 960 960 960 960 960 4,800 0 TBR – Needs disaggregating by type of training and beneficiary group 110. Question 1: This is an output indicator. The baseline value is zero. 111. Question 2: As with the indicator above, the rationale for the target is not evident. A clear description of how sensitisation activities will be conducted with community members, farmer organisations or PTAs as well as key messages would be helpful. The types of training and beneficiary groups should to be specifically outlined by WFP. b) Percent of transfers made to the school inspectors as a % of planned Custom ind. # 3101 Percent of transfers made to the school inspectors as a % of planned Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 60% 100% 100% 100% 100% 100% 0 TBR – A percentage is not specific enough for this indicator 112. Question 1: This is an output indicator. The baseline value is zero. 113. Question 2: According to the field mission, the transfers referred to in the indicator consist of giving petrol to inspectors for field travel. It’s unclear why, during the first year, only 60 percent of the transfer is targeted. The ET suggests that the absolute value (amount in USD) is included in the yearly targets instead of the percentage and that the number of beneficiaries is defined. c) Number of textbooks and other teaching and learning revised materials (based on revised curriculum) provided to teachers/schools as a result of USDA assistance Custom ind. # 4 Number of textbooks and other teaching and learning revised materials (based on revised curriculum) provided to teachers/schools as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 1,038 0 0 1,038 0 2,076 0 0 – The activities planned do not reflect the expect activities related to this indictor (see comments below) 114. Question 1: This is an output indicator. The baseline value is zero. 115. Question 2: The last revision of the national primary curriculum occurred in 2000. At the time of the baseline there was no revised curriculum in Congo and subsequently no revised textbooks or other teaching and learning materials. In line with the Education Sector Strategy 2015-2025, revision and implementation of a new curriculum is one component of the PRAASED102 supported by the World Bank that will be implemented during the 2016-2023 period. Hence, the ET does not see this indicator being feasible under the MGD programme. 101 In its PMP, WFP considered custom indicator #3 as being linked to the foundational results 1.4.4/2.7.4. However, as this activity goes to school inspectors for petrol, the ET considers that this indicator is linked to the foundational result 1.4.3 (see the graph on page 37). 102 Projet d'appui à l'amélioration du sytème éducatif. Baseline report – Second draft – July 2018 27 | Page d) Percentage of student who indicate they are attentive or very attentive during class/instruction Custom ind. # 5 Percentage of student who indicate they are attentive or very attentive during class/instruction Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project TBD 60% 65% 70% 75% 80% 80% 73.8% See comments below Female / Male TBD 60%/60% 65%/65% 70%/70% 75%/75% 80%/80% 80%/80% 77%/70.6% See comments below Baseline for ORA MGD schools 75% Baseline for non-MGD￾supported schools 72.7% Female / Male Baseline for ORA MGD schools 71.5%/78.5% Baseline for non-MGD￾supported schools 74.9%/70.5% 116. Question 1: The ET decided to collect data on the attentiveness through asking teachers rather than pupils; in the ET’s experience, it is unlikely that children will admit to a lack of concentration in class. In that sense, the ET suggest WFP to change slightly the title of the indicator. To collect attentiveness rates the ET concluded that it is more efficient to collect inattentiveness rate (see table below – for disaggregated figures, refer to 0). Hence attentiveness rate is calculated by subtracting the inattentiveness rate from 100. Table 7: Average perc. of pupils reported to be inattentive (every day or sometimes) Every day Sometimes n* Mean Std. dev103 Lower CI Upper CI Mean Std. dev Lower CI Upper CI Non-ORA schools 187 14.7% 12.0% 13.0% 16.4% 12.0% 13.0% 10.1% 13.9% MGD supported schools 97 13.9% 12.5% 11.4% 16.4% 12.3% 14.0% 9.5% 15.2% Non-supported schools 90 15.6% 11.5% 13.2% 17.9% 11.7% 11.9% 9.2% 14.1% ORA schools 23 13.7% 9.9% 9.4% 18.0% 11.3% 12.5% 5.9% 16.7% * n = number of teacher reports 117. Question 2: The question of attentiveness needs to be related to the root causes of tiredness and loss of concentration. The ET asked children, whether they ate before coming to school and whether they took any food to school. According to the ET's experience, coming to school having had a proper breakfast has more impact on attentiveness that having a proper lunch after school. This indicator can be used as a proxy for inattentiveness (see section on ET indicators). 118. The rationale behind the five percent increase in attentiveness over the years is not clear to the ET. WFP should avoid setting targets that are not rationally identifiable. As stated in the USDA indicator handbook, “the higher the increase the better” can be an acceptable outcome target in such a case. e) Number of government staff trained as a result of USDA assistance Custom ind. # 6 Number of government staff trained as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 300 300 300 300 300 300 103 Std. Dev stands for standard deviation, a measure of variation or dispersion. It quantifies how much population group members’ values differ from the mean value for the group. Lower CI and Upper CI refer to the lower and upper limits of the confidence interval for the mean. The confidence interval is the range of values within which the true mean value lies, with a specific level of certainty (set at either 90 or 95% in this study). Baseline report – Second draft – July 2018 28 | Page 0 300 300 300 300 300 300 Female / Male 0/0 120/180 120/180 120/180 120/180 120/180 120/180 0/0 Needs to be disaggregated by type of training and beneficiary group 119. Question 1: This is an output indicator. The baseline value is zero. 120. Question 2: According to the field mission, it appears that this indicator is linked to Foundational Results under capacity building. In order to properly monitor this indicator, further details are required regarding which government staff will be trained (e.g. staff of the National School Feeding Directorate, Inspectors, School administrators) and what kind of training they will receive (e.g. study visits abroad such as the Brazilian Centre of Excellence, workshops on specific matters linked to school feeding). Evidently all 300 staff members will receive training every year as the figure does not change across the life of the project. As with some indicators above, the percentage of women remains at 40 percent throughout the programme. f) Percentage of school days missed due to illness (target < 3%) Custom ind. # 7 Percentage of school days missed due to illness (target < 3%) Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project TBD104 TBD TBD TBD TBD 3% 3% 4% See comments below 3% 3% Baseline for ORA MGD schools 1% Baseline for non-MGD￾supported schools 4% 121. Question 1: As requested by the PMP, the baseline figure has been determined by the ET (see table below). Table 8: Percentage of days missed due to sickness (according to pupils) Number of students surveyed Total days in a month (20 per student) Total number of days absent reported due to sickness % days missed due to sickness Non-ORA schools 1,590 31,800 1,226 4% MGD supported schools 791 15,820 591 4% Non-supported schools 799 15,980 635 4% ORA schools 219 4,380 29 1% 122. Question 2: According to the ET yearly target do not need to be defined here. As stated by the USDA handbook on indicators: “the lower the better”. However, to meet the MGD objectives, WFP and its sub-recipient partners should aim to a final year target lower than 3 percent. 5.1.1.3. Evaluation Team indicators a) Number of day teacher have been absent over the last 4 weeks ET ind. # 1 Number of day teacher have been absent over the last 4 weeks 104 “To be Determined », as per the ToR. Baseline report – Second draft – July 2018 29 | Page Baseline for MGD schools 2.37 days Baseline for ORA MGD schools 0.4 days Baseline for non-MGD schools 1 day 123. Teacher absenteeism can be a factor of poor school attendance and could have an impact on poor instruction quality. Whether and how the MGD programme affects teacher absenteeism would be valuable information. The baseline data has been collected through the school administrators’ questionnaires and can be found in the table below: Table 9: Average perc. of days that teachers are absent (over the last month) # schools # teachers Mean Std. dev. Lower CI Upper CI Non-ORA schools 86 322 8.2% 36.4% 4.3% 12.2% MGD supported schools 41 153 11.8% 51.3% 3.6% 20.0% Non-supported schools 45 169 5.0% 11.3% 3.3% 6.8% ORA schools 10 23 0.2% 1.0% -0.2% 0.7% b) What level of education and what specific teaching diploma/training does the teacher have? ET ind. # 2 What level of education and what specific teaching diploma/training does the teacher have? Baseline for MGD schools See tables below Baseline for ORA MGD schools See tables below Baseline for non-MGD schools See tables below 124. Poor school results can also be attributed to the poor academic level and training of teachers. Educational and professional diploma at baseline is described in both tables below: Table 10: Highest educational diploma Non-ORA MGD supported Non-ORA non-supported ORA MGD supported All Males Females All Males Females All Males Females N 97 73 24 90 61 29 23 14 9 College certificate 46.4% 54.8% 20.8% 61.1% 67.2% 48.3% 65.2% 64.3% 66.7% Baccalaureate 41.2% 38.4% 50.0% 20.0% 21.3% 17.2% 26.1% 35.7% 11.1% Bachelor’s degree 1.0% 1.4% 0.0% 4.4% 4.9% 3.4% 0.0% 0.0% 0.0% Master’s degree 9.3% 5.5% 20.8% 13.3% 4.9% 31.0% 4.3% 0.0% 11.1% Ph. D 2.1% 0.0% 8.3% 1.1% 1.6% 0.0% 4.3% 0.0% 11.1% TOTAL 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Table 11: Highest professional diploma Non-ORA MGD supported Non-ORA non-supported ORA MGD supported All Males Females All Males Females All Males Females N 97 73 24 90 61 29 23 14 9 None 59.8% 71.2% 25.0% 58.9% 68.9% 37.9% 73.9% 92.9% 44.4% Baseline report – Second draft – July 2018 30 | Page CFFEN105 36.1% 24.7% 70.8% 38.9% 29.5% 58.6% 17.4% 0.0% 44.4% CAPEL 1.0% 1.4% 0.0% 1.1% 1.6% 0.0% 4.3% 7.1% 0.0% CAPCEG 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 4.3% 0.0% 11.1% CAPES 3.1% 2.7% 4.2% 1.1% 0.0% 3.4% 0.0% 0.0% 0.0% TOTAL 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 5.1.2. MGD SO2: Increased Use of Health and Dietary Practices 5.1.2.1. Standard indicators a) Number of individuals who are trained in / and who make use of child health and nutrition as a result of USDA assistance Standard ind. # 18 Number of individuals trained in child health and nutrition as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 591 440 290 290 290 1,901 0 TBR – Unclear in UNICEF’s workplan 0 0 0 0 TBR Female / Male 0/0 236/355 176/264 116/174 116/174 116/174 760/1,141 0/0 Gender disaggregation needs to be stated in MoU 125. Question 1: This is an output indicator. The baseline value is zero. 126. Question 2: The original PMP figures shown in the table above need clarification. UNICEF is the implementing partner for this indicator. In their one-year work plan UNICEF expects to train 290 people in WASH and 800 people in nutrition, covering 60 schools in the Likouala department. It is not clear if the 290 people trained in WASH are included in the 800 trained in nutrition. From a nutrition perspective, it would be preferable not to disconnect WASH and nutrition as WASH is an integral part of nutrition. According to the indicator, child health should also be part of the training, but it does not appear in those activities. 127. Whilst gender disaggregated information is shown in the table above, UNICEF is not providing these data at the planning stage. However. It must be noted that the proportion of female trainees remains at 40 percent throughout the project’s life. Whether this percentage reflects the actual percentage of female teachers or committee members needs to be verified. Standard ind. # 19 Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 384 (65%) 308 (70%) 218 (75%) 232 (80%) 247 (85%) 1,388 0 TBR according to Standard ind. #18 128. Question 1: This indicator follows directly from the previous indicator. Its baseline value is zero. 105 CFEEN (Certificat de Fin d’Études des Écoles Normales) / CAPES (Certificat d’Aptitude au Professorat de l’Enseignement du Second degré) / CAPEL (Certificat d’Aptitude au Professorat de l’Enseignement des Lycées) / CAPCEG (Certificat d’Aptitude au Professorat des Collèges d’Enseignement Général) Baseline report – Second draft – July 2018 31 | Page 129. Question 2: Yearly targets of this indicator need to be redefined once standard indicator #18 has been revised. In addition, as explained in paragraph 74, each annual objective should aggregate the targets of the past year(s). Furthermore, the ET believes, if school children are to be the final beneficiaries, the numbers should be much higher. b) Number of individuals trained in safe food preparation and storage as a result of USDA assistance Standard ind. # 20 Number of individuals trained in safe food preparation and storage as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 960 960 960 960 960 4,800 0 954 954 954 954 954 TBR Female / Male 0/0 384/576 384/576 384/576 384/576 384/576 1,920/2,880 130. Question 1: This is an output indicator. The baseline value is zero. 131. Question 2: The yearly target (number of SFC members trained per year) given by WFP’s PMP for this indicator is based on the estimated number of schools (350) originally slated for assistance.106 The value across the life of the project equals 4,800; this means that 13 to 14 individuals would be trained per school over five years. This is more than the number of people in each SFC. Hence, the ET does not understand the rationale behind this figure. In addition, it would make more sense for all SFCs to be trained from the very beginning of the project as this training will contribute directly to the proper storage and use of the food provided by WFP. Refresher training should also be considered. 132. The female/male ratio (one third female / two thirds male) does not increase over time. A more gender transformative approach with the integration of more female beneficiaries over the years could be considered. c) Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance Standard ind. # 21 Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 624 (65%) 672 (70%) 720 (75%) 768 (80%) 816 (85%) 3,600 0 TBR TBR TBR TBR TBR TBR 133. Question 1: This indicator follows directly from the previous indicator. Its baseline value is zero. 134. Question 2: Yearly targets should be cumulative according to the ET as explained in paragraph 74. The indicator will need to be recalculated once the number of trained individuals is clearly defined as per standard indicator #20. d) Number of schools using an improved water source Standard ind. # 22 Number of schools using an improved water source Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 40 74 82 86 91 91 77 (30.6%)107 77 + 0 77 + 0 77 + 0 77 + 0 77 + 0 77 MoU does not reflect the initial PMP 106 As planned in WFP’s proposal to USDA: WFP Republic of Congo – FY2017 McGovern-Dole Proposal; 2017 107 Values are absolute number estimates to fit with the requirements of the USDA indicator. Data collection provided values in percentage. Baseline report – Second draft – July 2018 32 | Page Baseline for ORA MGD schools 33 (50%) Baseline for non-MGD￾supported schools N/A (however we have an estimate of 26.5%) Year 1 33+5 135. Question 1: According to the MGD indicator handbook, this indicator measures the total number of project schools that actually have an improved water source and not only those supported by MGD.108 Hence, if the baseline figure is zero as in the WFP PMP, this means that no school uses an improved water source. However, according to the baseline survey, 30.6 percent of MGD schools benefit from an improved water source; this extrapolates to 77 of the 253 schools having an improved water source. For the ORA schools, we can estimate that 33 out of the 65 schools have an improved water source. 136. Whilst the baseline survey provides the overall statistics on water sources drawn from a representative sample, it would be good for WFP to provide a school inventory for each of the schools enrolled under MGD. It would provide WFP and its partners with a specific baseline per school and allow better follow-up over the programme’s lifespan. Table 12: Water source characteristics n Improved water sources* Unimproved water sources** Unknown Total num. of schools Total num. of schools with improved water source (estimated) Non-ORA schools 98 28.6% 71.4% 0.0% N/A N/A MGD supported schools 49 30.6% 69.4% 0.0% 253 77 Non-supported schools 49 26.5% 73.5% 0.0% N/A N/A ORA schools 12 50.0% 0.0% 50.0% 65 33 *Improved drinking water sources include piped water into dwelling or yard, public tap or standpipe, tube-well or borehole, protected dug well, protected spring and rainwater. **Unimproved water sources are bottled water, surface water, tanker-truck, cart with tank/drum, unprotected dug well, and unprotected spring. 137. Question 2: Construction/rehabilitation of water sources directly linked to MGD will be done by UNICEF in the first year of programme implementation during which five water sources will be rehabilitated or constructed in the Likouala department. This does not reflect what WFP’s PMP projected across the programme’s lifespan. In year one, the target value according to the ET is the initial 77 (non-ORA) + 33 (ORA) improved water sources plus the extra five that will be built during the first year. Activities related to water sources are undefined for years 2, 3, 4 and 5. e) Number of schools with improved sanitation facilities Standard ind. # 23 Number of schools with improved sanitation facilities Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 29 58 65 72 79 79 108 (42,9%) 108 + 6 108 + 6 108 + 6 108 + 6 108 + 6 108 + 6 MoUs do not reflect the initial PMP 108 According to the MGD handbook on indicators, improved water sources are: Piped water into dwelling, plot, or yard; Public tap/standpipe; Tube well/borehole; Protected dug well; Protected spring; and Rainwater collection Baseline report – Second draft – July 2018 33 | Page Baseline for ORA MGD schools 5 (8,3%)6 + 5 in year 1 Baseline for non-MGD￾supported schools 57,1% 138. Question 1: As per standard indicator #22, this indicator measures the total number of existing improved sanitation facilities109 in schools. Please refer to the comments made for the previous indicator. Table 13: Latrines' characteristics School with latrines School with functional latrines separated by sex Latrine conditions N Perc. Lower CI Upper CI Perc. Lower CI Upper CI Improved Un￾improved Unknown Non-ORA schools 98 68.4% 59.2% 77.6% 25.5% 16.9% 34.1% 50.0% 13.3% 5.1% MGD supported schools 49 61.2% 47.6% 74.9% 30.6% 17.7% 43.5% 42.9% 12.2% 6.1% Non￾supported schools 49 75.5% 63.5% 87.6% 20.4% 9.1% 31.7% 57.1% 14.3% 4.1% ORA schools 12 83.3% 62.2% 100.0% 33.3% 6.7% 60.0% 8.3% 50.0% 25.0% 139. Question 2: UNICEF will construct or rehabilitate five latrines in the Likouala department during the first year and ACTED will construct five and rehabilitate one. As per the comment for the indicator above, the yearly targets should reflect the total number of schools with improved sanitation facilities regardless of MGD assistance. 140. In addition, UNICEF plans to distribute 210 hand-washing kits (three per school) in two departments. This activity should go under a separate custom indicator. f) Number of students receiving deworming medication(s) Standard ind. # 24 Number of students receiving deworming medication(s) Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 53,750 53,750 53,750 53,750 53,750 96,750 0 53,750 TBR TBR TBR TBR TBR 141. Question 1: This is an output indicator. The baseline value is zero. 142. Question 2: The yearly targets should take into consideration the percentage increase in beneficiaries over the years (see standard ind. #8). g) Number of child health and nutrition policies, regulations, or administrative procedures in each of the following stages of development as a result of USDA assistance Standard ind. # 25 Number of child health and nutrition policies, regulations, or administrative procedures in each of the following stages of development as a result of USDA assistance: - Stage 3: Presented for legislation/decree 109 According to the MGD indicator handbook, and in addition to separate latrine, improved sanitary facilities are: Flush or pour/flush facilities connected to a: piped sewer system, a septic system or a pit latrine; Pit latrines with slab; Composting toilets; and ventilated improved pit latrines. Baseline report – Second draft – July 2018 34 | Page - Stage 4: Passed/Approved Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 0 0 0 0 1 1 0 Contribution of MGD programme to the NSFP development is not planned according to ET findings 143. Question 1: This is an output indicator. The baseline value is zero. 144. Question 2: According to the findings gathered during the field mission, this indicator refers to the National Food Security and Nutrition Policy that has been drafted by the WHO and UNICEF as a result of the second International Conference on Nutrition (ICN2) and financed by FAO. The validation process is already under way. The ET thinks this indicator is not relevant and should be removed since findings from data collection show that contribution of MGD programme to the NSFP development is not planned. 5.1.2.2. Custom Indicators a) Number of PTAs, communities members; farmers organization trained or sensitized about the importance of Health and Hygiene Practices Custom ind. # 1 Number of PTAs, communities members; farmers organization trained or sensitized about the importance of Health and Hygiene Practices Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 960 960 960 960 960 4,800 0 Knowledge, Attitudes and Practices (KAP) under way. Indicator will need to be disaggregated by type of training and beneficiary group 145. Question 1: This is an output indicator. The baseline value is zero. 146. Question 2: Discussions with implementing partner (ACTED) did not clarify the rationale for the setting of the 960-yearly target. However, a KAP (Knowledge, Attitudes and Practices) survey will be undertaken by ACTED in order to evaluate the needs on health and hygiene practices sensitisation. This KAP survey will inform the PMP and yearly targets according to the needs evaluated. For the moment, it stays unclear if the sensitisation will be done to community members, farmer organisations or PTAs. Accordingly, this indicator needs to be disaggregated. b) Number of fuel-efficient stoves provided and rehabilitated Custom ind. # 8 Number of fuel-efficient stoves provided and rehabilitated Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Life of project 0 60 0 0 60 0 120 0 60 0 0 60 0 120 Rational on timing and school selection not clear (see comments below) 147. Question 1: This is an output indicator. The baseline value is zero. 148. Question 2: This indicator reflects an activity which will cover a gap that has been observed by the ET in the field.110. The ET found no assessment or analysis to support these targets. Nor could it determine how many should be rehabilitated and how many should be provided. Likewise, it found no explanation for targeting only 120 schools out of 318, or why some schools benefit the first year and others only the fourth year. 110 Gap also noted in the mid-term evaluation of the CP20648 in 2017. Baseline report – Second draft – July 2018 35 | Page 5.1.2.3. Evaluation team suggested indicators a) Percentage of children always/sometimes/rarely/never having had breakfast before coming to school ET ind. # 3 Percentage of children always/sometimes/rarely/never having had breakfast before coming to school Baseline for MGD schools See tables below Baseline for ORA MGD schools See tables below Baseline for non-MGD schools See tables below 149. School feeding could have the unintended effect of parents not feeding their children in the morning, knowing they will receive food at school. Taking a small meal before going to school could directly affect pupils’ attentiveness. Therefore, this indicator can be used as a proxy for inattentiveness as school meals are provided only at lunch, which does not address concentration and attentiveness during the morning hours. Baseline figures can be found in the table below: Table 14: Food before school. Do you usually eat before school? ORA Schools Non-ORA schools All students Boys Girls All students Boys Girls MGD supported schools Non￾supported schools N 219 109 110 1590 816 774 791 799 Everyday 51.1% 57.8% 44.5% 29.4% 29.4% 29.5% 29.7% 29.2% Sometimes 23.3% 19.3% 27.3% 15.5% 17.2% 13.8% 14.8% 16.3% Rarely 8.7% 0.9% 16.4% 5.0% 5.4% 4.5% 5.7% 4.3% Never 16.9% 22.0% 11.8% 50.1% 48.0% 52.2% 49.8% 50.3% 150. Even if the ET considers breakfast as an important factor to attentiveness111, other factors influencing attentiveness can be considered such as the number of pupils per classroom, the temperature in the classroom, how well pupils sleep during the night, family problems, etc. On the other hand, the ET points out that a meal given after school has only a limited effect on school attentiveness (in the schools assisted by MGD, pupils go to school in the morning only). b) Dietary diversity of school age children ET ind. # 4 Dietary diversity of school age children Baseline for MGD schools See tables below Baseline for ORA MGD schools See tables below Baseline for non-MGD schools See tables below 151. The purpose of this indicator is to assess the change in dietary practices in MGD￾assisted schools compared to non-MGD assisted schools. It is envisaged that dietary diversity will improve in MGD-assisted schools, due to nutrition education (training of teachers, nutrition module in new curriculum). Dietary information has been collected through the pupil questionnaire asking each child what food they eat during the day and 111 The study “Breakfast Eating Habit and its Influence on Attention-concentration, Immediate Memory and School Achievement”, 2008 also suggests strong correlation between breakfast and attention/concentration at school : “Regular habit of eating breakfast as opposed to irregular consumption or skipping breakfast altogether had beneficial influence on attention-concentration, memory and school achievement”. Baseline report – Second draft – July 2018 36 | Page classifying this food consumption using the 7 suggested food groups (see Annex 26 for more information on food groups). Results can be found in table 16 of the Annex 21. Table 15: Overall dietary diversity*, 0-7 scale (7 food groups) Population group n Mean Std. Dev Lower CI (90%) Upper CI (90%) Non-ORA schools (5 depts, G1) 1590 2,94 0,91 2,74 2,81 MGD-supported schools 791 2,78 0,86 2,73 2,83 Non-supported schools 799 2,77 0,90 2,71 2,82 Boys 817 2,74 0,87 2,69 2,79 Girls 773 2,81 0,89 2,76 2,86 MGD-supported school boys 404 2,72 0,85 2,65 2,91 MGD-supported school girls 387 2,84 0,86 2,77 2,99 Non-supported school boys 413 2,76 0,88 2,69 3,08 Non-supported school girls 386 2,77 0,93 2,70 3,07 ORA schools (2 depts, G2) 219 2,81 0,92 2,71 2,92 Boys 109 2,94 0,78 2,81 3,06 Girls 110 2,69 1,03 2,53 2,85 *Overall dietary diversity does not include school meal c) Percentage of children washing hands after going to the toilet ET ind. # 5 Percentage of children washing hands after going to the toilet Baseline for MGD schools 86.3% Baseline for ORA MGD schools 72.3% Baseline for non-MGD schools 83.1% 152. The purpose of this indicator is to assess the change in hygiene practices in MGD￾assisted schools as compared to non-MGD assisted schools. Baseline data is extracted from the following table: Table 16: Hand washing after toilet ORA Schools Non-ORA schools All students Boys Girls All students Boys Girls MGD supported schools Non￾supported schools n 112 57 55 1230 630 600 609 621 Yes 72.3% 70.2% 74.5% 84.9% 84.3% 85.5% 86.7% 83.1% No 27.7% 29.8% 25.5% 15.1% 15.7% 14.5% 13.3% 16.9% d) Percentage of children washing hands before and after eating ET ind. # 6 Percentage of children washing hands before and after eating Baseline for MGD schools See tables below Baseline for ORA MGD schools See tables below Baseline for non-MGD schools See tables below Rationale: The purpose of this indicator is to assess the change in hygiene practices in MGD-assisted schools compared to non-MGD-assisted schools. Baseline report – Second draft – July 2018 37 | Page 153. As the indicator above, the purpose of this indicator is to assess the change in hygiene practices in MGD-assisted schools as compared to non-MGD assisted schools. Baseline data is extracted from the following tables: Table 17: Hand washing before eating ORA Schools Non-ORA schools All students Boys Girls All students Boys Girls MGD supported schools Non￾supported schools n 218 109 109 1537 787 750 753 784 Yes 81.7% 89.0% 74.3% 91.0% 90.7% 91.2% 92.0% 89.9% No 18.3% 11.0% 25.7% 9.0% 9.3% 8.8% 8.0% 10.1% Table 18: Hand washing after eating ORA Schools Non-ORA schools All students Boys Girls All students Boys Girls MGD supported schools Non￾supported schools n 112 57 55 1243 638 605 622 621 Yes 77.7% 78.9% 76.4% 88.2% 87.9% 88.4% 88.7% 87.6% No 22.3% 21.1% 23.6% 11.8% 12.1% 11.6% 11.3% 12.4% e) Percentage of children absent from school within the last four weeks and why ET ind. # 7 Percentage of children absent from school within the last four weeks and why Baseline for MGD schools See tables below Baseline for ORA MGD schools See tables below Baseline for non-MGD schools See tables below 154. Based on experience, the ET decided to collect the information related to the reason for absenteeism directly to pupils (through the pupil questionnaire) rather than asking teachers. It is important for the programme to understand the main reasons for absenteeism, as illness (as measured in the custom indicator # 7) might not be the main reason. In MGD-supported schools it might be that sick children still come to school to benefit from the school meal instead of staying at home and recuperating. This would directly affect pupils’ attentiveness. The comparison between MGD-assisted and non￾assisted schools on this indicator can provide useful information on programme impact. Table 19: Reasons for absence Non-ORA schools (5 depts, G1) Students Boys Girls MGD supported schools Non-supported schools n 687 348 336 317 370 Sickness 58.8% 56.0% 61.7% 60.3% 57.6% Distance 2.6% 3.2% 2.1% 3.8% 1.6% Work at home 1.3% 1.4% 1.2% 0.9% 1.6% Financial reasons 29.4% 30.5% 28.3% 27.1% 31.4% Baseline report – Second draft – July 2018 38 | Page Hunger 1.3% 2.0% 0.6% 1.6% 1.1% Family moved out 3.1% 2.9% 3.2% 3.8% 2.4% Work in the fields 0.6% 0.9% 0.3% 0.9% 0.3% No response 0.9% 1.4% 0.3% 0.6% 1.1% Other 2.0% 1.7% 2.4% 0.9% 3.0% ORA schools (2 depts, G2) Students Boys Girls n 22 15 7 Sickness 54.5% 60.0% 42.9% Distance 9.1% 6.7% 14.3% Work at home 18.2% 26.7% 0.0% Financial reasons 4.5% 6.7% 0.0% Hunger 4.5% 0.0% 14.3% Family moved out 4.5% 0.0% 14.3% Work in the fields 4.5% 0.0% 14.3% No response 0.0% 0.0% 0.0% Other 0.0% 0.0% 0.0% f) Percentage of female in school feeding committees having a lead position ET ind. # 8 Percentage of female in school feeding committees having a lead position Baseline for MGD schools See tables below Baseline for ORA MGD schools See tables below Baseline for non-MGD schools See tables below 155. WFP encourages SFCs to integrate women for them to access decision making positions. ET ind. #8 can track over time the increase of female/male ratio and also track evolution of the different positions female have within those SFCs. Table 20: FSC size and female presence Non-ORA schools n Average Std. Dev Lower CI* Upper CI # parents in FSC 42 5.8 3.1 4.8 6.8 % female parents 42 27.3% 25.0% 19.5% 35.1% ORA schools n Average Std. Dev Lower CI Upper CI # parents in FSC 13 4.1 2.4 2.6 5.5 % female parents 13 59.7% 32.1% 40.3% 79.1% * All confidence intervals (CI) in the present FSC data analysis are of level=95% Table 21: Position of female in SFCs Non-ORA schools ORA schools n % Lower CI Upper CI n % Lower CI Upper CI % female presidents 42 2.4% 0.0% 7.0% 13 30.8% 5.7% 55.9% % female vice-presidents 38 23.7% 10.2% 37.2% 11 63.6% 35.2% 92.1% % female treasurer 38 52.6% 36.8% 68.5% 9 77.8% 50.6% 100.0% % female vice-treasurer 27 22.2% 6.5% 37.9% 5 40.0% 0.0% 82.9% % female administrators 36 16.7% 4.5% 28.8% 5 60.0% 17.1% 100.0% Baseline report – Second draft – July 2018 39 | Page 156. The first two indicators will help identify potential gaps related to the literacy level of school-age children since it is widely accepted that the quality of teaching has a direct bearing on children’s learning. 157. Through the next five indicators, the evaluation team will capture how much of an impact the MGD programme has had on children’s increased use of improved health, nutrition and dietary practices looking directly at children’s habits. Food diversity is very low in Congo and is one of the reasons for limited nutrition outcomes. Therefore, the ET advises to collect data on the pupils’ dietary diversity. A review of dietary diversity in Congo (as well as between implementing partners) revealed huge differences and weak coherence. The four food groups112 which were commonly used in Congo are not really conducive to describe dietary diversity. The ET suggested to use the seven food groups,113 as described in Annex 26. It is hoped that children’s dietary diversity will improve during the project, given the fact that nutrition education will highlight the importance of dietary diversity. Food-based dietary guidelines for use in Congo are needed. 158. The last indicator will allow to see the eventual change of women’s empowerment within the SFCs. Key findings and conclusions – Question 1 • Based on evidence obtained during the data collection of the baseline, MGD programme partners do not seem to have fully integrated the PMP indicators to which they and WFP will be held accountable for programme implementation. In addition, the programme design seems to be fragmented into individual (partner) interventions; each responds to its own logic of intervention and monitoring systems, rather than to an integrated Results Framework and PMP shared by all implementing stakeholders. • New custom indicators should be added to either follow activities that are not reflected in the standard indicators (e.g. actual activity linked to Standard ind. #2) or disaggregated indicators that are not specific enough (e.g. Standard ind. #12 or Custom ind. #1). The more precise the indicator the easier the M&E. • Additional baseline indicators have been considered and collected through the INS survey to better monitor expected effects (e.g. change in health and dietary practices, change in women empowerment) or unexpected effects (e.g. decrease in teachers’ absenteeism) over the duration of the programme. In the opinion of the evaluation team, ET ind. #1 and 2 can help monitor changes that influence the programme’s performance. • Gender disaggregated indicators are operational, but the PMP could also add indicators allowing to measure women empowerment (see ET ind. #8 for example). WFP has foreseen and 50-50 gender balance regarding beneficiary pupils. This does not match the figures in the document signed by the MoE on the 10th of April 2018 listing the number of schools and pupils that will benefits from the MGD programme. 112 Group 1: Basic energy foods (source of carbohydrates); Group 2: Building foods (source of protein); Group 3: Complementary energy foods (source of lipids); and Group 4: Protecting foods (source of vitamins and minerals) 113 Group 1: Grains, roots and tubers; Group 2: Legumes and nuts; Group 3: Dairy products; Group 4: Flesh foods; Group 5: Eggs; Group 6: Vitamin-A rich fruits and vegetables; and Group 7: Other fruits and vegetables. Baseline report – Second draft – July 2018 40 | Page Key findings and conclusions – Question 2 • Changes over time of yearly targets are sometimes in contradiction between each other (e.g. the number of pupils enrolled in school increases over the years whereas the number of meals distributed does not). • Certain fundamental activities will not actually be conducted by MGD implementing partners (Standard indicators #2, #5, #6, #25 and custom indicator #4). It will therefore be impossible to attribute their outcomes “as a result of USDA assistance”. • Yearly targets need to consider the past targets achieved and should not be only focused on the activities done during the current year (see paragraph 74). Otherwise it would imply that programme benefits are only expected to last a year. • MoU between WFP and sub-recipient partners are based on one year which prevents the forecasting of targets over the life of the programme. Furthermore, this lack of visibility may lead to negative efficiency on the programme’s implementation. • Certain indicators need to be disaggregated by type of recipients (e.g. farmers, pupils, communities, teachers) or activity (e.g. petrol distribution, radio broadcast, distribution of leaflets, distribution on mathematic school books) in order to be specific and easily measurable across the life of the MGD programme. • Indicators on improved water source and improved sanitation facilities are not defined “as a result of USDA assistance” and are supposed to be collected as quantities rather than percentages. This means that it would be preferable if these indicators were done through an individual follow-up of each school at baseline, as well as during the whole programme. This has not been put in place. 158.1. Success factors for efficient and effective M&E of the MGD programme (Q. 3) 159. Key success factors for efficient and effective M&E for RF1 related indicators. The diversity of activities and partners involved in the MGD programme implementation implies the functioning of a strong M&E system which, at the time of the baseline, has not yet been established. According to the separate contribution agreements signed between WFP and key sub-recipients of MGD funds (UNICEF, UNESCO and ACTED), each partner will use its corporate M&E system to monitor specific activities implemented as part of the MGD programme. Reporting by each sub-recipient will be compounded with both monthly and quarterly narrative reports on the activities (including quantitative data), and interim and final financial statements. Each sub-recipient will prepare an annual work plan that will be submitted to WFP for approval prior to launching activities. This annual work plan will notably include an organisational chart indicating post titles and functions of staff committed to the implementation of MGD activities within the organisation. Monitoring costs represent 13 percent of the total budget of each contribution agreement signed. The strategic objective (SO1) of the RF1 is the Improved Literacy of School-Age Children. Achievement of this SO is dependent upon the achievement of three 'results streams' related to improved student attendance, improved quality of literacy instruction, and improved attentiveness. These “results streams” will be achieved through the effective implementation of complementary activities conducted by the main MGD implementing partners that will constitute an “Essential Package”. The ET was informed that WFP has recruited a senior M&E officer who will be in charge of designing and implementing an M&E system to cover the entire MGD programme. Baseline report – Second draft – July 2018 41 | Page 160. When examining the indicators from the approved PMP, it appears that several fundamental activities will not be conducted by “internal” MGD implementing partners but through complementary and/or parallel activities of “external” contributors, particularly the World Bank and the PRAASED. This is notably the case with activities related to the revision and implementation of a new national curriculum focused on literacy and numeracy, the provision of teaching and learning materials according to this new curriculum, and the training of teachers to use these new tools and methods (See Standard indicators #2, #6; Custom indicator #4). These latter activities are however essential to the achievement of MGD SO1, and their implementation by “external” contributors needs to be coordinated. Effective and efficient M&E of such indicators can only be done through tight coordination between the relevant actors avoiding overlap and duplication.114 161. Key success factors for efficient and effective M&E for RF2 related indicators. To achieve MGD SO2 (Increased Use of Health and Dietary Practices) in a measurable way, the following factors need to be considered: (1) Nutrition education is part of the interventions from UNICEF and UNESCO. Whilst UNESCO is about to employ a consultant who will be responsible for the development of key messages, UNICEF has not yet started the process but will modify the messages they have developed for infant and young child feeding. It is very important that the two organisations collaborate and coordinate. As has been shown in Annex 26, a harmonised approach on food groups and dietary diversity should be a key component of these messages. In addition, it is necessary that messages on hygiene and sanitation be included in those messages at the same time. (2) Specific target groups for the training of trainers remain unclear. Whilst UNESCO will train 160 people, out of which only 52 are teachers in the MGD primary schools,115 UNICEF will target 800 people in 2 districts in the first year. For UNESCO’s activity, it remains unclear how the ToT will ensure that a subsequent training of more people can be guaranteed as UNESCO’s support will stop after the first year. Thus, any impact on increased use of health and dietary practices will only be seen if harmonisation and coordination between the partners can be assured and if all MGD schools will profit from health and nutrition education. 162. Furthermore, it will be important that all MGD-supported schools have improved infrastructure in terms of access to safe water and availability of improved latrines by the programme’s conclusion. The already mentioned inventory of each school will provide the basis for monitoring the improvements, either due to MGD assistance or from other partners. It will also be easier to judge the relevance of interventions and the choice of targeted schools for the above-mentioned improvements. 162.1. Feasibility of the evaluation design and evaluation questions (Question 3) 163. The ET considers the evaluation design is feasible only if the WFP M&E system puts in place a systematic follow-up of each school. KonTerra’s baseline survey, mid-term review and final evaluation will monitor the programme using a sample of the targeted schools. This sample permits a general understanding of the situation but cannot replace a systematic school by school follow-up. For example, it is essential to have an initial inventory of the physical condition of buildings, latrines, water sources, etc., for each school participating in the programme. If such a baseline is not carried out, the only baseline will consist of the figures of the sample collected during the present baseline 114 This issue was raised by the World Bank during the field mission: World Bank's regulations forbid payment to an organisation for an activity that has already been requested (and payed) by another agency. 115 The remaining people are teachers in pre-schools or adult literacy classes Baseline report – Second draft – July 2018 42 | Page survey. This would not be appropriate for an efficient and effective M&E system for the McGovern-Dole programme. 164. The evaluation questions proposed in the ToR for the mid-term and final evaluations are based on the OECD-DAC evaluation criteria of relevance, effectiveness, efficiency, impact, and sustainability which are considered overall appropriate. As stated in the ToR, the evaluation questions will be further developed during the Inception Phase of both the mid-term review and end line evaluation of the MGD programme to be conducted in 2020 and 2022 respectively. Key findings and conclusions – Question 3 • Even if each sub-recipient is supposed to use its own M&E system to monitor their activities and report on a regular basis to WFP, a strong M&E system which coordinates the diversity of activities and partners involved in the MGD programme is not yet in place. A senior M&E officer will/has been116 recruited by WFP for this task. • Fundamental activities linked to educational training and elaboration of new school books based on a revised curriculum are not being conducted by “internal” MGD implementing partners. Hence coordination with external MGD actors, namely the World Bank, is essential to ensure that those activities will directly have an impact on MGD beneficiaries. • The approach on dietary practices does not seem to be coordinated between implementing partners and other actors in the country. If all implementing partners do not use/have the same approach on dietary practices (same food groups, same key messages, etc.), a common M&E system will not be easily applicable. • Efficient M&E could be better assured if annual targets are clearly defined at baseline for the duration of the programme. • The evaluation design will be more feasible if WFP’s M&E follows each targeted school in a systematic way, starting with a baseline picture for each school. 5.4. Gender equality and women empowerment throughout the programme 165. In its Gender Policy 2015-2020, WFP commits to integrate gender equality and women’s empowerment into all of its work and activities. To achieve this goal, the policy establishes four objectives: food assistance adapted to needs, equal participation, decision making by women (empowerment) and gender protection. Note that WFP does not consider gender only as the condition of being male or female. WFP’s definition of gender refers to socially constructed roles, behaviours and expectations. 166. In addition, the Gender Policy Minimum Standards for gender mainstreaming establish that outcomes and indicators of WFP’s Strategic Plan are updated to stimulate and track gender outcomes. If this is taken to the level of programming, under the assumption that tracking of programme outcomes will inform Country Strategic Plan monitoring, the ET concludes that the MGD M&E plan should include adequate gender￾related indicators that allow to report on corporate gender commitments of WFP. As it currently stands, the MGD M&E plan only contains indicators that reflect participation. The ET proposes the following suggestions to cover the gap in light of the WFP Gender Policy commitments. 116 Will be amended at final version of report Baseline report – Second draft – July 2018 43 | Page 167. Equal participation is monitored throughout the programme by disaggregating data. While ensuring that girls and boys, are equally benefiting from the programme, this will allow WFP to easily identify eventual unexpected effects related to gender disparity and act accordingly. Regarding equal participation on training, WFP’s activities are not based on equal participation (e.g. see Standard ind. #18 or 20 in which 40 percent of beneficiaries are women). The ET understands that this may reflect the local context and that WFP cannot impose equal participation during initial stages of programme implementation. However, the fact that the same participation ratio (40%-60%) remains for the duration of the programme suggests that through the programme, WFP is not investing specific efforts to promote equal gender participation by gradually increasing women’s participation in order to attain as soon as possible a 50%-50% ratio. 168. Promoting decision-making positions for women is integrated in WFP school feeding programmes by systematically sensitising SFCs to incorporate women. However, no indicator has been put in place to assess how much women are actually integrated. Furthermore, as stated in the mid-term evaluation of the ongoing Country Programme 200648, for this corporate objective to be verified, WFP should also assess and analyse the extent to which the nature of the positions assumed by women in SFC allow them to effectively influence decision-making. The fact that only 2.4 percent of women in non￾ORA-school SFCs are presidents (against 30.8% for ORA schools) shows that the main decision-making positions still seem to be held by men in non-ORA. 169. WFP has a potential to contribute to women empowerment through the work that women conduct as cooks of the assisted schools. In this sense, WFP’s Gender Policy establishes that WFP should aim at reducing women’s unpaid workloads linked to WFP food assistance.117 However, WFP Congo does not pay cash or in-kind to the female cooks/mothers that prepare meals in the assisted schools. This is a major concern that emerged during discussions with SFCs. In certain schools visited, SFCs authorise cooks to keep some of the food they prepare. The ET understands the idea that giving time for cooking school meals is modest compared to the gain of having children fed at school. However, women’s empowerment has to do with recognising the particular burdens that women face due to traditional roles and behaviours that impose constraining norms to their rights to achieve choice, power and control over their lives. In this sense, imposing as a condition of school feeding programmes that women (men are not likely to accept this position) cook for free in school canteens, works against women empowerment, since it adds a considerable amount of work to women’s daily workload with no counter benefit for women, diminishes the value of that extra effort done by women and undermines cooks’ work recognition in the school, in the community and in cooks’ households. For these reasons, WFP should reconsider the free work requested to cooks.118 170. Discussions with school children during the school visits confirmed that boys and girls reported on the violence they observe either in their own homes or in the neighbourhood. Some girls reported as well that they are being harassed by boys in class and boys confirmed that assertion. These findings correlate with the 2008 UNICEF study on violence at school in Congo that reported school violence as an important issue.119 WFP’s Gender Policy establishes that “In accordance with the “do no harm” approach, WFP programmes and interventions must not create, exacerbate or contribute to gender 117 Minimum Standard for targeted interventions letter m) of WFP’s Gender Policy establishes that “WFP uses technologies, services and infrastructure that help to reduce women’s unpaid workloads linked to WFP food assistance”. The Minimum Standards have been developed to assist country offices in closing gender gaps. 118 Other humanitarian agencies/NGOs do retribute in different ways cooks in school feeding programmes (ex: IPHD in Congo). In other countries, WFP does retribute school cooks. 119 Stratégie Sectorielle de l’Eduction 2015-2025; page 57. Baseline report – Second draft – July 2018 44 | Page inequalities or discrimination and must mitigate risk of GBV.” In addition, the policy states that “Food assistance should not harm to the safety, dignity and integrity of the women, men, girls and boys receiving it, and is provided in ways that respect their rights”. Aware that GBV violence at school and out of school already exists, in accordance with its Gender Policy, WFP’s MGD programme should prevent all forms of violence and ensure food assistance does not negatively affect child protection; indicators should be developed to track protection and violence and GBV targets. The MGD programme could benefit from existing tools for GBV prevention, such as the module on gender equality for teacher training120 developed in 2017 by UNESCO and the Congolese Government. Key findings and conclusions – Gender • MGD M&E plan expects to track equal participation-related objectives but it does not foresee indicators that relate to other WFP Gender Policy objectives which should be mainstreamed in the MGD programme, in particular in relation with empowerment, decision-making and protection. • Whenever relevant, gender disaggregated indicators report on participation. While participation in certain activities is not gender balanced at baseline due to contextual matters, yearly targets do not show that the activities are aiming towards reducing this gender gap. • WFP Gender Policy calls for the integration of women in SFCs in decision-making positions. By adding indictors that can monitor such objective (such as ET ind. #8) WFP could monitor the impact the MGD programme has on women’s empowerment. In addition, the Gender Policy calls for WFP to reduce women’s unpaid workloads linked to WFP food assistance. The ET suggests reconsidering the free contribution of cooks and the development of an indicator that tracks cook’s empowerment through MGD programme. • Violence in and out of school occurs at a relatively high rate in Congo. Considering that the Gender Policy calls for WFP’s programmes to mitigate risk of GBV, as well as to ensure food assistance does not harm the safety, dignity and integrity of recipients, WFP should add an indicator to specifically target the extent to which these objectives are met. Existing tools such as a UNESCO-Government developed module on gender is newly available. 6. Conclusions and Recommendations 171. Based on the findings presented in the previous section, an overall assessment that responds to the baseline questions is provided below. This is followed by six recommendations of how WFP and its implementing partners could act to improve the PMP and the M&E system. 6.1. Overall Assessment and Conclusions 172. The graph on the following page gives the global overview of the degree of relevance of the M&E system to monitor the MGD programme, based on the feasibility assessment of each standard and custom indicator as presented in section 2. Each indicator has been 120 Module sur l’égalité des Genre. 2017. Due to financial constraints this module has not yet been utilised. Baseline report – Second draft – July 2018 45 | Page Figure 1: Overview of the degree of relevance of the M&E system using the PMP indicators Baseline report – Second draft – July 2018 46 | Page associated to the RFs as well as to the Foundational Results to illustrate the intended dynamic of the MGD theory of change and to show how its expected results can be achieved. The graph synthesises baseline questions 2 and 3 in the sense that it shows how realistic each indicator is, considering the nature of the activities they are intended to measure, and reflects gaps that need to be covered in order to obtain efficient and effective M&E of the MGD programme. 173. To illustrate the strength of the indicator and the likelihood that it will provide the expected data to track results, the same colour coding as in the individual analysis (Chapter 2) per indicator has been used (Green: Indicator may need minor change but is considered appropriate by the ET / Orange: Indicator definitely needs to be changed, but not major changes according to ET / Red: Indicator needs major change according to the ET / Purple: Indicator not linked to any MGD activity according to the ET). It is important to understand that this graph is not tracking the likelihood of the activities to be properly implemented, but rather the likelihood that the indicator can properly monitor and evaluate the activity to which it is linked. Out of 33 indicators, eight are green, fifteen are orange, four are red and five are purple. The suggested ET indicators have not been included in the graph. 174. Some indicators/activities will actually not be implemented by the MGD programme (coloured purple in the diagram below). For example, even if WFP’s PMP shows activities and indicators linked to the Improved Quality of Literacy Instruction Result stream (see graph below), the findings of the field mission reveal that those indicators/activities do not correspond with the actual expectations of USDA regarding this Result stream. However, considering the context of Congo and the activities currently in progress by the World Bank (mainly through PRAASED), it makes absolute sense that WFP and its partners concentrate on other aspects of the MGD programme whilst they liaise strongly with the World Bank regarding the Improved Quality of Literacy Instruction Result stream which is very important for the achievement of SO1. The ET acknowledges that a partnership was planned in the agreement signed by USDA and WFP, but according to field findings, no thorough coordination on the matter is evident. 175. For most of the indicators coloured in green and orange the evaluation team found a difference between the programme’s yearly objectives as outlined in WFP’s PMP and the reality of what WFP and sub-recipients are planning to implement as specified in the different MoUs. Whilst sometimes the differences in figures is only a matter of readjusting numbers of beneficiaries according to the latest selection of schools selected for the programme,121 the differences in some instances are much more significant. This is often due to the fact that sub-recipients have only been able to forecast activities for one or two years instead of five, due to budget constraints.122 176. Most of the indicators in orange and red on the graph need a more specific approach. Beneficiaries and activities need to be properly identified for the indicators to be usable for the programme’s M&E. For example, it is important to distinguish literacy school books from educational leaflets (Standard ind. # 2 and Custom ind. #4), or school inspector training from school feeding directorate staff training (Custom ind. #6), or educational training from nutrition training (Standard ind. # 4), etc. However, it appears from the field findings that the lack of specificity of the indicators is a direct consequence of the difference of interpretation and understanding of the MGD theory of change among implementing partners. This has also led to certain indicators not matching the actual 121 The final list of schools was agreed between WFP and the Ministry of Education in March 2017 during the inception field mission. 122 The PMP is based on an estimated budget done in 2017 for WFP’s proposal to USDA. However, the budget initially going to sub￾recipient partners went down from $6,895,965 to $2,302,378 which represent a budget decrease of 66%. Baseline report – Second draft – July 2018 47 | Page activity it is supposed to measure (Custom ind. # 6). As a result, the internal coherence of the whole programme is at stake; while being theoretically designed according to a robust rationale, the present MGD programme looks rather like a juxtaposition of activities and even pre-existing activities (see for example § 81). This may be the result of the drastic budget revision which occurred between the initial WFP budget proposal and the USDA￾approved budget, as already mentioned above (see § 173 and related footnote) and requires careful attention by the CO. 177. Regarding SO2, no common approach has been used by cooperating partners regarding sensitisation on the use of improved health and dietary practices. This is primarily reflected by the absence of a common definition of food groups which provides the basis on which M&E can measure changes on dietary practices over time (see Annex 26 for more information). The lack of a harmonised approach on the implementing side leads to the lack of a harmonised M&E system. According to field findings, this lack of harmonisation is partly the result of weak coordination between implementing partners.123 178. Key success factors for an efficient and effective M&E system mainly depend on close coordination between cooperating partners as well as on WFP staff capacity. The MGD programme goes beyond a School Feeding Programme as stated in its official title: “McGovern-Dole International Food for Education and Child Nutrition Programme”.124 This means that WFP needs to work in close coordination with partners’ M&E divisions that have an expertise in different areas of intervention such as education, nutrition, health, sanitation, capacity building, etc. The ET noted that in order to do so, WFP’s M&E staff will be strengthened by additional staff before the programme actually begins. Common understanding and use of indicators is also essential. If for example partners125 do not work with a common approach regarding food groups, data collection and/or indicators will not be harmonised, which may lead to inefficiency when comparing data between partners. 179. As it currently stands, the M&E system only plans to monitor equal participation￾related objectives of the MGD programme (i.e. boys receiving food vs. girls, or males trained vs. females). However, M&E does not foresee indicators that relate to other WFP Gender Policy objectives, which should be mainstreamed in the programme, evidencing a gap on gender programming and a low gender-sensitive M&E system. Indeed, there are no indicators foreseen to capture aspects related to women’s empowerment, decision-making or protection and mitigation of GBV in a context where gender-related violence is known and reported, both in the context of the school and at household level. 6.2. Recommendations 180. All of the following recommendations are addressed to the CO but will require strong support from the RBJ which has a technical support role in the Southern Africa region. They should be addressed before the implementation of the programme at the beginning of the coming school year (2018-2019). Recommendations aim at helping WFP and its cooperating partners to actualise the standard and custom indicators at the start of the programme, so as to enhance the M&E system of the MGD programme for the next five years. They are written below in priority order: 123 Lack of coordination has been confirmed by various partners/stakeholders during semi-structured interviews and has also be observed by the ET. 124 The title of this baseline report in identical to that in the ToR as requested by the Baseline Report Quality Checklist, however, the ET suggest to WFP to revise this title to better reflect MGD’s objectives. 125 Including Ministry of Health Baseline report – Second draft – July 2018 48 | Page 181. Recommendation 1: The ET recommends126 that WFP, the sub-recipient partners and other main stakeholders in the education and school feeding sector (such as the Directorate of School Feeding, the World Bank and the ASPC) enhance coordination before the start of the MGD programme in September/ October 2018 in order to respond to the weaknesses in the planning of the programme as identified in this baseline report. These refer mainly to insufficient common understanding of programme outcomes between WFP and cooperating partners, lack of appropriate activities planned to reach the “Improved Quality of Literacy Instruction” Result Stream, insufficient details and disaggregation of the implementation plan, and nonalignment between sub-recipient partners’ MoUs and the initial WFP/USDA agreement. Steps to implement recommendation 1 Priority Timeline Responsible Together with sub-recipient partners, revisit the intervention logic (theory of change and RFs). This will ensure that all partners understand how their respective work links to the overall MGD Strategic Objectives, and that everyone has integrated in their corresponding results frameworks and M&E systems the relevant PMP indicators. Very high Before programme starts WFP CO Revisit the intervention logic underpinning Improved Quality of Literacy Instruction Result Stream (see recommendation 3) N/A N/A N/A Ensure that all MoUs reflect the MGD activities needed to lead to the SOs. Ideally, those MoUs should reflect engagement over the whole duration of the programme. High Before programme starts WFP CO; sub￾recipient partners 182. Recommendation 2: Following the baseline findings, several indicators need to be modified either through further disaggregation or through re-definition in order to better match the activity they intend to measure. Moreover, new indicators should be incorporated to ensure that adequate information is tracked to properly measure planned activities. Yearly targets need better definition and as much as possible they should be articulated throughout the programme’s duration. Those indicators should be directly linked to activities as per agreed MoUs with WFP. Steps to implement recommendation 2 Priority Timeline Responsible Examine each PMP indicator and make sure:  they match the activities they are supposed to measure  they are properly disaggregated Very high Before programme starts WFP CO 126 This recommendation can be considered as beyond the mandate of the baseline report, but these foundation steps needs to be addressed in order for the MGD programme to start on a strong basis. Baseline report – Second draft – July 2018 49 | Page  custom indicators are added to properly reflect activities implemented  yearly targets are defined for the whole length of the programme 183. Recommendation 3: Observing that the fundamental activities linked to the Improved Quality of Literacy Instruction will not be conducted by MGD sub-recipients but rather fall under the PRAASED World Bank programme, the ET recommends that MGD works in tight collaboration with PRAASED by elaborating a common framework in order to enhance programme complementarities, avoid overlap and allow MGD’s monitoring system to establish the necessary connections with PRAASED for an effective monitoring of the MGD programme. This is particularly important regarding the “Improved Quality of Literacy Instruction” result stream. Steps to implement recommendation 3 Priority Timeline Responsible Engage in early coordination with the World Bank and its PRAASED programme to identify complementarities between PRAASED and MGD regarding “Improved Quality of Literacy Instruction” Result Stream. It is possible that the PRAASED already covers most of this Result Stream. Very high Before programme starts WFP CO With PRAASED objectives in mind, reconsider how UNESCO’s work regarding the “Improved Quality of Literacy Instruction” result stream can better fit the MGD requirements. High Before programme starts WFP CO 184. Recommendation 4: WFP and its sub-recipient partners should apply the Congolese Food Based Dietary Guidelines to overcome the inconsistencies found across various documents on the use of food groups, allowing for a unified understanding and measurement of activities and indicators linked to nutrition. In addition, the key messages on the improved use of health and dietary practices that will be used in schools, health centres and agricultural extension should be harmonised among all implementing partners. Steps to implement recommendation 4 Priority Timeline Responsible Call for a multi-stakeholder working group (sub-recipient partners and relevant ministries) in charge of defining unified way to measure dietary diversity practices. This working group should also elaborate common key Very high Before programme starts WFP CO Baseline report – Second draft – July 2018 50 | Page messages/documents/tools on the use of improved heath and dietary practices. Ensure application of the use of this unified food groups for monitoring the MGD programme. High Within next 6 months WFP CO 185. Recommendation 5: WFP’s M&E system needs to increase its capacity through the recruitment of staff and more detailed tracking of activities in order to be able to monitor MGD programme implementation performance. According to the baseline findings, monitoring the MGD programme should include:  close and detailed cooperation between all involved partners,127 including a coordinated and aligned M&E system;  an individual follow-up of each MGD-assisted school which will allow reporting on specific needs rather than relying on broad indicator values as collected by the INS survey. Steps to implement recommendation 5 Priority Timeline Responsible Recruitment of M&E staff in order to increase the WFP CO’s M&E capacity. Very high Before programme starts WFP CO Together with all implementing partners (ACTED, UNESCO, UNICEF, government, ASPC, World Bank, etc.) establish a M&E task force to ensure that all partners have same understanding of indicators to track: if M&E systems are aligned; what type of data is appropriate for each level of indicators; how data collected by partners will be properly processed and input into end system (WFP) on a regular basis; how any monitoring challenges will be reported to WFP in time to be able to act upon; etc. This task force only needs to operate over a certain period of time (first implementation year for example). Very high Before programme starts WFP CO Ensure each school can be followed individually, allowing close-up assessment of school needs and M&E activities. This can be done through a tight collaboration with the Directorate of School Feeding / Ministry of Education by High Before programme starts WFP CO 127 Not only sub-recipients, but also implementing partners such as the World Bank and the ASPC. Baseline report – Second draft – July 2018 51 | Page adapting the government’s monitoring system already in place to the MGD programme. 186. Recommendation 6: In order to meet Gender Policy commitments, the WFP CO should develop specific indicators that monitor the extent to which the programme promotes participation of women in School Feeding Committees (SFCs) in decision￾making positions and others to prevent unforeseen protection issues. Attention should be given on how MGD activities can promote women’s empowerment, and how to promote gender equality in schools. Steps to implement recommendation 6 Priority Timeline Responsible Elaborate additional indicators to measure women’s empowerment throughout the programme plus indicators to anticipate protection issues that might appear as a result of programme implementation. High Before programme starts WFP CO with the help of RB Establish a working group on how school feeding can enhance women’s empowerment through food preparation for school canteens and how women can access leading positions in school feeding committees. High Before programme starts WFP CO Put in place a working group on how UNESCO’s Gender Equality Module could be integrated in the MGD programme. The Ministry of Education and the Ministry of Women should ideally take the lead in this working group. High Before programme starts WFP CO Baseline report – Second draft – July 2018 52 | Page Annexes Baseline report – Second draft – July 2018 53 | Page Annex 1.Terms of reference substitutable Evaluation of the WFP McGovern Dole Funded School Feeding Programme in the Republic of Congo Terms of Reference Decentralized evaluation for evidence-based decision making Baseline report – Second draft – July 2018 54 | Page Terms of Reference EVALUATION of McGovern Dole School Feeding Program in the Republic of Congo (2018 to 2022) WFP Congo, Republic of Table of Contents 1. Introduction........................................................................................................................... 1 2. Reasons for and Objectives of the Evaluation ......................................................................... 1 2.1 Rationale/Purpose of the Evaluation .....................................................................................1 2.2 Objectives ................................................................................................................................2 2.3 Stakeholders and Users ..........................................................................................................2 3. Context and subject of the Evaluation.................................................................................... 3 3.1 Context ....................................................................................................................................3 3.2 Subject of the evaluation: School Feeding .............................................................................5 4. Evaluation Approach............................................................................................................. 6 4.1. Scope........................................................................................................................................6 4.2. Evaluation Criteria and Questions .........................................................................................6 4.3. Data Availability, Reliability and Validity ..............................................................................8 4.4. Methodology ...........................................................................................................................8 4.5. Quality Assurance and Quality Assessment........................................................................ 11 5. Phases and Deliverables .......................................................................................................11 6. Organisation, Management and Conduct of the Evaluation................................................. 12 6.1. Organisation and Management........................................................................................... 12 6.2. Evaluation Conduct.............................................................................................................. 13 6.3. Team composition and competencies ................................................................................. 13 6.4. Ethical Considerations......................................................................................................... 14 6.5. Security Considerations....................................................................................................... 14 7. Roles and Responsibilities of Stakeholders ......................................................................... 15 8. Communication and budget ................................................................................................ 16 8.1. Communication.................................................................................................................... 16 8.2. Budget................................................................................................................................... 17 Annex 1 Map of Targeted Areas ............................................................................................. 18 Annex 2 McGovern Dole Results Framework......................................................................... 19 Annex 3 Preliminary Stakeholder analysis (Interests, uses, means of engagement)...21 Annex 4 Performance Indicators and Targets........................................................................ 23 Annex 5 Evaluation Schedule (Baseline, Mid-term and Final Evaluations) ............................ 25 Annex 6 Membership of the Evaluation Committee ............................................................... 27 Annex 7 Membership of the Evaluation Reference Group ...................................................... 27 Annex 8 Communication and Learning Plan......................................................................... 28 Acronyms ........................................................................................................................................ 29 WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 1 | P a g e 1. Introduction 1. These Terms of Reference (TOR) are for the evaluation of the World Food Programme (WFP) school meals programme funded by McGovern-Dole (MGD) Food for Education and Child Nutrition Program in Republic of Congo. The United States Department of Agriculture (USDA) has awarded WFP Congo a total of US$ 30 million to be implemented from 2018 to 2022. The Program aims to support education, child development and food security through school feeding and related activities. The program provides to WFP USA produced agricultural commodities and financial assistance for implementation of school feeding as well as capacity development of the government and enhanced monitoring and reporting by WFP and partners. It will reach 54,000 /schoolchildren (27,270 boys and 26,730 girls) with school meals and other beneficiaries through other activities as outlined in section 3.2 of this TOR. Sustainability is an important consideration, and WFP Congo will work to support government and community ownership. 2. These TOR were prepared by the WFP Congo Country Office with support from the WFP regional Bureau (RB), based on an initial document review, consultation with stakeholders and following a standard template. The purpose of the TOR is threefold. First it outlines how WFP will actualise the evaluation plan as approved by USDA over the course of the five year programme; secondly, it provides key information to stakeholders about the proposed evaluation; and thirdly, it provides key information to the evaluation team and helps guide them throughout the evaluation process. 3. These TOR are informed by the WFP evaluation policy and USDA’s Monitoring and Evaluation policy. The evaluation is expected to follow and meet the requirements outlined in these policies as appropriate. 2. Reasons for and Objectives of the Evaluation 2.1 Rationale/Purpose of the Evaluation 4. In line with the agreement signed between WFP and USDA, this evaluation will be commissioned by WFP Congo Country Office and will include baseline in 2018, mid-term evaluation in 2020 and final evaluation in 2022. The baseline seek to assess the situation before the beginning of the programme and the evaluations will seek to assess progress towards achievement of intended outcomes. 5. The grant agreement between WFP and USDA incorporates specific results and performance indicators against which performance of the program will to be measured (Annex 2). The agreement also includes the evaluation plan, in which WFP committed to conducting a baseline study, mid-term and final evaluation. This evaluation is therefore commissioned to systematically establish benchmarks at baseline (2018) upon which performance will be assessed; assess progress to towards achievement of the results at mid-term (2020) to inform adjustments to programming and/or implementation and course correction as appropriate; and measure assess performance at the end of the program (2022). 6. For USDA, the purpose of the baseline is to establish benchmarks; and the purpose of the evaluations is to critically and objectively review and take stock of the program implementation experience within the implementing environment of Congo, assess whether targeted beneficiaries are receiving services as expected, assess whether the project is on track to meeting its stated goals and objectives, review the results frameworks and assumptions, document initial lessons learned, and discuss necessary modifications or mid-course corrections that may be necessary to effectively and efficiently meet the stated goals and objectives.1 7. Utility: The findings from the baseline will be used to adjust targets if/as appropriate; the findings from the mid-term evaluation will be used for course-correction if/as appropriate and the final evaluation findings will inform design of future interventions. 1 USDA Monitoring and Evaluation Policy, 2013 WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 2 | P a g e 2.2 Objectives 8. The evaluation will serve the two mutually reinforcing objectives of accountability and learning. • Accountability for actions and results: The objective of the baseline is to establish baseline values for the indicators as outlined in the Performance Monitoring Plan (PMP) upon which performance will be measured and the basis on which WFP will account for results achieved and resources utilised. The objective of the mid-term evaluation is to account for the progress made in achievement of results, comparing with baseline and assessing whether these results account for actions taken and resources utilised so far. The final evaluation will account for the achievement of the results and long-term effects of the program (intended, unintended, negative or positive) on targeted girls, boys, men and women, communities and institutions. • Learning and adjusting based on lessons: The baseline will provide evidence on whether the targets set in the PMP are realistic (not too low/high). WFP and USDA will use this evidence to decide whether to adjust the targets. The mid-term evaluation will determine the reasons why certain results were achieved or are in progress of being achieved or not. It will draw lessons, derive good practices and pointers for learning. This will inform operational and strategic decision-making, including any course correction measures by WFP and/or USDA. The final evaluation will generate lessons on what has worked in achieving positive long term effects and what factors may have led to any negative effects. 9. To enhance learning, the baseline study will make recommendations on most efficient approach to monitoring the program based on the indicators in the PMP; the mid-term evaluation will make recommendations on what is needed to strengthen and improve project implementation for the remaining period. The mid-term evaluation will also assess whether recommendations made during the baseline study were integrated into programme implementation and if so, whether these recommendations were successful in strengthening the programme implementation. Likewise, the final evaluation will generate recommendations to inform future design and implementation, while also assessing the extent to which recommendations made at mid-term were implemented and to what effect. 10. The evaluation reports will be actively disseminated, and the findings incorporated into relevant knowledge management systems within WFP and USDA to ensure wider organisational learning. 2.3 Stakeholders and Users 11. A number of stakeholders both inside and outside of WFP have interests in the results of this evaluation and some of these will play a role in the evaluation process. Annex 3 provides a preliminary stakeholder analysis, briefly describing the interests of each stakeholder and suggestions on means of engaging them during the evaluation. The evaluation team will further deepen this analysis during the inception phase and provide a more detailed map of stakeholders and means/plan of engaging them. This will include proposals on how the findings of the evaluation will be disseminated to the beneficiaries. 12. WFP is committed to ensuring that gender dimensions are addressed throughout the evaluation process, with participation and consultation of women, men, boys and girls. 13. The primary users of this evaluation will be: • WFP Congo and its partners (see Annex 3) in decision-making, adjustments and course-correction at mid-term; and generating lessons for future from the final evaluation; • Given the core functions of the RB, it is expected to use the evaluation findings to provide strategic guidance, programme support, and oversight; • WFP HQ may use evaluations for wider organizational learning and accountability; • WFP office of evaluation (OEV) may use the evaluation findings to feed into evaluation syntheses as well as for annual reporting to the Executive Board progress in evaluation policy implementation; • USDA will use the findings from the evaluation to generate lessons to inform future design and implementation of MGD programs in other contexts. All evaluation reports will be made publicly available on the Foreign Agriculture Service (FAS) website. USDA expects that facilitation and WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 3 | P a g e exchange of lessons learned and good practices from these evaluations, will lead to improved program design and effectiveness of current and future efforts in food assistance and capacity building; • The Government of Republic of Congo (GRoC), who is one of the donors of this program will use the findings and recommendations from this evaluation to support policy development and implementation decisions, particularly those related to the national school feeding policy (NSFP). • Stakeholders including UNICEF and ACTED who are sub-recipients for the MGD funding will use the results of this evaluation to improve aspects of the program that are directly under their responsibilities. 3. Context and subject of the Evaluation 3.1 Context 14. Macro Environment: The Republic of Congo has a population of 4.2 million people, of which about 64% live in the urban areas of Brazzaville and Pointe-Noire. The Country is rich in natural resources (oil, timber) and fertile land. Oil exports contribute approximately 70% of government revenues and about 95% of export earnings. The downward trend in oil prices in recent years is having a negative impact on the local economy. Classified as a lower middle- income country, Congo’s GDP contracted from of USD 14.4 billion in 2011 to 7.8 billion in 2016.2 Likewise, the GDP per capita contracted from US$ 5,538 in 2014 to US$ 5,301 in 2016 (See figure).3 According to a World Bank report, the country moved from a surplus of 9.6% of GDP in 2010/13 to a deficit of 14.2% in 2015/16).4 According to the World Bank, economic prospects are weak, and GDP growth will average about 0.9% over 2017-2019, despite increased oil production with the entry of new oil fields.5 15. Poverty (SDG 1) and Food insecurity (SDG 2): Income in Congo is unevenly distributed, reflected by a Gini coefficient of 0.43. Roughly, 48% of Congolese live on less than USD 1.25 per day, while up to 77% live below US$ 1.9 per day. Congo’s Global Hunger Index score in 2017 is 25.6 placing the country at a moderate hunger level.6 More than 121,000 households –14% of the population–suffer from food insecurity. According to FAO, the proportion of undernourished population in Congo averaged 28.2% between 2014 and 2016 (see figure).7 Food production is below the national requirements. Only 2% of arable land is currently under cultivation, producing less than 30% of the population's food needs. The country imports most of its food, making it vulnerable to food prices fluctuations. 16. Nutrition and Health: According to the Multiple Indicator Cluster Survey (MICS) 2015, rates of severe acute malnutrition stood at 2.6%, global acute malnutrition at 8.2%, stunting at 21.2% and underweight at 12.3%. Approximately 12% of women suffer from acute malnutrition, 8% of women suffer from clinical vitamin A deficiency (history of night blindness during most recent pregnancy) and 70% of 2 https://data.worldbank.org/indicator/NY.GDP.MKTP.CD?locations=CG 3 http://www.fao.org/faostat/en/#country/46 4 World Bank, (2017), Africa’s Pulse: An Analysis An analysis of issues shaping Africa’s economic future, Volume 16, October 2017 5 http://www.worldbank.org/en/country/congo/overview 6 https://www.ifpri.org/publication/2017-global-hunger-index-inequalities-hunger 7 http://www.fao.org/faostat/en/#country/46 WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 4 | P a g e pregnant women have iron and folic acid deficiencies. The national HIV/AIDS prevalence is 3.2%, with urban areas having a higher prevalence than rural areas (3.3% and 2.8% respectively). 17. WFP operations in RoC include a Country Programme (CP 200648) and the Emergency Operation targeting refugees and IDPs. Through the country programme, WFP aim is to promote access to education, provide capacity building to the Government for school feeding policy development to define the vision and provide guidance as well as a handover strategy. WFP also aim to support the Government to establish a social safety net for vulnerable households. On nutrition, WFP aims to provide specialized food to children under the age of five, pregnant and breastfeeding women to prevent chronic malnutrition and improve nutritional recovery among people living with HIV/AIDS or TB and receiving treatment. So far, the country program has faced critical resource shortfalls and some of the planned activities have been suspended since October 2016. 18. Refugees: Following the crisis in the Central Africa Republic (CAR), WFP has been providing assistance to refugees in Congo through a regional EMOP, which spans five countries [CAR, Cameroon, Chad, DRC and Congo]. This EMOP started in 2015 and will end in December 2017. The situation in CAR remains volatile and unpredictable, and sporadic outbreaks of extreme violence continue to occur. As the prospects for self-sufficiency for CAR refugees in Congo remain limited, WFP assistance to the refugees in the northern part of Republic of Congo (Likouala) and IDPs in the Pool department will continue in 2018 through a county level EMOP. In the Pool region, the Government is facilitating peace building through demobilization and disarmament and a reintegration program with the local authorities. 19. Donors and Aid: The Net Official Development assistance (ODA) as a percent of Gross National Income (GNI) dropped from dropped from 14.6% in 2010 to just 1.2% in 2015.8 GRoC is the main donor for the country programme and committed to fund 60 percent of the USD 56.8 budget. From 2010 to 2014 it contributed more than 12 million USD. Since then, no contribution was received from due to serious budget gaps occasioned by the drop in the oil prices. Donors for refugees assistance include the United States, Japan and Brazil. 20. Government Policies and Priorities: The National Development Plan (NDP 2012-2016) provides the current national planning framework; the new NDP (2017- 2021) is under preparation. The NDP embodies the country’s vision and aims to create greater shared prosperity to expedite the modernisation and industrialisation of Congo. It is broken down into five pillars namely governance; transformation of the economy; development of economic and social infrastructures; social development and inclusion; and balanced development that is sustainable and thus enduring. The UNDAF (2014-2018) was developed in alignment with three of the five pillars of the NDP: governance; social development and inclusion; and balanced, sustainable development. Congo is a member of the Scaling Up Nutrition (SUN) movement and in October 2013 it adopted a strategic framework for combating malnutrition. The inter￾ministerial “Congolese Feed Congolese” (Le Congolais nourrit le Congolais) initiative linking school feeding with local agricultural production was developed in 2012 with WFP’s assistance. 21. School Feeding Policy Framework: In 2014, a national capacity assessment and planning workshop on school feeding allowed for a diagnosis of national capacities in school nutrition. In 2015, Congo conducted a Systems Approach for Better Education Results (SABER), which produced a set of actions towards the development of school feeding in the Country. In 2016, Congo adopted a new national school feeding policy (NSFP) which was developed with support of WFP. While this is a big step towards national ownership of school feeding, setting up the structures to implement this policy remains a major challenge for the coming years. 22. Gender: Despite laws guaranteeing gender equality, the ratification of international instruments and the creation of a specific ministry, women in Congo continue to suffer legal and practical discrimination and inequalities and the country does not yet have a policy against gender-based violence. 9 The country scores 0.617 on the Gender Inequality Index and there are significant legal and policy gaps 8 https://data.worldbank.org/indicator/DT.ODA.ODAT.GN.ZS?locations=CG 9 WFP/EB.2/2014/7/3 COUNTRY PROGRAMME THE CONGO 200648 (2015–2018), page 7. WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 5 | P a g e relating to issues of gender protection.10 Women have limited access to education, limited participation in the labour market, vulnerability to pregnancy related deaths, and high adolescent birth rates. It is estimated that 63 percent in Bouenza and 47 percent in Pool of the female headed households are food insecure while 52% and 42% for male headed households respectively. 3.2 Subject of the evaluation: School Feeding 23. Through the Development project (DEV 2011-2014) approved in December 2011, WFP supported primary schools in the regions of Cuvette, Lekoumou, Plateaux and Pool through provision of daily hot meals to primary school children in participating schools. The project was aimed at increasing primary school access, enrolment, attendance, retention and completion while also reducing micronutrient deficiencies of primary school children in the most food-insecure regions of the country. Under this DEV project, WFP reached up to 92,000 beneficiaries.11 In 2014, the programme was expanded when WFP got the approval from the Government to assist the Observe, React, Act (ORA) schools, targeting some additional 3,500 indigenous children in 53 schools in the Likouala department. 24. The DEV project was succeeded by the Country Programme (CP 2015-2018) approved by the Executive Board in November 2014. Through the CP in 2016, WFP provided school meals to 67,776 beneficiaries in 574 rural public schools in the departments of Cuvette, Lekoumou, Plateaux, Bouenza and Pool. This included 45 ORA non-public schools in the Likouala department to encourage and promote education for indigenous children. In addition to food distribution, WFP provided non-food items, including 500 cooking pots to participating schools. However, due to lack of resources, the number of feeding days was reduced from 180 school feeding days to 65 in 2016.12 25. In September 2017, USDA signed an agreement to fund the WFP to implement a USD 30,022,053 school meals program from 2018 to 2022. The objective of the program is to improve literacy and nutrition of boys and girls. This objective will be achieved through a broad set of activities and inputs over five years including provision of hot lunches served at mid-day to 54,000 primary school children (27,270 boys and 26,730 girls) in six departments (Pool, Bouenza, Cuvette, Plateaux, Lekoumou, and Likouala). The McGovern Dole funded school feeding programme will target some of the same schools and children assisted under the CP 200648. The details of the targeting (overlaps and new additions) will be discussed in details with the evaluation team during the inception phase to inform the design of the evaluation. Each child will receive a meal consisting of fortified rice, split yellow peas, and vegetable oil. The meal will be supplemented by iodized salt provided by the GRoC and canned fish provided by Japan which will be integrated into school feeding pramme. 26. The program will use McGovern-Dole commodities and cash funding to contribute directly towards the two McGovern-Dole program’s highest-level Strategic Objectives namely Improved Literacy of School-Aged Children (MGD 1) and Increased Use of Health, and Dietary Practices (MGD 2). This contribution will be achieved through the following activities: • Improve Student Enrolment by raising awareness on the Importance of Education; • Distributing food to provide School Meals to school children • Promoting improved health by Building/Rehabilitation of Latrines; Building/ Rehabilitation Water Stations and Hand Washing Kits; and Deworming; • Supporting improved literacy by: Distributing School Supplies; supporting Revision of National Curriculum, distribution and training on the revised curriculum; Promoting Teacher Attendance; Training of Teachers; and Training of School Administrators and Officials; • Promote Improved Nutrition by: Training and Raising Awareness on Good Health and Hygiene Practices; and Training and Raising Awareness on the Importance of Improved Nutrition, Health and Dietary Practices; 10 http://hdr.undp.org/en/composite/GII 11 WFP Congo Dev 200144, Standard Project Report, 2014. 12 WFP Congo CP 200648, Standard Project Report, 2016. WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 6 | P a g e • Support Improved safe food preparation and storage by: Building/ Rehabilitation of Kitchens and Storerooms; and providing Energy Saving Stoves and Kitchen Utensils. 27. The program also has a strong focus on institutional capacity building to ensure sustainability and contribute to MGD foundational results namely increased capacity of Government institutions; improved policy and regulatory framework; increased Government support and engagement of local organizations and community groups. This will be achieved through the following activities: • Building capacity by Support the Implementation of the Systems Approach for Better Education Results (SABER) Action Plan and Government National School Feeding Policy (NSFP); • Establish/Strengthen local Agriculture and school communities to support graduation through the implementation of the national home-grown school feeding program. • Promote improved health by Training on Commodity Management, Food Preparation, and Storage. 28. For a graphical representation of the project’s theory of change, including the linkages among key activities and results, and the names of partners with whom WFP will work with under each activity, see the results framework in Annexes 2. Annex 3 provides the list of indicators for monitoring progress and assessing achievement of the objectives. These two elements will be central to the evaluation and will need to be analysed during the inception phase when designing the evaluation. 29. Partnerships: The implementation of these activities is in partnership with Key sub-recipients of the MGD funds (UNICEF, UNESCO and ACTED). Field implementation involve NGOs including ASPC, Pioneer Hospital, Initiative Development and private sector entities (NG Enterprise, Minoco). Capacity building is targeted to the Ministries of Agriculture, Education and Health. 30. Gender Analysis in the context of school feeding: During the establishment of food management committees for school feeding in targeted schools under the country programme, local communities were encouraged to aim for gender parity to foster the involvement of women in decision-making. Women representation increased from 30% to 35%.12 Communities have recognized the participation of volunteers as a key component in school feeding activities and were seen as an important contribution to local development. This was particularly important for women from indigenous groups, as their participation has promoted their integration and acceptance into other groups. While men were continuously encouraged to volunteer, very few proved willing to help in the preparation of school meals. Instead, most male volunteers assisted with constructing and maintaining school infrastructures, such as kitchens, warehouse, toilets and others facilities. 4. Evaluation Approach 31. This is a multi-year evaluation, with three phases that will produce three deliverables over a period of five years: baseline study in 2018, mid-term evaluation in 2020 and final evaluation in 2022. These phases will be conducted by the same evaluation team and following the same methodological approach. The products will be delivered in accordance with the timelines agreed upon with WFP and USDA. 4.1. Scope 32. The evaluation will cover all activities implemented through the MGD funding. The inception period will establish and confirm appropriate sampling frames, sampling strategy and survey instruments for the baseline, midterm and final evaluations. In terms of the period to be covered, the baseline will focus on collecting the latest values for all the indicators in the PMP as at February 2018 before commencement of the activities. For those indicators whose source is secondary (from monitoring data, government or other partners), the baseline will use the latest available figures. The mid-term evaluation will cover three years (February 2018-February 2021). The final evaluation will cover five years (February 2018 to February 2023). 4.2. Evaluation Criteria and Questions 33. The baseline study will answer three key questions: WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 7 | P a g e Q1: What are the baseline values for each indicators in the PMP? Q2: Given these baseline values, the objectives of the MGD program and within the context of Congo, are the targets set for each indicator realistic? Are any of them too low or too high? Q3: Given the objectives and activities of MGD and the context of Congo, what are the key success factors for efficient and effective M&E of the program? Are the evaluation design and evaluation questions planned at inception feasible. 34. The mid-term and final evaluations will apply international evaluation criteria of relevance, effectiveness, efficiency, impact and sustainability.13 Gender Equality and empowerment of women will be mainstreamed throughout these five criteria, with specific evaluation questions where appropriate. 35. Under each criteria, the evaluation will address a number of evaluation questions to enable assessment of the performance of the program and the impact on targeted individuals and institutions. Table 1 provides a preliminary list of questions, which will be further developed by the evaluation team during the inception phase. Collectively, the questions aim at highlighting performance, results and key lessons of the MGD funded program. Table 1: Criteria and evaluation questions Key Questions at Mid Term Evaluation Key Questions at Final Evaluation Relevance •Is the program’s strategy relevant to the beneficiaries’ needs, including girls, boys, men, women and other groups such as indigenous people? •Is the program aligned with national government’s education and school meals policies and strategies? •Does the program design and implementation arrangements complement other donor-funded and government initiatives? •Is the program designed to reach the right people with the right type of assistance? •Was the program designed to reach the right people with the right type of assistance? •Did the program’s implementation lead to meeting the intended beneficiaries’ needs with the right mix of assistance? •Is the program aligned with national government’s education and school meals policies and strategies? •Did the program complement other donor-funded and government initiatives? Effectiveness •What is the progress of program implementation–is the program on track to carry out all activities as planned? •To what degree has the program resulted (or not) in the expected results (outputs and outcomes) for girls, boys, men and women? •What internal and external factors affect the program’s achievement of intended results? •Are any changes required to increase the programeffectiveness? •To what degree have the interventions resulted (or not) in the expected results (outputs and outcomes as perthe PMP), for girls, boys, men and women? •What internal and external factors affect the program’s achievement of intended results? Efficiency •How efficient is the targeting? •Did assistance reach the right beneficiaries (girls, boys, men and women) in the right quantity, quality and at the right time? •Is the program efficient in terms of costs and costs/ beneficiary? •How efficient is the targeting? •Did assistance reach the right beneficiaries in the right quantity and quality at the right time? •Is the program efficient in terms of costs and costs/ beneficiary? Impact •To what degree has, the program outcomes made progress toward positive long-term effects on targeted beneficiaries(girls, boys, men and women), households, Communities and institutions? •Have there been any unintended outcomes (positive, negative? •What internal and external factors affected the program’sresults from leading to intended impact on targeted beneficiaries? •What are the long-term effects of the interventions on targeted beneficiaries’ lives, households, communities and institutions? •Were there unintended outcomes, (positive, negative)? •What internal and external factors affected the program’s results from leading to intended impact on targeted beneficiaries? Sustainability 13 For more detail see: http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 8 | P a g e •Is the program sustainable/is there strategy for sustainability, sound policy alignment; stable funding/budgeting; quality program design; institutional arrangements; local production & sourcing; partnerships & coordination; community participation & ownership? •What progress has the government made toward developing a nationally owned school meals program? •How are local communities involved in and contributingtoward school meals? •What needs to be done within the remaining period in orderto transition to a nationally owned school meals program? •To what extent is it likely that the benefits of the program will continue after the end of the program? •What are the key factors that affect the likelihood of sustainability of the results of the program? General •What are lessons noted from the program up to this point? •Are there any recommendations for mid-course corrections to improve the program’s relevance, efficiency, effectiveness, impact, and/or sustainability? •What are lessons learned from the program? •How can WFP improve future programming, in the context of these lessons noted? •How can USDA improve future MGD funding in the context of these lessons noted? 4.3. Data Availability, Reliability and Validity 36. The MGD program has measurable objectives, twenty-five quantifiable indicators and targets as outlined in Annex 3. The results framework presented in Annex 2 lists a number of critical assumptions that have to hold true for the success of the program. The detailed PMP provides sources of data for each indicator as well as the frequency of collection. 37. During the inception phase, the evaluation team will review the PMP in detail, verify the data sources and programme design and design the evaluation. The design will ensure that: a) The baseline study collects and analyses data for all indicators from the most appropriate sources; b)The mid-term evaluation will be able to utilise the baseline data to assess progress in achieving the program objectives and c) The final evaluation will be able to utilise the baseline and mid-term evaluation data to assess the performance of the program and effects on targeted individuals and institutions. 38. The evaluation design should allow utilization of existing data and collection of primary data only where needed. Existing data includes past studies such as school feeding Cost Benefit Analysis (CBA), 2014 SABER and UNICEF studies such as the MICS. 39. To ensure reliability and validity of data, and credibility of the evaluation, evaluation team will: • At inception: Assess data availability and reliability for all those indicators for which sources are indicated as secondary in the PMP and make a determination on whether these sources are sufficient to provide reliable data. This assessment will inform the design of primary data collection. • Throughout the evaluation: systematically check accuracy, consistency and validity of all data collected and acknowledge any limitations/caveats that should be born in mid when drawing conclusions or interpreting the findings presented in the evaluation reports. 4.4. Methodology 40. The methodological approach for the baseline study, mid-term evaluation and the final evaluation will be designed in accordance with WFP Decentralized Evaluation Quality Assurance System (DEQAS) as well as USDA’s Monitoring and Evaluation Policy. The methodology will be developed during inception phase by the evaluation team. Based on the requirements described in these TORs, the evaluation team will, during the inception phase, carry out a detailed document review, consult key stakeholders and formulate an appropriate evaluation design, methods, approaches, and sampling strategy for the baseline, mid-line and final surveys. This will be presented in the Inception Report (IR), including a detailed evaluation matrix with evaluation questions and sub-questions and data sources. 41. The evaluation will employ a mixed methods approach with quantitative baseline, end line and final surveys complemented by qualitative elements. The survey design, sampling frame and data WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 9 | P a g e collection methods will be informed by program coverage, context and the list of indicators as per the PMP and the most appropriate and reliable sources of data for each indicator. The design will ensure pre-post comparisons at mid-term and final evaluations. Noting that the schools targeted by the program are not randomly selected, the evaluation team will use quasi-experimental approach bearing in mind ethical and technical considerations in identifying comparison groups for humanitarian and development interventions14. The team will identify which indicators in the PMP will be measured to compare the change in outcomes for schools not targeted with the change for the schools that receive the intervention. It is important to note that in five departments, WFP has already been providing school meals and therefore the USDA funding is used to continue the provision of meals rather than a completely new programme. If it is not possible to identify suitable comparison schools the inception report must clearly indicate this noting the reasons why attribution cannot be measured. In this case, the inception report must clearly set out whether and how contribution to these indicators will be measured. This will be discussed with stakeholders including USDA before the inception report is approved. The methodology must also clearly set out how the joint provision of salt and fish by Japan, Brazil and the Government will be measuredand accounted for. At inception, the team will assess whether it is possible to identify the contribution of the McGovern Dole provided commodities to the school meal programme or whether the evaluation will account for the results of the joint provision of the hot meal that combines MGD, Government and other donors inputs. For example if not all schools receive the commodities provided by other donors in addition to USDA commodities, then the schools that do not receive can be used as control for assessing the effects of these additional commodities. 42. The evaluation team will discuss with key stakeholders which indicators can be meaningfully assessed using this approach and which indicators a simple pre-post analysis will be sufficient. Given the emphasis on learning from this evaluation, the focus should be on a careful analysis of the contribution the program activities have on the higher objectives. 43. The quantitative surveys will be complemented by key informant interviews and/or focus group discussions with key stakeholders including USDA (DC-based program analysts and regional agricultural staff), UNICEF, World Bank, WHO, UNESCO, Ministry of Education, Ministry of Agriculture, Ministry of Health, Ministry of Economy, ACTED, FAO, Farmer’s organizations, Parents, Students and School management committees, WFP school feeding and nutrition officers, UNICEF nutrition and education officers, school inspectors, school administrators, and teachers, school children and their families, cooks, and farmers. The findings from these interviews will be used to put quantitative data into context and provide guidance for program implementation, communication of results, and formulation of action plans to address any weaknesses while enhancing strengths. 44. During inception phase the team will expand the above methodological approach to ensure it: • Employs the relevant evaluation criteria as outlined in table 1, ensuring the right balance between depth and breadth of analysis under each criteria; • Sets out transparently how the contribution of the WFP school-meals programme is identified and measured; • Demonstrates impartiality and lack of biases by triangulating data and information from a variety sources (variety of documents, interview of a variety of stakeholder groups, including triangulating views of men and women on same aspects; national and district level data); • Uses a transparent sampling, data collection and analysis processes, stating any limitations explicitly; • Ensures that women, girls, men and boys from different stakeholder groups participate and their voices are heard and reflected in the final report; This should be informed by a gender analysis the parameters of which the team should outline during the inception phase and deepen during the baseline study. This analysis should be used/revisited during the mid-term and final evaluations. • Mainstreams gender equality and women’s empowerment in the way the evaluation is designed, data is collected and analysed, findings are reported, and conclusions and recommendations are made. This will enable the team to reflect on lessons and recommendations that are gender responsive; 14 WFP 2017, Technical Note on Impact Evaluations WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 10 | P a g e • Includes ethical considerations throughout the evaluation process and that appropriate clearances are sought as necessary and as per the UNEG Ethical Guidelines • Includes an analytical framework, showing how existing data and primary data collected will be analysed and used to answer the evaluation questions. If the methodology used includes use of comparisons groups, the analytical framework will include use of difference-in-difference analysis for key indicators (to be agreed at inception). • Uses an evaluation matrix as the organizing tool to ensure all key evaluation questions are addressed and the conclusions are based on credible evidence 45. An initial analysis of potential risks is outlined in table 2. The evaluation team will deepen this risk analysis and identify additional mitigation measures. This should be reflected in the inception report. Table 2: Potential Risks, causes, effects and proposed mitigation actions Potential Risk Underlying causes Effects Mitigation actions Unforeseen contextual changes over the course of 5 years 5 years is a long time to plan and design an evaluation and a lot can change, within the Congo context, WFP and the context of the firm that will be contracted15 The evaluation is not conducted as initially designed; or resources allocated at the time of contracting are not sufficient; -At baseline stage, the plans for mid-term and final evaluations to be considered tentative liable for revisions if necessary; -Contract for mid-term evaluation to be based on performance at baseline, and contract for final evaluation to be based on performance at mid-term. Secondary data sources turn out not to be reliable for some indicators PMP was created at proposal stage indicates secondary sources of data for some indicators, before in-depth data reliable assessment If these are left out of the primary data collection, the baseline report will be less reliable OR incomplete Evaluation team to spend some time during inception assessing reliability of the secondary data sources. The result to inform what indicators will be included in primary data collection and which will be addressed from secondary sources Logistical difficulties in getting access to some beneficiaries If data collection is undertaken during rainy season which would reduce accessibility in areas with poor infrastructure Incomplete data collection; voices of some affected populations not heard; in some cases overreliance on secondary sources Data collection schedules informed by the season to the extent that this does not affect overall objectives of the evaluation; Use technology to collect data, with local research assistants who can transmit the data from remote sites (WFP sub-offices to support in this regard) Difficulties in getting access to relevant institutional partners and representatives The nature of government ministries is such that different departments are relevant for different aspects of the program. Not everyone from a ministry will necessarily be relevant for all topics The contribution of the institutions is limited if the right persons are not engaged (e.g. MOA in the discussions of sustainability and linkages to home grown school feeding) -Deepen the stakeholder analysis and identify relevant representatives from different institutions/ministries; -When inviting stakeholders for forums/sessions through the ERG, be specific on what the topic is and what inputs are expected so that institutions can identify the most relevant persons Security constraints that limit access to some of the targeted areas Some of the areas targeted by the program currently have some security issues, which has resulted in presence of internally displaced persons (IDPs); though the government is currently engaged in peace building efforts Voices of some of the affected populations is not heard; If the places are accessible at baseline but not so at mid-term for final evaluation it will make it difficult to collect comparable data using the same methodology. -WFP to share information on the situation with the contracted firm as often as needed; -Contracted firm to use that information to assess the impact on the design of the evaluation and identify mitigation measures; -Data collection to use technology and to the extent possible local data collectors that can remotely submit data (WFP sub￾offices to support in this regard) 15 From contracted firm point of view, the biggest risk is the extent to which the firm can guarantee the same team to conduct the three evaluations. From WFP point of view, is the risk of a firm contracted for the three evaluations and underperforming in the baseline or mid-term evaluation. WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 11 | P a g e 4.5. Quality Assurance and Quality Assessment 46. WFP’s DEQAS defines the quality standards expected from this evaluation and sets out processes with in-built stepsfor Quality Assurance, templates for evaluation products and Checklists for their review. DEQAS is based on the UNEG norms and standards as well as good practice of the international evaluation community. It aims to ensure that the evaluation process and products conform to best practice. 47. DEQAS will be systematically applied to this evaluation. The Evaluation Manager will be responsible for ensuring that the evaluation progresses as per the DEQAS Process Guide and for conducting a rigorous quality control of the evaluation products ahead of their finalization. 48. WFP has developed a set of Quality Assurance Checklists for its decentralized evaluations. This includes Checklists for assessing the quality for each of the evaluation terms of reference, inception report and evaluation report. The relevant Checklist will be applied at each stage, to ensure the quality of the evaluation process and outputs. 49. To enhance the quality and credibility of this evaluation, an outsourced quality support (QS) service directly managed by WFP’s Office of Evaluation in Headquarter provides review of the draft inception and evaluation report (in addition to the same provided on draft TOR), and provide: a) Systematic feedback from an evaluation perspective, on the quality of the draft reports; b)Recommendations on how to improve the quality of the final inception/evaluation report. 50. The evaluation manager will review the feedback and recommendations from QS and share with the team leader, who is expected to use them to finalise the inception/ evaluation report. To ensure transparency and credibility of the process in line with the UNEG norms and standards[1], a rationale should be provided for any recommendations that the team does not take into account when finalising the report. 51. The quality assurance process as outline above does not interfere with the views and independence of the evaluation team, but ensures the evaluation provides the necessary evidence in a clear and convincing way and draws its conclusions on that basis. 52. The evaluation team will be required to ensure the quality of data (validity, consistency and accuracy) throughout the analytical and reporting phases. The evaluation team should be assured of the accessibility of all relevant documentation within the provisions of the directive on disclosure of information. This is available in WFP’s Directive CP2010/001 on Information Disclosure. 53. The regional bureau, through the regional evaluation officer, will systematically support to the country office to ensure that the evaluation delivers quality process and product in line with both WFP and USDA policies and that the resulting products are useful and used. 54. The mid-term and final evaluation reports will be subjected to a post hoc quality assessment by an independent entity through a process that is managed by OEV. The overall rating category of the reports will be made public alongside the evaluation reports. 5. Phases and Deliverables 55. This is a multi-year evaluation that will proceed through five phases with key deliverables and deadlines for each phase are as follows as shown in figure 1. Figure 1: Summary Process Map [1] UNEG Norm #7 states “that transparency is an essential element that establishes trust and builds confidence, enhances stakeholder ownership and increases public accountability” •Terms of Reference •Inception Report •Baseline Report •MTE Report • FinalEvaluation •Contract • Management • Management Response Report Response • ManagementResponse 5. Final Evaluation 4. Mid-Term Evaluation 3. Baseline study 2. I ti 1. Preparation WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 12 | P a g e 56. Phase 1: Preparation (October-December 2017): This phase will be the responsibility of WFP country office with RB support to deliver final evaluation TOR. This will be the master document to guide the evaluation over the five years. This phase will also deliver the contract for the management and conduct of the evaluation. 57. Phase 2: Inception (January-February 2018): This phase will be the led by the evaluation team and will focus on the design of the evaluation. It will deliver the inception report, which will contain a) the full evaluation approach and methodology for the three deliverables (baseline, mid-term evaluation and final evaluation), b) stakeholder analysis and mapping; c) data collection process and tools; d) Analytical framework; e) evaluation matrix; f) review of the MGD results framework and clear indication of how the evaluation team will use it; g) confirm the evaluation questions for the mid-term and final evaluation, including proposing additional sub-questions if necessary. 58. Phase 3: Baseline Study (February-June 2018): This will be led by the evaluation team, and will answer three key questions outlined in section 4.2. The study will deliver a baseline report with recommendations on a) whether any of the targets needs to be adjusted; b) key actions required to ensure efficient and effective M&E of the program. The WFP country office will respond to these recommendations by preparing a management response with actions and timeline within which these actions will be taken. The Baseline study should also confirm that the evaluation design provided at inception remains feasible and the evaluation approach and methodology is on track to provide a high￾quality mid-term and final evaluations. 59. Phase 4: Mid-term Evaluation (November 2020-June 2021): This will be led by the evaluation team, and will build on the previous phases by starting with a review and update of the inception report to reflect any contextual changes and incorporate lessons coming from the implementation of the programme. The evaluation team will deliver an updated inception report and a mid-term evaluation report with recommendations. The WFP country office will response to these recommendations by preparing a management response with actions and timelines for implementing the recommendations. 60. The evaluation team will provide a 2-3 pages stand-alone brief describing the evaluation design, methodology, key findings, conclusions and recommendations. It will serve to inform stakeholders of the midterm evaluation, and should be written in a language easy to understand by non-evaluators and with appropriate graphics and tables. This mid-term evaluation brief will be prepared after the main report has been approved. 61. Phase 5: Final Evaluation (November 2022-June 2023): This will be led by the evaluation team, and will build on the previous phases by starting with a review and update of the inception report to reflect any contextual changes and incorporate lessons coming from the implementation of the programme so far. The evaluation team will deliver an updated inception report and a final evaluation report with recommendations. The WFP country office will response to these recommendations by preparing a management response with actions and timelines for implementing the recommendations. 62. The evaluation team will provide a 2-3 pages stand-alone brief describing the evaluation design, methodology, key findings, conclusions and recommendations. It will serve to inform stakeholders of the final evaluation, and should be written in a language easy to understand by non-evaluators and with appropriate graphics and tables. This final evaluation brief will be prepared after the main report has been approved. 63. Annex 5 provides a detailed evaluation schedule. This schedule will be reviewed in details during the inception phase and included as an annex in the inception report. 6. Organisation, Management and Conduct of the Evaluation 6.1. Organisation and Management 64. Evaluation Manager: The evaluation will be managed by an external evaluation manager appointed by the firm that will be contracted by WFP Country office in line with the terms and conditions of the Long-term agreement for provision of evaluation services. The manager must have prior experience WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 13 | P a g e in managing similar evaluations and a deep understanding of evaluation standards in general and WFP DEQAS in particular. She/he must be able to dedicate time to perform all the tasks of an evaluation manager as outlined in the DEQAS and summarised in the evaluation schedule. The external evaluation manager will be answerable to the evaluation committee. 65. Evaluation Focal Point: To ensure smooth coordination between the external evaluation manager, WFP internal stakeholders and other in￾country stakeholders, the WFP country director will appoint a staff who is not part of the day-to-day implementation of the school meals programme as evaluation focal point (EFP). Preferably this should be a member of the country office M&E team. The EFP will be answerable to the evaluation committee chair who will be the WFP deputy country director. 66. The EM and EFP will work together to ensure that the appropriate safeguards for impartiality and independence of the evaluation are applied throughout the process. The WFP regional evaluation officer will provide additional support to WFP staff in this regard. The structure above shows how the evaluation management will be structured. This structure will be maintained throughout the 5-year process. 6.2. Evaluation Conduct 67. The evaluation will be conducted by a team of independent consultants who will not have been involved in the design or implementation of the school feeding programme or have any other conflicts of interest. Potential conflicts of interest will be assessed prior to hiring of the evaluation team. All team members will be expected to sign the code of conduct for evaluators, act impartially and respect the code of conduct of the evaluation profession. The conduct of the evaluation will be guided by the evaluation schedule in Annex 5, ensuring that deliverables are available within the planned timeline. 6.3. Team composition and competencies 68. Team composition: The evaluation team will be composed of 3 consultants, including the team leader. The team will include a mix of national and international evaluators, be gender-balanced, geographically and culturally diverse with appropriate skills to assess gender dimensions of the subjectas specified in the scope, approach and methodology sections of these TOR. At least one team member should have experience in evaluating WFP work, preferably in Congo. 69. Team Competencies: The team will be multi-disciplinary and include members who together bring an appropriate balance of expertise and practical knowledge in the following areas:  Education policies and programmes, including school feeding;  Nutrition-sensitive programming, including nutrition education and linkages with education;  Capacity development, especially in education and health;  Gender expertise/good knowledge of gender issues in education and health;  Evaluation methods, specifically use of mixed methods (quantitative and qualitative). 70. All team members should have strong analytical and communication skills, evaluation experience and familiarity with the republic of Congo. The working languages for this evaluation will be English and French. The evaluation team should collectively have excellent oral and written French. The evaluation products will be prepared in English and once approved will be translated to French. The cost of translation should be included in the evaluation budget. 71. The team leader will have technical expertise in some of the areas listed above as well as expertise in designing evaluation methodology and data collection tools. He/she must have demonstrated experience in leading similar evaluations. She/he will also have leadership, analytical and communication skills, including a record of accomplishment in writing and presenting in French. WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 14 | P a g e 72. Her/his primary responsibilities will be: i) defining the evaluation approach and methodology; ii) guiding and managing the team; iii) leading the evaluation mission and representing the evaluation team; iv) drafting and revising, as required, the inception report, the end of field work (i.e. exit) debriefing presentation and evaluation report in line with DEQAS. 73. The team members will bring together a complementary combination of the technical expertise required and have a record of accomplishment of written work on similar assignments and in French. They will: i) contribute to the methodology in their area of expertise based on a document review; ii) conduct field work; iii) participate in team meetings and meetings with stakeholders; iv) contribute to the drafting and revision of the evaluation products in their technical area(s). 6.4. Ethical Considerations 74. The evaluation must be conducted in line with the UNEG ethical guidelines. This will include: respect for dignity and diversity; fair representation of the views of different stakeholders; compliance with ethics in research involving young children and/or vulnerable groups; confidentiality; avoidance of harm and appropriate referrals in situations of risk/protection concerns. During the design of evaluation at inception, specific safeguards must be put in place to protect the safety (physical and psychological) of respondents and those collecting the data. Data collection tools must be designed to be culturally (and age) appropriate. Data collection visits must be planned in collaboration with the relevant stakeholders and organized at the appropriate time and place to minimize risk or inconvenience to respondents. 75. Informed Consent and contact with children/vulnerable groups: Data collection training must include research ethics including how to ensure that all participants are fully informed about the nature and purpose of the evaluation and their involvement. Only participants who have given informed written or verbal consent should be included in the study. Noting that this evaluation includes possible contact with children, women and other vulnerable groups (e.g. indigenous people) recruitment of data collectors should assess suitability to work with these groups within the Congo context. With respect to involvement of children, this guidance is useful when training the data collection staff. Reports should not bear names of respondents and qualitative data must be reported in a way that will not identify respondents. 76. Comparison groups: As noted earlier, the targeting of school meals program (districts or schools within districts) is not random. If methodology will include use of comparison groups of districts, schools or individuals not targeted by the program, there should be considerations on whether and how the participants will be informed about the program, explanations of why they are not targeted, the purpose of the evaluation and why they are being contacted. This should be discussed during the training of data collection staff and potential risks/issues and mitigation measures identified prior to start of data collection. 6.5. Security Considerations 77. As noted under the risks, there are some security concerns in some of the areas where WFP will be implementing the school meals program especially the Pool region. The contracted firm will have to keep contact with WFP in Brazzaville to monitor any changes that may affect the conduct of the evaluation. Any implications should be discussed and documented as appropriate. 78. Security clearance where required is to be obtained from Brazzaville. As an ‘independent supplier’ of evaluation services to WFP, the evaluation company is responsible for ensuring the security of all persons contracted, including adequate arrangements for evacuation for medical or situational reasons. The consultants contracted by the evaluation company do not fall under the UN Department of Safety & Security (UNDSS) system for UN personnel. 79. However, to avoid any security incidents, the Evaluation Manager will ensure that: • WFP CO registers the team members with the Security Officer on arrival in country and arranges a security briefing for them to gain an understanding of the security situation on the ground. • The team members observe applicable UN security rules and regulations such e.g. curfews. WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 15 | P a g e 7. Roles and Responsibilities of Stakeholders 80. The WFP Congo Country Office Management (Director/Deputy Director) will take responsibility to: contract an independent firm to manage and conduct the evaluation; establish the internal evaluation committee (EC) and the evaluation reference group (ERG); appoint Evaluation Focal Pointfor the evaluation; approve the final TOR, inception, baseline, mid-term and final evaluation reports; ensure the independence and impartiality of the evaluation at all stages; participate in discussions with the evaluation team on the evaluation design and the evaluation subject, its performance and results with the Evaluation Manager and the evaluation team; organise and participate in internal and external debriefings; and oversee dissemination and follow-up processes, including the preparation of a Management Response to the evaluation recommendations; 81. The External Evaluation Manager, who will be answerable to the evaluation committee will: • Manage the evaluation process through all phases in accordance with DEQAS16 • Ensure quality assurance mechanisms are operational • Consolidate and shares comments on draft inception and evaluation reports with the evaluation team • Ensure expected use of quality assurance mechanisms (checklists, quality support • Ensure that the team has access to all documentation and information necessary to the evaluation; facilitates the team’s contacts with local stakeholders; sets up meetings, field visits; provides logistic support during the fieldwork; and arranges for interpretation, if required. • Organise security briefings for the evaluation team and provides any materials as required. 82. The WFP Evaluation Focal Point, who will be answerable to the evaluation committee, will coordinate between the external evaluation manager and internal WFP stakeholders including briefing the external EM on WFP internal processes relevant to the evaluation. He/she will also support the external EM with any tasks related to evaluation management that cannot be accomplished remotely such as delivering invitations to stakeholders for meetings, organising meeting rooms, collecting any signed documents etc. Before contracting is completed, the differences between the roles of external EM and the WFP focal point will be outlined in detail. 83. An internal Evaluation Committee has been established as part of ensuring the independence and impartiality of the evaluation. The committee comprises the WFP deputy country director, head of the program, M&E and VAM. The EC will oversee the evaluation process, by making decisions, giving advice to the evaluation manager and clearing evaluation products submitted to the EC Chair for approval. Annex 5 provides the list of members of the committee. 84. An Evaluation Reference Group has been established, composed of the members of the evaluation committee mentioned above, representatives from relevant government ministries, key project partners, and other relevant stakeholders, including USDA and WFP Regional Bureau and OEV. The ERG members will review and comment on the draft evaluation products and act as key informants in order to further safeguard against bias and influence. Annex 3 provides a list of the ERG members. The ERG members will review and comment on the draft evaluation products and act as key informants in order to further safeguard against bias and influence. 85. The WFP country office staff will brief the evaluation team; gather and share relevant documents and data for desk review; assist with field visit preparation and logistics; act as key informants during the field work; provide feedback on draft TORs, inception and evaluation reports; attend debriefing sessions; disseminate evaluation reports; consult with major stakeholders regarding evaluation findings; and use the evaluation findings in the implementation of the program. 86. The Regional Bureau, through the regional evaluation officer (REO) will provide technical support throughout the evaluation process to ensure that the evaluation is conducted in line with appropriate guidelines and the provisions for impartiality are upheld. In addition, relevant RB staff will: 16 The DEQAS under each step explains what the EM should do, and with whom she/he should coordinate. It is expected that the external EM will follow these steps which are distinct from the responsibilities of the contracted firm WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 16 | P a g e  Be active members of the ERG to provide expert advisory;  Participate in discussions with the evaluation team on the evaluation design and on the evaluation subject as relevant;  Provide comments on the draft TOR, Inception, baseline, mid-term and final Evaluation reports;  Review and comment on the Management Response to the evaluation and track the implementation of the recommendations. 87. WFP HQ school feeding unit will:  Discuss with the evaluation team WFP strategies, policies or systems in in relation to school feeding;  Comment on the evaluation TOR, inception and evaluation reports, as required. 88. Government, NGOs and UN agencies Partners will, through their membership in the ERG, review and comment on draft evaluation products (TOR, inception, mid-term and final evaluation report), attend briefing and debriefing meetings; and be interviewed as key informant interviews. 89. The Office of Evaluation (OEV), through the Regional Evaluation Officer, will advise the Evaluation Manager and provide support to the evaluation process as/when required. OEV will provide access to the outsourced quality support service that will review and provide feedback on draft ToR, inception and evaluation reports from an evaluation perspective. It accomplish this role by maintaining a functional help desk function available to the country office and RB. 90. The independent and external Evaluation Team will be responsible for the conduct of the baseline study, mid-term and final evaluations. The team will be responsible for document review, design of surveys including sampling; conducting all fieldwork, including quantitative surveys, focus groups, etc; drafting and finalizing evaluation report with findings and recommendations; 91. United States Department of Agriculture (USDA) will be involved in the evaluation throughout all the phases, starting with the approval of these TOR. Relevant staff members of USDA (Program Analyst and M&E Lead) review of the Terms of Reference; serve as a member of the Evaluation Reference Group, and participate in stakeholder meetings, be interviewed as key informants and participate in the presentation of the evaluation findings; 92. The WFP Partnerships Officer (Washington Office) will work closely with the WFP CO, RB, OEV and the USDA to ensure smooth communication and submission of key evaluation deliverables, according to project timelines. The Partnerships Officer will review evaluation deliverablesfor adherence to USDA policy and facilitate communication with USDA; Provide feedback on draft TORs and draft evaluation report; coordinate with USDA to seek feedback of TORs, inception and evaluation reports; share evaluation findings and discuss the management response; Disseminate evaluation reports and findings to relevant stakeholders. 93. Beneficiaries, including boys, girls, men and women (teachers, administrators) in targeted districts and schools will be key participants in the evaluation to provide feedback and information regarding the program. Depending on the nature of findings and recommendations from the evaluations, they may be responsible for taking action to implement those recommendations. 8. Communication and budget 8.1. Communication 94. A draft communication plan is outlined in Annex 8. The Evaluation manager, in consultation with the evaluation committee, will develop the communication and learning plan to detail the processes and channels of communication and responsibilities. The evaluation manager will be responsible for: • Sharing all draft products including TOR, inception report and evaluation report with internal and external stakeholders to solicit their feedback; The communication will specify the date by when the feedback is expected and highlight next steps; • Documenting systematically how stakeholder feedback has been used in finalised the product, ensuring that where feedback has not been used a rationale is provided; WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 17 | P a g e • Informing stakeholders (through the ERG) of planned meetings at least one week before and where appropriate sharing the agenda for such meetings; • Informing the team leader in advance the people who have been invited for meetings that the team leader is expected to attend/present and sharing the agenda; • Sharing final evaluation products (TOR, inception and Evaluation report) with all internal and external stakeholders for their information and action as appropriate; 95. To ensure a smooth and efficient process and enhance the learning from this evaluation, the evaluation team will place emphasis on transparent and open communication with all key stakeholders. The evaluation team leader will be responsible for: • Discussing with the evaluation manager additional communication and learning strategies; • Communicating the rationale for the evaluation design decisions (sampling, methodology, tools) in the inception report; • Working with the evaluation manager to ensure a detailed evaluation schedule is communicated to stakeholders before field work starts, and it is annexed to the inception report; • Sharing a brief PowerPoint presentation prior to the internal and external debriefings to enable stakeholders joining the briefings remotely to follow the discussions; • Including in the final report the list of people interviewed, as appropriate (bearing in mind confidentiality and protection issues)17; • Systematically considering all stakeholder feedback when finalising the evaluation report, and transparently provide rationale for feedback that was not used; 96. As part of the international standards for evaluation, WFP requires that all evaluations are made publicly available following the approval of the final evaluation report; and the links circulated to key stakeholders as appropriate. The evaluation manager will be responsible for sharing the final report and the management response with the regional evaluation officer, who will upload it in the appropriate systems. OEV will upload the final products on the WFP intranet and public website. 8.2. Budget 97. This evaluation will be funded from the M&E budget line as outlined in the approved budget for MGD program. The evaluation will be contracted to a firm with whom WFP has a long-term agreement for provision of evaluation services. The actual budget will be determined by agreed rates for the firm that will provide the most financially competitive and technically sound proposal. The proposed budget should include all data collection activities, including transport, field-level research assistants and translation. More discussions on these elements may be held with the firms prior to their submission of technical and finalise proposals if required. For more information, please send any queries to:  Gisele GALESSAMI, Programme Officer, WFP Congo gisele.galessami@wfp.org,  Corneille OKO, Programme Officer, WFP Congo corneille.oko@wfp.org  Koffi AKAKPO, Deputy Country Director, WFP Congo, koffi.akakpo@wfp.org 17 For example, omitting names of people where appropriate, and instead stating the name of the organisation WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 18 | P a g e Annex 1 Map of Targeted Areas WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 19 | P a g e Annex 2 McGovern Dole Results Framework WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 20 | P a g e WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 21 | P a g e Annex 3 Preliminary Stakeholder analysis (Interests, uses, means of engagement) Stakeholder Name What are interest in the evaluation and likely uses of evaluation report to this stakeholder How will they be involved in the evaluation process and what are the means of engagement? INTERNAL STAKEHOLDERS WFP Country Office (CO) Republic of Congo Responsible for the country level planning and implementation of the programme, WFP CO is the primary stakeholder and has a direct stake in the evaluation and an interest in learning from experience to inform decision-making and adjustments for better results. WFP CO also is expected to account internally as well as to externally to the donor, beneficiaries and partners for performance and results of its this programme; The CO management will commission the evaluation and oversee its conduct. The CD/DCD will be briefed by the team at the start of the evaluation, and one of them will chair the evaluation committee and reference group, through which they will oversee the process. The WFP country office staff will brief the independent evaluation team; gather and share relevant documents and data for desk review; assist with field visit preparation and logistics; act as key informants during the field work; provide feedback on draft TORs, inception and evaluation reports; attend debriefing sessions; disseminate evaluation reports and consult with major stakeholders regarding evaluation findings. WFP Regional Bureau (RB) Johannesburg RB is responsible for both oversight of COs and technical guidance and support. Through the Regional Evaluation Officer (REO), the RB supports the country offices to ensure quality, credible and useful evaluation. As such, the RB management and staff have an interest in having an independent and impartial account of the operational performance as well as in learning from the evaluation findings to apply this learning to other country offices. The REO will provide technical support throughout the evaluation process. She will be consulted on technical aspects of the process, including on appropriate application of both WFP and USDA policies to this evaluation. The RB programme staff specifically those in charge of nutrition, school feeding and social protection will be engaged through the evaluation reference group, and will provided an opportunity to review draft evaluation products and attend debriefing meetings WFP evaluation committee (EC) (temporary mechanism) Internal Evaluation Committee (EC) will be formed as part of ensuring the independence and impartiality of the evaluation; it will be composed of key CO staff and the Regional Evaluation Officer. The EC has an interest in ensuring that the evaluation process remains as impartial as possible, while making efficient use of available resources (human and financial). The EC also has an interest in ensuring that the evaluation meets the expectations of the key stakeholders including USDA, WFP and sub￾recipients of the MGD funding. The EC will oversee the evaluation process, by making decisions, giving advice to the evaluation manager and clearing evaluation products submitted to the EC Chair for approval. The EC will therefore be the main body supervising the conduct of the evaluation, and providing the evaluation team with support and direction. WFP HQ Social protection and safety nets division, and specifically school feeding Other HQ divisions including performance monitoring (RMP) WFP HQ social protection and safety net division is responsible for issuing and overseeing the rollout of normative guidance on corporate programme themes, activities and modalities, as well as of overarching corporate policies and strategies related to the areas covered by this evaluation. They have an interest a credible account of the extent to which the appropriate normative guidance is applied in the conduct of WFP work in Congo and the results. They also have an interest in the lessons that emerge from this evaluation, as they may have relevance beyond the Congo. The staff of the HQ division will be given an opportunity to join debriefing sessions and review and comment on evaluation products. Depending on the nature of recommendations that emerge from the evaluation, other divisions will be consulted during the preparation of the management response, and asked to respond to any recommendations directly targeted at HQ. Office of Evaluation (OEV) OEV has a stake in ensuring that all evaluations in WFP are credible, of quality and useful. It is responsible for setting the normative framework and guidance for decentralized evaluations including a comprehensive Decentralized Evaluation Quality Assurance System (DEQAS). It therefore has an interest in ensuring this evaluation adheres to the OEV operates a help desk that is accessible to the country office and RB throughout the evaluation process. The help desk will be consulted on any issues related to application of the normative framework as appropriate. OEV also manages an independent quality support (QSS) service that reviews draft evaluation products and provides feedback for further improvement. The draft TOR, inception and final report for this evaluation will be submitted to QSS for review and feedback. WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 22 | P a g e normative framework and guidance and that the evaluation is credible of good quality and useful. WFP Washington office The WFP Washington Office coordinates communication between WFP and USDA on all matters related to the funding and implementation of the programme that is the subject of this evaluation. The office therefore has an interest in ensuring that this evaluation, which is part of the agreement between the WFP and USDA is commissioned and conducted as per that agreement, and that it meets the expectations of USDA. The Partnerships Officer (Washington Office) will work closely with the WFP CO, RB, OEV and the USDA to ensure smooth communication and submission of key evaluation deliverables, according to project timelines and the agreement. The Partnerships Officer will review evaluation deliverables for adherence to USDA policy and facilitate communication with USDA; he/she will provide feedback on draft TORs and draft evaluation report; coordinate with the donor (USDA) to seek feedback on TORs, inception and evaluation reports; share evaluation findings and discuss the management response to evaluation recommendations; WFP Executive Board (EB) The WFP governing body has an interest in being informed about the effectiveness of all WFP operations, as well as progress in the implementation of the evaluation policy. This is closely linked with the involvement of OEV in this evaluation as outlined above, as it has the responsibility of reporting to the EB. While this evaluation will not be presented to the EB, its findings may feed into evaluation syntheses and other corporate learning processes. Furthermore, it will contribute to the contents of the annual evaluation report that will be presented to the board. EXTERNAL STAKEHOLDERS Beneficiaries (boys, girls, women and men); teachers; members of the school management committees (SMCs); Parent teacher Associations (PTAs) and other education administrators As the ultimate recipients of food assistance and capacity building efforts, beneficiaries have a stake in WFP determining whether its assistance is appropriate and effective. As such, the participation in the evaluation of women, men involved in the education sector such as teachers, administrators and parents; and boys and girls will be important. As part of commitment to affected population, which stipulates that people should be involved in decisions and actions that affect them, the evaluation will make deliberate effort to involve the beneficiaries in evaluation process. The means by which this will be done will be determined at inception phase. This involvement should not only be during the conduct of evaluation (collecting data from beneficiaries) but should also include dissemination of the findings from this evaluation. The evaluation team will determine means of engaging beneficiaries in the dissemination of the results. United States Department of Agriculture (USDA) As the funder of the program being evaluated, USDA’s interest is to ensure that the evaluation provides an independent, credible and useful account of the performance of the program; while accounting for the resources it has provided to WFP. Relevant staff members of USDA (Program Analyst and M&E Lead) were consulted for approval of the Evaluation Plan, which laid the framework for this evaluation. USDA staff will review the Terms of Reference; serve as a member of the Evaluation Reference Group, and participate in stakeholder meetings and presentation of the evaluation findings as appropriate; Government (Ministries of education, agriculture, health and population; social affairs) Government of Congo through its various ministries and institutions has a direct interest in knowing whether WFP activities in the country are aligned with its priorities, harmonised with the action of other partners and meet the expected results towards achievement of national development objectives. The Government also has an interest in knowing the extent to which the interventions are sustainable or likely to be sustainable over time. Specifically, The ministries of education, agriculture and health are partners in the design and implementation of the programme, and will be interested in knowing the extent to which the program is contributing to the objectives in their respective mandates and drawing lessons. Key staff from the ministries of education, agriculture and health will be members of the evaluation reference group to ensure systematic engagement of the Government in the evaluation process. They will attending team briefing and debriefing meetings, review and comment on evaluation draft products (inception and evaluation reports), and be consulted on the responses to evaluation recommendations as appropriate. Some of them will be interviewed as key informants during the data collection process. They will receive the final evaluation report. As a key stakeholder, they will be consulted when preparing the management response to the recommendations. Sub-recipients of MGD United Nations agencies; NGOs UNICEF, UNESCO, ACTED (sub-recipients of the MGD funding) and the World Bank are key WFP partners in the implementation of the programme. As such, they have a direct interest Representatives from these agencies will be members of the evaluation reference group to ensure that they are systematically engaged throughout the evaluation process. They will attend briefing and debriefing meetings, be WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 23 | P a g e and the World Bank in the evaluation process as it relates to the performance of the specific aspects of the program under their responsibilities. interviewed as key informants and review and comment on draft evaluation products. They will receive the final evaluation report. As a key stakeholder, they will be consulted when preparing the management response to the recommendations. NGOs partners Field implementation of the program involve NGOs including Association des Spiritains au Congo (ASPC), Pioneer Hospital, Initiative Development; Partnership for child Child Development (PCD), Autochthone’s advocacy group. These partners have a direct interest in the process of the evaluation as well as the findings given that the results of the evaluation may influence future implementation modalities, strategic orientations and partnerships. Representatives from these partners will be members of the evaluation reference group to ensure that they are systematically engaged throughout the evaluation process. They will attend briefing and debriefing meetings, be interviewed as key informants and review and comment on draft evaluation products. They will receive the final evaluation report. If there are recommendations that related to their responsibility in the implementation of the programme, they will be consulted when preparing the management response to the recommendations. Donors Japan In addition to USDA funding, the Government of Japan and the host government of Congo provide complementary support to supplement the provision of school meals to schoolchildren. They have interest in knowing whether their funds have been spent efficiently and if WFP’s work has been effective and contributed to the intended objectives. These donors will be consulted as key informants, given the opportunity to attend debriefing meetings and to review and comment on draft evaluation products. They will receive the final evaluation report. If there are recommendations that related to their responsibility in the implementation of the programme, they will be consulted when preparing the management response to the recommendations. Private sector actors To deliver the programme, WFP will work with private-sector entities such as NG Enterprise, Minoco ,CIB, Likouala Timber, ENI congo These actors will be consulted as key informants and given the opportunity to attend debriefing meetings. If there are recommendations that related to their responsibility in the implementation of the programme, they will be consulted when preparing the management response to the recommendations. Annex 4 Performance Indicators and Targets # Activity Performance Indicator Targets Per Year Life of 2018 2019 2020 2021 2022 Project 1 Improve Student Enrolment Number of students regularly (80%) attending USDA supported classrooms/schools 43,000 45,150 47,408 49,778 52,267 92,920 2 Number of students enrolled in school receiving USDA assistance 53,750 56,438 59,259 62,222 65,333 102,651 3 Distribute Food Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance 9,675000 9,675,00 0 9,675,00 0 9,675,00 0 9,675,00 0 48,375,00 0 4 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance 53,750 53,750 53,750 53,750 53,750 96,750 5 Number of individuals benefiting directly from USDA-funded interventions 56,261 55,950 55,800 55,800 55,800 108,061 6 Number of individuals benefiting indirectly from USDA-funded interventions 268,750 268,750 268,750 268,750 268,750 483,750 7 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance 53,750 53,750 53,750 53,750 53,750 96,750 8 Support Improved Safe Food Preparation and Storage Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/ constructed as a result of USDA assistance 18 18 7 7 7 57 9 Number of schools using an improved water source 40 74 82 86 91 91 WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 24 | P a g e 10 Promote Improved Health Number of schools with improved sanitation facilities 29 58 65 72 79 79 11 Number of students receiving deworming medication(s) 53,750 53,750 53,750 53,750 53,750 96,750 12 Number of individuals trained in safe food preparation and storage as a result of USDA assistance 960 960 960 960 960 4,800 13 Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance 624 672 720 768 816 3,600 14 Promote Improved Nutrition Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance 470 470 470 470 470 470 15 Number of individuals trained in child health and nutrition as a result of USDA assistance 591 440 290 290 290 1,901 16 Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance 384 308 218 232 247 1,388 17 Support Improved Literacy Number of textbooks and other teaching and learning materials provided as a result of USDA assistance 50,000 10,000 12,000 14,000 17,000 103,000 18 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text 50% 60% 70% 80% 80% 80% 19 Number of teachers/educators/ teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance 120 0 0 0 0 120 20 Number of teachers/ educators/ teaching assistants trained or certified as a result of USDA assistance 160 0 0 0 0 160 21 Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance 520 560 600 640 680 3,000 22 Number of school administrators and officials trained or certified as a result of USDA assistance 800 800 800 800 800 4,000 23 Build Capacity Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance: Stage 1: Analysed Stage 2: Drafted and presented for public/stakeholder consultation Stage 3: Presented for legislation/decree Stage 4: Passed/Approved Stage 5: Passed for which implementation has begun 1 0 0 0 0 1 24 Number of child health and nutrition policies, regulations, or administrative procedures in each of the following stages of development as a result of USDA assistance: Stage 1: Analysed Stage 2: Drafted and presented for 0 0 0 0 1 1 WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 25 | P a g e public/stakeholder consultation Stage 3: Presented for legislation/ decree Stage 4: Passed/Approved Stage 5: Passed for which implementation has begun 25 Establish/Str engthen Local Agriculture and school communities to promote Graduation Value of new public and private sector investments leveraged as a result of USDA assistance 0 0 100,000 0 0 100,000 Annex 5 Evaluation Schedule (Baseline, Mid-term and Final Evaluations) # Phases, Deliverables and Timeline Key Dates By Who Phase 1-Preparation 1 Document review and draft TOR as per approved evaluation plan Sept/Oct 17 CO with RB 2 Submit draft TOR to the outsourced quality support service (QS) 27th Oct 17 CO EFP 3 Receive and review feedback from QS, and discuss with RB if necessary 2nd Nov 2017 CO EFP 4 Review draft TOR based on DE QS feedback to produce draft 2 10th Nov 2017 CO EFP 5 Circulate draft 2 of TOR for review and comments to ERG and other stakeholders 13th–20th Nov 2017 CO EFP 6 Review draft 2 of TOR based on stakeholders comments 20th Dec 2017 CO EFP 7 Approve the final TOR 21st Dec 2017 EC 8 Submit Final TOR to USDA for approval 22nd Dec -5th Jan USDA 9 Share the final approved TOR with key stakeholders, and with LTA firms 8th Jan 2018 CO EFP 10 Request for proposals from firms with LTA with WFP 8th- 15th Jan 2018 CO EFP 11 Select and recruit evaluation team (including raising the purchase order) 27th Jan 2018 EM Phase 2 - Inception Key Dates By Who 12 Brief the evaluation team on expectations, the TOR and process 29th Jan 2018 EM+CO EFP 13 Desk review, evaluation design and drafting of the inception report 30th Jan-13th Feb ET 14 Submit draft 1 of the inception report (IR) to the EM 14th Feb 2018 TL 15 Review draft 1 of the IR and if it is complete submit to QS 15th Feb 2018 EM 16 Receive and review QS feedback and submit to the evaluation team 21st Feb 2018 EM 17 Revise draft 1 of IR based on QS feedback and produce draft 2 IR 22nd Feb -1st Mar ET 18 Submit draft 2 of IR to the evaluation manager 2nd Mar 2018 19 Circulate draft 2 IR for review and comments to ERG and other stakeholders 2nd -8th Mar 2018 EM 20 Consolidate stakeholder comments and submit to evaluation team 8th Mar 2018 EM 21 Revise draft 2 IR based on stakeholder comments & produce draft 3 10th -17th Mar 2018 ET 22 Submit draft 3 (final) IR to the evaluation manager 18th Mar 2018 TL 23 Review and if OK Submit the final IR to the evaluation committee for approval 19th Mar 2018 EM 24 Approve the inception report 23rd Mar 2018 EC 25 Share final inception report with key stakeholders 23rd Mar 2018 WFP CO Phase 3–Baseline Study Key Dates By who 26 Brief evaluation team and train data collection assistants 26th Mar 2018 TL 27 Collect data (primary + secondary for all indicators) 27th Mar-17th Apr ET 28 Data analysis + drafting of the baseline report 20th Apr-1st May ET 29 Submit Draft 1 of the baseline report to the EM 2nd May 2018 TL 30 Review draft 1 of baseline report and if complete submit to QS 3rd May 2018 EM 31 Receive the QS feedback and forward to the evaluation team 9th May 2018 32 Revise baseline report based on QS feedback and produce draft 2 16th May 2018 ET 33 Submit revised baseline report to evaluation manager 17th May 2018 TL 34 Circulate draft baseline report for review and comments to ERG and other stakeholders 18th -25th May 2018 EM 35 Consolidate stakeholder comments and submit to team leader 26th May 2018 EM 36 Revise draft baseline report based on stakeholder comments 27th– 3rd Jun 2018 ET 37 Submit final revised Baseline Report to the evaluation manager 4th Jun 2018 TL 38 Submit BR to the evaluation committee for approval 5th Jun 2018 EM 39 Approve the final baseline report 8th Jun 2018 EC WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 26 | P a g e 40 Submit final Baseline Report to USDA 11th Jun 2018 WFP CO 41 Share final Baseline Report with key stakeholders 24th Jun 2018 WFP CO 42 Prepare management response to the recommendations 24th Jun 2018 WFP CO 43 Review and provide feedback on the management response 30th Jun 2018 WFP RB 44 Finalize the management response based on RB comments 10th July 2018 WFP RB 45 Share the final BR and MR with OEV for publication 15th July 2018 RB Phase 4–Mid-term Evaluation Key Dates By who 46 Assemble evaluation team (same team that conducted the baseline study) Nov 2020 EM 47 Review documents, baseline report and data sets and monitoring reports 1st -15th Dec 2020 ET 48 Review and update the Inception Report that was prepared during baseline 15th - 30th Dec 2020 ET 49 Submit draft 1 of updated inception report (IR) to the EM 31st Dec 2020 TL 50 Review draft 1 of the IR and if it is complete submit to QS 3rd Jan 2021 EM 51 Receive and review QS feedback and submit to the evaluation team 9th Jan 2021 EM 52 Revise draft 1 of IR based on QS feedback and produce draft 2 IR 10th - 15th Jan 2021 ET 53 Submit draft 2 of IR to the evaluation manager 16th Jan 2021 54 Circulate draft 2 IR for review and comments to ERG and other stakeholders 17th -24th Jan 2021 EM 55 Consolidate stakeholder comments and submit to evaluation team 26th Jan 2021 EM 56 Revise draft 2 IR based on stakeholder comments & produce draft 3 27th - 31st Feb 2021 ET 57 Submit draft 3 of IR to the evaluation manager 1st Feb 2021 TL 58 Submit the final IR to the evaluation committee for approval 2nd Feb 2021 EM 59 Approve the inception report 3rd Feb 2021 EC 60 Share final inception report with key stakeholders 4th Feb 2021 WFP CO 61 Prepare for field work 5th Feb 2021 TL 62 Brief evaluation team and train data collection assistants 5th Feb 2021 TL 63 Collect data (primary + secondary for all indicators) 6th - 19th Feb 2021 ET 64 Data analysis + drafting of the mid-term evaluation (MTE) report 20th Feb -4th Mar ET 65 Submit Draft 1 of the MTE report to the EM 5th Mar 2021 TL 66 Review draft 1 of MTE report and if complete submit to QS 7th Mar 2021 EM 67 Revise MTE report based on QS feedback and produce draft 2 14th Mar 2021 ET 68 Submit revised MTE report to evaluation manager 15th Mar 2021 TL 69 Circulate draft MTE report for review and comments to ERG and other stakeholders 16th -23rd Mar 2021 EM 70 Consolidate stakeholder comments and submit to team leader 26th Mar 2021 EM 71 Revise draft MTE report based on stakeholder comments 27th – 1st 2021 ET 72 Submit of final revised MTE report to the evaluation manager 2nd Apr 2021 TL 73 Submit MTE report to evaluation committee for approval 5th Apr 2021 EM 74 Approve the final MTE report 9th Apr 2021 EC 75 Submit final MTE to USDA 10th Apr 2021 WFP CO 76 Share final MTE report with key stakeholders 24th Apr 2021 WFP CO 77 Prepare management response to the recommendations 24th May 2021 WFP CO 78 Review and provide feedback on the management response 30th May 2021 WFP RB 79 Finalize the management response based on RB comments 10th Jun 2021 WFP RB 80 Share the final MTE report and MR with OEV for publication 15th June 2021 RB Phase 5–Final Evaluation Key Dates By who 81 Assemble evaluation team (same team that conducted mid-term evaluation) Nov 2022 EM 82 Review documents, baseline report and data sets and monitoring reports 1st -15th Dec 2022 ET 83 Review and update the Inception Report that was prepared during MTE 15th - 30th Dec 2022 ET 84 Submit draft 1 of updated inception report (IR) to the EM 31st Dec 2022 TL 85 Review draft 1 of the IR and if it is complete submit to QS 3rd Jan 2022 EM 86 Receive and review QS feedback and submit to the evaluation team 9th Jan 2022 EM 87 Revise draft 1 of IR based on QS feedback and produce draft 2 IR 10th - 15th Jan 2022 ET 88 Submit draft 2 of IR to the evaluation manager 16th Jan 2022 89 Circulate draft 2 IR for review and comments to ERG and other stakeholders 17th -24th Jan 2022 EM 90 Consolidate stakeholder comments and submit to evaluation team 26th Jan 2022 EM 91 Revise draft 2 IR based on stakeholder comments & produce draft 3 27th - 31st Feb 2022 ET 92 Submit draft 3 of IR to the evaluation manager 1st Feb 2022 TL 93 Submit the final IR to the evaluation committee for approval 2nd Feb 2022 EM 94 Approve the inception report 3rd Feb 2022 EC 95 Share final inception report with key stakeholders 4th Feb 2022 WFP CO 96 Prepare for field work 5th Feb 2022 TL WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 27 | P a g e 97 Brief evaluation team and train data collection assistants 5th Feb 2022 TL 98 Collect data (primary + secondary for all indicators) 6th - 19th Feb 2022 ET 99 Data analysis + drafting of the final evaluation report 20th Feb -4th Mar ET 100 Submit Draft 1 of the final evaluation report to the EM 5th Mar 2022 TL 101 Review draft1 of final evaluation report and if complete submit to QS 7th Mar 2022 EM 102 Revise final evaluation report based on QS feedback and produce draft 2 14th Mar 2022 ET 103 Submit revised final evaluation report to evaluation manager 15th Mar 2022 TL 104 Circulate draft evaluation report for review and comments to ERG members 16th - 23rd Mar 2022 EM 105 Consolidate stakeholder comments and submit to team leader 26th Mar 2022 EM 106 Revise draft final evaluation report based on stakeholder comments 27th – 1st 2022 ET 107 Submit of final revised final evaluation report to the EM 2nd Apr 2022 TL 108 Submit final evaluation report to the EC for approval 5th Apr 2022 EM 109 Approve the final evaluation report 9th Apr 2022 EC 110 Submit final evaluation report to USDA 10th Apr 2022 WFP CO 111 Share final evaluation report with key stakeholders 24th Apr 2022 WFP CO 112 Prepare management response to the recommendations 24th May 2022 WFP CO 113 Review and provide feedback on the management response 30th May 2022 WFP RB 114 Finalize the management response based on RB comments 10th Jun 2022 WFP RB 115 Share the final evaluation report and MR with OEV for publication 15th Jun 2022 RB 116 Publish the Evaluation report and MR and share the links 30th Jun 2020 OEV Annex 6 Membership of the Evaluation Committee The Internal Evaluation Committee for this evaluation will be composed of the following: 1. Koffi AKAKPO, DCD: (Chair of the evaluation committee) 2. TBC: Evaluation Manager [Name to be confirmed after contracting is completed] 3. Gisele GALESSAMI, Programme Officer, McGovern Dole program 4. Corneille OKO, Programme Officer 5. Gautier MASSAMOUNA 6. Grace Igweta, WFP Regional Evaluation Officer, WFP Regional Bureau Annex 7 Membership of the Evaluation Reference Group The Evaluation Reference Group for this evaluation will be composed of the following: 1. Koffi AKAKPO, DCD: (Chair of the evaluation committee) 2. TBC: Evaluation Manager [Name to be confirmed after contracting is completed] 3. Gisele GALESSAMI, Programme Officer, McGovern Dole program 4. Corneille OKO, Programme Officer 5. Gautier MASSAMOUNA, VAM 6. Representative from UNICEF 7. Representative from UNESCO 8. Representative from ACTED 9. Representative from Ministry of Education 10. Representative from Ministry of Health; 11. Representative from Ministry of Agriculture 12. Representative from Ministry of Social affairs 13. TrixieBelle NICOLLE, School Feeding; WFP Regional Bureau 14. Anna HENTTINEN, Evaluation Consultant, WFP Office of Evaluation Annex 8 Communication and Learning Plan Internal Communication When: Evaluation phase (month/year) What: Communication product To whom: Target group or individual What: Organizational level of communication From whom How: Communication means (meeting, interaction, etc.) Why: Purpose of communication Preparation (Nov 2017) Terms of Reference (TOR) Evaluation committee (EC) Programme/technical level Evaluation focal point (EFP) Consultations, meetings and written exchanges Draft TOR for comments Final for information Inception (Dec 2017 to Jan 2018) Team Briefing + Inception Mission Inception Report Country office staff; RB staff; HQ staff Operational and management level EM + Evaluation Team Leader (TL) Written exchange; consultations on phone and in person in Brazzaville -Understand expectations, clarify design; -Draft Inception report for review and comments; -Final inception report for information Fieldwork: baseline study debrief (Feb 2018) PowerPoint presentation CO, RB, HQ, stakeholders Operational and management level TL + other team members Meeting in person and/or /Teleconference For information and verbal feedback on preliminary findings Reporting for baseline (March/April 2018) Draft baseline report CO, RB, HQ, stakeholders Operational level EM Written exchanges with reports attached (+ matrix of comments) for written comments; Final baseline report CO, RB, HQ, stakeholders EM Written exchanges with report attached for information Dissemination (May/June 2018) Management response to recommendations Final baseline report CO, RB, HQ stakeholders All levels EM Written message with the intranet and internet links to the documents Dissemination of findings, conclusions and the actions that will be taken to implement the recommendations Note: The mid-term and final evaluations will take the above as minimum communication, and adjusted based on the context at the time. External Communication When Evaluation phase What: Communication product To whom Target org. or individual What Organizational level From whom How: Communication means Why: Purpose of communication Preparation (Nov 2017) Draft TOR ERG members Operational and management; Evaluation focal point; Email with attached draft For review and comments on draft TOR Cleared TOR USDA Technical and Management Country office management Email with attached draft TOR For review and approval of TOR Final TOR ERG members and other stakeholders All levels Evaluation focal point Email with attached final TOR For information Inception (Dec 2017 to Jan 2018) Inception Mission ERG members Operational and management level Evaluation team leader + EM Written exchange; consultations on phone and in person Understand expectations, clarify design; Draft Inception report ERG members and other stakeholders Operational level EM Email with attached draft TOR + comments matrix Draft Inception report for review and comments; Final inception Report ERG members and other stakeholders Operational and management levels EM Email with attached final TOR for information Field Work (Feb 2018) PowerPoint presentation ERG members Operational level Team leader + team members Meeting in person and/or /Teleconference For information/verbal feedback on preliminary findings Reporting (Mar/Apr 2018) Draft baseline report ERG members Operational level EM Email with reports attached (+ matrix of comments) for review and written comments; Final baseline report Key Stakeholders All levels EM Email with report attached for information Dissemination (May/Jun 2018) Final report and management response to recommendations Key Stakeholders All levels EM Written message with the internet links to the documents Dissemination of findings, conclusions and the actions that will be taken to implement the recommendations WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 28 | P a g e Acronyms ACTED Agence d'Aide à la Coopération Technique Et au Développement ASPC Association des Pères Spiritains du Congo CAR Central Africa Republic CP Country Programme DEQAS Decentralized Evaluation Quality Assurance System DEV Development Project DRC Democratic Republic of Congo EFP Evaluation Focal Point EM Evaluation Manager ER Evaluation Report FAO Food and Agriculture Organisation FAS Foreign Agriculture Service GNI Gross Domestic Income GRoC Government of Republic of Congo IDPs Internally Displaced Persons IR Inception Report MGD McGovern Dole NDP National Development Plan NSFP Government National School Feeding Policy ODA Offical Development Assistance OEV Office of Evaluation, WFP ORA Observe, Reflect and Act PMP Performance Monitoring Plan RB Regional Bureau ROC Republic of Congo SABER Systems Approach for Better Education Results SUN Scaling Up Nutrition TOR Terms of Reference UNDSS United Nations Department of Safety and Security UNEG United Nations Evaluation Group UNESCO United Nations Educational, Scientific and Cultural Organization UNICEF United Nations Children's Fund USDA United States Department of Agriculture USDA Unites States Dollar WHO World Health Organisation WFP Congo Rep. McGovern Dole School Feeding Evaluation TOR: Update on 20th December 2017 29 | P a g e Annex 2. Map of intervention Annex 3. Resource requirements and funding situation The donor for the McGovern-Dole is the US Department of Agriculture through FAS for a total budget of $30,022,053. It is split as follows:  $5,620,250 for commodity cost (food provided by USDA)  $4,776,000 for international freight cost  $19,625,803 for project operations (see Figure 2) According to the agreement between USDA and WFP, the government of Congo is expected to provide iodised salt and the government of Japan128 will provide canned fish to complement the school meals. Nothing has been agreed for the moment with those partners. Additional food inputs that may materialise during the programme will be considered by the evaluation methodology the same way as USDA’s food input. Budget per recipient (see Annex 4 for further precision per activity) ACTED: $423,034.49 / UNICEF: $1,756,626.93 / UNESCO: $122,716.33 / WFP: $991,396.36 128According to Japan development plan in Congo of April 2017 “In order to improve the living conditions of the population in the Republic of Congo, Japan is expanding its assistance to improve access to primary education, basic health services, and water by combining the donation with micro-projects contributing to human security, Internships and Multilateral Cooperation by collaborating with international organisations. Japan also aims to use Non-Refundable Donations, such as a Program for Economic and Social Development and Food Aid”. Figure 2: Details of the project operating costs Annex 4. Detailed description of all MGD activities according to the project agreement between WFP and USDA Activity 1: Build capacity WFP ($129,116.23) will:  train government staff, organise workshops and provide advocacy;  Organise visit studies in order to learn from other countries. WFP envisions government staff visiting other countries that have been implementing a SABER action plan successfully in order to discuss it and learn from active HGSF programmes. Activity 2: Improve pupil enrolment Under this activity, WFP will execute subrecipient agreements with: ACTED ($13,257.78):  to include discussions of the importance of education in the training modules that it provides to farmers organisations in Bouenza. UNICEF ($38,901.85):  to develop material such as leaflets;  organise training to school committees and PTAs;  organise awareness raising sessions and radio broadcast set up. Activity 3: Establish/Strengthen local agriculture and school communities to promote graduation Under this activity, WFP ($106,908.93) will:  organise for studies in Bouenza;  organise study visit for government officials to the WFP Centre of Excellence in Brazil;  the study visit will be used to provide training and capacity building activities for the implementation of the Home-Grown School Feeding Programme. Activity 4: Distribute food Under this activity, WFP ($593,617.00) will distribute food in all the schools participating in the MGD programme. Activity 5: Promote improved health Under this activity, WFP will execute sub recipient agreements with: UNICEF ($755,504.12):  for building and rehabilitating 49 latrines in schools attended by indigenous children in Likouala;  to procure and distribute 630 handwashing kits (3 per school) in five of the six departments where WFP will implement the MGD programme. ACTED will cover Bouenza. ACTED ($258,354.64):  for the building and rehabilitating of latrines. (22 new, 22 rehabilitated);  to procure and distribute 168 handwashing kits (3 per school) in Bouenza. Each school receiving handwashing kits will have their water source/station built/rehabilitated. WFP ($92,755.19) will:  use communication material that illustrate (pictures and words) appropriate care and usage of the new or rehabilitated latrines;  ensure logistics and distribution expenses of deworming tablets provided by the government through WHO and UNICEF;  develop trainings WFP for transporters, school management committees, PTAs, school teachers and communities on commodity management, food preparation and storage. Activity 6: Support improved literacy Under this activity, WFP will execute sub recipient agreements with: UNICEF ($923,319.11):  to provide indigenous children attending ORA school with a school kit. This school kit will also be distributed to schools located in Bouenza. The school kit will be distributed to 20,000 children in Bouenza and to 5,000 children in the ORA schools in the Likouala in Year 1. For the other years, only the ORA schools will be targeted for school supplies distribution (5,000 ORA pupils in Year 2, up to 8,500 in Y5);  to train 800 school administrators in all the departments targeted by McGovern-Dole. MGD schools will select one to two school administrators to participate. In the 53 ORA schools, every school administrator (one school administrator/ORA school) will be invited to participate. UNESCO ($122,716.33):  to provide one ToT training in pedagogy and life skills at the national level for up to 160 selected teachers from MGD funded schools on nutrition education. This training is designed to include teachers for all twelve RoC departments;  to work closely with other partners for the development of the new national curriculum to ensure a balance between the need to refocus the primary curriculum on literacy, and numeracy. WFP ($28,711.33) will:  ensure the procurement of 40 mobile tablets and training to the government staff on the use of these tablets. These tablets will help monitors handover monitoring to government inspectors that will track and promote teacher attendance. Activity 7: Promote improved nutrition Under this activity, WFP will execute sub recipient agreements with: UNICEF ($38,901.85):  to develop a nutrition guide;  organise awareness campaigns on good health, nutrition and hygiene;  support the Ministry of Health to establish a nutrition monitoring system;  as well as to develop material such as leaflets, organise training to school committees and PTAs (4210 persons);  awareness raising sessions and radio broadcast set up. ACTED ($13,257.78):  to include discussions of the importance of good health and hygiene in communities. WFP ($40,287.04) will:  develop a nutrition guide;  organise awareness campaigns on good nutrition and hygiene;  support the Ministry of Health to establish a nutrition monitoring system. Activity 8: Support improved safe food storage and preparation Under this activity, WFP will execute sub recipient agreements with: ACTED ($138,164.29):  to construct/rehabilitate kitchens in 60 schools (60 schools in Bouenza department);  to construct/rehabilitate storerooms in 60 schools (22 schools in Bouenza department);  to provide efficient stoves to 60 schools in Bouenza and kitchen utensils (dishes, goblets, marmites). Annex 5.Literacy Result Framework (RF 1) Annex 6. Health, Nutrition and Dietary Practices Results Framework (RF 2) Annex 7.Foundational Results Annex 8. Outputs according to actual PMP All data has been copied/pasted from the approved PMP Standard /#Custom indicator number Performance Indicator Baseline Targets Life of project Year 1 Year 2 Year 3 Year 4 Year 5 Oct1, 2017- 30 Sep, 2018 Oct1, 2018- 30 Sep, 2019 Oct1, 2019- 30 Sep, 2020 Oct1, 2020- 30 Sep, 2021 Oct1, 2021- 30 Sep, 2022 To be confirmed during baseline evaluation (May 2018) Target for mid￾term evaluation (end 2020) Target for final evaluation (end 2022) 1 Number of pupils regularly (80%) attending USDA supported classrooms/schools 40 000 43 000 45 150 47 408 49 778 52 267 92 920 Number of pupils regularly (80%) attending USDA supported classrooms/schools (female) 20 000 21 500 22 575 23 704 24 889 26 133 46 460 Number of pupils regularly (80%) attending USDA supported classrooms/schools (male) 20 000 21 500 22 575 23 704 24 889 26 133 46 460 2 Number of textbooks and other teaching and learning materials provided as a result of USDA assistance 0 50 000 10 000 12 000 14 000 17 000 103 000 4 Number of school administrators and officials trained or certified as a result of USDA assistance 0 800 800 800 800 800 4 000 6 Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance 0 160 0 0 0 0 160 7 Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance 0 18 18 7 7 7 57 8 Number of pupils enrolled in school receiving USDA assistance 51 190 53 750 56 438 59 259 62 222 65 333 102 651 Number of pupils enrolled in school receiving USDA assistance (female) 25 595 26 875 28 219 29 630 31 111 32 667 51 325 Number of pupils enrolled in school receiving USDA assistance (male) 25 595 26 875 28 219 29 630 31 111 32 667 51 325 9 Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance 0 470 470 470 470 470 470 15 Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance 0 9 675 000 9 675 000 9 675 000 9 675 000 9 675 000 48 375 000 16 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance 0 53 750 53 750 53 750 53 750 53 750 96 750 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance (female) 0 26 875 26 875 26 875 26 875 26 875 48 375 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance (male) 0 26 875 26 875 26 875 26 875 26 875 48 375 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance (new) 0 53 750 0 0 0 11 200 64 950 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance (continuing) 0 0 56 000 56 000 56 000 56 000 67 200 17 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance 0 53 750 53 750 53 750 53 750 53 750 96 750 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (female) 0 26 875 26 875 26 875 26 875 26 875 48 375 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (male) 0 26 875 26 875 26 875 26 875 26 875 48 375 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (new) 0 53 750 0 0 0 11 200 64 950 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance (continuing) 0 0 56 000 56 000 56 000 56 000 67 200 18 Number of individuals trained in child health and nutrition as a result of USDA assistance 0 591 440 290 290 290 1 901 Number of individuals trained in child health and nutrition as a result of USDA assistance (female) 0 236 176 116 116 116 760 Number of individuals trained in child health and nutrition as a result of USDA assistance (male) 0 355 264 174 174 174 1 141 20 Number of individuals trained in safe food preparation and storage as a result of USDA assistance 0 960 960 960 960 960 4 800 Number of individuals trained in safe food preparation and storage as a result of USDA assistance (female) 0 384 384 384 384 384 1 920 Number of individuals trained in safe food preparation and storage as a result of USDA assistance (male) 0 576 576 576 576 576 2 880 22 Number of schools using an improved water source 0 40 74 82 86 91 91 23 Number of schools with improved sanitation facilities 0 29 58 65 72 79 79 24 Number of pupils receiving deworming medication(s) 0 53 750 53 750 53 750 53 750 53 750 96 750 27 Number of individuals benefiting directly from USDA -funded interventions 0 56 261 55 950 55 800 55 800 55 800 103 901 Number of individuals benefiting directly from USDA -funded interventions (female) 0 27 879 27 755 27 695 27 695 27 695 51 280 Number of individuals benefiting directly from USDA -funded interventions (male) 0 28 382 28 195 28 105 28 105 28 105 52 621 Number of individuals benefiting directly from USDA -funded interventions (new) 0 56 261 2 200 2 050 2 050 13 250 75 811 Number of individuals benefiting directly from USDA -funded interventions (continuing) 0 0 56 261 58 311 60 361 73 611 86 861 28 Number of individuals benefiting indirectly from USDA - funded interventions 0 268 750 268 750 268 750 268 750 268 750 483 750 #1 Number of PTAs, communities members ; farmers organisation trained or sensitised about the importance of Health and Hygiene Practices 0 960 960 960 960 960 4800 #2 Number of PTAs, communities members; farmers organisation trained or sensitised about the importance of education 0 960 960 960 960 960 4 800 #3 Percent of transfers made to the school inspectors as a % of planned 60,0% 100% 100% 100% 100% 100% 100% #4 Number of textbooks and other teaching and learning revised materials (based on revised curriculum) provided to teachers schools as a result of USDA assistance 0 1 038 0 0 1 038 0 2 076 #5 Percentage of pupil who indicate they are attentive or very attentive during class/instruction TBD 60% 65% 70% 75% 80% 80% Percentage of pupil who indicate they are attentive or very attentive during class/instruction (female) TBD 60% 65% 70% 75% 80% 80% Percentage of pupil who indicate they are attentive or very attentive during class/instruction (male) TBD 60% 65% 70% 75% 80% 80% #6 Number of government staff trained as a result of USDA assistance 0 300 300 300 300 300 300 Number of government staff trained as a result of USDA assistance (female) 0 120 120 120 120 120 120 Number of government staff trained as a result of USDA assistance (male) 0 180 180 180 180 180 180 #7 Percentage of school days missed due to illness (target < 3%) TBD TBD TBD TBD TBD 3% 3% #8 Number of fuel -efficient stoves provided and rehabilitated 0 60 0 0 60 0 120 Annex 9. Outcomes according to actual PMP All data has been copied/pasted from the approved PMP Standard /#Custom indicator number Performance Indicator Baseline Targets Life of project Year 1 Year 2 Year 3 Year 4 Year 5 Oct1, 2017- 30 Sep, 2018 Oct1, 2018- 30 Sep, 2019 Oct1, 2019- 30 Sep, 2020 Oct1, 2020- 30 Sep, 2021 Oct1, 2021- 30 Sep, 2022 To be confirmed during baseline evaluation (May 2018) Target for mid￾term evaluation (end 2020) Target for final evaluation (end 2022) 3 Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance 0 520 560 600 640 680 3 000 5 Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance 0 120 0 0 0 0 120 11 Value of new public and private sector investments leveraged as a result of USDA assistance 0 0 0 100 000 0 0 100 000 12 Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance: 129 Stage 1: Analysed Stage 2: Drafted and presented for public/stakeholder consultation Stage 3: Presented for legislation/decree Stage 4: Passed/Approved Stage 5: Passed for which implementation has begun 0 1 0 0 0 0 1 19 Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance 0 384 308 218 232 247 1 388 129 The educational policy this indicator is related to is the National School Feeding Policy adopted in 2016. 21 Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance 0 624 672 720 768 816 3 600 25 Number of child health and nutrition policies, regulations, or administrative procedures in each of the following stages of development as a result of USDA assistance:130 - Stage 1: Analysed - Stage 2: Drafted and presented for public/stakeholder consultation - Stage 3: Presented for legislation/decree - Stage 4: Passed/Approved - Stage 5: Passed for which implementation has begun 0 0 0 0 0 1 1 26 Percent of pupils who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text 43% 50% 60% 70% 80% 80% 80% Percent of pupils who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (female) 43% 50% 60% 70% 80% 80% 80% Percent of pupils who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (male) 43% 50% 60% 70% 80% 80% 80% 130 This refers to the National Food Security and Nutrition Policy that has been drafted by the WHO and UNICEF Annex 10. Analytical Framework Annex 11. Baseline interview guides The following questions / themes are general guidelines to ensure consistency of qualitative data collection between members of the evaluation team. Interviews will also focus on issues related to education and nutrition according to the needs of the ET and the interviewee. Relevance/Coherence  At baseline, how relevant is the programme?  Is it in alignment with government policy and expectations? (NFSP)  How does the programme consider gender equality and empowerment of women (GEEW)? How will/can empowerment of women be assessed? What indicators will/can be used? Effectiveness, efficiency and impact  What are the main challenges regarding effectiveness and efficiency of the programme?  How are / can /will those challenges be addressed?  What is the expected impact of the programme? How realistic is it?  How does the MoE intend implementing their NSFP within the programme? Sustainability  At baseline, what is the Government’s capacity to take over the programme / parts of the programme in 5 years’ time (human resources, technical resources, financial resources, etc.)?  How big is the school feeding committee’s involvement in the project?  Once MGD funding ends, what capacity does the SFC have for sustaining the programme? Annex 12. Evaluation matrix Q1 / What are the baseline values for each indicator in the Performance monitoring plan? No. Sub-questions Measure/Indicat or of progress Main sources of information Data collection methods Data analysis methods Evidence availability and reliability 1 Each of the 33 indicators can be considered as sub-question. For ease of use, the evaluation matrix will treat those indicators together instead of individually. set baseline figure INS surveys; PASEC survey; WFP reports; school attendance sheets; implementing partners. Secondary and primary data Descriptive statistics mostly strong Q2 / Given the baseline values, the objectives of the MGD programme and within the context of Congo, are the targets set for each indicator realistic? Are any of them to low or too high? No. Sub-questions Measure/Indicat or of progress Main sources of information Data collection methods Data analysis methods Evidence availability and reliability 2 Each of the 33 indicators can be considered as sub-question. For ease of use, the evaluation matrix will treat those indicators together instead of individually. N/A Secondary data; WFP staff; government staff; implementing partners Desk review, interview with WFP staff and government staff Qualitative analysis, triangulation between multiple key informants mostly strong Q3 / Given the objectives and activities of MGD and the context of Congo, what are the key success factors for efficient and effective monitoring and evaluation of the programme? Are the evaluation design and evaluation questions planned at inception feasible? No. Sub-questions Measure/Indicat or of progress Main sources of information Data collection methods Data analysis methods Evidence availability and reliability 3.1 What are the key success factors for efficient and effective monitoring and evaluation of the programme? N/A Secondary data, WFP staff (M&E), implementing partners Desk review, interview with WFP staff and partners (M&E) Qualitative analysis, triangulation between multiple key informants Strong 3.2 Are the evaluation design and evaluation questions planned at inception feasible? N/A Internal (ET); implementing partners Desk review, interview with WFP staff and government staff Qualitative analysis, triangulation between multiple key informants N/A Annex 13. Team schedule during field mission Schedule for evaluation team during field mission (from the 7th of May to the 18th of May 2018) Morning Morning Afternoon Afternoon 5th / 6th of May Team arrives in Congo Monday 7h 8.30 WFP nutrition & gender officer (Friederike) 9.10 Security brief with UNDSS (all team) 10.00 Meeting with ref group (all team) Informal interview with MoH (Friederike) Meeting with PASEC (Jean-Pierre & Jean-Pépin) Informal discussion with MoA (Ian) Informal discussion with M of Planning (Ian) 14.00 meeting INS (all team) Interview with WFP programme managers according to availability and needs (Jean-Pierre) Tuesday 8th Training with INS on questionnaires (Friederike, Jean-Pépin & Ian + remote supervision from Cova) 9.30 UNICEF – education officer, WASH officer (Jean-Pierre) 11.30 Direction of School Feeding (Jean-Pierre & Ian) 15.00 - Meeting WFP programme officers (Jean-Pierre & Friederike) Wednesday 9th INS staff training on electronic devises (Friederike, Jean-Pépin & Ian + remote supervision from Cova) 11.00 - Meeting with Ministry of Women (Friederike) Meeting with the Programme national pour l’évaluation des acquis scolaires (Jean-Pierre) 16.00 Meeting with UNESCO (Jean￾Pierre & Friederike) Thursday 10th (bank holiday) Team meeting / work Meeting with (M&E, gender, VAM) WFP staff (Friederike & Jean-Pierre) Work with INS (Friederike, Jean-Pépin & Ian) Friday 11th INS testing of questionnaires in field (Friederike, Jean-Pépin & Ian) INS finalise tools (Friederike + remote supervision from Cova) 10.00 - Meeting with World Bank (Jean-Pierre) 14.00 - Meeting with DEP (Jean￾Pierre) Saturday 12th Field trip Brazzaville - Owando (Friederike & Ian) Sunday 13th Field trip Owando - Ouesso (Friederike & Ian) Meeting with the ASPC in Ouesso Field trip Brazzaville - Nkayi (Jean-Pierre & Jean-Pépin) Monday 14th Visit of 2 ORA schools in Seka-Sembé ( Kedick & Egnabi) Meeting with the DDEPSA131 of the Sangha department Meeting with ACTED Meeting with WFP sub-office Meeting with the DDEPSA Tuesday 15th Visit of 2 ORA schools in Mokeko-Mambili ( Friederike & Ian) Meeting with the DDEPSA of the Cuvette department Meeting with a school director and an inspector Trip to Sibiti Meeting with DDEPSA and the inspector Visit to 1 or 2 schools in Komono Wednesday 16th Visit of 1 schools Ngo – Plateaux department Trip back to Brazzaville Trip to Brazzaville Thursday 17th Team work Meeting with UNICEF nutritionist (Friederike) Team work to prepare debrief Internal debriefing Meeting with WFP M&E officer Friday 18th Team work External debriefing Meeting with PASEC team Jean-Pierre and Ian leave country at 8.45pm – Air France 897 Friederike leave country at 12.10pm - Ethiopian airlines 861 19th of May until 15th of June Jean-Pépin (national consultant) follows up on INS and PASEC. 131 Direction départementale de l’éducation primaire, secondaire et de l’alphabétisation (departmental direction of primary and secondary education and alphabetisation) 50 Annex 14. Teacher questionnaire MINISTERE DU PLAN, DE LA STATISTIQUE ET DE L’INTEGRATION REGIONALE REPUBLIQUE DU CONGO ********** Unité*Travail*progrès INSTITUT NATIONAL DE LA STATISTIQUE ********** ********** ENQUETE DE BASE SUR LES CANTINES SCOLAIRES QUESTIONNAIRE ENSEIGNANTS Coopération INS- KonTerra-PAM QUESTIONNAIRE N° |__|__|__| TYPE D’ENQUETE: 1=Enquête de Reference 2= Enquête de Mi-Parcours 3=Evaluation Finale du Programme |__| Ce questionnaire a un but purement statistique. Il sert uniquement à recueillir les informations utiles au programme MGD du PAM. Les informations collectées au cours de cette enquête sont strictement confidentielles au terme de la loi N° 8-2009 du 28 octobre 2009 sur la statistique. AA. AMÉLIORATION DE L'ATTENTION DES ÉLÈVES 0. INFORMATIONS GENERALES A. IDENTIFICATION ET LOCALISATION DE L’ECOLE S0.1 Département : ________________________________________ |__|__| S0.2 Commune/District : ____________________________________ |__|__| S0.3 Quartier /village : ________________________ |__|__| S0.4. Numéro de l’enquêteur : ______________________________________________ |__|__| B. EQUIPE DE COLLECTE S0.5 Fonction Noms et prénoms Date de l’interview S0.5.1 Enquêteur S0.5.2 Chef d’équipe |__|__| |__|__| |__|__|__|__| S0.5.3 Superviseur C. INFORMATION SUR L’ECOLE ET CARACTERISTIQUES DE L’ENSEIGNANT S0.6. Nom de l’école : ___________________________________________________ S0.7. Type d’enseignement : 1 = Primaire 2 = ORA |__| S0.8. Type de l’école : 1= École avec soutien du PAM 2= École rurale publique sans soutien du PAM |__| S0.9. Noms et prénoms de l’enseignant : ___________________________________________________ S0.10. Sexe : 1=Masculin 2=Féminin |__| S0.11. Age |__|__| S0.12. Quel est le dernier diplôme obtenu par l’enseignant : 1= BEPC 2=BAC 3=Licence 4= Maitrise 5=Doctorat 7=Autre à préciser_________ |__| S0.13. Quel est le dernier diplôme professionnel de l’enseignant 1=CFFEN 2=CAPEL 3=CAPCEG 4= CAPES 5=Aucun 6= Autre à préciser S0.11. Niveau d'enseignement (si chargé d'enseignement) cochez la réponse correspondante :  aucun a) Si Enseignement primaire : CP2 CE1 CE2 CM1 CM2 b) Si Enseignement ORA : Niveau 1 Niveau 2 Niveau 3  internat Présence : Si l’enseignant prend plusieurs classes, demandez à ce qu’il donne la situation d’une des classes qu’il tient. 51 AA1. Y a combien d'élèves au total dans votre classe (y compris les absents) ? |__|__|__| AA2. Y a combien d’élèves filles au total dans votre classe (y compris les absentes) ? |__|__|__| AA3. Y a combien d'élèves absents dans votre classe aujourd'hui ? |__|__|__| AA4. Y a combien d'élèves filles absentes dans votre classe aujourd’hui ? |__|__|__| AA 5. Pourquoi ces élèves sont-ils souvent absent(e)s ? malade la maison est éloignée de l’école travail à la maison raison financière Faim ne sais pas A. ATTENTION A1. Veuillez estimer le nombre d'enfants qui sont inattentifs tous les jours (somnolents, inactifs) en classe |__|__| A2. Veuillez estimer le nombre d'enfants filles qui sont inattentives tous les jours (somnolentes, inactives) en classe |__|__| A3. Veuillez estimer le nombre d'enfants qui sont inattentifs parfois ou par moment (somnolents, inactifs) en classe |__|__| A4. Veuillez estimer le nombre d'enfants filles qui sont inattentives parfois ou par moment (somnolentes, inactives) en classe |__|__| A5. Est-ce que cela change selon la saison/ mois ? 1= Oui 2= Non si non, Fin du questionnaire: |__| A6. Si oui, veuillez préciser le mois ou la saison durant laquelle la plupart des élèves sont inattentifs : a) grande saison des pluies : b) petite saison sèche  : c) petite saison des pluies : d) grande saison sèche  52 Annex 15. School administrator questionnaire MINISTERE DU PLAN, DE LA STATISTIQUE ET DE L’INTEGRATION REGIONALE REPUBLIQUE DU CONGO ********** Unité*Travail*progrès INSTITUT NATIONAL DE LA STATISTIQUE ********** ********** ENQUETE DE BASE SUR LES CANTINES SCOLAIRES QUESTIONNAIRE DIRECTEUR D’ECOLE Coopération INS- KonTerra-PAM QUESTIONNAIRE N° |__|__|__| TYPE D’ENQUETE: 1=Enquête de Reference 2= Enquête de Mi-Parcours 3=Evaluation Finale du Programme |__| Ce questionnaire a un but purement statistique. Il sert uniquement à recueillir les informations utiles au programme MGD du PAM. Les informations collectées au cours de cette enquête sont strictement confidentielles au terme de la loi N° 8-2009 du 28 octobre 2009 sur la statistique. SECTION 1 : INFORMATIONS SUR LA CARTE SCOLAIRE SECTION 0: INFORMATIONS GENERALES B. IDENTIFICATION ET LOCALISATION DE L’ECOLE S0.1 Département : ________________________________________ |__|__| S0.2 Commune/District : ____________________________________ |__|__| S0.3 Quartier/village : ________________________ |__|__| S0.4. Numéro de l’enquêteur |__|__| B EQUIPE DE COLLECTE S0.5 Fonction Noms et prénoms Date de l’interview S0.5.1 Enquêteur S0.5.2 Chef d’équipe |__|__| |__|__| |__|__|__|__| S0.5.3 Superviseur C. INFORMATION SUR L’ECOLE ET CARACTERISTIQUES DE L’ADMINISTRATEUR S0.6. Nom de l’école :___________________________________________________ S0.7. Type d’enseignement : 1=Primaire 2=ORA |__| S0.8. Type d’école : 1=Ecole primaire avec soutien du PAM 2= École rurale publique sans soutien du PAM |__| S0.9. Noms et prénoms de l’administrateur : ________________________________________________Tel : S0.10. Sexe : 1=Masculin 2=Féminin |__| S0.11. Avez-vous une classe en charge 1= oui 2= non S1.1. Nombre total d'élèves dans cette école |__|__|__|__| S1.2. Nombre total d'élèves filles dans cette école |__|__|__|__| S1.3. Nombre d'élèves inscrits durant cette année scolaire, par niveau et par sexe (Entrez 999 Si inconnu) Niveau Nombre d'élèves inscrits 2017-2018 Nombre d'élèves promus (de l'année dernière) 2016-2017 Nombre de redoublants 2016- 2017 Nombre d'abandons (depuis le début de l'année) 2017-2018 Total Dont Filles Total Dont Filles Total Dont Filles Total Dont Filles École Primaire |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| CP1 |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| 53 CP2 |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| CE1 |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| CE2 |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| CM1 |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| CM2 |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| Ecole ORA |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| Niv 1 |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| Niv 2 |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| Niv 3 |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| Internat |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| |__|__|__| Fréquentation scolaire S1.4. L'école a-t-elle des cahiers/feuilles de présence (registre d’appel) pour cette année scolaire ? (veuillez observer par vous-même ou demander au directeur de l'école de les montrer) 1=Oui, observable 2=oui, non observable 4=Non |__| SECTION 2 : AMÉLIORATION DE L'ÉQUIPEMENT SCOLAIRE ET DES INFRASTRUCTURES (Veuillez observer par vous-même et vérifier auprès de personnes appropriées) Fournitures S2.1. L'école a-t-elle suffisamment de matériel didactique pour un enseignement efficace cette année scolaire ? (Remarque pour l'enquêteur : observez la classe)  Affiche incluant des lettres (matériel de lecture)  Affiche incluant des chiffres (matériel de mathématiques)  Image descriptive (corps humain, animal)  Affiche liée à la science  Autres  Aucun Latrines S2.2. L’école a-t-elle des latrines 1= Oui 2=Non si non aller à S2.12 |__| S2.3. Si oui, combien y en a-t-elles ? |__|__| S2.4. Combien de latrines sont réservées pour les élèves ? |__|__| S2.5. Parmi ces latrines combien sont Fonctionnelles ? : |__|__| S2.6. Les latrines fonctionnelles pour les élèves sont-elles séparées pour les garçons et les filles ? 1= Oui 2=Non |__| S2.7. Si oui, combien de latrines fonctionnelles pour les garçons |__|__| S2.8. Si oui, combien de latrines fonctionnelles pour les filles : |__|__| S2.9. Les latrines pour les filles comportent-t-elles un espace spécifique pour les informations sur les menstruations et des kits d'hygiène ? (si oui l'enquêteur doit procéder à une observation) 1= Oui 2=Non |__| S2.10. Nombre de latrines fonctionnelles non séparées |__|__| S2.11. Veuillez caractériser les latrines (par observation directe) : (1=Oui 2=Non) a) chasse d'eafu ou système d'évacuation raccordé à un réseau d'égouts, un système septique ou une latrine à fosse |__| b) chasse d'eau ou système d'évacuation sans système d'égout |__| c) Latrines à fosse avec dalle |__| d) Latrines à fosse sans dalle/ouvertes |__| e) Toilettes à compost |__| f) Latrines à fosse améliorées ventilées |__| g) Latrines seau |__| h) Toilettes/latrines suspendues |__| i) Autre, à préciser : _______________ S2.12. Comment gérez-vous et entretenez-vous les latrines ? (cochez les réponses à choix multiples) a) Forme les élèves et organise des rotations pour le nettoyage des latrines |__| b) Assure la présence de savon et de matériel pour le lavage des mains à l'intérieur ou à proximité des toilettes |__| c) Ferme les latrines pendant les vacances scolaires |__| d) S'assure que le lavabo est plein d'eau |__| e) Invite les utilisateurs à laisser leurs chaussures à l'extérieur des latrines |__| f) Autre, spécifiez : _______________ |__|__|__| 54 S1.5. Combien d’absences ont été enregistrées au cours de la semaine dernière ? (entrez 999 Si inconnu) Lavage des mains S2.13. L'école a-t-elle un poste de lavage des mains? (l'enquêteur doit procéder à une observation) 1=Oui 2= Non S2.14. Si oui, y a combien de postes de lavage des mains dans l'école ? S2.15. Si oui, quelles sont les conditions actuelles de la station de lavage des mains ? 1= Bon état et bon fonctionnement toute l'année 2= Bon état et bon fonctionnement seulement pendant la saison des pluies 3=Mauvais état mais fonctionne toute l'année 4= Mauvais état mais ne fonctionne que pendant la saison des pluies 5= Hors d'état de service, ne fonctionne pas 6=Autre, à préciser__________________ Source d'eau améliorée S2.16. L'école a-t-elle une source d'approvisionnement en eau ? 1=Oui 2= Non---> élimination des déchets |__ S2.17. Veuillez décrire la source d'approvisionnement en eau (observation et cochez les réponses correspondantes) : a) Eau canalisée dans les locaux, la parcelle, ou la cour  b) Robinet/tuyau public  c) Puits canalisé/forage  d) Puits creusé protégé  e)Puits creusé non protégé  f) Source protégée  g) Source non protégée  h) Collecte des eaux pluviales  i)Chariot avec petit réservoir/tambour  j) Camion-citerne  k)Eau de surface (rivière, barrage, lac, étang)  l) Eau de bidon/gourdes  m) Autre à préciser  :____________________ S2.18. Préciser la source principale (exemple b qui représente le robinet/tuyau public) S2.19. L'eau est-elle normalement disponible à partir de cette source d'eau ? 1=Oui 2= Non, si non aller à S2.21 S2.20. Est-ce que l'eau n'a-t-elle pas été disponible à partir de cette source d'approvisionnement au cours des 2 dernières semaines durant au moins une journée? 1=Oui 2= Non 3=Ne sais pas Amélioration de la qualité de l'enseignement du français S1.6. Nombre d'enseignants (y compris des bénévoles) cette année scolaire : (entrez 99 si inconnu) |__|__|: S1.7. Nombre d'enseignants femmes (y compris des bénévoles) cette année scolaire : (entrez 99 si inconnu) |__|__|: S1.8. Nombre d'enseignants bénévoles, le cas échéant (en-dehors des enseignants fonctionnaires) :(entrez 99 Si inconnu) |__|__|: S1.9. a. Nombre d'enseignants bénévoles femmes, le cas échéant (en-dehors des enseignants fonctionnaires) : (entrez 99 Si inconnu) |__|__|: S1.10. Présence des enseignants durant le dernier mois Prénoms de l’enseignant Sexe 1=Masculin 2=Féminin Nombre de journées d'enseignement du dernier mois Nombre de journées d'absence du dernier mois 1 |__| |__|__|__| |__|__| 2 |__| |__|__|__| |__|__| 3 |__| |__|__|__| |__|__| 4 |__| |__|__|__| |__|__| 5 |__| |__|__|__| |__|__| 6 |__| |__|__|__| |__|__| 7 |__| |__|__|__| |__|__| 8 |__| |__|__|__| |__|__| 9 |__| |__|__|__| |__|__| 10 |__| |__|__|__| |__|__| 55 ENGAGEMENT ACCRU DES GROUPES LOCAUX ET COMMUNAUTAIRES S1.11. L'école a-t-elle un Comité pour l'Alimentation Scolaire (CAS) ? 1=Oui 2=Non si non, aller à la section suivante |__| S1.12. Est-t-il actif/opérationnel à l'école ? 1=Oui 2=Non |__| SECTION 3 : UTILISATION ACCRUE DE PRATIQUES ALIMENTAIRES SAINES ÉLIMINATION DES DÉCHETS SOLIDES S3.1. L'école a-t-elle des poubelles ou d'autres équipements pour gérer les déchets solides ? 1=Oui observable 2=oui observable 3=Non |__| S3.2 à quelle fréquence les déchets solides sont-ils collectés ? 1=tous les jours 2=tous les deux ou trois jours 3=toutes les semaines |__| S3.3. Comment les déchets solides sont-ils éliminés§ 1=incinérés 2=jetés hors de l’école 3=enterrés 4=vider aux environs de l’école 5=jetés dans la na0ture 56 Annex 16. School feeding committee questionnaire MINISTERE DU PLAN, DE LA STATISTIQUE ET DE L’INTEGRATION REGIONALE REPUBLIQUE DU CONGO Unité*Travail*progrès ********** ********** INSTITUT NATIONAL DE LA STATISTIQUE ********** ENQUETE DE BASE SUR LES CANTINES SCOLAIRES QUESTIONNAIRE COMITE POUR L’ALIMENTATION SCOLAIRE Coopération INS- KonTerra-PAM QUESTIONNAIRE N° : |__|__|__| TYPE D’ENQUETE: 1=Enquête de Reference 2= Enquête de Mi-Parcours 3=Evaluation Finale du Programme |__| Ce questionnaire a un but purement statistique. Il sert uniquement à recueillir les informations utiles au programme MGD du PAM. Les informations collectées au cours de cette enquête sont strictement confidentielles au terme de la loi N° 8-2009 du 28 octobre 2009 sur la statistique. 57 Infrastructures S1.10. L'école a-t-elle une cuisine ? (à observer par l'enquêteur) 1= Oui 2 Non si non aller à la question S1.13 |__| SECTION 0 : INFORMATIONS GENERALES C. IDENTIFICATION ET LOCALISATION DE L’ECOLE S0.1 Département : ________________________________________ |__|__| S0.2 Commune/District : ____________________________________ |__|__| S0.3 Quartier/village : ________________________ |__|__| S0.4. Numéro de l’enquêteur |__|__| B. EQUIPE DE COLLECTE S0.5 Fonction Noms et prénoms Date de l’interview S0.5.1 Enquêteur S0.5.2 Chef d’équipe |__|__| |__|__| |__|__|__|__| S0.5.3 Superviseur C. INFORMATION SUR L’ECOLE ET REPRESENTANT DU COMITE S0.6. Nom de l’école : ___________________________________________________ S0.7. Type d’enseignement : 1=Primaire 2=ORA |__| S0.8. Type d’école : 1= École avec soutien du PAM 2= École rurale publique sans soutien du PAM |__| S0.9. Noms et prénoms du représentant du comité : ___________________________________________________ S0.10. Sexe : 1=Masculin 2=Féminin |__| SECTION 1 : INFORMATIONS GENERALES Taille S1.1. Nombre total de personnes qui sont membres du comité Entrez 99 si inconnu |__|__| S1.2. Nombre total de femmes qui sont membres du comité Entrez 99 si inconnu |__|__| S1.3. Sexe du Président 1=Masculin 2=Féminin |__| S1.4. Sexe du Vice-Président 1=Masculin 2=Féminin 3= N/A |__| S1.5 Sexe du Trésorier 1=Masculin 2=Féminin 3= N/A |__| S1.6 Sexe du Vice-trésorier 1=Masculin 2=Féminin 3= N/A |__| S1.7 Sexe de l’Administrateur 1=Masculin 2=Féminin 3= N/A |__| S1.8. Administrateur adjoint 1=Masculin 2=Féminin 3= N/A |__| S1.9 Autre à préciser ;_________________________ 58 S1.11. Si oui, quelles sont les conditions actuelles de la cuisine ? (Observer) a) Bon état b) Manque d'ustensiles de cuisine c) Équipement pour cuisiner et pour manger propre d) Fuites dans le toit e) Inondé pendant la saison des pluies f) Utilisation de pierres en guise de fours g) Autre, préciser : ……......................... h) Non observable 1=Oui 2=Non |__| |__| |__| |__| |__| |__| |__| |__| S1.12. Si oui, est-ce que l'école a des fours à économie d'énergie ? (observer) 1=Oui 2=Non si non aller question S1.13 |__| S1.12.a Quel est l'état actuel des fours à économie d'énergie ? 1=Bonne condition et bon fonctionnement 2= Mauvais état, mais toujours utilisé 3= Cassé, ne fonctionne pas 4=Autre, à préciser : 5= Non observable |__| S1.13. L'école a-t-elle une réserve (ou un endroit où entreposer des aliments) ? (Observer) 1=Oui 2=Non Si non fin du questionnaire |__| S1.14. Si oui, quelles sont les conditions actuelles des locaux d'entreposage des vivres ? a) Bien nettoyé (Observer) b) Le plancher est sec (Observer) c) Palettes pour le stockage des aliments(Observer) d) La porte est bien verrouillée e) Garde de sécurité pendant la nuit/pendant les vacances scolaires f) Les aliments sont stockés en bon ordre (Observer) g) Toitures qui fuient h) Fenêtres/portes cassées (Observer) i) Murs endommagés (Observer) j) Pas de murs k) La nourriture a été stockée hors sol l) La réserve avait une ventilation m) Autre, préciser : ....................... 1=Oui 2=Non |__| |__| |__| |__| |__| |__| |__| |__| |__| |__| |__| |__| |__| 59 Annex 17. Pupil questionnaire MINISTERE DU PLAN, DE LA STATISTIQUE ET DE L’INTEGRATION REGIONALE REPUBLIQUE DU CONGO ********** Unité*Travail*progrès INSTITUT NATIONAL DE LA STATISTIQUE ********** ********** ENQUETE DE BASE SUR LES CANTINES SCOLAIRES QUESTIONNAIRE ELEVE Coopération INS- KonTerra-PAM QUESTIONNAIRE N° |__|__|__| TYPE D’ENQUETE: 1=Enquête de Reference 2= Enquête de Mi-Parcours 3=Evaluation Finale du Programme |__| Ce questionnaire a un but purement statistique. Il sert uniquement à recueillir les informations utiles au Programme MGD du PAM. Les informations collectées au cours de cette enquête sont strictement confidentielles au terme de la loi N° 8-2009 du 28 octobre 2009 sur la statistique. S0. INFORMATIONS GENERALES D. IDENTIFICATION ET LOCALISATION DE L’ECOLE ID.1 Département : ________________________________________________________________ |__|__| ID.2 Commune/District : ________________________________________________________________ |__|__| ID.3 Quartier/ village : __________________________________________ |__|__| ID.4 Numéro de l’enquêteur : |__|__| E. EQUIPE DE COLLECTE S0 Fonction Noms et prénoms Date de l’interview S0.1 Superviseur S0.2 Chef d’équipe |__|__| |__|__| |__|__|__|__| S0.3 Enquêteur C. INFORAMTION SUR L’ECOLE ET CARACTERISTIQUES DE L’ELEVE S0.4. Nom de l’école : ___________________________________________________ S0.5. Type d’enseignement : 1= Primaire 2= ORA |__| S0.6. Type de l’école : 1= École avec soutien du MGD 2= École rurale publique sans soutien du PAM |__| S0.7. Noms et prénoms de l’élève : ___________________________________________________ S0.8. Sexe : 1= Masculin 2= Féminin |__| S0.9. Age |__|__| S0.10. Niveau scolaire : c) Si Ecole primaire : 1=CP1 2=CP2 3=CE1 4=CE2 5=CM1 6=CM2 d) Si Ecole ORA : 7= Niveau 1 8= Niveau 2 9=Niveau 3 10=internat |__|__| AA. AMÉLIORATION DE L'ATTENTION DES ÉLÈVES S1. Présence S1Q1. As-tu manqué l'école au cours des 4 dernières semaines ? 1= Oui 2= Non 3= Ne sait pas si non ou ne sait pas aller à S2Q1 |__|__| S1Q2. Si oui, combien de jours as-tu manqué l'école ? Essayez d'aider l'enfant à estimer le nombre. Entrez 999 si inconnu |__|__|__| 60 S1Q3. Pourquoi as-tu été absent(e) ? 1=malade 2= la maison est éloignée de l’école 3= travail à la maison 4=raison financière 5=Faim 6=déplacement familiale 7= travaux de champs 8=pas de réponse 9= autre à préciser :____________ |__| S1Q4. Si tu as été malade, quelle maladie as-tu souffert ? cochez les réponses déclarées par l’enquêtée a) fièvre  b) vomissement  c) faiblesse fatigue  d) mal au ventre e) mal à la tête f) mal au corps  g) réactions diarrhéiques  h) rhume i) froid j) autre |__| à préciser : _______________________ S2. Faim à court terme S2Q1. Manges-tu habituellement quelque chose à la maison avant de venir à l’école le matin ? 1=Tous les jours 2=Parfois 3=rarement 4= Jamais |__| S2Q2. Est-ce que cela change selon la saison ? 1= Oui 2= Non si non à la S2Q4 |__| S2Q3. Si oui, précise le mois/ la saison durant lequel/ laquelle tu ne manges pas avant d'aller à l'école (cochez la réponse correspondante au choix multiple) 1) grande saison des pluies  : 2) petite saison sèche  : 3) petite saison des pluies : 4) grande saison sèche | : S2Q4. Qu'as-tu mangé ce matin avant de venir à l'école (cochez la réponse correspondante) a) Manioc(Chikwangue)/Foufou, Riz, Pain ou beignets, autres céréales (pates, etc.), Autres tubercules (Patate douce/Pomme de terre, Igname,) et banane plantain, sucre, miel et autres sucreries  b) Haricot, Pois, lentilles, niébé, courge, sojà, Pâte d’arachide, arachide, sésame et autres légumineuses  c) Légumes (Amarantes, Feuilles de manioc , Endives, épinards, Oseille…) , Autres légumes (aubergines, courgette …)  d) Légumes oranges (courge, carottes, patate douce à chair rouge…), Fruits oranges (papaye, mangue, oranges), fruits sauvages, Autres fruits frais  e) Viande d’élevage et viande de brousse (gibiers), Volailles, escargot, crevettes, poissons (frais, salé et fumé), Insectes comestibles (Chenilles…), abats etc  f) Œufs  g) Lait/Fromage/Yaourt/Produits laitiers  S2Q5. As-tu apporté de la nourriture à manger pendant que tu es à l'école ? 1= Oui 2= Non si non, aller à S2Q7 |__| S2Q6. Si oui, qu'as-tu apporté aujourd'hui : cochez les réponses correspondantes à la déclaration _______________________________________________ a) Manioc(Chikwangue)/Foufou, Riz, Pain ou beignets, autres céréales (pates, etc.), Autres tubercules (Patate douce/Pomme de terre, Igname,) et banane plantain, sucre, miel et autres sucreries  b) Haricot, Pois, lentilles, niébé, courge, sojà, Pâte d’arachide, arachide, sésame et autres légumineuses  c) Légumes (Amarantes, Feuilles de manioc , Endives, épinards, Oseille…) , Autres légumes (aubergines, courgette …)  d) Légumes oranges (courge, carottes, patate douce à chair rouge…), Fruits oranges (papaye, mangue, oranges), fruits sauvages, Autres fruits frais  e) Viande d’élevage et viande de brousse (gibiers), Volailles, escargot, crevettes, poissons (frais, salé et fumé), Insectes comestibles (Chenilles…), abats etc  f) Œufs  g) Lait/Fromage/Yaourt/Produits laitiers  S2Q7. Reçois-tu des repas scolaires : 1=Oui 2= Non |__| S2Q8. Combien de repas manges-tu à la maison après ton retour de l'école 1= Rien 2=Un repas 3= deux repas 4=Seulement des casse-croûtes |__| S2Q9. Quel type de nourriture as-tu reçu à la maison, après ton retour de l'école (la veille ou le jour d'école précédent) : ________________________________________ 61 a) Manioc(Chikwangue)/Foufou, Riz, Pain ou beignets, autres céréales (pates, etc.), Autres tubercules (Patate douce/Pomme de terre, Igname,) et banane plantain, sucre, miel et autres sucreries  b) Haricot, Pois, lentilles, niébé, courge, sojà, Pâte d’arachide, arachide, sésame et autres légumineuses  c) Légumes (Amarantes, Feuilles de manioc , Endives, épinards, Oseille…) , Autres légumes (aubergines, courgette …)  d) Légumes oranges (courge, carottes, patate douce à chair rouge…), Fruits oranges (papaye, mangue, oranges), fruits sauvages, Autres fruits frais  e) Viande d’élevage et viande de brousse (gibiers), Volailles, escargot, crevettes, poissons (frais, salé et fumé), Insectes comestibles (Chenilles…), abats etc  f) Œufs  g) Lait/Fromage/Yaourt/Produits laitiers  S2Q10. As-tu l’habitude de laver les mains avant le repas ? 1=Tous les jours 2=Parfois 3=rarement 4= Jamais |__| S2Q11. As-tu l’habitude de laver les mains après le repas ? 1=Tous les jours 2=Parfois 3=rarement 4= Jamais |__| S2Q12. As-tu l’habitude de laver les mains après la toilette ? 1=Tous les jours 2=Parfois 3=rarement 4= Jamais |__| 62 Annex 18. National Institute of Statistics (INS) overview Created by Decree No. 2010-805 of 31 December 2010, replacing the National Centre for Statistics and Economic Studies (CNSEE), the National Institute of Statistics (INS) is a public institution with financial autonomy. The INS’s mission is to contribute to the provision of statistical data related to economic, demographic, social and cultural fields. As such, it collects, processes, analyses, disseminates, stores and updates information in order to: • coordinate the activities of the national statistical system of which the INS is the central body; • produce national accounts and macroeconomic indicators; • to produce statistical data and analyses that comply with recognised international standards; • centralise, manage all economic, demographic, social and cultural databases; • publish and disseminate quality information on the economic, demographic, social and cultural situation; • assess the impact of national development strategies, policies, programmes and projects; • promote cooperation, and a culture of evidence. INS is known for its large statistical operations such as: - the General Census of Population and Housing (RGPH) in 1974, 1984, 1996, 2007 and that of 2018. - Congolese household surveys (ECOM 2005, 2011-2012); - Demographic and Health Surveys (ESDC 2005 and 2011 - 2012); - The Multiple Indicator Cluster Survey (MICS 2015) - The Seroprevalence and HIV / AIDS Indicators Survey (ESISC 2009); - The Survey of Employment and the Informal Sector (EESIC 2009 and 2011); - Global Analysis on Food Security and Vulnerability (AGSAV 2013); - Evaluation Survey of Health Centres (EECSIC 2010); - Business surveys (structural and cyclical surveys); Data collection of all the above projects was conducted in collaboration with United Nations agencies (UNICEF, UNFPA, WFP) and the World Bank. The INS is comprised of a Directorate General and five departments: - Department of economic research - Department of Demographic and Social Studies; - Department of statistical programming, coordination and harmonisation; -Department of Surveys and Censuses; -Department of Administrative and Financial Affairs. The Institute employs 460 executives and agents distributed throughout the country including 242 in Brazzaville. 63 Annex 19. Additional note to the quantitative survey strategy and approach Notes and comments on other tools to be applied under the quantitative component of this evaluation To assess the level of achievement of certain outputs in a limited number of schools, we propose to randomly sample a number of these schools and verify the percentage of them in which the output achievement is confirmed. For instance, in the 22 schools in which latrines are repaired, we propose to sample and visit 10. These 10 visits will be part of the qualitative research.132 This would lead to an estimation of the percentage of schools where the output was achieved, with 20% assumed error. Table 22: Estimation of error in the output achievement indicator by number of schools visited Information extracted from CONFEMEN’s PASEC 2014 report The pieces of information presented here below are extracted from the PASEC 2014 report elaborated by the CONFEMEN for the Congo. The report emphasizes the existence of significant disparities in terms of socio-economic contexts (Table 23) and literacy test results (Table 24) between the identified regions. These are presented in the Map 2 below. All of them include departments where MGD is implemented, except for the Pointe Noire region. 132 The quantitative survey, which will be conducted beforehand, may or may not include one of the 22 schools. 64 Map 1: Regions or zones identified as homogeneous strata in the PASEC 2014 report Source: CONFEMEN’s PASEC 2014 report. Figure 3.1, page 42. Table 23: Main characteristics of enrolled children by region/zone (beginning on school life) Source: CONFEMEN’s PASEC 2014 report. Tableau 3.7, page 43. Table 24: Percentage of enrolled children by literacy level (as issued from the literacy test) and region/zone. Level <1 Level 1 Level 2 Level 3 Level 4 % passed* Brazzaville 1.3 15.2 26.4 28.6 28.6 57.2 Pointe -Noire 0.3 8.0 28.7 28.7 34.3 63.0 Niari 9.3 60.7 25.4 4.3 0.4 4.7 Plateaux 18.1 31.0 34.1 12.6 4.2 16.8 Cuvette–Likouala 1.6 18.4 34.8 35.8 9.3 45.1 National average 4.8 28.6 28.6 21.4 16.6 38.0 Source: CONFEMEN’s PASEC 2014 report. Tableau B3.9, page 139. * Level 3 or above, calculated 65 Annex 20. Training outline Training of surveyors included one day of in-room training, during which survey objectives, materials and techniques were presented and discusses, and one day of survey pilot in the field, to test suitability of survey tool and surveyors’ capacities. After field pilot, a session on difficulties and hints was held. During the training, special emphasis was put in training surveyors to be able to introduce survey objectives, explain participation conditions (including confidentiality) and obtain consent from participants. Surveyors were trained to create a relaxed ambiance in which survey participants would not feel evaluated, but invited to share their personal views, experiences and thoughts. Matériels/Outils Num. Outil 1 Présentation PowerPoint (Français) 3 Projecteur 5 Flipchart et feutres 6 Questionnaires en papier 7 Tablettes avec outils de collecte en CSPro 8 Guide de phrases clés Nombre de participants : 24 dont 6 seront chefs d’équipe Nombre de formateurs : 2 – 4 selon session Duration : 1 journée, 8 heures (de 8h30 à 17h) 66 Nu m Objectifs Méthodes et contenu Activités/outils Duration 133 Heures 1 Comprendre les acteurs engagés et ses rôles Présenter les différentes alliances qui existent : - USDA+PAM+MEPSA pour le programme d’alimentation scolaire - INS+KONTERRA pour l’évaluation Expliquer le rôle externe/indépendant de l’INS+Konterra et donc des enquêteurs eux mêmes Présentation [diapo 3] 10 08:30 – 08:40 2 Comprendre les objectifs du programme MGD Présenter le contexte, objectifs et activités du programme Présentation [diapos 4 (vidéo) et 5] 15 08:40 – 09 :00 3 Comprendre les objectifs de l’étude ainsi que l’importance de ses conclusions et qu’elles soient basées sur des données de qualité Présenter les objectifs de l'enquête. Expliquer comment les données collectées éclaireront la prise de décision pour des programmes au futur La qualité de la recherche est essentielle pour assurer un soutien pertinent aux bénéficiaires. Présentation [diapos 6, 7] 10 09 :00 – 09:05 133 Exprimée en minutes 67 Nu m Objectifs Méthodes et contenu Activités/outils Duration 133 Heures 4 Connaitre le plan d’enquête Expliquer le plan d’enquête : équipes de terrain, lieux à visiter, procédures de travail quotidiennes et horaires Présentation [diapos 9, 10] 15 09:05 – 09:20 Questions des participants 10 09:20 – 09:30 Jour 1, Partie 2: Présentation des objectifs de l’enquête aux participants (1 heure) 5 Connaitre les méthodes de sélection de participants sur place Présenter brièvement les 4 questionnaires et expliquer l’existence des questions à demander et questions à observer Expliquer la méthode de sélection des participants, surtout pour les enfants. IMPORTANT : Faire un test, comment sélectionneraient nous 16 enfants parmi les participants à cette formation Présentation [diapo 12] Paper questionnaires Présentation [diapo 13] Dés 20 09:30 – 09:50 6 Être en mesure de présenter l'entrevue à des participants potentiels à l'enquête et promouvoir la participation sincère et la confiance 6.1. Être capable de présenter le projet et sa valeur Premier aperçu de l'introduction de l'enquête - éléments clés. Ensuite, expliquer les principaux concepts du langage corporel. Présentations d’introduction : Présentation [diapos 14 – 17] 10 09:50 – 10 :00 68  Phrases de remerciement pour la participation  Phrases pour présenter le but de l'enquête 6.2 Être capable d’expliquer les conditions de participation Expliquer les conditions de participation (différentes selon le type de participant) Présentation [diapo 18,19] 10 10 :00 – 10:35 Exercice: Simulation de presentations du projet aux participants de différents types (2 personnes font une simulation devant le reste, ensuite autres 2 personnes différentes font pareille…) Extraire des phrases clés pour expliquer les conditions et écrivez-les sur un tableau à feuilles mobiles et laissez-les en place pendant la formation. Exercise (discussion) 25 PAUSE 15 10:35 – 10:50 Jour 1, Partie 3: Lecture détaillée du questionnaire (3h) 7 Lecture détaillée des questionnaires question par question Lire collectivement les questionnaires. Discutez de la façon dont les questions clés fonctionneront et quelles informations supplémentaires pourraient être nécessaires pour qu'elles soient mieux comprises. 1. Les participants à l'enquête tournent pour lire les questions et les réponses en plénière. 2. Les formateurs facilitent la compréhension en expliquant la signification des questions et réponses, proposent des synonymes, expliquent ce que la question NE signifie PAS Lecture collective S’appuyer der photos dans la présentation 180 10 :50 – 12:00 Pause de 10 min si nécessaire 12:10 – 13 :00 69 3. Les formateurs résument les différents types de questions de l'enquête, en fonction de leurs réponses. 4. Explication détaillée des éléments à observer LUNCH 60 13 – 14h Jour 1, Partie 4: Simulations d’entretiens (2h35 min – including 15 minutes break) 8 . Être en mesure de conduire des entretiens avec des différents types de participants Exercice : Simulations d’entretiens (deux personnes devant l’audience pour ensuite discuter sur le déroulement de la simulation, quoi améliorer, où sont les difficultés, etc) Exercise (Simulations d’entretiens) 60 14:00 – 15:20 BREAK 15 15:20 – 15:35 8 Idem Simulations d’entretiens Exercise (Simulations d’entretiens, continuation) 60 15:35 – 16:35 Clôture Questions de participants Questions de l’audience 10 16:35 – 16:45 70 Annex 21. Survey data analysis report Please find the Survey Data Analysis Report in attachment 71 Annex 22. Table of all Standard and Custom indicators (WFP PMP vs. ET findings) USDA Standard Indicators Performance Indicator Baseline Targets Life of project Year 1 Year 2 Year 3 Year 4 Year 5 1 Number of students regularly (80%) attending USDA supported classrooms/schools WFP indicators 40,000 43,000 45,150 47,408 49,778 52,267 92,920 ET proposed amendments 91.5% of total num. of MGD￾supported schools pupils <43,200 <45,360 <47,628 <50,009 <52,510 <93,101 2 Number of textbooks and other teaching and learning materials provided as a result of USDA assistance WFP indicators 0 50,000 10,000 12,000 14,000 17,000 103,000 ET proposed amendments 0 0 – The activities planned do not reflect the expect activities related to this indicator 3 Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance WFP indicators 0 520 560 600 640 680 3,000 ET proposed amendments 0 TBD – Need to consider previous annual target when defining actual yearly target 4 Number of school administrators and officials trained or certified as a result of USDA assistance WFP indicators 0 800 800 800 800 800 4,000 ET proposed amendments 0 960 800 800 800 800 4160 Indicators needs to b disaggregated according to the topic of training 5 Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance WFP indicators 0 120 0 0 0 0 120 ET proposed amendments TBR TBR according to standard ind. # 6 6 Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance WFP indicators 0 160 0 0 0 0 160 ET proposed amendments 0 0 - The activities planned do not reflect the expect activities related to this indictor 7 Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance WFP indicators 0 18 18 7 7 7 57 ET proposed amendments 0 11 5 0 0 0 16 8 Number of students enrolled in school receiving USDA assistance WFP indicators 51,190 53,750 56,438 59,259 62,222 65,333 102,651 ET proposed amendments 53,750 56,438 59,259 62,222 65,333 102,651 9 Number of Parent-Teacher Associations (PTAs) or similar “school” governance structures supported as a result of USDA assistance WFP indicators 0 470 470 470 470 470 470 ET proposed amendments 0 318 318 318 318 318 318 Need to disaggregate by type of support 11 Value of new public and private sector investments leveraged as a result of USDA assistance WFP indicators 0 0 0 100,000 0 0 100,000 ET proposed amendments TBR TBR – Needs to be based on anticipated agreements or negotiations 12 Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance: WFP indicators 0 1 0 0 0 0 1 72 Stage 1: Analysed Stage 2: Drafted and presented for public/stakeholder consultation Stage 3: Presented for legislation/decree Stage 4: Passed/Approved Stage 5: Passed for which implementation has begun ET proposed amendments 0 TBR – Needs to be more specific/disaggregated (see report) 15 Number of daily school meals (breakfast, snack, lunch) provided to school￾age children as a result of USDA assistance WFP indicators 0 9,675,000 9,675,000 9,675,000 9,675,000 9,675,000 48,375,000 ET proposed amendments 0 9,720,000 10,206,000 10,716,300 11,252,160 11,814,840 53,709,300 16 Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance WFP indicators 0 53,750 53,750 53,750 53,750 53,750 96,750 ET proposed amendments 0 53,750 TBR TBR TBR TBR TBR 17 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance WFP indicators 0 53,750 53,750 53,750 53,750 53,750 96,750 ET proposed amendments 0 53,750 53,750 53,750 53,750 53,750 96,750 ET suggests to remove indicator (see main report) 18 Number of individuals trained in child health and nutrition as a result of USDA assistance WFP indicators 0 591 440 290 290 290 1,901 ET proposed amendments 0 TBR – Unclear in UNICEF’s workplan 0 0 0 0 TBR 19 Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance WFP indicators 0 384 308 218 232 247 1,388 ET proposed amendments 0 TBR according to Standard ind. #18 20 Number of individuals trained in safe food preparation and storage as a result of USDA assistance WFP indicators 0 960 960 960 960 960 4,800 ET proposed amendments 0 954 954 954 954 954 TBR 21 Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance WFP indicators 0 624 672 720 768 816 3,600 ET proposed amendments 0 TBR TBR TBR TBR TBR TBR 22 Number of schools using an improved water source WFP indicators 0 40 74 82 86 91 91 ET proposed amendments 77 (30.6%) 77 + 0 77 + 0 77 + 0 77 + 0 77 + 0 77 MoU does not reflect the initial PMP 23 Number of schools with improved sanitation facilities WFP indicators 0 29 58 65 72 79 79 ET proposed amendments 108 (42,9%) 108 + 6 108 + 6 108 + 6 108 + 6 108 + 6 108 + 6 MoUs do not reflect the initial PMP 24 Number of students receiving deworming medication(s) WFP indicators 10 53,750 53,750 53,750 53,750 53,750 96,750 ET proposed amendments 0 53,750 25 Number of child health and nutrition policies, regulations, or administrative procedures in each of the following stages of development as a result of WFP indicators 0 0 0 0 0 1 1 73 USDA assistance: - Stage 1: Analysed - Stage 2: Drafted and presented for public/stakeholder consultation - Stage 3: Presented for legislation/decree - Stage 4: Passed/Approved - Stage 5: Passed for which implementation has begun ET proposed amendments 0 Contribution of MGD programme to the NSFP development is not planned according to ET findings 26 Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text WFP indicators 43% 50% 60% 70% 80% 80% 80% ET proposed amendments 9.90% TBR – Yearly targets to ambitious (see report) 27 Number of individuals benefiting directly from USDA-funded interventions WFP indicators 0 56,261 55,950 55,800 55,800 55,800 108,061 ET proposed amendments 0 TBR – This “aggregate indicator” needs to be update (see report) 28 Number of individuals benefiting indirectly from USDA-funded interventions WFP indicators 0 268,750 268,750 268,750 268,750 268,750 483,750 ET proposed amendments 0 TBR (see report) WFP Custom Indicators Performance Indicator Baseline Targets Life of project Year 1 Year 2 Year 3 Year 4 Year 5 1 Number of PTAs, communities members ; farmers organization trained or sensitized about the importance of Health and Hygiene Practices WFP indicators 0 960 960 960 960 960 4,800 ET proposed amendments 0 Knowledge, Attitudes and Practices (KAP) under way. Indicator will need to be disaggregated by type of training and beneficiary group 2 Number of PTAs, communities members; farmers organization trained or sensitized about the importance of education WFP indicators 0 960 960 960 960 960 4,800 ET proposed amendments 0 TBR – Needs disaggregating by type of training and beneficiary group 3 Percent of transfers made to the school inspectors as a % of planned WFP indicators 0 60% 100% 100% 100% 100% 100% ET proposed amendments 0 TBR – A percentage is not specific enough for this indicator 4 Number of textbooks and other teaching and learning revised materials (based on revised curriculum) provided to teachers schools as a result of USDA assistance WFP indicators 0 1,038 0 0 1,038 0 2,076 ET proposed amendments 0 0 – The activities planned do not reflect the expect activities related to this indictor (see main report) 5 Percentage of student who indicate they are attentive or very attentive during class/instruction WFP indicators TBD 60% 65% 70% 75% 80% 80% ET proposed amendments 73.80% See main report 6 Number of government staff trained as a result of USDA assistance WFP indicators 0 300 300 300 300 300 300 ET proposed amendments 0 300 300 300 300 300 300 Needs to be disaggregated by type of training and beneficiary group 7 Percentage of school days missed due to illness (target < 3%) WFP indicators TBD TBD TBD TBD TBD 3% 3% ET proposed amendments 4% See main report 3% 3% 8 Number of fuel-efficient stoves provided and rehabilitated WFP indicators 0 60 0 0 60 0 120 ET proposed amendments 0 60 0 0 60 0 120 Rational on timing and school selection not clear (see main report) 74 Annex 23. Rationale on the indicator measuring improved literacy Improved Literacy of School-Age Children (MGD SO1) A key objective of USDA McGovern-Dole funded programmes is to improve literacy amongst school age children. The outcome indicator to assess this objective is the "percent of pupils who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text". Learning outcomes are not tracked at regular intervals in Congo. The only evaluation of pupil learning in recent years was done in 2012–13 within the framework of the PRAEBASE project,134 assessing learning outcomes at the beginning of grade 6 of primary education (CM2), covering all regions of Congo and all types of schools (including public, subsidised religious, and private). A total of 1,942 pupils in 100 schools were evaluated in French and mathematics. In French, the average score was 33 percent across the country, with regional average scores ranging from 21 to 61 percent. In mathematics, the national average was 26 percent, with regional scores ranging from 18 to 49 percent. Although informative about weak pupils’ level of academic performance, this evaluation cannot inform the MGD baseline as it was conducted five years before the launching of the present MGD programme and did not focus on the end of grade 2 (CP2). Another source of information to document the literacy MGD indicator could be the results of quarterly and annual tests conducted by teachers in all primary schools throughout the country. Results of these examinations determine notably whether pupils pass to the next grade or must repeat grade. On one hand, quality, rigor, and transparent administration of these tests cannot be taken for granted; on the other hand in the absence of an operational EMIS135 based on computerised systems, data collection currently conducted by the central Studies and Planning Directorate (Direction des Etudes et de la Planification, DEP) proved to be laborious, costly, and not sustainable. Data presented in the last education statistical yearbook published in 2015 did not enable an effective analysis of the education system. The National Institute for Statistics (Institut National de la Statistique, INS) is responsible for the preparation of a computerised education statistics yearbook whose first issue covering the 2017-18 school year is expected to be completed and published in June 2018. As far as pupils’ learning outcomes are concerned, data will not be disaggregated by items but presented as an average for the whole class, grade by grade. Based on lessons learnt from the PRAEBASE, the new Education Sector Support Project (PRAASED136) agreed in 2016 between the RoC Government and the International Development Association (IDA) aims at improving education outcomes of primary and lower secondary school children. In particular, the PRAASED supports a revision of all primary and lower secondary school curricula, with a focus on literacy and numeracy and strengthening the teaching of science and mathematics. It also includes the design and launching of a national assessment in French and mathematics which would be conducted at least twice during the project 134 PRAEBASE, Projet d'Appui à l'Education de Base, co-funded by the RoC Government and the World Bank (IDA-financed project 2004-2009 and additional financing 2009-2012). 135 Education Management Information System. 136 PRAASED 2016-2023, Projet d'Appui à l'Amélioration du Système Educatif, co-funded by the RoC Government and the World Bank. 75 implementation period. The project will support activities that address weaknesses in national examinations and tests and measure learning outcomes. The Programme for the Analysis of Education Systems (Programme d'Analyse des Systèmes Educatifs de la CONFEMEN,137 PASEC) has produced data and analysis on the performance of education systems and the factors that contribute to education quality, through large-scale surveys of primary-level pupils’ learning outcomes. Since 2012, PASEC’s mission has evolved to better respond to the expectations of countries and the international community for better measurement of learning outcomes. The added value of the new approach is to focus on the comparability of results across national assessments. The measurement of different countries’ pupil competencies on a common scale, at the beginning (Grade 2) and end of primary (Grade 6), now enables better analysis and understanding of the effectiveness and equity of education systems, in line with other international programmes such as OCDE/PISA,138 PIRLS,139 IEA/TIMSS140 or SACMEQ.141 PASEC launched its first international assessment in 2014 in ten countries (Benin, Burkina Faso, Burundi, Cameroon, Chad, Congo, Côte d’Ivoire, Niger, Senegal and Togo). The sample included close to 40,000 pupils overall, surveyed in over 1,800 schools.142 PASEC tests are an external measurement of education systems and are not designed to measure the degree to which specific objectives inherent to each education system are met, nor those of a particular academic year. The PASEC2014 tests taken at the beginning of the primary cycle were administered individually to Grade 2 pupils to diagnose their abilities in the language of instruction and mathematics at this early stage, and to carry out an initial assessment of their core competencies. The test also aims to identify the main difficulties encountered in the learning process before they become more substantial and lead to failure and drop-out. In each country a national team has been responsible for implementing the programme. In Congo, the Director of the Studies and Planning Directorate (DEP) within the Ministry of Education143 has been assigned as national PASEC team leader. In 2017, the national PASEC2014 report for Congo was made available.144 It indicated that 62 percent of pupils do not reach the threshold "sufficient" in language at the beginning of schooling. Among all pupils at the beginning of schooling, about 4.8 percent do not demonstrate any of the most basic skills measured by this test in the language of schooling. In mathematics, 29.1 percent of pupils do not reach the "sufficient" skill threshold at the beginning of schooling. Of all the pupils at this level, 3.5 percent do not demonstrate any of the skills measured by this test in mathematics in the language of schooling, not even the most basic. Only 43 percent of pupils starting grade 6 achieved the minimum acceptable score in French assessment, and only 34 percent reached that minimum level in mathematics. This means that most pupils leaving primary school did not have sufficient foundational literacy and numeracy skills. When compared to other Francophone countries participating in this 137 Conférence des ministres de l'Éducation des États et gouvernements de la Francophonie. 138 PISA: Programme for International Pupil Assessment. 139 PIRLS: Progress in International Reading Literacy Study. 140 IEA: International Association for the Evaluation of Educational Achievement. TIMMS: Trends in International Mathematics and Science Study. 141 SACMEQ: The Southern and Eastern Africa Consortium for Monitoring Educational Quality. 142 PASEC2014, Education System Performance in Francophone Sub-Saharan Africa, Competencies and Learning Factors in Primary Education, Programme d'Analyse des Systèmes Éducatifs de la CONFEMEN, Dakar (Sénégal), 2015. 143 Ministère de l'Enseignement Primaire, Secondaire et de l'Alphabétisation, MEPSA. 144 PASEC (2017). PASEC2014 - Performances du système éducatif congolais : Compétences et facteurs de réussite au primaire. PASEC, CONFEMEN, Dakar 76 international assessment, Congo did not fare better than average despite its middle￾income status. The trial period for the PASEC2019 tests is due to start in April. A team of education specialists, who have worked on evaluation activities either under the previous PRAEBASE project or as part of the PASEC team, have now become members of an evaluation team (Programme national pour l’évaluation des acquis scolaires, PNEAS). It is intended to be the basis for the creation of a permanent evaluation team that will head an independent, expert commission for evaluating education system performance in Congo. As part of the present phase of the MGD evaluation (baseline), the ET plans to meet with the PNEAS to examine the extent to which this expert commission is already operational and would consider a possible cooperation with WFP to assess literacy learning outcomes throughout the implementation of the USDA MGD programme. Meanwhile, the ET has developed (in French) and submitted to WFP the so called "Termes de reference du test MGD de langue de fin de 2ème année de scolarité primaire" (TOR for the MGD French test by the end of 2nd grade of primary school) as a possible framework for discussion between WFP and MEPSA regarding the assessment of improved literacy of school age children. This document clearly indicates that the MGD test does not attempt to perform a diagnostic assessment for remediation but is a summative evaluation focusing specifically on improved literacy of school age children by the end of two grades of primary schooling as a result of USDA assistance. 77 Annex 24. Report from the PASEC literacy test REPUBLIQUE DU CONGO Unité* Travail* Progrès ------------- MINISTERE DE L’ENSEIGNEMENT PRIMAIRE, SECONDAIRE ET DE L’ALPHABETISATION ------------- CABINET ------------- EQUIPE NATIONALE DU PROGRAMME D’ANALYSE DES SYSTEMES EDUCATIFS DE LA CONFEMEN ------------- Test McGovern-Dole Année scolaire 2017/2018 ------------- PERFORMANCES DES ELEVES DE DEUXIEME ANNEE DU PRIMAIRE EN COMPREHENSION DE L’ECRIT Brazzaville, juin 2018 78 Équipe nationale PASEC du Congo Rédacteur • Monsieur Patrice NDOUDI, Statisticien, Chef de service de la planification à la Direction des études et de la Planification. Sous la supervision de : • Monsieur Jean Paulin ITOUA, Responsable national de l’équipe PASEC, Directeur des études et de la Planification ; • Bertin GOULAMIELE, Pédagogue didacticien, Inspecteur pédagogique ; • Raphael WANDO, Pédagogue didacticien, Directeur de l’Enseignement Primaire. 79 Le bureau d’étude KonTerra a été recruté par le Programme Alimentaire Mondial (PAM) pour suivre l’évolution du programme McGovern-Dole (MGD) en république du Congo. La mise en œuvre de ce programme qui permettra au PAM de continuer ses activités de cantines scolaires pour une période de cinq (5) ans, bénéficie de la contribution financière du Département de l’Agriculture des Etats-Unis d’Amérique (USDA). Il implique la participation, entre autres, du Ministère de l’Enseignement Primaire, Secondaire et de l’Alphabétisation (MEPSA). Le présent rapport donne les résultats de l’étude de référence préalable à la mise en œuvre de ce programme. Cette étude ne concerne que les élèves de fin de deuxième année du primaire (CP2). Elle permettra d’identifier les capacités et les difficultés des élèves en langue notamment en compréhension de l’écrit à travers une série de quatre (4) exercices. Pour cela, le niveau de langue des élèves de fin de CP2 doit être mesuré en début, milieu et fin de programme. C’est ainsi qu’une étude de référence est faite en 2018, puis viendra une évaluation à mi-parcours en 2020 et une évaluation finale en 2022. Ce rapport est composé de quatre sections : (i) la méthodologie du test ; (ii) le test de l’évaluation ; (iii) les compétences des élèves en compréhension de l’écrit ; (iv) la synthèse du constat, une piste de réflexion et les difficultés rencontrées. I. METHODOLOGIE La méthodologie a été conçue dans le but d’évaluer les compétences des élèves en langue française. Elle a suivi la démarche suivante : l’élaboration des items, la validation des items, l’élaboration des supports de passation, le recrutement des administrateurs de test, la formation des administrateurs de test, l’administration du test sur le terrain, la vérification des données, la saisie des données et l’analyse. I.I. Elaboration des items Quatre (4) enseignants titulaires de la deuxième classe du primaire ont été désignés pour l’élaboration des items dans le but de présenter un échantillon qui réponde à la progression des programmes afin de mettre sur une même échelle les résultats des élèves des établissements scolaires concernés pour cette étude. Ces items ont été testés dans quatre (4) classes de deuxième année du primaire. Les observations et les suggestions faites par les enseignants ont permis à l’équipe nationale du Programme d’Analyse des Systèmes Educatifs de la CONFEMEN (PASEC) d’avoir une série de quatre (4) exercices à soumettre à l’administration du test de langue. I.2. La validation des items Ces items ont été soumis à une pré-enquête administrée par quatre (4) Inspecteurs chargés des Activités Pédagogiques (ICAP) auprès des élèves de deuxième année du primaire dans les 80 écoles qui sont sous leur responsabilité. Les résultats de cette enquête préliminaire ont permis à l’équipe nationale, de concert avec les inspecteurs, de finaliser et valider les items. I.3. L’élaboration des supports de passation C’est à la suite de la validation des items que les supports de passation ont été rédigés. Ces supports sont au nombre de cinq (5) : - support de l’élève ; - cahier de l’élève ; - cahier de l’administrateur ; - fiche de suivi élève ; - fiche de suivi école. 1.4. Recrutement des administrateurs du test L’équipe nationale du Programme d’Analyse des Systèmes Educatifs (PASEC) a adressé une correspondance à l’Inspection Générale de l’Enseignement Primaire, Secondaire et de l’Alphabétisation (IGEPSA) qui a mis à sa disposition vingt-sept (27) Inspecteurs chargés des Activités Pédagogiques (ICAP), de préférence ceux qui avaient participé à l’évaluation PASEC2014. 1.5. La formation des administrateurs du test Les Inspecteurs chargés des Activités Pédagogiques ont suivi une formation de deux (2) jours selon le programme ci-après : - Imprégnation des supports de passation ; - Simulation sur les cas pratiques de terrain. 1.6. L’administration du test sur le terrain Vingt-sept (27) administrateurs ont été déployés sur le terrain pour une durée de cinq (5) jours dans sept (7) départements du pays à savoir : Lékoumou, Bouenza, Pool, Plateaux, Cuvette, Sangha et Likouala. Les administrateurs du test ont été responsables de la collecte de données au niveau des écoles. Le test a été administré dans la matinée. Afin de garantir la comparabilité des données recueillies, les administrateurs étaient formés à respecter scrupuleusement les consignes. Le test a été administré individuellement aux élèves, selon des consignes d’administration standardisées. L’administrateur a procédé à l’évaluation de seize (16) élèves au maximum par classe. Les élèves ont répondu oralement aux questions, avec des réponses très brèves. 81 1.7. Vérification des données La vérification des données a été faite au fur et à mesure que les administrateurs sont rentrés du terrain. Au dépôt des colis, les administrateurs ont été soumis à un contrôle sur la cohérence des informations contenues dans les deux types de fiches, le cahier de l’élève et aussi sur la grille des résultats. 1.8. Saisie des données Le masque de saisie a été conçu dans le programme Epidata comme dans le PASEC2014. Il a porté sur les variables suivantes : nom de l’élève, prénom de l’élève, sexe, âge, nom de l’école, le département, la circonscription scolaire, la note pour décoder le sens des mots, la note pour lire et comprendre des phrases, la note pour la compréhension du texte 1, la note pour la compréhension du texte 2. La saisie a mobilisé dix (10) opérateurs de saisie qui ont travaillé pendant 4 jours. Le dizième (1/10) des cahiers saisis par chaque opérateur dans la journée a été soumis à une double saisie pour minimiser les erreurs. II. LE TEST DE L’EVALUATION Les différents exercices proposés à l’évaluation ont été construits sur la base de la progression du programme. Pour éviter les disparités entre les élèves évalués, les notions étudiées au troisième trimestre de l’année scolaire ont été écartées. C’est ainsi que nous avons retenu les mots contenant les diphtongues en, an, on, ai, ec, ou, ch, gu, qu supposés vus aux premier et deuxième trimestres. Le test a été administré en français. Pour la plupart des élèves, le français n’est pas la langue maternelle. Les exercices soumis à l’évaluation du test sont présentés en annexe de ce rapport. II.1. Le test Le test a été administré individuellement aux élèves de deuxième année du primaire pour diagnostiquer les capacités acquises au cours des premiers apprentissages de la langue d’enseignement, notamment en compréhension de l’écrit, afin de déterminer s’ils disposent des connaissances et compétences suffisantes et établir au plus tôt un premier bilan en début du programme McGovern-Dole. Quatre (4) exercices ont été proposés et comprennent chacun une particularité : • Exercice 1 : Décoder le sens des mots. 82 L’élève lit à voix haute ou basse les mots proposés et montre l’image qui va avec ces mots. Il n’est pas évalué sur sa capacité à lire le mot mais uniquement sur sa capacité à identifier le sens du mot écrit. • Exercice 2 : Lire et comprendre des phrases. L’élève lit à voix haute les phrases proposées et répond à une question de compréhension pour chaque phrase. • Exercices 3 et 4: Comprendre un texte. L’élève lit à voix haute ou basse le texte proposé et répond à une série de quatre (4) questions de compréhension. Il est évalué uniquement sur sa capacité à répondre correctement aux questions, la précision de sa lecture n’est pas évaluée ici. Encadré II.1 : Présentation du sous domaine disciplinaire évalué dans le test. Domaine évalué Exercices et compétences évaluées Temps de passation approximatif (consignes et exemples compris) Compréhension à l’écrit • Décoder le sens des mots • Lire et comprendre des phrases • Comprendre le texte 1 • Comprendre le texte 2 • 2 minutes • 4 minutes • 6 minutes • 6 minutes Compréhension de l’écrit : La compréhension de l’écrit est évaluée à travers des situations de lecture de mots et phrases isolés et de textes dans lesquels les élèves sont amenés à retrouver, à combiner et à interpréter des informations. Le développement des compétences dans ce domaine permet aux élèves de lire de façon autonome dans des situations quotidiennes variées pour développer leurs savoirs et participer à la vie en société. II.2. L’échantillon et le taux de réponse II.2.1. L’échantillon Les données de l’évaluation ont été collectées à partir d’un échantillon de quatre-vingt (80) écoles dont trente-six (36) écoles bénéficiaires des produits PAM, trente-six écoles (36) non bénéficiaires des produits PAM et huit (8) écoles ORA (Observer-Réfléchir-Agir), écoles fréquentées par les élèves autochtones. Ce choix a été réalisé par le Programme Alimentaire Mondial. Au niveau enquêté, une classe de l’école a été sélectionnée aléatoirement pour participer à l’enquête puis on a tiré un échantillon aléatoire simple de seize (16) élèves dans cette classe pour passer le test en passation individuelle. II.2.2. Le taux de réponse 83 Après la collecte des données, 79 écoles sur les 80 échantillonnées ont effectivement été enquêtées d’où un taux de participation des écoles de 98,75 %. Tableau II.1 : Échantillons d’écoles et d’élèves prévus et réalisés et taux de participation Département Echantillon d’écoles Echantillon d’élèves Prévu Réalisé Taux de participation Prévu Réalisé Taux de participation Bouenza 17 16 94,12 272 240 88,79 Cuvette 6 6 100 96 91 94,79 Lékoumou 14 14 100 224 202 90,18 Likouala 8 8 100 128 120 93,75 Palteaux 14 14 100 224 177 79,02 Pool 18 18 100 288 200 69,44 Sangha 3 3 100 48 39 81,25 Total 80 79 98,75 1280 1069 83,52 Sur les 80 écoles prévues, 79 ont participé à l’évaluation du test, soit un taux de participation de 98,75%. Pour l’école absente, c’est au niveau de la circonscription scolaire que l’administrateur s’est rendu compte que l’école était fermée. Celui-ci n’a pas appelé son superviseur pour qu’il procède à un remplacement d'école. Il sied également de noter que l’administration du test s’est faite pendant la semaine des compositions de fin d’année. Les élèves de CP2 n’étaient plus à l'école. C’est grâce à une large communication que le taux de participation des élèves de 83,52% a été atteint. Ainsi, les effectifs d’élèves administrés se répartissent comme indiqué dans le tableau ci￾dessous. Tableau II.2 : Effectifs d’élèves administrés par type d’école, par sexe et selon le département Les garçons (52,01%) ont été plus présents au cours du test que les filles (47,99%). III- LES COMPETENCES DES ELEVES EN COMPREHENSION DE L’ECRIT Garçons Filles Garçons Filles Garçons Filles BOUENZA 66 62 128 55 57 112 240 CUVETTE 21 22 43 22 26 48 91 LEKOUMOU 55 49 104 57 41 98 202 LIKOUALA 62 58 120 120 PLATEAUX 51 36 87 50 40 90 177 POOL 43 46 89 56 55 111 200 SANGHA 18 21 39 39 Total 236 215 451 240 219 459 80 79 159 1069 Ecoles PAM Département Ecoles NON PAM Ecoles ORA Total Total Total Total 84 Cette section a pour objet de décrire les compétences et les principales difficultés rencontrées par les élèves en début de scolarité en langue, notamment en compréhension de l’écrit, telles que mesurées par le test. III.1. Les niveaux de compétence Tableau III.1 : Description du test selon le total des points, et le niveau de compétence Total des points Niveaux de compétence Pourcentage d’élèves Description des compétences 15 à 20 Niveau 4 3,0 Les élèves ont atteint un niveau de compréhension à l’écrit qui leur permet de comprendre des informations dans des mots, des phrases et des textes d’une vingtaine de mots. 10 à 14 Niveau 3 6,9 Les élèves sont capables d’identifier le sens de mots isolés, de comprendre des phrases courtes et accusent des difficultés à la compréhension de textes d’une vingtaine de mots. 5 à 9 Niveau 2 17,3 Les élèves sont capables d’identifier le sens de mots isolés, de comprendre des phrases courtes et accusent des difficultés à la compréhension de textes. 2 à 4 Niveau 1 27,2 Les élèves n’ont pas les premiers contacts avec le langage écrit. Ils connaissent de grandes difficultés dans le déchiffrage et l’identification des lettres et syllabes. 0 à 1 Niveau 0 45,6 Les élèves éprouvent d’importantes difficultés dans les premiers contacts avec le langage écrit. III.2. Pourcentage d’élèves par niveau de compétences 45,6% d’élèves éprouvent d’importantes difficultés dans les premiers contacts avec le langage écrit. Seulement 3,0% des élèves ayant passé le test ont atteint un niveau de compréhension à l’écrit qui leur permet de comprendre des informations dans des mots, des phrases et des textes. III.2.1. Selon le type d’école Tableau III.2 : Pourcentage d’élèves par niveau de compétence et selon le type d’école Type d’école Note par niveau Total 0 1 2 3 4 Ecole non PAM 20,8% 11,9% 6,1% 2,6% 1,6% 42,9% Ecole ORA 4,0% 3,8% 4,2% 2,0% 0,8% 14,9% Ecole PAM 20,8% 11,5% 7,0% 2,3% 0,6% 42,2% Total 45,6% 27,2% 17,3% 6,9% 3,0% 100,0% Sur les 45,6% d’élèves qui éprouvent d’importantes difficultés dans les premiers contacts avec le langage écrit, les écoles PAM et non PAM comptent chacune d’elle 20,8% d’élèves. 85 Le pourcentage le plus élevé au niveau 4 est atteint par les écoles non PAM (1,6%). Par contre, les écoles PAM sont celles qui ont moins d’élèves (0,6%) à ce niveau. III.2.2. Selon le département Tableau III.3 : Pourcentage d’élèves par niveau de compétence et selon le département Département Note par niveau Total 0 1 2 3 4 BOUENZA 16,3% 3,9% 1,6% 0,5% 0,2% 22,5% CUVETTE 2,9% 1,8% 2,9% 0,7% 0,3% 8,5% LEKOUMOU 4,5% 6,6% 5,1% 2,2% 0,4% 18,9% LIKOUALA 2,2% 2,9% 3,6% 1,8% 0,7% 11,2% PLATEAUX 7,7% 4,6% 2,4% 1,4% 0,5% 16,6% POOL 10,2% 6,5% 1,0% 0,2% 0,8% 18,7% SANGHA 1,8% 0,9% 0,6% 0,2% 0,2% 3,6% Total 45,6% 27,2% 17,3% 6,9% 3,0% 100,0% Le département du Pool a atteint le pourcentage d’élèves le plus élevé au niveau 4. Il est de l’ordre de 0,8%. Cependant, la Bouenza a le plus d’élèves (16,3%) qui sont au niveau 0. III.2.3. Selon le genre Tableau III.4 : Pourcentage d’élèves par niveau de compétence et selon le genre Genre de l’élève Note par niveau Total 0 1 2 3 4 Masculin 22,8% 14,3% 9,2% 3,9% 1,8% 52,0% Féminin 22,7% 12,9% 8,1% 3,0% 1,2% 48,0% Total 45,6% 27,2% 17,3% 6,9% 3,0% 100,0% On constate que les différences de performance en compréhension de l’écrit entre garçons et filles sont peu significatives. III.2.4. Selon l’âge Tableau III.5 : Pourcentage d’élèves par niveau de compétence et selon l’âge Tranche d’âge Note par niveau Total 0 1 2 3 4 Age inférieur à 8 ans 15,4% 8,3% 4,4% 1,4% 0,8% 30,4% 8 ans 17,3% 10,9% 5,5% 2,4% 0,7% 37,0% Age supérieur à 8 ans 12,8% 8,0% 7,4% 3,1% 1,4% 32,6% Total 45,6% 27,2% 17,3% 6,9% 3,0% 100,0% 1,4% d’élèves au niveau 4 ont un âge supérieur à 8 ans. 86 III.3. Pourcentage d’élèves ayant obtenu au moins la moyenne des points dans chaque exercice III.3.1. Par département Tableau III.6 : Pourcentage d’élèves ayant obtenu au moins la moyenne des points dans chaque exercice et par département L’exercice 1 a été noté sur 4. Plus de la moitié d’élèves ont obtenu les notes 2, 3 ou 4 dans les départements de la Likouala (69,17%) et La Cuvette (59,34%). Les autres départements sont en dessous de la moyenne de l’effectif présenté. L’exercice 2 a été noté sur 8. C’est le département de la Lékoumou où 21,67% d’élèves ont obtenu la note allant de 8 à 4. Le département de la Bouenza a eu le pourcentage d’effectifs le plus faible (3,75%). L’exercice 3 a été noté sur 4. Le pourcentage d’élèves qui ont obtenu la note allant de 2 à 4 est de 35,15% (pourcentage le plus élevé) dans le département de la Likouala et de 7,50% (le plus faible) dans le département du Pool. L’exercice 4 a été noté sur 4. Le pourcentage d’élèves qui ont obtenu la note allant de 2 à 4 est au-dessus de 25%. Le pourcentage le plus élevé a été obtenu dans la Likouala (24,17%) et le plus faible dans la Bouenza (2,92%). III.3.1. Par type d’écoles Tableau III.7 : Pourcentage d’élèves ayant obtenu au moins la moyenne des points dans chaque exercice et par type d’école. Exercice Type d’école PAM NON PAM ORA Exercice 1 41,02 39,22 61,01 Exercice 2 7,98 13,29 20,75 Exercice 3 17,52 15,25 24,53 Exercice 4 4,88 10,89 20,13 Les élèves de l’école ORA ont obtenu le pourcentage d’effectifs le plus élevé dans les quatre (4) exercices. Cela veut dire que dans l’exercice1, 61,01% d’élèves ont obtenu les notes allant de 2 à 4 points. Dans l’exercice 2, les notes 4, 5, 6, 7 ou 8 ont été obtenu par 20,75% d’élèves. Dans les exercices 3 et 4, les notes allant de 2 à 4 ont été obtenu respectivement par 24,53% et 20,13% d’élèves. Bouenza Cuvette Lékoumou Likouala Plateaux Pool Sangha Exercice 1 23,75 59,34 47,52 69,17 42,94 41,00 35,90 Exercice 2 3,75 10,99 16,83 21,67 17,51 6,50 17,95 Exercice 3 7,08 24,18 35,15 27,50 13,56 7,50 15,38 Exercice 4 2,92 9,89 13,37 24,17 10,17 5,50 7,69 Département Exercice 87 IV. SYNTHÈSE DU CONSTAT ET PISTE DE RÉFLEXION Cette section soulève le principal constat relevé au cours de la section précédente et propose une piste de réflexion pour renforcer l’acquisition des compétences clés pour tous les élèves des cours préparatoires (CP). Etant entendu que l’échantillon n'est pas représentatif de l’ensemble des élèves de CP2, cette étude nous a toutefois permis de mieux comprendre le fonctionnement de notre système éducatif du point de vue de la qualité de l’apprentissage de la langue française au cours des deux premières années du primaire. En effet, la proportion des élèves en dessous du niveau 3 est très importante. Elle est de l’ordre de 90,1%. Comme nous le savons, en fin de deuxième année du primaire, les programmes de l’enseignement primaire indiquent que tous les apprentis lecteurs devraient être capables de lire et de comprendre un message court, simple et familier. La compréhension de supports écrits implique pour l’élève d’avoir atteint un premier niveau de déchiffrage pour reconnaître les mots familiers, automatiser les mécanismes liés au déchiffrage de l’écrit et posséder un niveau de compréhension orale suffisant dans la langue d’enseignement. C’est ainsi que la piste de réflexion suivante est proposée pour une amélioration de la qualité de l’apprentissage de la langue : mettre en place les mesures d’accompagnement qui permet de s’assurer que les élèves en difficulté bénéficient réellement d’un accompagnement adéquat qui leur permet de rattraper leur retard. IV. DIFFICULTES RENCONTRES ET PROPOSITIONS DE SOLUTIONS IV. 1. DIFFICULTES RENCONTRES Les difficultés rencontrées sont d’ordre organisationnel et financier. Du point de vue de l’organisation, le mois de juin est le mois des examens. Une étude réalisée au cours de cette période ne saurait atteindre l’objectif fixé. Pour cela, une large sensibilisation a été faite pour la réussite de cette activité ; dans certains cas, les directeurs d'école sont allés au village chercher les élèves pour passer le test. Les administratrices de la Sangha et de la Likouala sont allées à plus de 150 km du chef-lieu du département. L’administratrice de la Sangha a fait sept (7) jours de collecte et celle de Likouala quatorze (14) jours. S’agissant des finances, les moyens financiers mis à notre disposition n’ont pas permis de couvrir les rubriques qui ont précédé la rédaction du rapport. IV. 2. PROPOSITIONS DE SOLUTIONS 88 Le programme McGovern-Dole (MGD) a prévu une revue à mi-parcours en 2020. Pour la réussite du test, nous souhaitons que les préparatifs commencent au mois de février 2020 et que l’activité se réalise au mois de mai de la même année. Pour permettre la réussite de l’activité, le PAM devrait verser le 2/3 du budget au début et 1/3 au dépôt du rapport de l’activité à la structure organisatrice Annexe A : Les items du test EXERCICE 1 : DECODER LE SENS DES MOTS vélo balle la ball pantalon 89 banane chat EXERCICE 2 : LIRE ET COMPRENDRE DES PHRASES émile dessine. magui découpe les légumes. samedi, safou défilera devant la fanfare. monique repasse ses habits. 90 EXERCICE 3 : COMPRENDRE UN TEXTE 1 papa est dans la forêt avec son ami. la forêt est humide. il ramène une biche. 1-qui est dans la forêt ? 2-avec qui est-il dans la forêt ? 3-comment est la forêt ? 4-comment s’appelle l’animal que ramène papa? EXERCICE 4 : COMPRENDRE UN TEXTE 2 catherine aime sa maman. à 6 heures, elle se lève et balaie la maison. elle va à la fontaine et allume le feu. elle est une enfant modèle. 1- qui aime sa maman ? 2- à quelle heure se lève catherine ? 3- que fait-elle à 6 heures? 4- comment est catherine? 91 Annexe B : Données de l’évaluation Tableau B.1 : Pourcentage d’élèves par genre Genre Effectifs Pourcentage Pourcentage valide Pourcentage cumulé Masculin 556 52,0 52,0 52,0 Féminin 513 48,0 48,0 100,0 Total 1069 100,0 100,0 Tableau B.2 : Pourcentage d’élèves par niveau de compétence Niveau de compétence Effectifs Pourcentage Pourcentage valide Pourcentage cumulé 0 487 45,6 45,6 45,6 1 291 27,2 27,2 72,8 2 185 17,3 17,3 90,1 3 74 6,9 6,9 97,0 4 32 3,0 3,0 100,0 Total 1069 100,0 100,0 92 Tableau B..3. Effectifs des élèves administrés par département et selon le genre et l’âge Genre et âge de l’élève DEPARTEMENT Total BOUENZA CUVETTE LEKOUMOU LIKOUALA PLATEAUX POOL SANGHA Masculin Age élève 6 3 2 0 0 0 0 0 5 7 47 22 27 2 35 23 2 158 8 42 13 49 2 40 46 0 192 9 21 4 22 7 15 21 3 93 10 8 2 12 9 7 8 3 49 11 0 0 0 8 3 1 2 14 12 0 0 2 18 0 0 8 28 13 0 0 0 7 1 0 0 8 14 0 0 0 3 0 0 0 3 15 0 0 0 3 0 0 0 3 17 0 0 0 3 0 0 0 3 S/total 121 43 112 62 101 99 18 556 Féminin Age élève 6 3 1 0 0 1 3 0 8 7 42 23 24 1 30 32 2 154 8 48 22 44 3 32 52 2 203 9 19 2 18 6 6 11 2 64 10 7 0 4 13 5 3 1 33 11 0 0 0 8 1 0 1 10 12 0 0 0 18 1 0 13 32 13 0 0 0 4 0 0 0 4 14 0 0 0 3 0 0 0 3 15 0 0 0 2 0 0 0 2 93 S/total 119 48 90 58 76 101 21 513 Total Age élève 6 6 3 0 0 1 3 0 13 7 89 45 51 3 65 55 4 312 8 90 35 93 5 72 98 2 395 9 40 6 40 13 21 32 5 157 10 15 2 16 22 12 11 4 82 11 0 0 0 16 4 1 3 24 12 0 0 2 36 1 0 21 60 13 0 0 0 11 1 0 0 12 14 0 0 0 6 0 0 0 6 15 0 0 0 5 0 0 0 5 17 0 0 0 3 0 0 0 3 Total 240 91 202 120 177 200 39 1069 94 Annex 25. Comparison of results to literacy test at the end of two grades of primary schooling Levels MGD 2018 PASEC2014 Description of Competencies Distribution of Pupils throughout the Levels of the Scale Description of Competencies Distribution of Pupils throughout the Levels of the Scale Level 4 Students have achieved a level of written comprehension that allows them to understand information in words, sentences and texts of about twenty words. 3.0% Intermediate reader: enhanced reading autonomy is bolstering their understanding of sentences and texts. Pupils have acquired written language decoding and listening comprehension competencies that enable them to understand explicit information in words, sentences and short passages. They can combine their decoding skills and their mastery of the oral language to grasp the literal meaning of a short passage. 16.6% Level 3 Students are able to identify the meaning of isolated words, to understand short sentences and have difficulties in understanding texts of around twenty words. 6.9% Novice reader: gradual improvement of written language decoding, listening comprehension and reading comprehension skills. Pupils have improved their listening comprehension and decoding skills and can concentrate on understanding the meaning of words. In listening comprehension they are able to understand explicit information in a short passage containing familiar vocabulary. They gradually develop links between the oral and written language and thus improve their decoding skills and expand their vocabulary. In reading comprehension, they are able to identify the meaning of isolated words. 21.4% Capacity to read and understand the meaning of grade level text "Sufficient" Competency Threshold145 145 For each competency scale, a “sufficient” threshold has been set, that makes it possible to determine the share of pupils who face a greater probability of mastering (above the threshold) or not mastering (below the threshold) the knowledge and abilities deemed indispensable to pursue their schooling in good conditions. The thresholds are defined on the basis of the concepts assessed in the PASEC tests and according to the priority 95 Level 2 Students are able to identify the meaning of isolated words, understand short sentences, and have difficulty understanding texts. 17.3% Emerging reader: gradual development of written language decoding skills and reinforcement of listening comprehension skills. Pupils have improved their listening comprehension skills and are able to identify a lexical field. They are in the process of developing the first basic links between the oral and written language and can perform basic graphophonological decoding, recognition and identification tasks (letters, syllables, graphemes and phonemes). 28.6% Level 1 Students do not have the first contact with written language. They have great difficulty in deciphering and identifying letters and syllables. 27.2% Early reader: first contact with the oral and written language. Pupils are able to understand very short and familiar oral messages to recognize familiar objects. They have great difficulty decoding written language and performing graphophonological identification (letters, syllables, graphemes and phonemes). 28.6% Below Level I Students experience significant difficulties in first contact with written language. 45.6% Pupils at this level do not display the competencies measured by this test. These pupils are in difficulty when it comes to Level 1 knowledge and competencies. 4.8% goals of school curricula in language-reading and mathematics, at the beginning and the end of the primary cycle. 96 Annex 26. Various concepts of dietary diversity in Congo During secondary data collection and informal interviews, the ET team realised that there is a high need to harmonise the approach to improved nutrition outcomes as envisaged in the conceptual framework (Result Framework 2) The following observations have been made in this context: 1. Dietary Diversity Working on improved dietary intake would need clear key messages which will be used for these activities. In this context, the documents reviewed by the ET show a lack of coherence. The Republic of Congo has used for a long time 4 food groups, which are shown in the box below: Food Groups and Main Nutrients Group I. Energy foods (carbohydrates): - Roots, tubers, starch (yams, cassava, potato, plantain, taro, …), - Cereals (rice, corn, millet, sorghum, pasta, bread, wheat semolina, … ) Group II. Body building nutrients (protein): -Animal origin (meat, fish, eggs, milk and milk products, poultry, insects, …) - plant origin (beans, lentils, peas, groundnuts, soy bean, squash seeds, sesame …) Group III. Complementary energy foods (fats): - Oils and fats (groundnut oil, palm oil, margarine, butter, palm kernel nut sauce, …) - protein (avocado, safou, peanut, soy, squash, sesame…) Group IV. Protective foods (vitamins and minerals): -Vegetables (tomatoes, aubergines, spinach, amaranth, « saka-saka », « matembélé », carrots, …) -Fruits (pineapple, mango, papaya, oranges, tangerines, bananas, wild fruits like malombos, tondolos, …) UNICEF has used 4 food groups as well in the booklet on key messages for community education. However, these food groups are very different – see image below: 97 This counselling card was prepared for children under 2 years of age and has therefore at the centre the continuation of breastfeeding. Looking at the 4 food groups it is evident, that there is a tremendous difference which is explained as well in the key messages under the picture. 1. The group of the most commonly eaten basic foods like grains, roots and tubers are used the same way 2. In the group of fruits and vegetables, there is a distinction between those rich in Vitamin A (on top) and other fruits and vegetables 3. The protein foods are grouped as well in (a) meat and fish, (b) eggs and (c) milk and milk products 4. There is one group on legumes which includes of groundnuts and sesame seeds, which are of course as well rich in plant-based oils. Fats and oils are not specifically mentioned The Ministry of Health provided another list of 8 food groups – see table below Food group Main nutrients 01 Cassava (Chikwangue)/Foufou, Rice, Bread or donuts, other cereals (pasta, etc.), other roots or tubers (sweet potatoes, Irish potatoes, Yams) and plantains, sugar, honey and other sweets Carbohydrates 02 Beans, peas, lentils Haricot, Pois, lentils, cowpeas, squash, soy bean, groundnut (paste), sesame and other legumes, amaranth Rich in protein (mix of protein and lipids) 98 03 Légumes (Amaranths, cassava leaves, endives, spinach, sorrel, aubergines, zucchini…) Minerals (iron, ...) 04 Orange fleshed vegetables (squash, carrots, orange fleshed sweet potatoes), orange fruits (papaya, mango, oranges), wild fruits, other fresh fruits Vitamin A, other vitamins 05 Meat including bush meat, poultry, snails, shrimps, fish (fresh, salted, smoked), edible insects, giblets etc. Protein and heme iron 06 Eggs Protein, minerals and vitamins 07 Milk, cheese, yogurt and other milk products Protein, calcium 08 Oil/Butter/Margarine Lipids Except the 8th food group (oil, butter, margarine), the 7 remaining food groups are completely identical with the ones used in the dietary diversity score for small children, which has been proposed by the World Health organisation – see list below. The ET decided to use the above mentioned 7 food groups to indicate dietary diversity among the school age children. They are close to the 8 food groups but are more in line with international recommendations. 99 WFP used in its CFSVA altogether 9 different food groups, including oil and even sugar. In addition, WFP did not make a distinction between Vitamin A rich fruits and vegetables. 2. Nutrition education should include education on WASH WASH and nutrition education are treated as two different activities. However, looking at the UNICEF conceptual framework of malnutrition, it is clear that improved nutrition outcomes will only be possible if both activities are put into place. The fact, that UNICEF will do nutrition training for 800 people as planned in the PMP and will do WASH training only for 290 people in the first year of implementation shows that WASH and nutrition education are not perfectly synchronised. UNICEF has worked out key messages for infant and young child feeding and they have included both in the document. (Livret de messages clés – Le paquet des Counseling Communautaire pour l’Alimentation du Nourrisson et du Jeune Enfant (ANJE)). 100 Bibliography / Desk Review Comfemen; PASEC 2014, Performance des systems éducatifs en Afrique subsahariènne francophone; 2014 FAS; Fiscal Year (FY) 2017 Food Assistance Proposal Guidance and Notice of Funding Opportunity; 2016 Republic of Congo; Document de stratégie pour la croissance, l’emploi et la réduction de la pauvreté (DSCERP 2012-2016); 2012 Repubic of Congo; Enquête démographique et de santé du Congo; 2011-2012 Republic of Congo; Politique nationale d’alimentation scolaire; 2016 Republic of Congo; Stratégie sectorielle de l’éducation 2015-2025; 2015 UNDP; Human Development Report 2016- Human Development for Everyone; 2016 WFP; Enquête FRAT sur la consommation de la farine de manioc en République du Congo; 2016 WFP; How School Meals Contribute to the Sustainable Development Goals – A Collection of Evidence; 2017 WFP; WFP Gender Policy 2015-2020; 2015 WFP; Nutrition Policy; 2017 WFP; School Feeding Policy – Promoting innovation to achieved national ownership; 2013 WFP; WFP Republic of Congo, FY2017 McGovern-Dole Proposal; 2017 WFP, USDA/FAS; Project Agreement Between the Foreign Agricultural Services and the World Food Programme for the Donation of Agricultural Commodities and Related Assistance Under the McGovern-Dole International Food for Education and Child Nutrition Programme; September 2017 [Name of commissioning Office] [Link to the website] [Place, Month and Year, Report number]