Foreign Agricultural Service United States Department of Agriculture FY 19 McGovern-Dole International Food for Education and Child Nutrition Project - The Future is Ours! Brakna and Gorgol Regions, Mauritania BASELINE ASSESSMENT REPORT December 2020 This publication was produced at the request of the United States Department of Agriculture. It was prepared independently by IMC Worldwide for Counterpart International. The Future is Ours! Baseline Performance Evaluation Report The McGovern-Dole International Food for Education and Child Nutrition project in Mauritania, named The Future is Ours!, seeks to improve literacy for school-aged children Mauritania’s Brakna and Gorgol regions. The project is funded by the United States Department of Agriculture (USDA) and implemented by Counterpart International (Counterpart) with support from local partners Economie et Devéloppement (EcoDev), Organisation Non Gouvernementale (ONG) Actions, and regional partner Associates in Research and Education for Development (ARED). Acknowledgements IMC would like to thank USDA and Counterpart International and The Future is Ours! project staff for the insight they provided into Mauritania and its educational system. Their guidance and reviews of report drafts were crucial contributions to this final baseline assessment report. IMC also thanks the Mauritanian Ministry of Education at central and regional levels and the project’s beneficiaries for facilitating the field work. Publication Information This report was prepared by IMC Worldwide under contract to Counterpart International as part of its work with the McGovern-Dole International Food for Education and Child Nutrition known as The Future is Ours! Agreement Number: FFE-682-2019/003-00-A Project Duration: 2019-2024 Implemented by: Counterpart International Evaluation Authored by: Jason Wares, Team Leader Christopher Root, Co-Evaluator Andrew Trembley, EGRA Specialist Rachel Hoffman, Project Manager Cover Photo Source: Counterpart International DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov Table of Contents EXECUTIVE SUMMARY......................................................................................................................................................... 1 1. INTRODUCTION........................................................................................................................................................... 6 PROJECT DESCRIPTION ..................................................................................................................................................... 6 BASELINE ASSESSMENT REPORT BACKGROUND SCOPE..........................................................................................................15 2. ASSESSMENT METHODOLOGY...................................................................................................................................16 DATA SOURCES AND DATA COLLECTION METHODS..............................................................................................................16 Early Grade Reading Assessment (EGRA).............................................................................................................16 Other Surveys and Qualitative Data Collection....................................................................................................19 Changes to The Assessment Implementation Due to COVID-19 ..........................................................................20 SAMPLING STRATEGY .....................................................................................................................................................20 Assessment Design Overview...............................................................................................................................20 EGRA Sample Size and Selection ..........................................................................................................................21 Other Survey Sample Size and Selection ..............................................................................................................23 DATA COLLECTION TRAINING...........................................................................................................................................23 Tool Refinement, Piloting and Debrief .................................................................................................................24 Data Analysis .......................................................................................................................................................24 Data Quality Control and Assurance....................................................................................................................24 LIMITATIONS ................................................................................................................................................................25 3. RESULTS ....................................................................................................................................................................25 KEY INDICATORS............................................................................................................................................................26 MGD SO1: Improved Literacy of School Aged Children ........................................................................................26 MGD 1.1 Improved Quality of Literacy Instruction ..............................................................................................28 MGD 1.2: Improved Attentiveness.......................................................................................................................29 MGD 1.3: Improved Student Attendance ............................................................................................................29 MGD 1.3.4: Improved Student Enrollment...........................................................................................................31 MGD SO2: Increased Use of Health, Nutrition and Dietary Practices..................................................................32 MGD 2.4: Increased Access to Clean Water and Sanitation Services...................................................................33 RESEARCH QUESTIONS ...................................................................................................................................................34 Project Level Performance ...................................................................................................................................34 School feeding and nutrition................................................................................................................................34 Education and Literacy Research .........................................................................................................................35 Health and Maternal & Child Health (MCH) ........................................................................................................35 Methodological Research Questions....................................................................................................................37 School Feeding Sustainability...............................................................................................................................39 4. CONCLUSIONS AND RECOMMENDATIONS ................................................................................................................41 RECOMMENDATIONS .....................................................................................................................................................41 Promoting School Feeding Sustainability.............................................................................................................41 Measuring Project Performance ..........................................................................................................................42 Improved Literacy of School Aged Children..........................................................................................................43 Improved Quality of literacy instruction ..............................................................................................................44 Improved Attentiveness.......................................................................................................................................44 Improved Student Attendance and Enrollment....................................................................................................45 CONCLUSION................................................................................................................................................................45 ANNEX 1: KEY INDICATORS AND BASELINE VALUES...........................................................................................................46 ANNEX 2: EGRA VALIDITY ..................................................................................................................................................49 ANNEX 3: EGRA EXPLANATION AND RESULTS....................................................................................................................50 ANNEX 4: BASELINE ASSESSMENT SCOPE OF WORK .........................................................................................................57 ANNEX 5: DATA COLLECTION INSTRUMENTS ....................................................................................................................69 ANNEX 6: RESULTS FRAMEWORK .....................................................................................................................................81 Tables Table 1 Quantitative Survey Sample Sizes……………………………………………………………………………………………..4 Table 2 EGRA Index Calculation................................................................................................................ 19 Table 3 EGRA sample composition ........................................................................................................... 22 Table 4 Quantitative Survey Sample Sizes................................................................................................ 23 Table 5 Percentage of students that can read 20 letters per minute in Arabic and in French .............. 266 Table 6 Reading proficiency; pilot vs non-pilot ........................................................................................ 27 Table 7 Proficiency by gender................................................................................................................... 27 Table 8 Percentage of teachers that had received training in the previous two years ........................... 28 Table 9 Quality of Teacher Training.......................................................................................................... 29 Table 10 Student attentiveness................................................................................................................ 29 Table 11 Most common reasons students miss school............................................................................ 30 Table 12 Percentage of students who wash their hands.......................................................................... 32 Table 13 Status of school latrines............................................................................................................. 33 Table 14 School sources of water............................................................................................................. 34 Table 15 Participation in extra-curricular activities………………………………………………………………………………35 Table 16 Percentage of schools that distribute deworming medication…………………………………………..….36 Table 17 Attendance at schools with and without good latrines………………………………………………………….37 Table 18 Attendance at schools with a working water source……………………………………………………………..37 Table 19 Percentages of boys and girls that miss school because of latrine or water quality……………….37 Table 20 Alpha ranges………………………………………………………………………………………………………………………….49 Table 21 Alpha statistics for the baseline dataset…………………………………………………………………….............49 Table 22 EGRA scores by task – French……………………………………………………………………..…………………………50 Table 23 EGRA scores by task – Arabic…………………………………………………………………………………………………51 Figures Figure 1 Student attendance by grade ..................................................................................................... 30 Figure 2 Enrollment by Grade................................................................................................................... 32 Acronyms and Abbreviations ARED Associates in Research and Education for Development Counterpart COVID Counterpart International Coronavirus disease CSB+ Corn-Soy Blend Plus DNES Direction de la Nutrition et de l’Education Sanitaire/ Department of Nutrition and Health Education DREN Direction Régionale de l’Education Nationale/ Regional Directorate of National Education Ecodev Economie et Développement EGR Early Grade Reading EGRA Early Grade Reading Assessment G grams GCNF Global Child Nutrition Forum GoM Government of Mauritania HQ Headquarters IDEN National Education Inspection Department IMC IMC Worldwide, Inc. McGovern-Dole McGovern-Dole International Food for Education and Child Nutrition MCH Maternal and Child Health MCN Maternal and Child Nutrition MEL Monitoring, Evaluation, and Learning MOE Ministry of National Education, Training and Reform MOH Ministry of Health NGO Non-Governmental Organization ONG Organisation Non Gouvernementale ONSER Office National des Services d’Eau en Milieu Rural/National Office for Water Services in Rural Areas PASEC Programme d’analyse des systems éducatifs de la confem PTA Parent Teacher Association PY Project Year SCC School Canteen Committees SMC School Management Committees SO Strategic Objective SSME Snapshot of School Management and Effectiveness TOT Trainer of Trainer TWG Technical Working Group UNICEF United Nations Children’s Emergency Fund US United States USD United States Dollar USDA WASH WFP United States Department of Agriculture Water, Sanitation and Hygiene World Food Program 1 Executive Summary The McGovern-Dole International Food for Education and Child Nutrition (McGovern-Dole) project, The Future is Ours!, seeks to improve literacy for school-aged children in Mauritania’s Brakna and Gorgol regions. The project is funded by the United States Department of Agriculture (USDA) and implemented by Counterpart International (Counterpart) with support from local partners Economie et Développement (EcoDev), Organisation Non Gouvernementale (ONG) Actions, and regional partner Associates in Research and Education for Development (ARED). The project agreement with USDA was signed on September 23, 2019 and is scheduled to run through September 30, 2024. The project’s total budget is $22.5 million USD. The Future is Ours! is focused on achieving the following objectives: • Improve student attendance rates by providing nutritious daily school meals; • Improve health and dietary practices by providing access to deworming medications; • Increase knowledge of nutrition, Water, Sanitation, and Hygiene (WASH), and safe food preparation and storage practices; • Improve school infrastructure and increase access to clean water and sanitation by building storerooms, latrines, and water station systems; • Improve the literacy of school age children through the implementation of a multi-layered pilot program; and • Increase capacity of national stakeholders to improve the quality of the primary school system, and to lead and maintain school feeding programs. The project aims to serve over 140,000 direct beneficiaries, specifically: • 127,100 students in 209 selected schools who will benefit from school meals provision. School meals will consist of a nutritious breakfast five days a week and mid-day school meal four days a week. Some students in the 51 pilot schools will also participate in literacy-focused activities, such as reading clubs. • 192 teachers in the 51-literacy pilot selected schools who will take part in the literacy-focused activities, such as training and coaching sessions. • 209 school administrators in the 209 selected schools, some of which will also participate in literacy-focused activities, such as training on Early Grade Reading Assessment (EGRA). • 10,265 Parent Teacher Association (PTA) participants, School Management Committees (SMC) members, pregnant and lactating mothers, and cooks who will participate in training in food preparation, storage practices, good health and nutrition practices, community mobilization for school canteens, and management of those canteens. • 3,247 children under five-years-old through Take-Home Rations (THR) provided to mothers participating in Maternal and Child Nutrition (MCN) activities. • 10 government officials who will participate in training and capacity building activities. Among the project’s innovations is a primary school literacy activity that uses best practices in EGRA. The pilot will be implemented in grades 1 – 3 in 51 of the project’s schools and will test a holistic approach to improve student learning outcomes. 2 Baseline Purpose and Methodology This baseline assessment report presents values for the project’s key indicators, answers to research questions included in the baseline scope of work, guidance for future evaluations, and recommendations for project implementation. The values for key indicators are presented in Annex 1. The research questions comprised 12 questions that the baseline evaluation sought to operationalize so that future evaluations will measure them consistently and accurately. These questions were the following: •Have program outputs and outcome targets been achieved? •What is the effect of school feeding on attendance, enrollment, and attention? •What is the interaction between school feeding and improved hygiene practices? •How effective are reading-oriented extra-curricular activities in improving literacy? •How effective are teacher trainings? •What is the effect of deworming medicine on student attendance? •What is the effect of latrine and water access on student attendance, especially for girls? •Is there behavioral change in handwashing for students? •What is the best way to measure the three undefined McGovern-Dole outcome indicators (MGD 1.1, 1.2 and 1.3.2)? •How reliable is school and government-collected attendance and enrollment data? How can the accuracy be improved? • What is the government capacity to manage school feeding at regional and national levels? • What commitment has the government shown regarding school feeding? (e.g., clearly defined roles for managing school feeding, plans to expand school feeding budget)? The baseline assessment followed a mixed methods approach for data collection and analysis that included secondary sources1, qualitative data collection, and quantitative data collection through 1 The evaluation used secondary sources to inform the background section of this report and to design the French and Arabic EGRA tools. Source: Counterpart International 3 stakeholder questionnaires. Specifically, the assessment collected data through the following types of surveys and interviews: 1. Student survey for grades 1-6 focused on collecting information about student absences and WASH practices. 2. Teacher survey focused on teacher absenteeism, student attentiveness, teaching materials, and teacher trainings received prior to The Future is Ours! 3. School director survey/school observation checklist included questions on attendance, enrollment, and water and sanitary facilities. 4. Qualitative interviews with parents focused on student WASH practices, food consumption, and school absences. 5. Qualitative interviews with government officials, as well as regional and departmental inspectors, focused on challenges to improving literacy and recommendations for The Future is Ours! The assessment also included a measurement of primary school student literacy using the EGRA methodology. The IMC Worldwide, Inc. (IMC) assessment team created two EGRA tools: one in French and another in Arabic. Those tools assessed student performance in literacy activities (known as sub￾tasks), beginning with the most basic literacy skills and progressing to the most difficult skills. The range of skills that the EGRA assessed, from easiest to most complex, are: 1. Phoneme Recognition 2. Character Recognition 3. Word Recognition 4. Reading Passage 5. Reading Comprehension 6. Writing The baseline assessment used a quasi-experimental design that allows for a strong comparison between literacy instruction plus school feeding (pilot) schools and just school feeding schools (non￾pilot schools). The assessment used a two-stage cluster design. In total, 146 of 209 McGovern-Dole schools (95 randomly selected non-literacy pilot schools, 51 literacy pilot schools) were included in this assessment. A summary of the sample size by group is shown in Table 1 below. 4 Table 1: Quantitative Survey Sample Sizes Interviews per school McGovern-Dole Pilot Expected/Actual McGovern-Dole non-Pilot Expected/Actual Total Expected/Actual Schools 51 51 95 95 146 146 EGRA 3.84 204 181 380 381 584 562 Non-EGRA Student 6 306 301 570 554 876 855 Teacher ~ 2.5 127 128 237 211 365 339 Director/checklist 1 51 51 95 95 146 146 Total 2,117 2,048 Limitations The baseline evaluation had the following limitations: Coronavirus disease (COVID)-19 restrictions: The COVID-19 pandemic may have created bias in the assessment results. Respondents may have been less willing to participate in interviews, or eager to cut interviews short because they were fearful of virus transmission. Students may have been less likely to attend school. COVID-19 also limited the degree to which the assessment team could conduct trainings in person and participate in face-to-face interviews in Mauritania. Dependence on school records for enrollment and attendance rates: Several baseline values, such as school attendance and enrollment rates, rely on school records. Measurement of changes in student attendance and enrollment rely on the discipline with which schools maintain these records consistently. Enumerators examined school records at each school that they visited. Every school visited had classes for students in grades 1 through 6. Data collectors did not collect information on differences in enrollment between male and female students, so discrepancies in enrollment and attendance rates for male and female students are not included in the report. That data will be collected by the project staff and reported on. Dependence on enrollment and attendance records for literacy assessment: Students were randomly selected from enrollment and attendance lists to participate in the EGRA. School directors indicated in interviews that occasionally students who did not have official birth certificates may not have appeared in school records. Students who did not appear on enrollment or attendance lists would have been excluded from the possibility of taking the EGRA. Reliance on teachers’ availability: The assessment team could not take a random sample of working teachers. The assessment relies on interviewing teachers who were available on the day of the assessment, which may create bias in teacher responses. Reliance on perceptions and memory: Many of the baseline assessment’s measurements rely on respondent’s perception of behaviors, such as frequency of handwashing. Others asked respondents to remember reasons for behaviors, such as missing school or missing taking attendance. While these perceptions provide valuable information, they are not as accurate as other measurements that observe behaviors. Lack of information about literacy standards: The assessment team could not find information on Mauritanian standards for primary school literacy, as described in the report. Mauritania lacks 5 information about literacy standards because the Ministry of National Education, Training and Reform (MOE) has never used literacy evaluation tools. When asked by the baseline evaluation team about EGRA, the general inspector of education, the director of Department of Nutrition and Health Education (DNES), and the director of basic education department mentioned that they had never used the tool, therefore, they could not provide the threshold for literacy standards. Fortunately, an education reform is ongoing in Mauritania and the current Minister of Education has already agreed to implement EGRA in all government schools in the country. Therefore, the standards for primary school literacy will be later identified. The assessment team recommends a measure of proficiency in the report that may not reflect the Future is Ours! project goals or the policy of the Mauritanian Government. Key Findings and Recommendations The assessment provides the following among its findings and recommendations. Sustainability: The Future is Ours! is in a strong position to work with the Government of Mauritania (GoM) to invest in school feeding sustainability. Among the project’s assets are the GoM’s commitment to promoting school feeding. The GoM’s School Feeding Department reported that it would benefit from assistance with managing funding, and monitoring and assessment of school feeding projects. The baseline assessment recommends three ways that the project could work with the GoM to increase the likelihood that school feeding projects will continue after The Future is Ours! ends. First, the project could assist the Department of School Feeding in working with other government departments and agencies to maximize the efficiency of government resources. Second, building the GoM’s monitoring and assessment capacities offers an opportunity to connect central government initiatives with local government officials to improve information flow and local ownership of school feeding initiatives. Last, the project could work with the GoM to promote initiatives that would tie the central government, local government, and communities together. For example, the project could work with the government and communities to create a school feeding challenge that incentivizes local government officials and school communities to stretch school feeding budgets while maximizing nutritional value of school meals. Student Literacy: The Future is Ours! aims to increase the percentage of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. The baseline assessment found that only 7 percent of students could meet the standard in Arabic, and only 2 percent of students could meet it in French. Consequently, the baseline study recommends adding an indicator that measures the percentage of students who, by the end of two grades of primary schooling, can read 20 letters per minute. Defining success as the percentage of students who can read 20 correct letters per minute puts the project’s focus on the skills that will most likely improve during the project’s implementation. The full report provides narrative and context around these findings. It also presents detailed findings around other key indicators and intermediate results for The Future is Ours! project team to consider when designing the project’s initiatives. 6 1. Introduction Project Description The McGovern-Dole project The Future is Ours! seeks to improve literacy for school-aged children Mauritania’s Brakna and Gorgol regions. The new project is funded by USDA and implemented by Counterpart, with support from local partners EcoDev, ONG Actions, and regional partner ARED. The project cooperative agreement with USDA was signed on September 23, 2019 and is scheduled to run through September 30, 2024. The project’s total budget is $22.5 million. The Future is Ours! aims to serve over 140,000 direct beneficiaries, specifically: • 127,100 students in 209 selected schools who will take part in school meals provision. School meals will consist of a nutritious breakfast five days a week and mid-day school meal four days a week. Some students in the 51 pilot schools will also participate in literacy-focused activities, such as reading clubs. • 192 teachers in the 51-literacy pilot selected schools who will take part in the literacy-focused activities, such as training and coaching sessions. • 209 school administrators in the 209 selected schools, some of whom will also participate in literacy-focused activities such as training on EGRA. • 10,265 PTA participants, SMC members, pregnant and lactating mothers, and cooks who will participate in training in food preparation, storage practices, good health and nutrition practices, community mobilization for school canteens, and management of those canteens. • 3,247 children under five-years-old through THR provided to mothers participating in MCN activities. • 10 government officials who will participate in training and capacity building activities. Evaluation COVID-19 Related Issues Although schools in Mauritania were closed from mid-March to August 31, 2020 due to COVID-19, they were open during the baseline evaluation. Schools re-closed from mid-December to mid￾January, and they have been open since then. In addition to the pandemic, there are other factors that might disrupt the project implementation such as a teacher strike and serious drought conditions that can negatively impact crop yield and household income, which might prevent parents from sending their children to school. Counterpart will monitor these factors and if they present a serious risk, Counterpart will produce monitoring evidence to USDA for decision-making. Local School Language in Mauritania 7 Mauritania has a bilingual school system with Arabic and French. Most people speak Hassania, followed by Peul, Wolof, and Soninké, in Mauritanian homes, while French is rarely spoken. Almost all young children spend time in a Quranic school for religious purposes where they learn the Arabic alphabet and memorize the Quran. Arabic is the language of instruction in Grade 1, and French is introduced as a second language in Grade 2. Nearly all Mauritanian students attend madrasahs from pre-school age and arrive at primary school with some foundational knowledge of the Arabic language. However, for most students, Hassania, Wolof, Peul, or another mother tongue language is their first language. The same scenario is true for teachers who are expected to teach in a language that is not their mother tongue. Moreover, both students and teachers have limited exposure to French. In 1999, the GoM started education system reforms that included standardizing the language of instruction. Before 1999, whether students were taught in Arabic or French depended on the location and priorities of each school. Since the 1999 reforms, schools all teach in Arabic, Mauritania’s official language, beginning in first grade. Instruction in French begins in students’ second year2. The languages spoken in Brakna and Gorgol, where the project is being implemented, are Hassania, Fulani, and Soninke. School Feeding Programs in Mauritania School feeding began in Mauritania in 1963 to support the enrollment of scholarship students in secondary schools. In 1970, the program expanded to include primary school students. In the second half of the 1980s, specifically in 1986, the program refocused exclusively on primary education and a few mahadras in rural areas most vulnerable to food insecurity. At the end of 1980, the global resource crunch prompted many countries to review the targets of school feeding programs. For Mauritania, as for several other countries, this revision resulted in the orientation of assistance towards poor rural areas vulnerable to food insecurity and with low education and nutrition indicators. In the early 2000s, the deepening of the global financial and energy crisis in donor countries, resulting in rising food prices and a food crisis in developing countries (2008), revealed the importance of school feeding as a social safety net when millions of children found themselves out of school and undernourished. World Food Program (WFP) has developed strategies for the sustainability of school canteens with the support of some countries, including Brazil, so that “no child goes to school hungry.”3 In this perspective, the GoM committed to progressively take charge of school feeding in the country. However, the deterioration of the economic situation, caused by recurrent droughts combined with other shocks, has not allowed the GoM to meet the challenge of taking charge of school feeding and, from 2011 to 2016, WFP, despite the reduction of its resources, has once again become the only 2 Moya, Solanash "The Current State of Education in Mauritania." Borgen Magazine, 17 June 2017 3 Page 7 of the Islamic Republic of Mauritania’s National School Feeding Policy, 2016 8 organization that supports school feeding in Mauritania. Many school children from poor rural families have been deprived of the multiple benefits of school meals. The GoM's school canteen program, supported by WFP, is facing funding shortfalls that expose thousands of beneficiary children to the risk of undernutrition and school dropout. To reverse the situation, in 2016, the GoM developed a school feeding program which aims to guarantee a sustainable school diet based on nutritious local products for all children enrolled in basic education schools by 2030. The goal of the national school feeding strategy is “by 2030 to have a sustainable school feeding program using local enriched food products for all students in preschool, primary school, rural middle schools, and religious schools (mahadras).”4 The GoM launched its national school feeding program in 2021 with a budget of $700,000 USD per year. The funding is used for procurement of kitchen utensils and local commodities (rice, beans, lentils, oil, and dry milk), transportation of commodities to the various schools, monitoring of results, and various trainings. The school feeding consists of breakfast and lunch without THR. Beneficiaries include students from primary schools and high schools in the following regions Adrar, Assaba, Dakhlet Nouadhibou, Gorgol, Guidimakha, Hodh Ech Chargui, Hodh El Gharbi, Inchiri, Tagant, Tiris Zemmour, and Trarza. The program is implemented by the GoM through Taazour, Commissariat à la Securité Alimentaire, and the School Feeding Department. Currently, WFP is the sole school feeding partner to the GoM, distributing daily snacks and lunches to 50,000 primary school children in the regions of Assaba, Guidimakha, and Hodh El Chargui. Additional services delivered to the schools including communication on nutrition, hygiene, and family practices, are funded by German cooperation. For Counterpart’s The Future is Ours!, school meals consist of a nutritious breakfast five days a week and midday school meal four days a week. Each student receives a daily breakfast ration of 100 grams (g) of corn-soy blend plus (CSB+) and 20 g oil. Students each also receive a lunch ration (Monday– Thursday) of 100 g fortified rice and 20 g peas or lentils and 20 g of oil. Peas and lentils are rotated to make the lunches tastier and more enticing to the children. In Year 5, only a lunch ration will be provided. The Future is Ours! Theory of Change The Future is Ours! theory of change is that improving student literacy and increasing use of health, nutrition, and dietary practices require a community focus on achieving those outcomes and building supportive structures to sustain the project’s activities beyond its implementation timeline. The project’s core activities directly integrate student literacy and nutrition outcomes. These activities are reinforced by other activities that link national decision makers to local school stakeholders and engage regional health and education actors to work together. The project’s work through schools, 4 Page 5 of the Islamic Republic of Mauritania’s National School Feeding Policy, 2016. 9 communities, and government changes behaviors and increases the possibility that the project’s efforts will be sustained beyond its implementation period. The project’s theory of change is supported by the causal links in its results framework, which is included in this report as Annex 6. The results framework is defined by two Strategic Objectives (SO): 1) improved literacy of school-aged children and 2) increased use of health, nutrition, and dietary practices. The two SO share policy results: increased capacity of government institutions, improved policy and regulatory framework, increased government support, and increased engagement of local organizations and community groups. The intermediate outcomes supporting these two SO include: improving the quality of literacy instruction, implementing activities that increase student attendance and improve in-class concentration, increasing students’ access to food, providing access to school supplies and materials, improving literacy instruction materials, improving school infrastructure, promoting knowledge of health and hygiene practices, and increasing access to water and sanitation services and preventative health interventions. Using best practices in EGRA, a pilot literacy project in 51 schools will test a holistic approach to improve student learning outcomes. Literacy activities focus on grades 1 to 3. Grade 1 literacy activities are held in Arabic. Grades 2 and 3 activities are in both Arabic and French. School feeding and other activities such as providing THR, improving school infrastructure, providing deworming medication, and PTA training and awareness campaigns will focus on all primary school grades (1 to 6). School Selection The 209 schools, including the 51 pilot schools, were selected among 400 schools in close collaboration with the GoM using the following criteria:  Schools with six cycles/grades (complete cycles/ full-scale5) were prioritized but the MOE highlighted that it is difficult to find 209 schools which meet this requirement especially in rural areas. Only 31 percent of primary schools in Mauritania are complete (full-scale) schools.  Schools that are close to local villages (1 to 2 kilometers). This increases enrollment of students and is in line with GoM policy.  Engagement and active participation of the community toward the project.  Schools accessible anytime for commodity transportation.  Schools with infrastructure such as storerooms, latrines, and water sources.  Schools with enrollment from 40 to more than 400 students. Once the 209 schools were selected, Counterpart and the MOE chose the 51 pilot schools using the criteria below: 5 A complete/full cycle primary school is one with all six grades. 10  Rural schools that are in proximity to 1 to 3 communes in Brakna and Gorgol and distance between schools does not exceed 5 kilometers.  Schools must have six grades with sufficient French and Arabic speaking teachers (30 to 40 percent Francophone teachers and 60 to 70 percent Arabic-speaking teachers).  The early grades 1, 2, and 3 have both French and Arabic ongoing literacy programs. The schools do not have a multi-grade system (two grades in one classroom) or double vacation (one grade in the morning and the other one in the afternoon).  Engagement and active participation of the PTA toward the project to increase parental support for education and literacy.  The SMC should be willing to sign a Memorandum of Understanding and engage actively in the pilot project. Project Activities The project has a total of 21 activities that work toward intermediate results in all 209 schools. Seven of these activities are literacy-related and will focus on the 51 literacy pilot schools. Below are the activity descriptions as written the in the project’s Attachment A: Plan of Operation. 1. Capacity Building: Local, Regional, and National Levels: To build government capacity, Counterpart will work with the MOE to form a Technical Working Group (TWG) of ministry officials that includes officials from the Division of School Nutrition and Health. Counterpart will coordinate training for the working group’s members. Counterpart will also arrange a “regional peer-to-peer mentoring program with Ministry of Education in Senegal and lead strategic exchange visits to Senegal for both the school feeding and literacy pilot.” Along with these efforts, Counterpart will work with the government to establish a fund for managing school canteens. 2. Building/Rehabilitation: Latrines: Counterpart will carry out a participatory assessment to identify which rural schools do not have latrines. Schools will be prioritized and selected based on the size of the school (number of students impacted) and financial mobilization. Community stakeholders will participate by providing materials and labor to build the latrines, and the SMC will help manage the process. Counterpart will lead a training on latrine cleaning, and the SMC will have a latrine maintenance plan. Accompanying the latrine will be a WASH training component. Counterpart will implement this activity in 50 schools in the Gorgol and Brakna regions.The final school selection will be determined after the assessment. 3. Building Rehabilitation: Warehouse and Storerooms: Counterpart will carry out a participatory assessment to identify which rural schools do not have storerooms. Schools will be prioritized and selected based on the size of the school (number of students impacted) and financial mobilization. Community stakeholders will participate by providing materials and labor to build the storerooms, and the SMC will help manage the process. In total, communities will be expected to contribute up to 50 percent of the cost of storeroom construction. Construction of a simple 4m x 4m model will be the starting point and then adapted to each school. Counterpart will implement this activity in 116 schools in the Gorgol and Brakna regions. The final school selection will be determined after the assessment. 11 4. Building/Rehabilitation: Wells and Water Station Systems: Counterpart will carry out a participatory assessment to identify which schools do not have water station systems. Counterpart will work with communities to address their water needs at school, assessing each school’s options to source water, while always respecting water quality norms and safety standards for point of consumption. Counterpart will work closely with MOE, municipal authorities, and National Office for Water Services in Rural Areas (ONSER), the agency responsible for rural water infrastructure and planning water station systems. Community stakeholders will participate by making in-kind contributions, while municipal government and/or ONSER will be asked for a financial contribution. Counterpart will implement this activity in 68 schools in the Gorgol and Brakna regions. The final school selection will be determined after the assessment. 5. Distribution of Deworming Medication, Vitamins and Minerals: Counterpart will provide deworming medicine to cover affected students at all schools for helminth infections and Schistosomiasis in all participating schools. This treatment will be provided in those localities identified by the Ministry of Health (MOH) as needing treatment. Counterpart will provide information for students about Schistosomiasis during the deworming campaign. Counterpart will work with the MOH and other development partners to advocate for a formal MOE/MOH policy on deworming in school-aged children. Through a partnership with Vitamin Angels, Counterpart also plans to distribute Vitamin A to students and multi-vitamins to pregnant women. 6. Promote Teacher Attendance: Counterpart will work with education authorities, community members, and teachers to reduce teacher absences. Counterpart, in collaboration with National Education Inspection Department (IDEN), school directors, and teachers, will assess the needs of the participating schools and identify appropriate methods to address teacher absenteeism. Counterpart will also promote teacher attendance through recognition events. Counterpart and IDEN will next identify a set of criteria to assess teacher excellence and put in place a process to recognize and reward teacher excellence. Once established, evaluation criteria and the process will be amply communicated to teachers. Annual recognition of an exemplary teacher will be based on best attendance record, and one teacher will be recognized at a public ceremony with the community. The PTA and SMC oversight role will be strengthened through trainings to improve monitoring of teacher absenteeism. Counterpart will establish a transparent monitoring process with clear reporting mechanisms in line with current MOE policies The PTA and SMC oversight role will be strengthened through trainings to improve monitoring of teacher absenteeism. Counterpart will establish a transparent monitoring process with clear reporting mechanisms in line with current MOE policies. 7. Provide Energy Saving Stoves: Counterpart will equip all 209 schools with locally-purchased stoves. Cooks will be trained on how to use and maintain them. 8. School Meal Provision: The school feeding program will start in program Year 2, expanding by 3 percent per year. School meals will consist of a nutritious breakfast five days a week and midday school meal four days a week. Each student will receive a daily breakfast ration of 100 g of CSB+ and 20 g oil. Students will each receive a lunch ration (Monday–Thursday) of 100 g fortified rice and 20 g peas or lentils and 20 g of oil. Peas and lentils will be rotated to make the lunches tastier and more enticing to the children. In Year 5, only a lunch ration will be provided. CSB+ will be used as a THR for mothers participating in the MCN activity. Counterpart will train 12 School Canteen Committees (SCC), along with the SMC and PTA, in their roles and responsibilities. Counterpart will create a training module in local languages to guide SCC trainings, as well as a manual for managing a school canteen adapted to the needs of the SCC. 9. Raising Community and Parental Awareness of the Importance of Education: Counterpart will raise awareness about (1) the expected impact of the canteen, (2) the leadership and oversight role of parents and community members around the school canteens, and (3) school enrollment. Counterpart will hold school meetings about these issues at the start of the school year. Counterpart will blanket target communities with training about schooling using local radio and community events. Approximately nine radio stations, one per department, will broadcast a show every two months, and Counterpart will organize 1-2 community gatherings per year at school with education themed festivities. Counterpart will reach out to all community members, especially members of minority ethnic groups, and design simple interactive radio emissions in local languages on different topics. Counterpart will work with each school community to select a group of 5-10 volunteer women to serve as a mothers’ mobilization group that will carry outdoor-to-door enrollment campaigns. At the end of each annual enrollment campaign, running from September to November, Counterpart will assess its success and make improvements for the following year. 10. Take Home Rations: Rations will also be provided to mothers participating in community-level MCN activities, specifically targeting pregnant and lactating women and mothers of children under five years of age. Women with children under the age of five will be invited to participate in routine education sessions under the existing health programs with education provided as described in Activity -Training: Good Health and Nutrition Practices. A THR of 5 kg of CSB+ will be offered on a quarterly basis as an incentive for participating in all education sessions offered and describing a qualifying message and/or change in behavior in a quarterly survey. The project will implement several trainings to support educational results, nutritional outcomes, and local coordination of project activities. 11. Training: Food Preparation and Storage Practices: Counterpart will train the SCC and school cooks in safe food preparation, proper food storage, and adherence to portion sizes. Cooks will learn about measuring portions with local serving materials, safety, and nutrition standards, and completing monitoring forms. Counterpart will put in place a well-understood first-in/first￾out and minimum-maximum inventory system at each school to ensure proper and safe storage of commodities. An effective school-wide oversight system will also prevent any waste or leakage of food commodities. 12. Training: Good Health and Nutrition Practices: Counterpart will work with existing health officers and community workers to carry out a participatory mapping of health actors and their activities in the targeted communities. Gaps in content and coverage will be identified and Counterpart will work with existing health systems and personnel to strengthen nutrition￾focused interventions to improve nutrition, health, and hygiene practices of mothers and their children under five, as well as pregnant and lactating women in targeted communities. Counterpart will adapt existing learning modules about nutrition and WASH for school-age children, created by United Nations Children’s Emergency Fund (UNICEF) and others, to the Mauritanian context to train teachers and SCC/PTA/SMC members on nutrition and WASH information. Once trained, these school stakeholders will be responsible for transferring the 13 information to students in a variety of ways—teachers will be encouraged to integrate the information into class discussions: schools can paint murals on the sides of newly built latrines or school warehouses and schools will hold two Health/Nutrition/WASH themed fairs over the course of the school year. Counterpart will provide education on the importance of dietary diversity, dietary resources, and supplementation with iron and folate, which is especially important during pregnancy and lactation. THR will be used as an incentive for behavior change by mothers around those improved practices. Parents and community members will be invited to attend fairs. Counterpart will complement one of these fairs with a deworming campaign. 13. Training: Parent Teacher Associations: Counterpart will train PTA, SMC, and SCC in their respective roles and responsibilities. The PTA will be trained in mobilizing parents to participate in the school canteen, school governance, and in school programs, etc. SMC representatives will be trained in school governance. The SCC will be trained in canteen management. Counterpart will use existing modules and make necessary modifications to adapt them to the needs of these trainings. A refresher training will be given in Project Year 4 for two days. Counterpart will monitor SMC, PTA, and SCC functioning and will offer practical guidance as needed. The SMC and PTA will be invited (in literacy pilot schools) to share aggregated student academic results with the larger community. Schools will be invited to submit a focused school improvement plan. 14. Training: School Administrators: Counterpart will train directors in managing a school, including personnel management, community engagement, administrative management, and resolving conflict. Counterpart will identify what directors need to know (based on MOE regulations and processes) to create a training module on knowledge and skills needed to succeed. Directors will be trained in four-day workshops; a two-day refresher training will be given the following year. Pilot Activities In addition to these activities, the project will implement a literacy pilot in 51 of the 209 primary schools as mentioned earlier. The pilot seeks to improve early grade reading (EGR) outcomes by conducting the following seven activities: 15. Distribution of School Supplies and Materials: Counterpart will provide textbooks and teachers’ guides in the literacy pilot schools. Counterpart will work with the SMC and PTA to ensure oversight and accountability of these school materials by teachers and students. Counterpart will work with the MOE to explore policy options for a sustainable supply of learning materials as textbooks for students and guides for teachers are fundamental to a successful school system. 16. Literacy Promotion: Counterpart will provide textbooks and teachers’ guides in the literacy pilot schools. Counterpart will work with the SMC and PTA to ensure oversight and accountability of these school materials by teachers and students. Counterpart will work with the MOE to explore policy options for a sustainable supply of learning materials as textbooks for students and guides for teachers are fundamental to a successful school system. 17. Organization of Extra-curricular Activities: Counterpart will work with select schools to establish reading clubs. Volunteer club leaders will be trained in maintaining a club and given a 14 menu of core activities to make the clubs interactive and fun; they will also be encouraged to innovate their own activities. In urban schools with access to internet, clubs will incorporate electronic materials to read texts online and play interactive reading games. 18. Production of Books and Supplementary Reading Materials: Counterpart will design and produce simple, low-cost textbooks aligned with the revised literacy curriculum. Textbooks in Year 1 are in Arabic only, textbooks in Years 2 and 3 are in Arabic and French. Counterpart will engage the TWG in a series of material development workshops to design the content. All materials will be adapted to the Mauritanian context. Counterpart will bring outside technical experts to assist and contribute to design work. Counterpart will work with schools and parents to develop high-quality reading materials that are adapted to the Mauritanian context. These books will contribute to book banks in the classroom and will also be used in the reading clubs. Counterpart and school stakeholders will work together to identify protection and storage methods. Teachers, students, parents, illustrators, and language specialists will participate in material development workshops to produce stories. 19. Curriculum Development: Counterpart will implement a literacy pilot in 51 schools. Counterpart will first design a scope and sequence across the first three grades for teaching letters and developing decodable words and text/stories. Secondly, Counterpart will produce teaching guides. The guides will support teachers to develop reading fluency and comprehension skills in Arabic in Grades 1-3, and in developing oral language skills in French as a second language in Grade 2, and initial reading skills in Grade 3. Teachers’ guides for each class will be produced and distributed to classroom for teacher use in years/grades 1-3. Year 1 guides are in Arabic while Years 2 and 3 are in French and Arabic. All work on the curriculum and materials will be complete by Y3 and then revised in the final year based on results and findings from the pilot. Counterpart will include information dissemination at key stages in the process both targeted towards decision makers in the GoM and to education system stakeholders (teachers, inspectors, etc.). 20. Training Teachers: Counterpart will cluster schools for the teacher training and will use a cascade training (ToT) approach. Counterpart will organize two initial trainings for Master trainers and literacy coaches who will then train teachers. Regional Directorate of National Education (DREN)/IDEN are included in the ToT as participants even though they might not necessarily give the training. Teacher training has been divided into four in-service sessions to cover the revised curriculum and the new teaching method and tools. These trainings will last for five days each, for a total of 20 training days. Counterpart will execute intensive in-class support for the first three years of the program. Teachers will have twice monthly coaching sessions with literacy coaches as well as pedagogical visits and support from DREN/IDEN. Coaching sessions will include feedback sessions with teachers. Counterpart will facilitate cluster-based peer learning and exchange starting late in PY2. 21. Training: Government Officials: Counterpart will create training modules that are aligned with the revised curriculum and the teacher training. Approximately 30 DREN/IDEN officials will be trained in the same content as teachers and will receive additional training on coaching and supervising skills. As part of this training, participants will be asked to produce a coaching calendar to plan for regular visits to literacy pilot schools. Counterpart will assist with transportation costs associated with the additional coaching and inspection visits to pilot schools. Counterpart will support DREN/IDEN to acquire the skills to carry out a “mini EGRA,” a 15 tool to provide a macro view of student reading levels in a department. Counterpart will coordinate IDEN inspectors to shadow the program’s independent evaluation team for the midline evaluation EGRA. Counterpart will train IDEN in the EGRA methodology and assist with practice exercises in schools, the results of which will be used to inform coaching and training sessions. Counterpart literacy coaches will provide in-classroom support during the pilot program, as the DREN/IDEN gradually assumes the coaching role. Counterpart will work with DNES to design a survey tool using both quantitative and qualitative data to capture the level of canteen performance and the level of “customer satisfaction” of end users—students, parents, the SCC, and cooks. A calendar of activities will be established to carry out the survey on an annual basis starting in PY3. Activities targeting all the 209 schools Activities ONLY in 51 Pilot schools Capacity Building: Local, Regional, and National Level Building/Rehabilitation: Latrines Curriculum Development Building/Rehabilitation: Warehouses and Storerooms Distribution: School Supplies and Materials Building/Rehabilitation: Wells and Water Station Systems Establish Activities to Promote Literacy Distribution: Deworming Medication, Vitamins and Minerals Extra-curricular Activities Promote Teacher Attendance Production of Books and Supplementary Reading Materials Provide Energy Saving Stoves Provide School Meals Training: Teachers Raising community and parental awareness of the importance of education Training: Government Officials Take-Home Rations Training: Food Preparation and Storage Practices Training: Good Health and Nutrition Practices Training: Parent Teacher Associations Training: School Administrators The project’s implementation plan includes data collection activities and learning activities to identify innovations, results, and opportunities for adaptation. This approach enables the project team to make evidence-based decisions and analyze results so that successful initiatives are scaled, and less successful initiatives are modified or dropped. Baseline Assessment Report Background Scope 16 Counterpart contracted IMC to conduct the baseline assessment for The Future is Ours! The purpose of the baseline assessment was to produce a baseline report containing descriptive statistics of baseline indicators that require non-zero measurements disaggregated by gender, department, and grade, and recommendations on methodology for implementing subsequent evaluations. The baseline assessment was also required to operationalize research questions included in the scope of work (Annex 4) so that they could be answered by midterm, final, and impact evaluations. These questions are addressed in section 3.2 of the full report. IMC completed the following tasks for the baseline assessment: • Reviewed all relevant project documents. • Produced an inception report with the methodology and approach of the baseline assessment and operationalized the research questions and indicators. • Trained the supervisors of Tenmiya, the local data collection firm. • Oversaw testing and piloting of the data collection tools. • Managed data collection activities, including surveys and interviews with students, teachers, directors, MOE officials, and parents to collect high quality data to provide baseline values for the research questions and for project indicators. • Conducted an EGRA for students completing grade 2 and entering grade 3 at the time of data collection. • Produced a baseline report containing (a) descriptive statistics of baseline values, where appropriate, disaggregated by gender, department, and grade; and (b) recommendations on the methodology for implementing subsequent assessments. 2. ASSESSMENT METHODOLOGY Data Sources and Data Collection Methods The baseline assessment followed a mixed methods approach to data collection and analysis that included secondary sources6, qualitative data collection in the form of semi-structured interviews, and quantitative data collection through stakeholder questionnaires. The assessment also took a measurement of primary school student literacy using the EGRA methodology. The assessment team developed tablet-based data collection tools for the EGRA assessment, surveys, and qualitative data collection guides. Early Grade Reading Assessment (EGRA) 6 The evaluation used secondary sources to inform the background section of this report and to design the French and Arabic EGRA tools. Secondary sources included the EGRA Toolkit and studies conducted by organizations such as confemen. The secondary sources are cited in footnotes throughout this section of the report. 17 The IMC assessment team created two EGRA tools: one in French and one in Arabic. The team designed the EGRA tools using the EGRA Toolkit, Second Edition7, and adjusted the difficulty level according to the advice from Counterpart’s The Future is Ours! project team. The EGRA tools assessed student performance in activities (known as sub-tasks), beginning with the most basic literacy skills and progressing to the most difficult skills. When implemented, the EGRA assessment asks students to demonstrate basic literacy skills before moving to more complex tasks. For example, a student who cannot recognize any words is not asked to read a passage or demonstrate reading comprehension. The range of skills that students are asked to complete, from easiest to most complex, are: 1. Phoneme Recognition 2. Character Recognition 3. Word Recognition 4. Reading Passage 5. Reading Comprehension 6. Writing The assessment team, with input from Counterpart, adjusted the difficulty of the EGRA’s standard reading passage and comprehension questions to reflect the reading abilities that a Mauritanian student may have achieved after two years of schooling. The team’s design ensured that both EGRA tools were regionally appropriate in terms of phoneme use, grammar norms, and cultural relevance. The baseline’s EGRA assessment measured students’ literacy achievement in Arabic and French. However, achievement in different languages is not directly comparable because patterns in literacy acquisition vary by language. For example, while French speakers need to learn both capital and lower￾case letters, Arabic letters do not contain upper and lower cases. Therefore, an Arabic-literate student may read more letters per minute than a French-literate student. However, that does not indicate that an Arabic-literate student has achieved a higher degree of literacy. Conversely, a student who has mastered isolate forms of the Arabic alphabet but is unable to recognize them in contextual forms may not read words as quickly as a student in French who had dedicated the same number of hours to learning. 7 The EGRA toolkit can be found here: https://ierc-publicfiles.s3.amazonaws.com/public/resources/EGRA Toolkit V2 2016.pdf Source: Counterpart International 18 The assessment team selected four students per school who were completing grade 2 for the 2019- 2020 school year to take the EGRA. For the baseline, students were randomly assigned so that half of the selected students took the French assessment and half took the Arabic assessment. The literacy indicator MGD 1: Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text, measures the proportion of students who meet a minimum threshold of literacy competency at each assessment point. The assessment team could not determine minimum thresholds of literacy found because information on Mauritania’s standardized assessment data is not released publicly, and the country does not participate in international student assessments. However, it ranks among the 10 lowest countries in youth literacy in the world (for which there is data).8 The most recent available student literacy data is the 2003-2004 Programme d’analyse des systems éducatifs de la confem (PASEC) from the Conference of Ministers of Education of French-Speaking Countries. 9 Although correct words per minute were not published for the assessment, the average reading score for second graders in Mauritania were lower than any other country that participated in the test. While school enrollment has dramatically increased since 2003, the recent evidence on youth literacy and PASEC scores suggests that the baseline assessment should use a lower threshold to measure literacy. Because of the paucity of external sources, the assessment team defined literacy proficiency based on the data, which showed that most students are still learning to recognize letters and phonemes. The assessment team defined proficiency for this indicator as, “the ability to read at least 20 letters per minute.” This measurement is based on the letter and phoneme recognition assessment, subtask 2 of the EGRA. The 20 letters per minute target was arrived at by understanding the distribution of student achievement at baseline: 42.3 percent of students tested in Arabic could not read any letters, 21.7 percent could read some letters, but fewer than 20 per minute, and 35.0 percent could read at least 20 correct letters per minute. Similarly, in French, most students (85.0 percent) could recognize at least one letter, but only 8.1 percent could read 20 letters in a minute. These figures, along with disaggregated data by gender and region, are included in Table 2. EGRA Index Calculation While each subtask has an independent score, there is no standardized guidance on how to create a final single EGRA score. For simplicity, an overall EGRA score was calculated using a weighted average of the five subtask scores, where subtasks 2, 3, and 4 (which contain 100, 50, and 50 items 8 Education Policy Data Center (2018). Mauritania National Education Profile. https://www.epdc.org/sites/default/files/documents/EPDC_NEP_2018_Mauritania.pdf 9 http://www.pasec.confemen.org/publication/la-qualite-de-leducation-en-mauritanie-quelles-ressources-pour-quels￾resultats/ 19 respectively) were given twice the weight of subtasks 1 and 5, which contain 5 items each (Table 2). To account for greater variation, correct words per minute was scaled to 50 items.10 Table 1 EGRA Index Calculation Subtask Calculated As Weighting 1. Sound Recognition Percent Correct (5 items) 12.5% 2. Phoneme Reading Correct letters per minute 25.0% 3. Devised Words Correct words per minute 25.0% 4. Oral Reading Correct words per minute 25.0% 5. Reading Comprehension Percent Correct (5 items) 12.5% Other Surveys and Qualitative Data Collection In addition to the EGRA, the assessment collected data through the following means: 1. Survey for students in grades 1-6 2. Survey for teachers 3. Survey for directors/school checklist 4. Qualitative interviews with parents 5. Qualitative interviews with government officials Each of the quantitative surveys used open- and closed-ended questions. The open-ended questions provided context and probed responses for more details. 6. Questions for students focused on absences and WASH practices. 7. Questions for teachers focused on student attentiveness, teaching materials, teacher trainings received prior to The Future is Ours! and teacher absenteeism. 8. Director interviews/school checklists included questions on attendance, enrollment, and water and sanitary facilities. 9. A short question guide for interviews with parents focused on student WASH practices, food consumption, and school absences. 10. A question guide for interviews with government officials, as well as regional and departmental inspectors, focused on challenges to improving literacy and recommendations for The Future is Ours! 10 For example, a student who scored on subtask 1 60%; subtask 2 20 letters per minute; subtask 3 50 cwpm; subtask 4 55 cwpm; and subtask 5 20%, their score would be calculated as: (12.5*0.6)+25*(20/50+50/50+50/50)+(12.5*0.2), or 70. 20 Changes to The Assessment Implementation Due to COVID-19 The COVID-19 pandemic required that the evaluation team change the way that this evaluation was conducted compared to the methodology the team has used in past evaluations of McGovern-Dole projects. Primarily, these limitations relate to the way international staff interacted with the data collection team and project stakeholders. In previous McGovern-Dole evaluations, at least two international team members travel to the project country to train enumerators and conduct qualitative interviews with all stakeholder groups. International team members also usually travel with enumerators to project schools to spot check data collection and troubleshoot any issues that arise. To limit the spread of COVID-19, the evaluation team took measures to reduce face-to-face contact among international team members, the data collection team, and project stakeholders. Only one team member, rather than two, traveled to Mauritania. That team member trained the data collection team while the EGRA specialist observed remotely using a Zoom connection. One team member was not sufficient to cover all the qualitative interviews that were required for the assessment, so the team member focused on government and project staff interviews. To make up for the inability to interview other stakeholder groups, the evaluation team included questions requiring qualitative responses in the teacher, parent, and school director questionnaires. While these types of questions do not present the same depth of narrative as full interviews, they do allow for the discovery of unexpected information that contributes to understanding. In addition to limiting travel, the evaluation team took other precautions to prevent the spread of the virus. Project teams were limited to four members per vehicle, as per Mauritanian regulations, which increased the costs of implementing the evaluation. Data collection teams were also provided with masks and hand sanitizer. Sampling Strategy Assessment Design Overview Source: Counterpart International 21 The baseline assessment used a quasi-experimental design that allows for a comparison between McGovern-Dole literacy pilot schools and McGovern-Dole schools that do not have the literacy pilot project. The assessment used a two-stage cluster design. The first stage comprised the selection of schools to participate in the assessment, and the second stage comprised the selection of students (for EGRA and the student survey), directors, teachers, and parents within each school. In total, 146 of 209 McGovern-Dole schools (95 randomly selected non-literacy pilot schools, 51 literacy pilot schools) were included in this assessment. The sample design allows for a strong comparison between literacy instruction plus school feeding (McGovern-Dole pilot) schools and just school feeding schools (McGovern-Dole non-pilot schools). This sample design is more important for comparison at midterm and final evaluations than in the baseline assessment. EGRA Sample Size and Selection The sample size for the EGRA is based on finding a minimum detectable effect of the McGovern-Dole indicator: SO1: Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. Since EGRA is only conducted for students at the end of grade 2 (only one grade out of six total), the student sample size for EGRA is different than the other indicators that look at all six grade levels. Two-Stage Cluster The sample strategy used a two-stage cluster design. The first stage randomly selected the schools for inclusion in the sample, and the second stage randomly selected four students entering grade 3 at each school to participate in the EGRA. To estimate the minimum sample size for learning outcomes, the assessment team assumed that 10 percent of the sample will meet a minimum literacy threshold at baseline, and the aim was to detect when 17.5 percent or more students pass the minimum learning requirement at midterm or final evaluations. The sample size of students was calculated using the standard parameters of 0.80 power and 0.05 significance level. An intra-cluster correlation coefficient of 0.15 was used in the calculation. The resulting cluster size of four students per school produced a sample size of 584 EGRA takers in 146 schools.11 This small cluster size allowed the assessment team to visit more schools which increased the statistical power of the director/school checklist survey (discussed below). Four students at the end of grade 2 were randomly selected from school enrollment lists at each school. At each school, survey teams used random number generators on their tablets to select four random numbers corresponding to numbered students on grade 2 enrollment lists. Two students were randomly assigned to take the French EGRA and two the Arabic version of EGRA. If a selected student was not present on the day of the assessment, then a replacement student was randomly selected. Sample Weighting 11 Sample sizes calculated with Stata and manually. 22 Although the values in Table 2 EGRA Index Calculation represent the true number of observations, statistical weighting was used to calculate means and percentages. The sample was designed to have a 1:2 ratio of pilot to non-pilot schools, though the actual ratio of pilot to non-pilot schools is 1:3. 12 By oversampling pilot schools, statistical differences between the two groups are more easily detected. However, to ensure that the results overall are unbiased, slightly higher weight was given to non-pilot schools so that non-pilot to pilot schools influence the means at the same ratio of actual pilot to non￾pilot schools. Sample Composition In total, the samples include 291 valid assessments in French and 271 valid assessments in Arabic. A valid assessment is an entire EGRA completed by a student in the correct grade. Students were 46.4 percent female in the French subset and 51.7 percent female in the Arabic subset. In limited cases, enumerators did not successfully complete two assessments in each language at each school. Also, a small number of assessments were excluded because they were completed by a student in the wrong grade. Table 2 EGRA sample composition French Arabic Group # % # % Total 291 100% 271 100% Pilot13 94 32.3% 87 32.1% Non-Pilot 197 67.7% 184 67.9% Female14 135 46.4% 140 51.7% Male 148 50.9% 129 47.6% Brakna 187 64.3% 176 64.9% Gorgol 103 35.4% 95 35.1% Slightly less than two-thirds of both EGRA samples were from the Brakna region, and the remainder from the Gorgol region. Chi-square tests of equality did not find significant differences between the two randomly selected samples in their composition by ethnicity, department, or sex. While these results do not guarantee 12 The weights are designed as a ratio of 51 pilot and 158 non-pilot schools. 13 Due to nonresponses, some subgroups may add up to less than the total sample. 14 While the assessments were given correctly and passed our quality review, 8 assessments did not include a response to gender, and an enumerator did not fill out the region and community it was conducted in. The assessment was still valid but missing this information. While the assessment intentionally included an option to not respond to the gender question, the option to not fill out the region and community was an oversight. 23 that the two samples are representative of the entire beneficiary group, it suggests there are no differences due to random error (please see Annex 2 for details). Other Survey Sample Size and Selection The other surveys’ (non-EGRA students, teachers, and directors) sample sizes are based on the 146 schools selected for the EGRA. The non-EGRA student survey includes one student from each grade (1 through 6) at each of these 146 schools, for a total of six students per school and 876 for the entire survey. This sample size allows detection of increases as small as from 1 to 1.3 handwashing occurrences per day with 0.80 power and 0.05 significance. Students from grades 1 through 6 were randomly selected using the enrollment lists at each school as described above. The team also conducted interviews with one director from each school and interviewed up to three teachers of grades 1 through 3 per school. Teachers were selected based on availability. The results from these surveys have smaller sample sizes and lower precision than the EGRA and student surveys. A summary of expected and actual sample size by group is shown in Table 4 below. Table 3: Quantitative Survey Sample Sizes Interviews per school McGovern-Dole Pilot Expected/Actual McGovern-Dole non-Pilot Expected/Actual Total Expected/Actual Schools 51 51 95 95 146 146 EGRA 3.84 204 181 380 381 584 562 Non-EGRA Student 6 306 301 570 554 876 855 Teacher ~ 2.5 127 128 237 211 365 339 Director/checklist 1 51 51 95 95 146 146 Total 2,117 2,048 Data Collection Training The eight-day training took place in two phases. In the first phase, the assessment team's co-evaluator conducted a three-day, in person, training in Nouakchott for the survey team’s eight field supervisors. In the second phase, eight field supervisors trained the 16 enumerators. After delivering the field supervisor training, the United States-based team members shared PowerPoint presentations and other training materials for the field supervisors to use with the enumerators. To take sufficient precautions considering the COVID-19 pandemic, the U.S.-based EGRA Specialist participated in online trainings to ensure information was covered thoroughly and to answer any questions that arose. The training included the following topics:  Introduction: Overview of project, purpose of research, main research questions, and objectives for the baseline assessment.  Overview of fieldwork: Overview of the sampling strategy, data collection and data entry procedures, and troubleshooting.  Research ethics and code of conduct in communities: Child protection and treatment of children during interviews, informed consent/assent, and confidentiality. 24  EGRA: A thorough training on understanding the purpose of EGRA and how to implement it using the tablet-based Tangerine application.  Walk through each questionnaire: A guided question by question walk through for the student, director, teacher, and parent questionnaires for trainees on their tablets.  Practice interviews: Practice interviews were held with fellow trainees. Tool Refinement, Piloting, and Debrief The assessment team finalized the data collection tools in two phases. During supervisor training, supervisors gave feedback on improving the tools. These were incorporated in time for piloting. Following the three-day supervisor training, supervisors conducted one day of pilot interviews at four schools near Nouakchott, which featured all the interview types described above. The day after the pilot, the team met again to debrief. During the debrief session, supervisors shared their experiences and made final suggestions on how the tools could be improved. These were incorporated into the questionnaires and a new version was released prior to the training of enumerators the following week. Data Analysis Quantitative data was analyzed using Stata statistical software. Data was disaggregated by demographic and geographic characteristics, and by school category and pilot versus non-pilot to determine if there are any differences at baseline that will need to be controlled/accounted for at midterm and final. In this report, tests of means are reported as significant if p < 0.05. Qualitative data came in two forms, from interviews and from short answers provided within questionnaires. Qualitative data from interviews, which were conducted with government officials, was coded using an Excel spreadsheet according to research question topic. The spreadsheets were also categorized according to more specific research questions and the evaluation criteria that each answer responded to. Coded items were grouped together and then analyzed on their own, particularly in the case of project sustainability, or grouped with similarly coded quantitative data to add context. The evaluation team analyzed qualitative data that came in the form of short answers in stakeholder questionnaires by reading through a random selection of responses to understand the themes that came up in each short answer. For example, a question about reasons students were late to school would produce “sickness,” “lack of transportation,” or “other duties to attend to” as themes. The short answers were then recoded according to these major themes, which generally covered 90 to 95 percent of all answers given. The remaining answers were then read to see if they fit into previously defined themes or merited a new category of response. Data Quality Control and Assurance While the survey team was often operating in remote areas with limited internet coverage, they were instructed to upload data to the server at least twice weekly. The assessment team monitored progress against targets daily using a dashboard. The lead supervisor in Mauritania communicated regularly 25 with the assessment team to inform them of challenges, such as unreachable schools, and to seek solutions. After the first week, the assessment team assessed the data quality. This assessment included examining interview durations, qualitative response word counts, and outlier values. A brief report flagging potential issues was provided to the lead supervisor in Mauritania who addressed these issues with his team. He then debriefed the assessment team on what he had learned from his field team about the issues flagged. Limitations COVID-19 restrictions: The COVID-19 pandemic may have created bias in the assessment results. Respondents may have been less willing to participate in interviews, or eager to cut interviews short because they were fearful of virus transmission. Students may have been less likely to attend school. COVID-19 also limited the degree to which the assessment team could conduct trainings in person and participate in face-to-face interviews in Mauritania. Dependence on school records for enrollment and attendance rates: Several baseline values, such as school attendance and enrollment rates, rely on school records. Measurement of changes in student attendance and enrollment rely on the discipline with which schools maintain these records consistently. Enumerators examined school records at each school that they visited. Every school visited had classes for students in grades 1 through 6. Data collectors did not collect information on differences in enrollment between male and female students, and so discrepancies in enrollment and attendance rates for male and female students is not included in the report. Dependence on enrollment and attendance records for literacy assessment: Students were randomly selected to participate in the EGRA from enrollment and attendance lists. School directors indicated in interviews that occasionally students who did not have official birth certificates may not have appeared in school records. Students who did not appear on enrollment or attendance lists would have been excluded from the possibility of taking the EGRA. Reliance on teachers’ availability: The assessment team could not take a random sample of working teachers. The assessment relied on interviewing teachers who were available on the day of the assessment, which may have created bias in teacher responses. Reliance on perceptions and memory: Many of the baseline assessment’s measurements rely on respondent’s perception of behaviors, such as frequency of handwashing. Others asked respondents to remember reasons for behaviors, such as missing school or missing taking attendance. While these perceptions provide valuable information about the degree to which respondents estimate behaviors take place with desirable frequency or quality, they are not as accurate as other measurements that observe behaviors. Lack of information about literacy standards: The assessment team could not find information on Mauritanian standards for primary school literacy, as described in the report. The assessment team recommends a measure of proficiency in the report that may not reflect The Future is Ours! project goals or the policy of the Mauritanian Government. 3. RESULTS 26 The results section of the report is divided into two sections. Section 3.1 presents the values for key indicators and discusses those values, providing context from key informant interviews and secondary research. Section 3.2 addresses the operationalization of research questions for the project’s midterm and final evaluations. Key Indicators Annex 1 provides a summary of the key indicator framework and baseline values. This section discusses the meaning of those values so that they may be more useful to the management of the project. MGD SO1: Improved Literacy of School Aged Children This section presents the baseline EGRA scores for all students in Arabic and French. The data is disaggregated by gender and pilot versus non-pilot schools. The complete description of EGRA scores is presented in Annex 3: EGRA Explanation and Results. Annex 3 also includes the methodology the assessment team used to determine the definition of proficiency. The SO1 indicator title and definition are: Improved Literacy of School Aged Children: Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. The baseline assessment team recommends changing the definition of this indicator to, “percent of students who, by the end of two grades of primary schooling, demonstrate reading proficiency.” The baseline assessment team recommends defining ‘proficiency’ as the ability to read 20 letters per minute. Table 5 shows the percentage of students who could read at least 20 letters per minute at the time of the baseline assessment. The baseline team recommends setting a target of increasing the proportion of students who can read 20 letters per minute at the end of the second grade by 3 percentage points per year of project implementation. This target is reasonable based on the low proficiency scores in both languages at baseline, but also the fact that a significant portion of students have learned to read some letters in both languages. Table 4 Percentage of students who can read 20 letters per minute in Arabic and in French Language Proficient Target Improvement: Arabic 32.2% +3% per year French 5.9% +3% per year Source: Counterpart International 27 Tracking proficiency level by pilot and non-pilot schools will reveal whether the school’s participation in the project correlates with reading improvements. Table 6 shows the current difference in levels of proficiency between pilot and non-pilot schools. Pilot schools show statistically significant, lower levels of Arabic proficiency than non-pilot schools. This is a surprising result, given that pilot schools were often selected because they were more physically accessible than non-pilot schools, which would suggest better access to resources such as trainings. Further analysis found that significant differences held even when controlling for ethnicity15 and language spoken at home. At baseline, students at non-pilot schools achieved higher rates of proficiency in both languages than students at pilot schools. Future assessments must measure increases in proficiency over these baseline values rather than aiming for a target level of proficiency for all schools. Table 5 Reading proficiency; pilot vs non-pilot Language Proficient Arabic: Non-pilot 36.4% Arabic: Pilot 17.2% French: Non-pilot 6.1% French: Pilot 5.3% In French, there was not a statistically significant difference between pilot and non-pilot schools. If scores in both languages were significantly higher in non-pilot schools, it may suggest that students learned better in one group of schools or the other. However, mixed results by language suggests that the difference is caused by other factors. Male students who took the EGRA test scored slightly higher than female students, though the difference was not statistically significant in either language. Table 6 Proficiency by gender Language Proficient Female: Arabic 30.4% Male: Arabic 33.7% Female: French 5.6% Male: French 6.0% 15 Schools include students from the Haratine, Beydane, Halpulaar, and Fulani ethnicities. 28 MGD 1.1 Improved Quality of Literacy Instruction The baseline assessment measured the percentage of teachers who have received training in literacy instruction in the last two years. Teachers who had received training were asked to rank the quality of the instruction they received and the quality of materials that were used in the training. Teacher Training Attendance The evaluation interviewed 339 teachers; 43 percent of whom were female and 57 percent were male. About 35 percent of all teachers said that they had received training in literacy instruction within the last two years. At baseline, 13.3 percent more teachers at pilot schools received training than teachers at non-pilot schools. The baseline also asked all teachers if they had ever received training, including longer ago than two years. Forty percent said they had never received training and 25 percent said they had attended a training over two years ago. The differences in the percentage of teachers who received training disaggregated by region or gender were not significant. Table 7 Percentage of teachers that had received training in the previous two years Measure Total result Pilot Non￾Pilot Difference/Significant Brakna Gorgol Female Male % of teachers receiving literacy related training in last 2 years 34.85 44.53 31.22 13.31/yes 31.63 41.7 36.47 33.7 Quality of Teacher Training Teachers were asked to rank how well they were trained on a scale from 1 to 5, with 1 representing not well and 5 representing very well. On average, teachers in pilot schools rated their training 3.52 and non-pilot rated their training 2.79, indicating that teachers in pilot schools may receive better training than teachers in non-pilot schools at baseline (Table 9). This significant difference at baseline between pilot and non-pilot schools reflects a confounding factor that will influence the statistical difference between pilot and non-pilot schools at midterm and final evaluations. The project will deliver training to pilot schoolteachers, so one would expect teachers to give their trainings higher ratings at future evaluations than at baseline, while non-pilot school trainings should stay the same. Therefore, we will compare the pilot schools and non-pilot schools at midterm and end of the project with the baseline values to determine the impact of the activity “Teacher Training.” The baseline assessment also asked teachers who had received training to rate the materials used in their training (Table 9). Teachers at pilot and non-pilot schools rated the materials used in training of about equally low quality; 1.83 at pilot schools and 1.78 at non-pilot schools. When asked why they rated the materials as they had, teachers said that materials were incomplete, overly relied on text￾based instruction, and were not easily transferable to lesson plans in the classroom, and that trainers lacked chalk. When asked who had provided the training, 75 percent of teachers said they received training from the MOE or local inspectors. The remainder were trained by various Mauritanian and international NGOs, 29 including Centre de Renforcement de l’Enseignement des Langues (Center for strengthening language teaching), The Alliance Franco Mauritania, and World Vision. Table 8 Quality of Teacher Training Measure Total result Pilot Non￾Pilot Difference/Significant Teacher ranking of training quality on a scale of 1 to 5 2.98 3.52 2.79 0.73/yes Teacher ranking of training materials quality on a scale of 1 to 5 1.8 1.83 1.78 4.65/no MGD 1.2: Improved Attentiveness The baseline assessment asked teachers how many times they must take action to refocus the class as a means of estimating student ability to pay attention. On average, teachers had to refocus the class 3.4 times. This number was consistent between pilot and non-pilot schools and region. Table 9 Student attentiveness Total result Pilot Non-Pilot Difference/ Significant Brakna Gorgol Difference/ Significant Number of times teachers must refocus class 3.38 3.37 3.39 -0.02/no 3.21 3.74 -0.53/no MGD 1.3: Improved Student Attendance The evaluation obtained attendance data from school directors for each grade in their school. The data, presented in Figure 1, shows that attendance rates are high, ranging from 13.43 percent in 5th grade absent on average, to 7.67 percent absent in second grade. Annex 1 provides complete data disaggregated by pilot and non-pilot schools, and by region where possible. 30 Figure 1 Student attendance by grade Parents were asked the main reason that their child or children miss school. Table 10 shows the most common reasons. Data collectors noted the reasons that each parent gave for their child or children’s absence. The most common reason parents gave for why their children missed school was illness (86.32 percent) and the second most common was teacher absence (17.54 percent) (Table 11). Table 10 Most common reasons students miss school Reason for Absence Percent Illness 86.32% Teacher absence 17.54% Lack School Materials/Clothes 9.12% Lack of Food 8.77% Weather/Heat 4.56% Distance 1.75% Agriculture/Home Work 1.40% Playing with Friends 1.40% Traveling 0.35% 90.31 92.33 90.99 88.56 87.57 89.22 0 10 20 30 40 50 60 70 80 90 100 Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Percent Attendance by Grade 31 The evaluation also asked parents if their children ever missed a portion of the school day, either arriving at school late or leaving school early. Fifty-one percent of parents reported that their children occasionally missed a portion of the school day to return home during recess to eat and return to school late. For student with homes far away, they may not return to school. A large portion (approximately 28 percent) of these parents also said that their children arrive late to school if their breakfast is delayed. Nearly all parents that the team interviewed said that their children would be more eager to go to school if there were a canteen. Some parents even joked that if their child’s school had a well￾maintained canteen, that “we would go to school instead of our children!” MGD 1.3.4: Improved Student Enrollment Annex 1 provides school director-provided enrollment data disaggregated by grade, region, and pilot versus non-pilot schools. The baseline assessment found that the average school has 317 students in grades 1-6, with about 52 students per grade. However, enrollment declines as grades advance. Schools have an average of 60 students in grade 1, and enrollment declines steadily per grade with an average of 44 students in grade 6; a 27 percent decline (Figure 2). Source: Counterpart International 32 Figure 2 Enrollment by Grade MGD SO2: Increased Use of Health, Nutrition, and Dietary Practices The definition of SO2 is: The percentage of beneficiaries who can demonstrate knowledge of effective health and hygiene practices as a result of USDA assistance. As the school feeding project had not started yet, there were no volunteer breakfast and lunch cooks or project participants that the baseline assessment team could survey about health and hygiene practices. For the baseline measurement, students were asked if they washed their hands at home and school, when they washed their hands, and what they used to wash their hands. The baseline found that about 13 percent of students said that they wash their hands at school and about 15 percent wash their hands at home with soap and water after eating and after using the latrine (Table 12 Percentage of students who wash their hands). The baseline showed that 7 percent more students wash their hands at pilot schools than non-pilot schools. Gender-based and regional differences in the percentage of students that report washing hands at school and at home were not significant. Table 11 Percentage of students who wash their hands Indicator Total result Pilot Non￾Pilot Diff./Sig nificant Brakna Gorgol Female Male % washing hands with soap and water before eating and after using the latrine (at school) 12.8 18.27 11.01 7.26/yes 13.46 11.63 12.66 13 % washing hands with soap and water before eating and after using the latrine (at home) 15.33 20.27 13.72 6.55/no 15.85 15 16.4 13.44 60.02 57.2 55.72 51.23 44.7 43.54 0 10 20 30 40 50 60 70 Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Number of Students Enrollment by Grade 33 The baseline assessment also asked parents about their handwashing behavior and parents’ observations of their children’s handwashing behavior. Nearly every parent said they washed their hands every day with soap and water. A small number mentioned using water and sand. Nearly all parents also said that their children washed their hands with soap and water every day. This finding likely demonstrates a high level of social desirability bias, especially when considered with student’s self-reporting about the regularity with which they wash their hands. However, parents’ responses demonstrate that they know one should wash hands with soap and water at least before eating and after using the latrine. MGD 2.4: Increased Access to Clean Water and Sanitation Services The baseline assessment team took an inventory of the quality of schools’ latrines and water sources. For latrines, the assessment recorded the percentage of schools that had at least two latrines that were marked for boys and girls and were at least somewhat clean and well-maintained (Table 13). Table 12 Status of school latrines Latrines Total Pilot Non-pilot Difference/ Significant % of schools with 2 or more latrines 84.72 80 86 -6/no % marked for boys and girls 72.11 58.33 75.58 -17.25/no % somewhat clean 59.86 58.54 60.22 -1.69/no % somewhat well-maintained 72.45 63.41 75 -11.59/no % connected to functioning water source 6.57 9.76 5.68 4.07/no % with source of water for handwashing 83.84 82.83 84.09 -1.16/no % with soap 32.19 26.83 33.72 -6.89/no % with clean towel 0 0 0 0/no The baseline also noted whether the latrines were connected to a water source, whether there was a source of water for washing hands, and whether soap and a clean towel were available. The assessment found that most schools met most criteria, with a large majority having two latrines, and Source: Counterpart International 34 slightly fewer with latrines marked for boys and girls. Only 6.57 percent had latrines connected to a water source, however. Also, even though 84 percent had a source of water for handwashing, only 32 percent had soap, and none had a clean towel. The baseline also measured the quality of school’s water sources. Sixty-four percent of schools had a source of water on the property. The remainder had an off-property, nearby source that took an average of less than three minutes to reach. Pilot schools off-property water sources were closer than those at non-pilot schools, but the difference, just over a minute, is marginal. Table 13 School sources of water Total Pilot Non-pilot Difference % with water source on property 64.03 68 62.77 5.23/no Distance of those off-property in minutes 2.87 2 3.11 -1.11/yes % water source well-functioning 45.09 42.22 45.98 -3.75/no Research Questions Project Level Performance The project level performance question is: • Have project outputs and outcome targets been achieved? Future evaluations will use the baseline assessment’s values to report on the project’s output and outcome-related achievements where appropriate. The recommendations of this report provide specific guidance for measuring results using pilot and non-pilot schools, and for indicators that showed a significant difference between pilot and non-pilot schools at baseline. School feeding and Nutrition The school feeding and nutrition research questions are: ● What is the effect of school feeding on attendance, enrollment, and attention? ● What is the interaction between school feeding and improved hygiene practices? The baseline assessment did not identify any schools that currently provide a school feeding program for their students besides the McGovern-Dole project. However, it was found that 31 percent of schools were part of a previous school feeding project implemented by WFP in Brakna and Gorgol regions from 2012 to 2017. This project was closed in 2018. More information on the history of school feeding in Mauritania is provided above. Counterpart’s current school meals activity consists of a nutritious breakfast five days a week and midday school meal four days a week. Each student receives a daily breakfast ration of 100 g of CSB+ and 20 g oil. Students each receive a lunch ration (Monday–Thursday) of 100 g fortified rice and 20 g peas or lentils and 20 g of oil. Peas and lentils will be rotated to make the lunches tastier and more enticing to the children. In Year 5, only a lunch ration will be provided. CSB+ will be used as a THR for mothers participating in the MCN activity. 35 Baseline values for attendance, enrollment, and attention are provided in Annex 1 and discussed in sections 3.1.3, 3.1.4, and 3.1.4. Values for hygiene practices are discussed in section 3.1.6. Education and Literacy Research Under education and literacy, future assessments will answer two research questions: • How effective are reading-oriented extra-curricular activities in improving literacy? • How effective are teacher trainings? Extra-curricular activities To support the measurement of whether or not extra-curricular activities are effective in improving literacy, the baseline assessment asked students whether or not they participated in extra-curricular reading activities at baseline. The baseline found that overall, 20.3 percent of students said that they participate in these activities. The percentage was about the same between pilot and non-pilot schools. However, a considerably higher percentage of students in Gorgol (34%) said they participated in extra-curricular reading activities than Brakna (12.6%). The baseline assessment did not take an inventory of the extra-curricular or measure their quality. Table 15 Participation in extra-curricular activities Measure Total result Pilot Non￾Pilot Difference/ Significant Brakna Gorgol Difference/ Significant Female Male % participating in extracurricular reading activities 20.31 18.94 20.76 -1.82/no 12.61 33.96 -21.35/yes 20.18 20.54 The baseline assessment found that percentage of female and male students that participate in extra￾curricular activities is similar. Teacher Trainings This report discusses teacher trainings, including the percentage of teachers who have received training in the last two years. The section also reports on teachers’ perception of the quality of those trainings. Health and Maternal & Child Health (MCH) Under Health and MCH, future evaluations will answer three research questions: ● What is the effect of deworming medicine on student attendance? ● What is the effect of latrine and water access on student attendance, especially for girls? ● Is there behavioral change in handwashing for students? 36 Deworming To measure the effect of deworming medicine on attendance, the baseline assessment measured the percentage of schools that distribute deworming medicine, as reported by school directors. The baseline found that 36 percent of schools overall currently distribute deworming medication. However, a larger percentage of pilot schools (52%) than non-pilot schools (30.5%) distribute the medication (Table 16). There is a risk of not being able to conclude whether the activities in the pilot schools are causing the higher performance of the pilot schools compared to the non-pilot ones since the pilot schools are already at an advantage. In that case, we will compare the pilot schools at baseline with values at midterm and at the end of the project. The evaluator will identify the various factors and conduct statistical analysis while taking into consideration confounding factors. Table 16 Percentage of schools that distribute deworming medication Measure Total result Pilot Non￾Pilot Difference/ Significant Brakna Gorgol Difference/ Significant % of schools where students received deworming medicine in the current school year 35.69 52 30.53 21.47/Yes 34.91 28.57 6.34/no Attendance for schools with deworming 94.31 88.31 96.65 -8.34/N/A 92.49 91.6 0.89/N/A Attendance for schools without deworming 88.95 92.44 88.39 4.05/N/A 96.09 82.44 13.65/N/A Latrine and Water Access A great majority of parents agreed in interviews that improved latrines would reduce their child’s school absences16. About 25 percent of parents disagreed. Parents who disagreed said that improved latrines would not increase attendance because their school already had good latrines. They also said that their children would only use latrines at home or lived close enough to school that children could go home easily or use the latrines in homes that neighbor the school. Still, the baseline assessment found that at the time of the report, attendance was not higher for schools at which data collectors observed good latrines than schools that did not have good latrines. The data shows that attendance was slightly higher at schools without good latrines, but the difference was not significant. 16 Schools were counted as having Improved Latrines if they had at least two latrines that were marked for boys and girls and were at least somewhat clean and well-maintained. 37 Table 17 Attendance at schools with and without good latrines Measure Percent Attendance with good latrine 88.57 Attendance without good latrine 91.24 Similarly, attendance did not differ at schools with or without a working water source. Table 148 Attendance at schools with a working water source Measure Percent Attendance with working water source at school 89.12 Attendance without working water source at school 90.75 The baseline assessment sought to understand students’ perspectives on the importance of well-maintained latrines and water sources. A relatively small percentage of boys and girls said that they miss school because of the latrines or quality of the water source. Table 19 Percentages of boys and girls that miss school because of latrine or water quality Measure Percent % girls reporting sometimes missing school because they don't like the latrines 8.84 % boys reporting sometimes missing school because they don't like the latrines 11.57 % girls reporting sometimes missing school because they don't want to drink the school water 9.04 % boys reporting sometimes missing school because they don't want to drink the school water 9.71 Handwashing Handwashing behavior values are reported in section MGD SO2: Increased Use of Health, Nutrition, and Dietary Practices. Methodological Research Questions The methodological questions included in the baseline assessment scope of work were: • What is the best way to measure the three undefined McGovern-Dole outcome indicators (MGD 1.1, 1.2 and 1.3.2)? • How reliable is school and government-collected attendance and enrollment data? How can the accuracy be improved? Measuring MGD 1.1, 1.2, and 1.32 MGD 1.1: Improved Quality of Literacy Instruction - The baseline assessment took a baseline measurement to support future evaluations of MGD 1.1 by asking teachers: • If they had received a training in the past two years. • To rank the quality of that training. 38 • To rank the quality of materials used in the training. The percentage of teachers that received training in the past two years and their ranking of the quality of the training is found above. Future evaluations should consider comparing improvements in reading proficiency to improvements in teacher training and quality, controlling for other variables such as changes in the school’s feeding program, deworming medicine distribution, location, number of extra-curricular activities, and size to understand the effectiveness of teacher trainings. MGD 1.2: Improved Attentiveness - The baseline study asked teachers to estimate the number of times they had to refocus the class to take a measure of students’ attentiveness. Midterm and final evaluation could ask this question again, with additional questions that ask for teachers’ and parents’ perception of changes in attentiveness since the project began. An objective measure of attentiveness, without investing time and resources into a sophisticated assessment methodology, remains elusive. The project could choose proxy indicators if it wished to track increases, like completeness of student assignments, or average grades on regularly taken tests. Tracking these indicators would also require time and resources. In lieu of a better alternative, teacher estimation of student attentiveness compared to data collected through interviews appears sufficient to measure improvement. MGD 1.3.2: Reduced Health Related Absences – The definition of MGD 1.3.2 is, Percentage of schools receiving USDA food security and health assistance which experience a decline in average student absenteeism. The baseline assessment has provided attendance rates for schools. The midterm and final evaluation will measure whether attendance has increased for schools that received USDA food security and health assistance. The baseline assessment also showed that attendance rates are already high at the project’s schools, but enrollment steadily declines as students’ progress from grades 1 to 6. Midterm and final evaluations should also measure changes in enrollment as schools receive USDA assistance. Reliability of Attendance and Enrollment Data The baseline assessment found that attendance and enrollment data that schools and the government collect seemed fairly accurate. School directors also reported that they considered the data accurate. Directors said that occasionally teachers do not take attendance, especially if there are not enough teachers present on a given day, but that missing attendance happens relatively rarely. Regarding enrollment, school directors said that inaccuracies occur when students leave because the family has moved, the student enrolls in a different school, or students are accepted off the record (this happens if the student does not have a birth certificate). One director said that some enrolled students left his school to attend neighboring schools that have canteens and offer school materials. Most school directors consider their enrollment data accurate. To ensure accuracy, the project should keep its own records of attendance and enrollment data. Project field staff should conduct regular, random headcounts at schools and compare the headcounts to attendance lists. The project should also compare enrollment numbers disaggregated by gender and grade to a list of eligible students in each school’s area of responsibility. Midterm and final evaluations 39 should seek to understand the degree to which increases in enrollment reflect a net increase of students attending school or students leaving non-project schools to schools that participate in the project. Counterpart’s Monitoring, Evaluation and Learning (MEL) Director will design questionnaires for the field agent to collect the number of students currently benefiting from the project who do not have birth certificates. The birth certificate is an essential document for the First Leaving School Certificate in grade 6. They will collect this information from the school directors and teachers. Counterpart will also collect the number of school-age children (five to eight-years-old) who do have birth certificates in each of the 209 schools project zone. Counterpart will collect this information from the communes, chief of the villages, PTA/SMC, and other resources persons from the villages. This information will allow Counterpart to understand the magnitude of the problem. Counterpart will then work with organizations such as the Ministry of Social Affairs, Childhood, and Family, the Ministry of Education, Save the Children, and UNICEF to address this issue. Counterpart will perform this activity yearly beginning in FY22. Table 19 Perception of attendance and enrollment data quality Measure Total Pilot Non-Pilot Difference/Significant % directors reporting attendance data is very accurate 96.24 97.44 95.96 1.45/no % directors reporting enrollment data is very accurate 92.15 88.64 93..07 -4.43/no % of teachers reporting taking attendance before every class 97.72 96.88 98.04 -1.16/no School Feeding Sustainability Research questions under sustainability include: • What is the government capacity to manage school feeding at regional and national levels? • What commitment has the government shown regarding school feeding? (e.g. does it have a school feeding policy, clearly defined roles for managing school feeding, and plans to expand school feeding budget)? 40 Mauritania’s recently elected president has reportedly prioritized education, including school feeding. After years of the WFP leading the international community’s contribution to Mauritania’s school feeding programs, in 2016, the GoM began leading its own school feeding program through its School Feeding Department. The program has a budget of $700,000 USD per year since 2019 and works in three regions. Whereas the WFP program focuses more on emergency relief, the government’s program focuses on school feeding throughout the school year. In 2021, the government launched its national school feeding program and plans to expand the program to additional regions not covered by Counterpart and WFP, though it is not clear if the government’s school feeding budget will also expand accordingly. 17 This program will reach 57,378 students in the regions of Hodh Charghi, Hodh Gharbi, Gorgol, Trarza, Adrar, Tagant, Dakhlet-Nouadhibou, Tiris-Zemmour, and Inchiri. The GoM school feeding program will include rice, beans, lentils, oil, and dry milk. The government’s School Feeding Department has 36 staff at headquarters and two staff in each region. Staff feel they have learned important lessons on planning, distribution, and training from the WFP. The School Feeding Department expressed confidence in its capacity to manage the program but see shortcomings in monitoring and assessment capabilities and would like support to improve in this capacity. The department would also like to learn how to allocate scarce school feeding funding more effectively, and more generally learn from government-run school feeding programs in other countries. One of the government’s motivations for school feeding is to centralize the school system. Government officials observe that schools are spread too thin, and as a result, so are education resources. Through school feeding, the government hopes to make it more practical for children to come from further away to attend more centralized, better resourced schools. That school feeding programs provide incentive for students to travel further to attend school was confirmed by a director who said that his school’s enrollment data was not accurate because students left his school to attend a school with a school feeding program. 17 The description of the GoM school feeding efforts and associated data obtained during interviews with government officials in support of the baseline assessment. Source: Counterpart International 41 4. Conclusions and Recommendations Recommendations Promoting School Feeding Sustainability Findings: The GoM is committed to promoting school feeding. The School Feeding Department reported that it would benefit from assistance with managing school feeding funding and monitoring and assessment of school feeding projects. Recommendation 1: The GoM’s expression of interest in receiving specific support in budget planning, monitoring, and assessment provides an opening for the Future is Ours! to build on the project’s sustainability. The project could work with the GoM in three ways to build the GoM’s capacity to manage school feeding and carry on the project’s activities after it ends. First, the project could assist the Department of School Feeding in working with other government departments and agencies to maximize the efficiency of government resources. USDA staff who manage McGovern-Dole initiatives report that school feeding projects are more sustainable in countries that have effective communication and coordination across government ministries. Along these lines, The Future is Ours! could work with the Department of School Feeding to allocate its resources and to understand how those resources are complemented by other governmental initiatives. Second, the GoM expressed an interest in improving its monitoring and assessment capacity. An initiative to improve this capacity also offers an opportunity to connect central government initiatives with local government officials. The project could better align local government data collection and reporting with central government monitoring needs and create incentives that would reward local governments for regular, high-quality reporting. Such an initiative would improve central government’s monitoring and assessment. Importantly, it would also provide incentives for local government officials to increase their awareness of the project and their involvement in it. Third, the project could work with the GoM to promote initiatives that would tie the central government, local government, and communities together. For example, the project could work with the government and communities to create a school feeding challenge. The initiative could challenge local government officials and school communities to stretch school feeding budgets while maximizing nutritional value of school meals. The initiative could reward innovative ideas of practices and result in a conference attended by school directors and parents from high performing schools. Source: Counterpart International 42 Counterpart agrees with this recommendation. The GoM recently launched an important national school feeding program funded under the national budget for 5.5 million in 2021. The GoM already requested that Counterpart join the country strategic board which includes three important entities – the Ministry of Education, Food Security Commission, and General Delegation for National Solidarity and the Fight against Exclusion, known as Taazour. Counterpart will also: • Coordinate DNES staff to attend the WFP Cote d’Ivoire-based Center of Excellence Against Hunger and to take a course on building a national financial plan and budget. • Coordinate a regional peer-to-peer mentoring program with the Ministry of Education in Senegal. • Coordinate DNES participation in the annual Global Child Nutrition Forum (GCNF), which brings together virtually and physically actors in child nutrition to share best practices. Counterpart will also lead strategic exchange visits to Senegal. • Finally, to ensure the continuity of school canteens in the project area, Counterpart will organize a sustainability/ graduation workshop in FY21 (as per project workplan). Measuring Project Performance Findings: • 13.3 percent more teachers at pilot schools than non-pilot schools had received a training in the last two years. • Teachers in pilot schools ranked the quality of their training higher than teachers at non-pilot schools. • Students at pilot schools were 7 percent more likely to say that they washed their hands with soap and water. • 11.5 percent more pilot schools than non-pilot schools distribute deworming medication to their students, and schools with deworming medicine distribution had higher attendance than those without deworming medicine distribution. Recommendation 2: The baseline assessment found that pilot schools perform better than non-pilot schools in the quantity of training that teachers receive, the quality of that training, the degree to which students report washing their hands, and the distribution of deworming medication and correlated influence on attendance rates. The findings indicate that pilot schools have some advantage over non-pilot schools in acquiring and using some resources efficiently. The project should take into consideration the advantages that pilot schools show at baseline when designing project activities and allocating resources. The midterm and final evaluations should measure overall improvement in these variables, and the distribution of that improvement to understand whether pilot schools’ evident advantage at baseline correlates with greater improvement than what is achieved at non-pilot schools. As mentioned above, future evaluations face a risk in measuring impact presented by the disadvantage that non-pilot schools face. The evaluator will compare pilot schools at midterm and end of the project with results at baseline and will conduct statistical analysis while taking into consideration confounding factors. 43 Improved Literacy of School Aged Children Finding: The designers of the EGRA suggest that students should be able to read 40 correct words per minute by the end of Grade 2: only 20 of 271 Arabic readers and 5 out of 291 French readers met this milestone. Most students are still learning to recognize letters and phonemes: we recommend setting a minimum proficiency level based on letter and phoneme recognition subtask. At baseline, 32.2 percent of students assessed in Arabic and 9.6 percent of students tested in French could read 20 letters per minute. Finding: Non-pilot school students were more proficient in Arabic than pilot school students by 19.2 percent. There was no significant difference in French proficiency between pilot school and non-pilot schools. Recommendation 3: The assessment team understands that the project prefers to keep the standards SO1 indicator to make sure that measurement is consistent across McGovern-Dole projects and to add an indicator tailored to measuring literacy improvements in the current The Future is Ours! project. The assessment team recommends that, in addition to the standard SO1 indicator, the project should add an indicator that would measure the percent of students who, by the end of two grades of primary schooling, can read 20 letters per minute. Defining success as the percentage of students who can read 20 correct letters per minute puts the focus on the skills that the project will most likely improve over the course of the project. Recommendation 4: The baseline team recommends that, in its efforts to improve student literacy, the project should aim to increase the percentage of students who, at the end of second grade, can read 20 letters per minute by 3 percentage points per year. This target is reasonable based on the low proficiency scores in both languages at baseline, but also the fact that a significant portion of students have learned to read some letters in both languages. Recommendation 5: If the project has sufficient resources, it should research why the proficiency gap between pilot and non-pilot schools exists in Arabic but not French. The project should use the results to tailor its interventions accordingly. For instance, if the project discovers that non-pilot school students are more likely to speak Arabic at home than pilot school students, the project could emphasize Arabic reading materials in pilot schools’ extra-curricular reading activities. Counterpart suspects that most of the students from the non-pilot schools are originally from koranic schools (mahadras) as they are more remote than pilot schools. Nevertheless, Counterpart will confirm this assumption by conducting a survey on the origin of the students in both pilot and non-pilot schools. The 209 schools, including the 51 pilot schools, were selected among 400 schools in close collaboration with the GoM using the following criteria:  Schools with six cycles/grades (completed cycles/ full-scale18) were prioritized but the Ministry highlighted that it is difficult to find 209 schools which meet this requirement especially in rural areas. Only 31 percent of primary schools in Mauritania are complete (full-scale) schools. 18 A complete/full cycle primary school is one with all six grades. 44  Schools that are close to local villages (1 to 2 kilometers). This increases enrollment of students and is in line with the Mauritania Government policy.  Engagement and active participation of the community toward the project.  Schools accessible anytime for commodity transportation.  Schools with infrastructure such as storerooms, latrines, and water sources.  Schools with enrollment from 40 to more than 400 students. Once the 209 schools were selected, Counterpart and the MOE chose the 51 pilot schools using the criteria below:  Rural schools that are in proximity of 1 to 3 communes in one department of Brakna and Gorgol.  Schools must have six grades with early grades 1, 2, 3 with one bilingual teacher and one classroom.  The early grades 1, 2, and 3 have both French and Arabic ongoing literacy programs. The schools do not have multi-grade system (two grades in one classroom) or double vacation (one grade in the morning and the other one in the afternoon).  Teachers are fluent in both French and Arabic.  Engagement and active participation of the PTA toward the project to increase parental support for education and literacy.  The SMC should be willing to sign a Memorandum of Understanding and engage actively in the pilot project. Improved Quality of literacy instruction Finding: Teachers reported that the trainings they had received were adequate and gave low ratings to the materials used. Recommendation 6: In interviews, teachers indicated that the trainings relied heavily on lecture and text. They also indicated they left trainings feeling that the lesson plans were not easily applied to their teaching environment. The project should ensure that trainings are dynamic and result in new skills and materials that teachers may transfer to their classroom. Improved Attentiveness Finding: Teachers reported that they had to refocus their class three to four times per session. Recommendation 7: Attentiveness is difficult to measure accurately and inexpensively. The baseline assessment provided one measure, stated in the findings above. In addition to asking about refocusing class, midterm and final evaluations should gather other perceptions of increased attentiveness. These could include asking students, parents, and teachers whether they perceived an improvement in their students’ focus during class time. Future evaluations could also review records of the student assignment completeness or improvement in test grades. Recommendation 8: If the project has sufficient resources, it should conduct a special study into how school feeding projects influence student attentiveness. There are many ways to design student attentiveness studies, and the project should select one that best fits its resources and relationship with schools. One suggestion is to have project staff who regularly engage with schools to randomly 45 select 2-3 classrooms per month for observation. The project staff member would then count the number of students who were actively listening to the teacher or engaged in the project activity in five￾minute intervals while the class was in session. The study should match data collected through these observations with the project’s school feeding data, schools’ demographic information, and qualitative data collected through interviews with students and teachers. The analysis would describe conditions under which school feeding projects improve student attentiveness. The analysis would be most meaningful if it could compare data collected at project schools to data from schools that do not have a school feeding project. While the project may not have access to schools that do not have school feeding projects, it should consider whether the government would be willing collect data at these schools to contribute to the study. The project should note that it is important in classroom observation studies to properly train classroom observers to ensure that the data is accurate and consistent. Improved Student Attendance and Enrollment Finding: According to baseline data, schools have high attendance rates. Finding: Enrollment declines as grade progresses, with about 60 students per class in first grade dropping to around 44 students per class in sixth grade. Caveat: The baseline data in this area may require correction because of COVID-19 effects on school attendance and data collectors’ ability of to check school attendance data. Recommendation 9: The project should collect and track attendance data for all the schools participating in the project to ensure that measurement of the impact of activities on attendance is measured accurately. Project staff should also conduct random head counts of boys and girls once a month, and compare that count against the day’s attendance to assess the quality of the school’s attendance data. Recommendation 10: The project should work with schools to understand why enrollment drops as students’ progress through grades 1 – 6. Future evaluations should seek to measure if the school feeding project reduces the dropout rate at higher grade levels. Conclusion There is ample room to improve the literacy of school-aged children in The Future is Ours! schools. The designers of the EGRA suggest that students should be able to read 40 correct words per minute by the end of grade two. At the time of the baseline assessment, only 20 of 271 Arabic readers and five French readers met this milestone. This suggests that, for the time spent in classrooms, students are not learning very efficiently. The qualitative and survey data can suggest what may be the key barriers to more efficient learning. The baseline assessment results indicate that The Future is Ours! activities will increase learning efficiency. Stakeholders indicated that school feeding, improved literacy instruction, and associated activities will invigorate the learning process. The findings provided in this report will provide comparative data to measure the degree to which the project’s goals and related intermediary results are achieved. 46 Annex 1: Key Indicators and Baseline Values Result Indicator Operationalized for Baseline Total Pilot Non￾pilot Difference/ Significant Male Female Difference/ Significant Brakna Gorgol Difference/ Significant MGD SO1: Improved Literacy of School Aged Children Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. Percentage of students who, by the end of two grades can read 20 correct letters per minute. French 8.1% 7.0% 8.3% -1.3%/no 7.9% 5.3% 2.6/no 6.5% 6.0% 0.5/no Arabic 35.0% 16.4% 37.5% -11.1%/no 35.9% 29.8% 6.1/no 33.3% 31.3% 2.0/no MGD 1.2: Improved Attentiveness Percent of teachers noting improved attentiveness in students participating in school feeding project. Number of times per class a teacher needs to take action to refocus class. 3.38 3.37 3.39 -0.02/no 3.21 3.74 -0/53/no MGD 1.3: Improved Student Attendance19 Average student attendance rate in USDA supported classrooms/schools. Average attendance rates at USDA supported schools as recorded in government and school attendance records. Grade 1 90.31 89.09 90.65 -1.56/no 92.44 86.69 5.75/no Grade 2 92.33 90.08 92.97 -2.89/no 94.4 88.7 5.7/no Grade 3 90.99 89.27 91.48 -2.21/no 94.75 84.35 10.4/yes Grade 4 88.56 89.28 88.35 0.93/no 93.14 80.68 12.46/yes Grade 5 87.57 90.05 86.74 3.3/no 93 77.95 15.05yes Grade 6 89.22 93.53 88.01 5.51/no 94.45 80.45 14/yes Mean – All grades 90.8 90.24 90.96 -0.71/no 93.63 85.92 7.71yes 19 Counterpart MEL team confirms that for the baseline enrollment data, we have 70122 students (34138 boys and 35984 girls). The attendance data are available at school level in each classroom. 47 Result Indicator Operationalized for Baseline Total Pilot Non￾pilot Difference/ Significant Male Female Difference/ Significant Brakna Gorgol Difference/ Significant MGD 1.3.4: Improved Student Enrolment Number of students enrolled in school receiving USDA assistance. Mean number of students enrolled in school receiving USDA Grade 1 assistance. 64.2 60.52 65.17 -4.65/no 63.36 65.67 -2.31/no Grade 2 57.38 57.43 57.37 .006/no 53.88 63.46 -9.58/no Grade 3 55.89 55.47 56 -0.53/no 54.87 57.66 -2.79/no Grade 4 51.66 52.91 51.33 1.58/no 52.83 49.62 3.21/no Grade 5 44.8 50.36 43.35 7.01/no 45.85 42.98 2.87/no Grade 6 43.91 46.86 43.13 4.74/no 43.64 44.39 -0.75/no Mean – All Grades 317.85 323.57 316.35 7.22/no 314.43 323.77 -9.34/no MGD SO2: Increased Use of Health, Nutrition, and Dietary Practices Number of beneficiaries who can demonstrate knowledge of effective health and hygiene practices because of USDA assistance. % of students washing hands with soap and water before eating and after using the latrine (at school). 12.8 18.27 11.01 7.26/yes 13 12.66 1.34/no 13.46 11.63 1.83/no % washing hands with soap and water before eating and after using the latrine (at home). 15.33 20.27 13.72 6.55/no 13.44 16.4 2.69/no 15.85 15.00 0.85/no 48 Result Indicator Operationalized for Baseline Total Pilot Non￾pilot Difference/ Significant Male Female Difference/ Significant Brakna Gorgol Difference/ Significant MGD 2.4: Increased Access to Clean Water and Sanitation Services Number of schools using an improved water source. % of schools with functional safe water source on property20. 27.51 32.26 26.15 6.1/no 22.9 38.85 -15.95/no Number of schools with improved sanitary facilities. % of schools with 2 marked latrines which are somewhat clean and well￾maintained, have a source of water for handwashing, and have soap. 17.33 11.76 19.15 -7.38/no 13 25.41 -12.41/no 20 Note that many schools use water sources that are off school property. 49 Annex 2: EGRA Validity The baseline assessment used Chronbach's alpha statistics for each complete EGRA and the subtasks that made up each EGRA to test for reliability of the assessment data. Alpha statistics test whether item-level responses are internally consistent. For example, students who can answer the first four reading questions correctly should be more likely to answer the fifth reading question correctly than students who got the first four wrong. Chronbach's alpha statistics are between 0 and 1. Table 20 Alpha ranges Alpha Range21 Finding 0.0-0.6 Not Reliable 0.6-0.8 Reliable 0.8-1.0 Very Reliable Table 21 presents that alpha statistics for the EGRA dataset. All Chronbach's alpha statistics are above 0.6, suggesting that the data are reliable. The lower reliability of the alpha statistic for the overall French score and the subtasks suggests that if a student’s skills are somewhat isolated within a given subtask, a student who scores high within one subtask may not be among the highest scorers on other subtasks. This underlines how different reading skills are. For example, students who did best at recognizing sound differences were not always the ones who read words most quickly. All the Chronbach's alpha scores remain well above the minimum acceptable threshold for validity. Table 21 Alpha statistics for the baseline dataset Arabic French # Items22 Alpha # Items Alpha 1. Sound Recognition 5 0.78 5 0.89 2. Phoneme Reading 100 0.90 100 0.97 3. Devised Words 50 0.88 50 0.97 4. Oral Reading Passage 72 0.97 97 0.97 5. Reading Comprehension 5 0.92 5 0.96 Overall 232 0.91 257 0.75 21 Source: Adhika, A. Improvement of Quality, Interest, Critical, and Analytical Thinking Ability of Students through the Application of Research Based Learning in Introduction to Stochastic Processes Subject International Electronic Journal of Mathematics Education 12(2):167-191 22Number of items on assessment. Some items were collinear and therefore excluded from the alpha statistic calculation. 50 Annex 3: EGRA Explanation and Results Section MGD SO1: Improved Literacy of School Aged Children describes the overall EGRA assessment results. This Annex provides the results three ways. First, it reviews scores according to each section of the EGRA, demonstrating the tasks that students could and could not perform. Second, the Annex presents overall scores using the EGRA Index, presenting overall scores on the entire assessment and illustrating differences between subgroups, such as students in pilot and non-pilot schools, gender, and department. The third section provides an explanation of the baseline assessment’s proposed indicator for MGD1, which is defined as the percentage of students who reach a certain level of reading proficiency. It also describes proficiency levels by subgroups. Scores by EGRA Section Subtask Scores: French The subtask scores show what preliteracy skills students possess. For example, in the first exercise, students were read three words out loud and asked which started with a different sound from the other two. Forty-three percent of students could not identify any of the differences; the average student completed 29 percent of the questions, or 1.5 out of the 5 questions. Table 22 EGRA scores by task - French Subtask Mean Score Units Percent of Zero Scores 1. Sound Recognition 29.0 Percent 43.0% 2. Phoneme Reading 5.8 Correct letters per minute 9.6% 3. Devised Words 0.7 Correct words per minute 82.2% 4. Oral Reading Passage 0.7 Correct words per minute 85.8% 5. Reading Comprehension 0.3 Percent 98.7% When asked to read a list of letters and phonemes, only 9.6 percent of students could not recognize any letters (91.4 percent could identify at least one). On average, students could recognize 5.8 letters in a minute. When given 60 seconds to read up to 50 devised words – that is, words that do not exist, but students could sound out, (such as nur, clo, and blu) – 82.2 percent of students could not sound out any words; 14.4 percent could sound out one to three words. On average, students could complete 0.7 words. When asked to read a short story out loud, students showed similar abilities as the devised words: 85.8 percent of students could not read any words; 12 percent could read one to five words. Only five of the students were able to read 10 or more words in the short story in a minute. The EGRA is based on a conceptual framework and evidence that students need to be able to read 40 words per minute or more to read for comprehension and sustain continual literacy 51 improvement in future grades.23 Despite EGRA’s use globally and in multiple languages, this is based primarily on research completed in English in the U.S. and may not be universal. When asked to answer questions that would reveal comprehension of a short story, 98.7 percent of the students could not answer any correctly. This is not a surprise given that the mean student read 0.7 words of the story correctly. Almost none of the students are reading words well enough to derive meaning from entire sentences. Subtask Scores: Arabic The Arabic literacy assessment showed greater student progress on many preliteracy skills than the French assessment. These scores may not imply greater learning in the classroom. Greater exposure to spoken and written Arabic and particularities of the language may make it easier for many students to develop these skills. Overall, the results follow a logical progression: on subtasks that require more complex skills, scores are lower and the percentage of students who cannot answer any correctly are higher. Table 15 EGRA scores by task - Arabic Subtask Mean Score Units Percent of Zero Scores 1. Sound Recognition 43.1 Percent 30.1% 2. Phoneme Reading 13.1 Correct letters per minute 43.9% 3. Devised Words 4.4 Correct words per minute 53.8% 4. Oral Reading Passage 5.2 Correct words per minute 54.5% 5. Reading Comprehension 6.7 Percent 83.8% The sound recognition score on the Arabic assessment was 43.1 percent, meaning students could hear the sound difference in 2.1 out of five questions on average. Just under one-third of students did not complete any of the sound recognition questions correctly. On average, students could read 13.1 letters or phonemes per minute. That average includes the 43.9 percent of students who could not read any letters or phonemes. Of the 56.1 percent of students who could read at least one letter, the average student could read 23.4 letters per minute. Students on average could read 4.4 devised words per minute. This is comprised of the 53.8 percent of students who could not read any devised words, and the remainder who read at an average of 9.6 correct devised words per minute. 23 RTI International. 2016. EGRA Toolkit, Second Edition. Washington, DC: United States Agency for International Development. https://pdf.usaid.gov/pdf_docs/pa00m4tn.pdf 52 When reading the short story, 54.5 percent could not read any words. On average, students could read 5.2 correct words per minute. While higher than French levels, these are very low compared to Arabic readers in other countries. Given that approximately half of students cannot read any words in Arabic, it is not surprising that 83.8 percent of students cannot answer any of the comprehension questions. On average, students could answer 6.7 percent of comprehension questions. While this is higher than the comprehension levels of the French EGRA, it is very low after two years of schooling. EGRA Index24 Group French Arabic Overall 7.2% 18.7% Pilot 7.7 18.4 Non-pilot 7.2 19.0 Female 7.5 17.7 Male 7.0 23.8 Brakna 7.0 9.0 Aleg 7.6 14.0 Babebe 7.0 13.7 Bogue 5.9 23.8 M'Bagne 10.1 20.0 Magtar Lahjar 5.7 21.0 Gorgol 7.7 13.7 Kaedi 7.9 22.6 M'bout 7.3 18.7 Mounguel 7.4 18.4 The numerical score of the EGRA index is useful to compare across subgroups and will be useful at future evaluation points to measure overall learning improvement. Despite considerable statistical power, there are no statistically significant differences in French EGRA index scores across gender, region, or department. This is likely due to the very low scores overall: half of the students answered five or fewer (of 257) questions correctly. The overall score on the EGRA index for all Arabic test takers is 18.7. While there is a slight difference in average scores across gender, it is not statistically significant. There is also no significant difference between the two regions, but considerable and statistically significant 24 The table does not include departments that had small sample sizes. 53 variation between departments. The most notable differences are the scores in Babebe, and M’Bout, which are substantially lower than the average. McGovern-Dole INDICATOR 1: Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. According to the definition, “measures of the indicator will be determined in consultation with the country, and informed by national (or regional, if applicable) curriculum standards, and by international experience.” The indicator requires using the percentage of students who reach a minimum level of proficiency, instead of a mean score. It was not possible to acquire copies of Mauritania’s primary school curriculum or student standards, and unclear whether they reflect actual teaching in the schools. According to reports from enumerators, most students said they had only been taught letters, not to how to write words. With a paucity of external bases to define proficiency, we recommend allowing the data collected to guide the definition of proficiency. While the original sampling design assumed a much higher level of literacy, adjusting the definition of proficiency to focus on ability to read letters and phonemes instead of words more accurately reflects the level of ability that is likely to change over the course of the project. The designers of the EGRA suggest that students should be able to read 40 correct words per minute by the end of grade two: only 20 of 271 Arabic readers and five French readers met this milestone. This suggests that, for the time spent in classrooms, students are not learning very efficiently. The qualitative and survey data can suggest what may be the key barriers to more efficient learning. Most students are still learning to recognize letters and phonemes: we recommend setting a minimum proficiency level based on letter and phoneme recognition subtask. At baseline, 32.2 percent of students assessed Arabic and 9.6 percent of students tested in French could read 20 letters per minute. Defining success as the percentage of students who can read 20 correct letters per minute targets the skills most likely to reasonably be improved over the course of the project. Language Zero Scores 0 clpm Not Proficient 1-9 clpm Approaching Proficient 10-19 clpm Proficient 20 clpm+ Arabic 43.9% 12.1% 11.9% 32.2% French 9.6% 76.6% 7.9% 5.9% Arabic Subgroups Subgroup Zero Scores 0 clpm Not Proficient 1-9 clpm Approaching Proficient 10-19 clpm Proficient 20 clpm+ Non-pilot 42.9% 8.7% 12.0% 36.4% Pilot* 47.1% 24.1% 11.5% 17.2% Female 47.9% 11.1% 10.6% 30.4% 54 Male 39.6% 13.4% 33.7% 33.7% Pilot schools show lower levels of Arabic proficiency than non-pilot schools at a statistically significant level. This is surprising, given that pilot schools were often selected due to being more accessible. Further analysis found significant differences even when controlling for ethnicity and language. Girls have slightly lower average Arabic proficiency, but not to a statistically significant degree. Zero Scores 0 clpm Not Proficient 1-9 clpm Approaching Proficient 10-19 clpm Proficient 20 clpm+ Brakna 44.4% 9.1% 12.2% 34.3% Aleg 29.5% 1.7% 17.8% 51.1% Babebe 68.9% 10.0% 6.7% 14.4% Bogue 43.2% 19.4% 16.2% 21.2% M'Bagne 57.0% 10.0% 5.0% 28.0% Magtar Lahjar 41.7% 2.8% 8.3% 47.2% Gorgol 42.8% 17.9% 11.2% 28.1% Kaedi 49.0% 16.2% 10.3% 24.5% M'bout 53.3% 6.7% 13.3% 26.7% Mounguel 20.8% 29.2% 12.5% 37.5% Although there are no statistically significant differences of proficiency between the two regions, there are statistically significant differences at the departmental level. The lowest levels of letter proficiency are found in Babebe, Bogue, M'bagne, and M'bout, which is in alignment with the findings on scores overall. Targeting schools in these departments may serve the greatest needs and are most likely to increase proficiency levels at future evaluation points. Ethnicity Zero Scores 0 clpm Not Proficient 1-9 clpm Approaching Proficient 10-19 clpm Proficient 20 clpm+ Overall 43.9% 12.1% 11.9% 32.2% Haratine 50.1% 13.1% 10.5% 26.3% Beydane 24.7% 11.0% 12.4% 51.6% Halpulaar 61.8% 9.1% 9.5% 19.6% Fulani 56.8% 18.5% 24.7% 0.0% Overall, ethnicity had a statistically significant difference on proficiency levels. Beydane students had higher levels of proficiency than all other ethnicities, followed by Haratine 55 students. In the recommendations section, this report discusses the way midterm and final evaluations should manage the significant impact that ethnicity has on proficiency. French Subgroups Unlike Arabic proficiency levels, French proficiency levels are not significantly higher in non￾pilot schools. Average letter recognition is slightly, but not significantly, lower in non-pilot schools than pilot schools, at 5.9 clpm and 5.7 clpm respectively. If scores in both languages were significantly higher in non-pilot, it may suggest that students learned better in one group of schools or the other, but because the results are mixed by language, it suggests that other factors (such as language exposure) are the cause. This is helpful for future evaluations, as it means that future differences in improvement are more likely attributable to the project, and not some other differences endemic to the schools. There were no significant differences across gender (mean scores were within one thousandth of a percent of each other). Subgroup Zero Scores 0 clpm Not Proficient 1-9 clpm Approaching Proficient 10-19 clpm Proficient 20 clpm+ Overall 9.6% 76.6% 7.9% 5.9% Non-pilot 9.6% 77.1% 7.1% 6.1% Pilot 9.6% 74.5% 10.6% 5.3% Female 13.1% 74.5% 6.8% 5.6% Male 7.0% 77.8% 9.3% 6.0% French proficiency scores were not significantly different by region or department. None of the students assessed in French reported French as their primary language. This aligns with the hypothesis that Arabic scores vary by department primarily because of exposure to the language, not differences in learning at schools. Despite having overall lower levels of proficiency than the Arabic assessment, more students were able to read at least one letter. This again may be due to exposure to a Latin alphabet, or that Arabic letter recognition requires understanding cursive and isolate letters. It suggests overall that even though French proficiency levels are lower, students at baseline have begun on the path of recognizing letters but are simply unable to do very quickly. This suggests there is a large group of students to focus on to bring from reading at 1-9 letters per minute into the 20 clpm proficiency range. Zero Scores 0 clpm Not Proficient 1-9 clpm Approaching Proficient 10-19 clpm Proficient 20 clpm+ Brakna 9.1% 75.4% 9.6% 5.9% Aleg 14.4% 59.6% 15.4% 10.6% Babebe 5.2% 89.6% 0.0% 5.2% Bogue 6.2% 85.5% 4.8% 3.5% M'Bagne 0.0% 80.0% 10.5% 9.5% 56 Magtar Lahjar 12.5% 70.0% 12.5% 5.0% Gorgol 10.67% 77.7% 5.8% 5.8% Kaedi 6.8% 80.0% 8.1% 5.1% M'bout 18.8% 68.8% 0.0% 12.5% Mounguel 16.7% 79.2% 4.2% 0.0% There are no statistically significant differences between ethnicities on French proficiency. There are minor variations in proficiency between ethnicities: most notably fewer zero scores among Halpulaar students, but ethnicity does not appear to be a significant determinant. Ethnicity Zero Scores 0 clpm Not Proficient 1-9 clpm Approaching Proficient 10-19 clpm Proficient 20 clpm+ Haratine 13.7% 79.4% 4.2% 2.7% Beydane 9.2% 70.7% 10.6% 9.5% Halpulaar 3.1% 78.4% 10.9% 7.5% Fulani 12.7% 78.4% 5.9% 2.9% Targets There appears ample and achievable improvement in literacy levels. Given a significant portion of both groups can read some letters, it is reasonable to assume a portion of students could move from not proficient to proficient. We recommend setting a target of increasing the proportion of students who can read 20 letters per minute 3 percentage points per year of project implementation. Language Proficient Target Improvement: Arabic 32.2% +3% per annum French 5.9% +3% per annum While we recommend maintaining formal indicator targets only for the two languages, reporting and tracking proficiency levels by pilot and non-pilot schools will be pivotal to understanding how the two groups are affected by the project. Given the differences in proficiency levels at baseline, it will be important that future evaluations focus on improvement of proficiency over the baseline levels (such as the 3 percent per annum target), and not an expectation that all types of schools reach a single level of proficiency. Language Proficient Arabic: Non-pilot 36.4% Arabic Pilot 17.2% French: Non-pilot 6.1% French Pilot 5.3% 57 Annex 4: Baseline Assessment Scope of Work Program Overview The Future is Ours! will be implemented by Counterpart International along with its local partners EcoDev, ONG Actions, and regional partner, ARED.  The program will be implemented in phases with different cohorts of beneficiaries and treatments for each phase.  Local partners EcoDev, ONG Actions, and ARED bring deep networks that anchor the evidence￾based set of program activities in local knowledge. Working in 209 schools, The Future is Ours! will link national level decision makers to local level school stakeholders by integrating improved student literacy and nutrition outcomes with school governance, engaging regional health and education actors, and building capacity of the MOE to improve the quality of primary schools. The program addresses the many challenges facing Mauritanian children and families by working through schools, communities, and government to strengthen institutions and change behaviors, both inside and outside the classroom.  The literacy activities will focus on grades 1- 3, with Arabic language instruction for grade 1 and Arabic and French for grades 2 and 3, and school feeding and other activities for all the grades (1 to 6). Using best practices in EGRA, a pilot literacy project in 51 schools will test a holistic approach to improve student learning outcomes. This pilot is a component of a larger set of activities that contribute to intermediate and higher-level outcomes as mapped out in the Results Framework in the next section.    Results and Activities Improved literacy of school aged children (SO 1) is to be achieved through a) improved quality of literacy instruction, b) greater student in-class concentration and c) increased attendance. Improved literacy will be achieved through improved access to school supplies and materials, improved literacy instruction materials and teachers’ attendance, and increased skill and knowledge of teachers and school administrators. Improved attentiveness will be achieved by increased access to food through school feeding. Improved student attendance and enrollment will result from increased access to food, reduced health-related absences, and improved school infrastructure. This pathway is illustrated in the results framework below. Policy results which support the implementation and sustainability of SO 1 are increased capacity of government institutions, improved policy and regulatory framework, increased government support, and increased engagement of local organizations and community groups. Increased use of health, nutrition, and dietary practices (SO 2) will be achieved through improved knowledge of health and hygiene practices; safe food preparation, storage, and nutrition; increased access to water and sanitation services; increased access to preventative health interventions; and increased access to food preparation and storage tools and equipment. The effectiveness and sustainability of SO 2 are supported by the same policy results as for SO 1. 58 Results Framework for Improved Literacy of School Aged Children 59 Results Framework for Increased Use of Health, Nutrition and Dietary Practices 60 Beneficiaries The program will reach approximately 141,023 direct beneficiaries over the life of the project as follows: • 127,100 students in 209 primary schools • 192 teachers/educators/assistants • 209 school administrators • 10,265 PTA, COGES, SMC, pregnant and lactating mothers, and cooks trained • 3,247 children under five-years-old • 10 government officials Scope of Work The awarded firm will work under the supervision of Counterpart International’s Program team based in Mauritania and the Washington D.C. area. The geographical focus of the program is on the Brakna and Gorgol regions in Mauritania. The firm is responsible for the quality of the data collection, data cleaning, analysis, and translation of findings into an accurate and high￾quality report. To successfully complete the baseline study, the awarded firm must complete the following tasks: • Operationalize research questions and indicators. Of special interest is the effectiveness of teacher trainings, as well as the research questions on sustainability that will be assessed in terms of government support and capacity. • Conduct a survey to collect high quality data from treatment and control schools to provide baseline values for the research questions and for program indicators. Data quality is to be ensured through data logic consistency checks and validation rules, as well as ongoing monitoring of data collected during the survey. Key aspects of fieldwork include the following tasks: o Identifying control schools o Operationalizing research questions for measurement • Produce a baseline report containing a) descriptive statistics of baseline values including, where appropriate, disaggregated by gender, department, and grade and b) In addition to the descriptive statistics, the report will include recommendations on methodology for implementing subsequent evaluations. The firm will also provide Counterpart with all data sets in both original and final (cleaned) version used for analysis. The firm will also provide all tools used as well as a survey manual that will inform survey implementation for subsequent evaluations. Do No Harm Counterpart International works on the assumption that ethics comes before evidence, in line with the principles of ‘Do No Harm,” ensuring that any kind of intervention does not inadvertently or in any way do harm or worsen the situation. It is essential that any interaction and work carried out as part of this evaluation do not in any way negatively impact the 61 individuals or communities involved. It is critical that during data collection the psychological impact of the research on participants is considered, as well as their physical security. Research Questions The following research questions will be answered cumulatively by the midterm, final, and impact evaluations. The baseline study therefore must assess the baseline status of these questions. The awardee will be responsible for operationalizing these research questions so that they can be consistently and accurately measured in subsequent evaluations. The operationalization of the research questions will be informed by review of similar evaluations and research, including other McGovern-Dole evaluations. McGovern-Dole Project Level Performance • Have program outputs and outcome targets been achieved? (see “McGovern-Dole Outcome Indicators that Require Baseline Values” table below) School Feeding and Nutrition • What is the effect of school feeding on attendance, enrollment, and attention? • What is the interaction between school feeding and improved hygiene practices? Education and Literacy • How effective are reading-oriented extra-curricular activities in improving literacy? • How effective are teacher trainings? Health and MCH • What is the effect of deworming medicine on student attendance? • What is the effect of latrine and water access on student attendance, especially for girls? • Is there behavioral change in handwashing for students? Methodology • What is the best way to measure the three undefined McGovern-Dole outcome indicators (MGD 1.1, 1.2 and 1.3.2)? • How reliable is school and government-collected attendance and enrollment data? How can the accuracy be improved? School Feeding Sustainability • What is the government capacity to manage school feeding at regional and national levels? • What commitment has the government shown regarding school feeding? (e.g., do they have a school feeding policy, clearly defined roles for managing school feeding, and plans to expand school feeding budget)? 62 McGovern-Dole Outcome Indicators that Require Baseline Values Result # Title in Result Framework Indicator MGD SO1 Improved Literacy of School Aged Children Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text MGD 1.1 Improved Quality of Literacy Instruction Number of schools whose teachers receive training and/or assistance to improve literacy instruction MGD 1.2 Improved Attentiveness Percent of teachers noting improved attentiveness in students participating in school feeding program MGD 1.3 Improved Student Attendance Average student attendance rate in USDA supported classrooms/schools MGD 1.3.2 Reduced Health-Related Absences Percentage of schools receiving USDA food security and health assistance which experience a decline in average student absenteeism MGD 1.3.4 Improved Student Enrollment Number of students enrolled in school receiving USDA assistance MGD 2.4 Increased Access to Clean Water and Sanitation Services Number of schools using an improved water source Number of schools with improved sanitary facilities MGD S02 Increased Use of Health, Nutrition, and Dietary Practices Number of individuals who demonstrate use of safe food preparation and storage practices because of USDA assistance Approach and Methodology Design Overview The consultant will collect data from a statistically representative sample of the 209 schools. The survey will include treatment and control schools. Treatment schools will be those where the program is implemented while control schools will be the ones outside of the program implementation area for both the literacy and school feeding components. The impact evaluation will compare impact on two aspects: • On the literacy component, we shall compare 51 schools that benefit from literacy programming to schools that will not benefit from the literacy intervention. The literacy component will only be implemented in Brakna and Gorgol. • On the school feeding component, we will compare schools that receive school meals to schools in the region that do not benefit from the project. The awardee will select control schools using the same criteria used for choosing treatment schools. The criteria will be discussed with the selected evaluator. Within sampled schools, interviews will be conducted with students, directors, teachers, and parents. The program seeks to identify any barriers that can hinder the success of the program or windows of opportunity that can be used to advance the goals. Additionally, will identify any behaviors, attitudes, perceptions among the education stakeholders e.g., teachers, directors, and parents to develop appropriate behavioral communication materials and messages. In 63 addition, interviews will be conducted with government officials to assess capacity and commitment for sustainability. Document Review Documents to be reviewed as part of the baseline study include the evaluation plan, the performance monitoring plan, the work plan, the proposal, and the USDA Monitoring and Evaluation policy, as well as baseline studies and evaluations for other MGD projects. Pilot Survey The awardee is required to conduct a pilot survey to test all questionnaires. The pilot is especially important because questions phrasing will be used for the subsequent evaluations and must elicit accurate responses. Sufficient time must be allocated for updating all instruments based on pilot findings. The local languages in which each data collection instrument should be fielded is Arabic and French. Survey Sampling The proposed sampling strategy should produce a minimal detectable effect size for the results of the student survey of 0.2, which is consistent with best practices. The evaluator will randomly select McGovern-Dole schools from the 209 participating in the project. The control sample will include randomly selected schools from those that match McGovern-Dole participation criteria, but which are not participating. This anticipated treatment to control school ratio is anticipated to be 60:40. Within each sample school the evaluator will also survey teachers, including one Head Teacher, as well as one director. The observation tool will be used in McGovern-Dole schools and control schools. Parents’ perceptions of changes in their children related to McGovern-Dole can be best assessed through qualitative interviews or focus groups. An illustrative sample for each survey is shown below and will be finalized in collaboration with USDA and Counterpart: Table: Proposed baseline sample size for SSME and EGRA Stakeholder McGovern￾Dole (Treatment) Control Total All Students grades 1-6 (x 9) 972 648 1,620 EGRA only students grades 1-3 (x 6) 648 432 1,080 Teachers including Head Teachers (x 3) 324 216 540 Directors (x 1) 108 72 180 Observation tool 108 72 180 In addition, Counterpart and the MOE team have agreed to the school selection criteria below. i. School with six (completed cycles/ full-scale) cycles/grades will be prioritized but the Ministry highlighted that it is difficult to find 209 school which meet this requirement, especially in rural areas. Only 31 percent of primary schools in Mauritania are complete (full-scale) schools. 64 ii. Schools that are in proximity of least 1 to 2 kilometers to local villages. This increases enrollment of students in that locality and is in line with the Mauritania Government policy. iii. Engagement and active participation of the community toward the project. iv. Schools shall be accessible anytime for commodity transportation. v. Schools will have infrastructure such as storerooms, latrines, and water sources. vi. Schools with enrollment from 40 to more than 400 students. Reading Tools The evaluations will use EGRA methodologies. EGRA is another reading assessment tool that has been used with McGovern-Dole evaluations in other countries. A baseline EGRA and Snapshot of School Management and Effectiveness (SSME) survey will inform the design of materials, teacher training, and community mobilization activities. The evaluator will be responsible for developing the tools and submit to Counterpart for approval. Data Analysis The local data collection firm is responsible for entering all data collected under this evaluation. This includes double entry of all survey data. Survey data must be entered in Excel files. Supervisor(s) will conduct thorough data checks and submit to Counterpart a final, clean dataset. The data collection firm will also prepare and deliver a codebook to accompany the final dataset. Survey data analysis will be carried out using an appropriate statistical package such as R or Stata. Analysis will include means and means comparisons by gender, grade, and department, along with tests of statistical significance. ** Firms with capacity to collect data using tablets or smart phones are encouraged to propose conducting electronic data entry using tools such as ODK, KoboToolbox, or others. While using electronic data collection, the firm must explain the methodology of programming/testing surveys, cleaning, and submitting electronic data. Counterpart will procure tablets (Androids 2.1 with sufficient memory) for data collection. Assignment Timeline, Place of Performance, and Other Conditions. Counterpart anticipates the preparatory work and review of relevant reports and documents to be completed according to the deadlines presented below. The first draft of the baseline evaluation report is due to Counterpart on or about January 5, 2021. Once the draft is submitted, Counterpart will have seven business days to review the report, raise concerns, provide comments, and send it back to the evaluator. The evaluator will then address Counterpart’s comments and concerns and submit a revised report to Counterpart for Counterpart and the donor’s review. The final version of the report is due no later than January 31, 2021 and should include a 2-3-page stand-alone brief describing the evaluation design, key findings, and other relevant considerations. It will serve to inform any interested stakeholders of the baseline evaluation and should be written in language easy to understand by non-evaluators and with appropriate graphics and tables. Should Counterpart still not be satisfied with the quality of the final baseline report, then both Counterpart and the evaluator will negotiate a no-cost extension to ensure both parties are satisfied with the result. 65 If some project activities just started or started concurrently with baseline data collection, we ask that the evaluators carefully document in their report these circumstances, explaining how much exposure to project activities the sampled participants have had and what effect on baseline numbers the evaluators predict this exposure has had. Throughout the baseline evaluation, there must be open communication between the evaluator and Counterpart through phone calls, emails, text messages, Skype calls, or face-to￾face meetings for effective coordination between both parties and to ensure that potential issues are resolved in a timely manner. Any anticipated changes to the plan during the evaluation must be submitted in writing and be approved by Counterpart. The timeline below assumes that travel restrictions will be lifted by end of May, but the global health situation is currently fluid, and restrictions may be in place for a longer, unknown period. Figure: Tentative workplan Phase I April￾September 2020 Phase II October￾November 2020 Phase III November 2020- January 2021 Kick-off meeting – April Desk Review- April-May Development of tools and evaluation matrix- May-June Development and Submission of Baseline Evaluation Workplan-June-July  Finalization of tools and translation in French and Arabic- August-September  Preparation for field visit-September In-Country Baseline workshop in Nouakchott with Counterpart and program staff-beginning of October Training of data collectors in Nouakchott- beginning of October Piloting of Tools in Brakna and/or Gorgol - beginning of October In-country data collection-October-early November Presentation of preliminary findings -early November Analysis-November Debriefing of Preliminary findings at Counterpart’s HQ-November  Preparation and submission of first draft of baseline report to Counterpart - early December  Incorporation of feedback and submission of second draft report to USDA for review-December before holidays  Finalization and submission of Baseline Evaluation Report-early January  Submission of data sets and survey manual and tools- early January  IMC Counterpart International  Deliverable 66 Staff Structure The evaluation team will comprise one international program evaluator (Team Leader), an Associate consultant, and two or more local or international consultants or members of a consulting firm selected for their technical expertise. To the extent possible, the evaluation team will be gender balanced. Deliverables The awardee will be responsible for the six deliverables below. 1. Inception report: Before starting fieldwork, the awardee will submit an inception report for approval by Counterpart. The inception report will contain a plan for fieldwork with a description and schedule for all activities. It will also include all tools to be used for data collection, including questionnaires. 2. In-country debrief: Before the fieldwork is completed, the awardee will complete a debrief with Counterpart team in Mauritania. The debrief presentation will cover what changes were made to questions based on the pilot, if there were any survey challenges, and preliminary findings. 3. Draft evaluation report: The report shall contain the following sections: 1) Introduction, 2) Methodology, 3) Results, and 4) Recommendations. The Results section will include both summary baseline values as well as description of qualitative results. The Recommendations will be both a) programmatic, based on relevant results and b) methodological for the subsequent evaluations. The report should detail and justify how key indicators and research questions were operationalized and measured. 4. Presentation to field and HQ staff: The awardee will prepare a PowerPoint presentation of the draft evaluation report and present it to Counterpart U.S. and field staff. 5. Datasets, tools, and survey manual: All datasets (should be cleaned up) and tools will be shared with Counterpart. Additionally, a survey manual explaining how the survey/s were carried out will be provided. 6. Final report: The final report will incorporate written feedback and comments from the presentation. It should not exceed 40 pages, exclusive of annexes. The report should also include a 2-3-page stand-alone brief describing the evaluation design, key findings, and other relevant considerations. It will serve to inform any interested stakeholders of the baseline evaluation and should be written in language easy to understand by non￾evaluators and with appropriate graphics and tables. The following table outlines requirements for the Final Baseline Report: Report Length Maximum of 40 pages, excluding the Table of Contents, Acronym List, and Annexes and should be written in Times New Roman font size 12. Illustrative Report Outline Acknowledgement Table of Contents Table of Exhibits Acronym List Executive Summary (in English and French) 67 Chapter 1. Evaluation Purpose and Research Questions Chapter 2. Project Background Chapter 3. Evaluation Method 3.1 Methodologies 3.2 Sampling Framework 3.3 Data Sources and Data Collection Methods 3.4 Field Work 3.5 Analysis Plan 3.6 Strengths and Limitations Chapter 4. Findings Chapter 5. Conclusion and Recommendations 5.1 Summary of Key Findings 5.2 Lessons Learned 5.3 Recommendations References Annexes Executive Summary Include an Executive Summary that provides a brief overview of the evaluation purpose, project background, evaluation questions, methods, findings, and conclusions. Questions Address all evaluation questions in the Scope of Work. Methods • Explain evaluation methodology in detail. • Disclose evaluation limitations, especially those associated with the evaluation methodology (e.g., selection bias, recall bias, etc.). NOTE: A summary of methodology can be included in the body of the report, with the full description provided as an annex. Findings • Tables with baseline results, disaggregated by gender, department, grade and/or pre-school, as appropriate. • Brief description of each table, including any context or explanation needed to aid the reader in interpreting and understanding. • Detailed description of findings for methodological research questions and operationalizing of key indicators as described in the Scope of Work. Recommendations • Support recommendations with specific findings. • Provide recommendations that are action-oriented, practical, and specific. Annexes Include the following as Annexes, at a minimum: • Terms of Reference. • All evaluation tools (questionnaires, checklists, discussion guides, surveys, etc.). • A list of sources of information (key informants, documents reviewed, other data sources). Only if applicable, include as an annex Statement(s) of Differences regarding any significant unresolved differences of opinion on the part of funders, implementers, and/or members of the evaluation team. 68 All final versions of international food assistance evaluation reports will be made publicly available. Evaluators shall provide a copy of the evaluation report that is free of personally identifiable information (PII) and proprietary information. Final versions of evaluation reports ready for publication should be accessible to persons with disabilities. For guidance on creating documents accessible to persons with disabilities, please see the following resources: https://www.section508.gov/create/documents; https://www.section508.gov/create/pdfs” Quality Control Assess reports for quality by including an in-house peer technical review with comments provided to the evaluator. Transparency An English report should be submitted in an electronic version to Counterpart for approval. All deliverables must be approved by Counterpart Roles and Responsibilities The evaluator will be responsible for all the deliverables listed in the Term of Reference and will be free to draw their own conclusions free from political or organizational pressure. The evaluator will coordinate with Counterpart staff including Senior Program Manager, Program Officer, Director of Program Evaluation and Learning, and Counterpart Mauritania Chief of Party with regards to the overall scope, direction, and completion of this assignment. USDA will provide guidance as needed along with feedback on the initial draft, to be included in the final report. USDA will also be consulted as a key informant prior to evaluation fieldwork. Counterpart staff will provide all relevant reports, data, and related information necessary to prepare the evaluator for the assignment. And as needed, Counterpart staff will facilitate field logistics, including potential meetings with all relevant stakeholders during the field visit in Mauritania. The U.S. point of contact will be the Senior Program Manager while the in-country point of contact will be the Chief of Party. Counterpart HQ Senior Program Manager is responsible for approving evaluation deliverables. 69 Annex 5: Data Collection Instruments 70 Student Survey Tool Question # Question name Question Responses Note to interviewer Logic starttime Start time Automatic startdate Start date Automatic note You have been selected to participate in a survey on health, nutrition and education as part of the The Future is Ours! project. Your participation in this study is entirely voluntary. You are under no obligation to participate. You have the right to refuse to answer questions and to withdraw from the study at any time. If you agree, please answer all questions as honestly as possible. If you are unable to answer one of the questions, you can ignore the question. All your answers are strictly confidential. consent Ask for the respondents consent. If the person agrees, continue the interview. Yes No interviewer_name Interviewer name Multiple choice tbd device_id deviceid region Region Brakna Gorgol school_type What type of school is this? MGD Pilot school MGD non-pilot school Control school department Department Aleg If region = Brakna, show Bababe Bouge M'Bagne Magtar Lahjar Kaedi If region = Gorgol, show M'Bout Maghama Monguel municipality Municipality Multiple choice tbd school Name of school Multiple choice tbd grade What is the grade of the student? 1st 2nd 3rd 4th 5th 6th gender What is the gender of the student? 1. Male 2. Female language What language do you speak most of the time at home? 1. Arabic 2. French 3. Berber 4. Other (specify) food_where Where do you eat when you're at school? 1. School canteen 2. Buy food from nearby 3. Go home to eat 4. Other (specify) food_quality Do you like the food you eat during school? 1. Yes 2. No food_absence Do you ever miss school to eat? 1. Yes 2. No food_absence_why Why do you miss school to eat? 1. Comes late after breakfast at home 2. Goes home for lunch and doesn't come back 3. Other (specify) 71 water_where Where do you get water from to drink during school? 1. From school multiple choice 2. From home 3. From nearby the school 4. I buy bottled water 5. I don't drink water during school water_quality What do you think about the school water quality? 1. It's good if water_where==1 2. It's not good water_quality_why Why isn't the water good? 1. Bad taste if water_quality==2 2. Water is dirty 3. Water area is not clean 4. Water is not reliable / not available all the time 5. Other (specify) water_absence Do you ever miss school because you don't want to drink the school water? 1. Yes 2. No washhands How many times per day do you wash your hands at school? Enter "0" if they do not wash hands washhands_not Why don't you usually wash you're hands at school? 1. There is no place to wash hands multiple choice if washhands==0 2. There is no water 3. There is no soap 4. the wash basin isn't clean washhands_when When do you usually wash your hands when you're at school? 1 Before eating multiple choice if washhands>0 2 Before touching or preparing food 3 Before giving food to someone else 4 When I have dirt on my hands 5 After touching something dirty 6 After touching an animal 7 After using the latrine 8 Other (Specify) washhands_what What do you usually wash your hands with when you're at school? 1 Water multiple choice if washhands>0 2 Ashes 3 Sand 4 Soap 5 Other (specify) washhandshome How many times per day do you wash your hands at home? washhandshome_not Why don't you wash your hands at home? 1. There is no place to wash hands multiple choice if washhandshome==0 2. There is no water 3. There is no soap 4. the wash basin isn't clean 5. Other (specify) washhandshome_when When do you usually wash your hands when you're at home? 1 Before eating multiple choice if washandshome>0 2 Before touching or preparing food 3 Before giving food to someone else 4 When you have earth on your hands 5 After touching something dirty 6 After touching an animal 7 After using the latrine 8. After changing clothes 9 Other (Specify) washhandshome_what What do you usually wash your hands with when you're at home? 1 Water multiple choice if washandshome>0 2 Ashes 3 Sand 4 Soap 5 Other (specify) 2. Bad color/not clean 72 latrines Are you comfortable using the school latrines? 1. I am comfortable using them 2. I am not comfortable using them 3. There is no latrine at this school latrines_comfort Why aren't you comfortable using the school latrines? 1. They are not clean multiple choice if latrines==2 2. I don't feel like I have privacy 3. There is not place to wash my hands 4. They are not safe 5. Other (specify) latrines_absense Do you ever miss school because you don't want to use the school latrine? 1. Yes 2. No extracurricular Did you participate in any extracurricular reading activities at school this past school year? 1. Yes 2. No extracurricular_what What kind of activities? 1. I had a private tutor 2. Small class with teacher or tutor 3. Other (specify) extracurricular_reason Why not? 1. Not offered at my school/not aware If extracurricular = 2 2. Don't have time/not allowed 3. Don't want to/not interested 4. Other 73 absence How many days did you not go to school last semester? <100 absence_reason What was the main reason you missed school last semester? 1. I was sick if absense !=0 2. Someone in my family was sick 3. I needed to work on the farm 4. I needed to do other work Missing 5. The latrine facilities at school are not good 6. Other (specify) leave How many days did you leave early or come late last semester? <100 leave_reason What is the the main reason for coming late or leaving early? 1. I was sick 2. Someone in my family was sick 3. I needed to work on the farm 4. I needed to do other work 5. I needed to eat at home 6. I needed water 7. The latrine facilities at school are not good 8. Other (specify) attention How easy is it for you to pay attention to your teacher in class? 1. Very Easy 2. Easy 3. Hard 4. Very hard attention_reason Why is it hard to pay attention? Write down the students response If attention>2 geopoint Geopoint GPS_location Please take the GPS coordinates of the interview location. endtime Interview end time Thank the student for their time and end the interview. 74 Teacher Survey Tool Question # Question name Question Responses Note to interviewer Logic starttime Start time Automatic startdate Start date Automatic note You have been selected to participate in a survey on health, nutrition and education as part of the The Future is Ours! project. Your participation in this study is entirely voluntary. You are under no obligation to participate. You have the right to refuse to answer questions and to withdraw from the study at any time. If you agree, please answer all questions as honestly as possible. If you are unable to answer one of the questions, you can ignore the question. All your answers are strictly confidential. consent Ask for the respondents consent. If the person agrees, continue the interview. Yes No interviewer_name Interviewer name Multiple choice tbd device_id deviceid region Region Brakna Gorgol school_type What type of school is this? MGD Pilot school MGD non-pilot school Control school department Department Aleg If region = Brakna, show Bababe Bouge M'Bagne Magtar Lahjar Kaedi If region = Gorgol, show M'Bout Maghama Monguel municipality Municipality Multiple choice tbd school Name of school Multiple choice tbd grade What grades does the teacher teach? 1st 2nd 3rd 4th 5th 6th gender What is the gender of the teacher? 1. Male 2. Female languageschool What languages do you teach in at school? 1. Arabic multiple choice 2. French 3. Berber 4. Other (specify) languagehome What language do you speak most of the time at home? 1. Arabic multiple choice 2. French 3. Berber 4. Other (specify) 75 attention In an average class this semester, how many times do you have to interrupt class to refocus distracted students? trainedwell On a scale of 1 (not well trained) to 5 (very well trained), how well trained do you feel to deliver your classes? 1. One (not well trained) 2. Two 3. Three 4. Four 5. Five (very well trained) traininglast When was the last literacy related training you received? 1. This semester 2. Last semester 3. 1 to 2 years ago 4. Over two years ago 5. Never trainingwho Who provided this training? text attendence Do you take attendence before every class? 1. Yes 2. No attendence_how How often do you give attendance data to the school director? 1. Every day 2. Every week 3. Every month 4. Other (specify) geopoint Geopoint GPS_location Please take the GPS coordinates of the interview location. endtime Interview end time Thank the teacher for their time and end the interview. 76 Director Checklist Tool Question # Question name Question Responses Note to interviewer Logic starttime Start time Automatic startdate Start date Automatic note You have been selected to participate in a survey on health, nutrition and education as part of the The Future is Ours! project. Your participation in this study is entirely voluntary. You are under no obligation to participate. You have the right to refuse to answer questions and to withdraw from the study at any time. If you agree, please answer all questions as honestly as possible. If you are unable to answer one of the questions, you can ignore the question. All your answers are strictly confidential. consent Ask for the respondents consent. If the person agrees, continue the interview. Yes No interviewer_name Interviewer name Multiple choice tbd device_id deviceid region Region Brakna Gorgol school_type What type of school is this? 1. MGD Pilot school 2. MGD non-pilot school 3. Control school department Department Aleg If region = Brakna, show Bababe Bouge M'Bagne Magtar Lahjar Kaedi If region = Gorgol, show M'Bout Maghama Monguel municipality Municipality Multiple choice tbd school Name of school Multiple choice tbd grade What grades are at this schoool? 1st multiple choice 2nd 3rd 4th 5th 6th gender What is the gender of the director? 1. Male 2. Female feeding Do you have a school feeding program at your school? 1. Yes 2. No feeding_cooks Currently, how many cooks are there in your school feeding program? numeric if feeding==1 feeding_cookstraining In the past year, how many of these have been trained AND are practicing new safe food preperation practices? numeric mark zero for none if feeding==1 feeding_management Currently and including yourself, how many people are involved in managing school commodities? numeric if feeding==1 feeding_managementtrain In the past year, how many of these people were trained in AND are practicing new safe food storage practices? numeric if feeding==1 deworming Did students at your school receive deworming medicine from the government during the current school year? 1. Yes 2. No note1 Tell the director you will ask about enrollment and attendence data for last year and this year 77 y y enrollment2019_yn Do you have enrollment data from 2019/2020? 1. Yes 2. No enrollment2019 What the enrollment rate for grades 1- 6 for the 2019/2020 school year? if enrollment2019_yn==1 enrollment2019_1 grade 1 if enrollment2019_yn==1 enrollment2019_2 grade 2 if enrollment2019_yn==1 enrollment2019_3 grade 3 if enrollment2019_yn==1 enrollment2019_4 grade 4 if enrollment2019_yn==1 enrollment2019_5 grade 5 if enrollment2019_yn==1 enrollment2019_6 grade 6 if enrollment2019_yn==1 enrollment2020_yn Do you have enrollment data from the 2020/2021 school year? 1. Yes 2. No enrollment2020 What the enrollment rate for grades 1- 6 for the 2020/2021 school year? if enrollment2020_yn==1 enrollment2020_1 grade 1 if enrollment2020_yn==1 enrollment2020_2 grade 2 if enrollment2020_yn==1 enrollment2020_3 grade 3 if enrollment2020_yn==1 enrollment2020_4 grade 4 if enrollment2020_yn==1 enrollment2020_5 grade 5 if enrollment2020_yn==1 enrollment2020_6 grade 6 if enrollment2020_yn==1 enrollmenthow How do you collect enrollment data? write down text response enrollmentconfident How confident are you in the quality of enrollment data? 1. Very confident 2. Somewhat confident 3. Not confident enrollmentquality Why are you only somewhat or not confident? write down text response if school_type!=3 & enrollme attendance2019 What the attendance rate for grades 1- 6 for the 2019/2020 school year? if attendance2019_yn==1 attendance2019_1 grade 1 if attendance2019_yn==1 attendance2019_2 grade 2 if attendance2019_yn==1 attendance2019_3 grade 3 if attendance2019_yn==1 attendance2019_4 grade 4 if attendance2019_yn==1 attendance2019_5 grade 5 if attendance2019_yn==1 attendance2019_6 grade 6 if attendance2019_yn==1 attendance2020_yn Do you have attendance data from the 2020/2021 school year? 1. Yes 2. No attendance2020 What the attendance rate for grades 1- 6 for the 2020/2021 school year? if attendance2020_yn==1 attendance2020_1 grade 1 if attendance2020_yn==1 attendance2020_2 grade 2 if attendance2020_yn==1 attendance2020_3 grade 3 if attendance2020_yn==1 attendance2020_4 grade 4 if attendance2020_yn==1 attendance2020_5 grade 5 if attendance2020_yn==1 attendance2020_6 grade 6 if attendance2020_yn==1 attendancehow How do you collect attendance data? write down text response attendanceconfident How confident are you in the quality of attendance data? 1. Very confident 2. Somewhat confident 3. Not confident attendancequality Why are you only somewhat or not confident? write down text response if attendanceconfident!=1 note2 Tell the director you would like to look at the latrines and water source latrine_num How many latrines does the school have for students? 1. No latrine for students if 1, skip to… 2. One latrine for students 3. Two or more latrines for students latrine_marked Are the latrines for boys and girls marked? 1. Yes 2. No 78 note3 If there are two latrines, review the one used by girls only. If there is not one for girls only, review either one. if latrine_num==3 latrine_water Is the latrine connected to water? 1. Yes 2. No latrine_waterwork Is the water connection working? 1. Yes if latrine_water==1 2. No latrine_clean How clean is the latrine? 1. Very clean 2. Somewhat clean 3. Not clean latrine_structure How well mainted is the latrine structure? 1. Very well maintained 2. Somewhat maintained 3. Not maintained latrine_sink Is the the latrine equiped with a sink for hand washing? 1. Yes 2. No latrine_sinkclean How clean is the sink area? 1. Very clean 2. Somewhat clean 3. Not clean latrine_soap Is there soap? 1. Yes 2. No latrine_towel Is there a clean towel or other material for drying hands? 1. Yes 2. No latrine_boys If there are two latrines, check the boys latrine. Is the the boys latrine in better, worse or the same overall condition compared to the girls latrine? 1. Better 2. The same 3. Worse water What is the source of drinking water at the school? 1. Piped water 2. Tubewell/borehole 3. Protected dug well 4. Unprotected dug well 5. Protected spring 6. Unprotected spring 7. Rainwater collection 8. Water tank filled by truck 9. Surface water (pond, lake, river, etc) 10. Other water_test Try to get some water. Do you have any problems accessing water or do you see any problems with the water? 1. Yes 2. No water_issues What are the problems? 1. Limit quantity if water_test==1 2. Water takes time to come 3. Equipment old/not working 4. Water does not look clean 5 . Area is not clean 6. Other (specify) water_property Is this water source on school property? 1. Yes 2. No water_distance How many minutes walking is the water source from the school? 1. < 2 minutes if water_property==2 2. >2 to 5 minutes 3. 5 to 10 minutes 4. > 10 minutes geopoint Geopoint Please take the GPS coordinates of the interview location. endtime Interview end time Thank the director for their time and end the interview. 79 Parent Qualitative Survey Tool Question # Question name Question Responses Note to interviewer Logic starttime Start time Automatic startdate Start date Automatic note You have been selected to participate in interviews on health, nutrition and education as part of the The Future is Ours! project. Your participation in this study is entirely voluntary. You are under no obligation to participate. You have the right to refuse to answer questions and to withdraw from the study at any time. If you agree, please answer all questions as honestly as possible. If you are unable to answer one of the questions, you can ignore the question. All your answers are strictly confidential. consent Ask for the respondents consent. If the person agrees, continue the interview. Yes No interviewer_name Interviewer name Multiple choice tbd device_id deviceid region Region Brakna Gorgol school_type What type of school is this? MGD Pilot school MGD non-pilot school Control school department Department Aleg If region = Brakna, show Bababe Bouge M'Bagne Magtar Lahjar Kaedi If region = Gorgol, show M'Bout Maghama Monguel municipality Municipality Multiple choice tbd school Name of school Multiple choice tbd gender What is the gender of the parent? 1. Male 2. Female children_num How many children do you have in grades 1 to 6? grade In which grades are the parents children? 1st multiple choice 2nd 3rd 4th 5th 6th languagehome What language do you speak most of the time at home? 1. Arabic multiple choice 2. French 3. Berber 4. Other (specify) latrine Do you think your children would miss less school if the school had better latrines? Explain. write down text responses water Do you think your children would miss less school if the school had better access to water? Explain write down text responses handwashing When do you wash your hands each day? write down text responses handwashing_what What do you use to wash your hands? write down text responses handwashing_children Do your children also wash their hands like you do? If not, why not? write down text responses food What do your children usually eat when they are in school? Is it enough? write down text responses food_where Where do they eat it? Do they ever miss school because they are eating? write down text responses absence When your children don't go to school, what are the main reasons? write down text responses absence_partial What about the reasons for when your children come to school late or leave early? write down text responses geopoint Geopoint Please take the GPS coordinates of the interview location. endtime Interview end time Thank the director for their time and end the interview. 80 Government Interview Guide Introduction Please read this to the interviewee: You have been selected to participate in interviews on school feeding, health, nutrition, and education as part of the USDA funded The Future is Ours! project. Your participation in this study is entirely voluntary. You have the right to refuse to answer questions and to withdraw from the study at any time. If you agree, please answer all questions as honestly as possible. If you are unable to answer one of the questions, you can ignore the question. All your answers are strictly confidential. Do you agree to participate? If they agree, start the interview. Respondent Background 1. Region and department: 2. Gender: 3. Office: 4. Interviewee position: 5. How long have you worked in this position? Commitment 6. Why is The Future is Ours! project important to for you and your office/department? 7. What is the government’s school feeding policy and programs? 8. How has it changed over the last 5 years, if at all? 9. Are there any changes planned for the next few years? What are they? Capacity 10. How does the government implement school feeding policy or programs? How does it monitor the program? 11. What are the strengths of the government’s school feeding policy and programs? 12. What are the weaknesses? 13. Besides funding, what are some challenges of school feeding in Mauritania? 14. One day, the U.S. government will want to hand over this school feeding program to the GoM. When that day comes, do you think will the GoM be able to continue the program? If not, what support would be needed to help the government continue it? Attendance 15. What is your office’s role in collecting and using school enrolment and attendance data? 16. Do you find the data to generally be reliable? Why or why not? 17. How could it be improved, if at all? Thank you for your time! 81 Annex 6: Results Framework Results Framework for Improved Literacy of School Aged Children 82 Results Framework for Increased Use of Health, Nutrition and Dietary Practices