TOGO McGovern-Dole International Food for Education and Child Nutrition Project Baseline Evaluation 3/26/2021 DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. STARS Baseline Evaluation Report Program: McGovern-Dole International Food for Education and Child Nutrition Agreement Number: USDA Award No. FFE-693-2019/016-00 Funding Year: Fiscal Year 2019 Project Duration: 2019-2024 Implemented by: Catholic Relief Services – United States Conference of Catholic Bishops (CRS) Evaluation Authored by: School-to-School International Candace Debnam, Crystal Chiglinsky, and Randy Tarnowski i Table of Contents List of Acronyms........................................................................................................................................... iv Executive Summary.......................................................................................................................................1 Project Background and Purpose..............................................................................................................1 Evaluation Design, Methods, and Limitations ..........................................................................................1 Findings and Conclusions..........................................................................................................................3 Recommendations....................................................................................................................................5 1. Introduction and Purpose.....................................................................................................................7 1.1. Project Context .............................................................................................................................7 1.2. Project Description........................................................................................................................8 1.3. Results Framework........................................................................................................................9 1.4. Purpose of the Evaluation...........................................................................................................11 2. Evaluation Design and Methodology..................................................................................................12 2.1. Evaluation Design........................................................................................................................12 2.2. Sampling methods ......................................................................................................................14 2.3. Data Collection Methods............................................................................................................15 2.4. Data Analysis Methods ...............................................................................................................19 2.5. Evaluation Limitations.................................................................................................................19 3. Findings...............................................................................................................................................21 Baseline Performance Indicators............................................................................................................21 Strategic Objective 1: School-Age Children in the Savanes and Kara Regions Have Improved Literacy 23 Initial Sound Identification..................................................................................................................25 Letter Sound Identification .................................................................................................................25 Nonword Reading ...............................................................................................................................25 Oral Reading Fluency and Reading Comprehension...........................................................................26 Listening Comprehension ...................................................................................................................27 IR 1.1: Improved Quality of Literacy Instruction.................................................................................28 IR 1.1.1 More Consistent Teacher Attendance...................................................................................29 IR 1.1.5 Increased Skills and Knowledge of School Administrators....................................................29 IR. 1.2: Improved attentiveness..........................................................................................................30 IR 1.2.1: Reduced Short-Term Hunger................................................................................................31 IR 1.3: Improved Student Attendance ................................................................................................32 ii IR 1.3.2: Reduced Health-Related Absences.......................................................................................33 IR 1.3.5: Increased Community Understanding of the Benefits of Education....................................34 Strategic Objective 2: Communities in the Savanes and Kara Regions have increased use of improved health, nutrition, and dietary practices..................................................................................................35 IR 2.4: Number of schools with improved sanitation facilities...........................................................35 IR 2.5: Number of schools using an improved water source..............................................................37 4. Conclusions.........................................................................................................................................37 5. Recommendations..............................................................................................................................39 Annexes.......................................................................................................................................................41 Annex A: Bibliography.............................................................................................................................41 Annex B: Updated Indicator Performance Tracking Table......................................................................42 Annex C: Results Framework for STARS Project .....................................................................................48 Annex D: Terms of Reference for the Evaluation ...................................................................................50 Annex E: Data collection instruments.....................................................................................................59 Annex F: Key Survey Frequency Tables...................................................................................................91 Classroom Observation – Teaching Practices.....................................................................................91 School Director Survey – Teacher Attendance ...................................................................................95 School Director Survey – Management Tools Present .......................................................................96 Annex G: Description of team members’ qualifications.........................................................................98 List of Figures Figure 1: Percentage of Students Receiving Zero Scores by Sex ..................................................................4 Figure 2: Mean EGRA Accuracy Scores by Sex ..............................................................................................5 Figure 3: Map of CRS Togo Intervention Prefectures...................................................................................7 Figure 4: STARS Theory of Change..............................................................................................................10 Figure 5: Percentage of Students Receiving Zero Scores by Sex ................................................................24 Figure 6: Mean EGRA Accuracy Scores by Sex ............................................................................................24 Figure 7: Histogram of Quality Literacy Instruction Score..........................................................................29 Figure 8: Proportion of School Officials Using All Ten Quality Supervision Tools.......................................30 Figure 9: Proportion of Attentive Classrooms Observed............................................................................31 Figure 10: Proportion of Student Attentiveness Levels in Observed Classrooms......................................31 Figure 11: Calculation of minimum acceptable diet indicator (WHO)........................................................32 Figure 12: Total number of students present and enrolled at sampled schools by gender.......................33 Figure 13: Proportion of parents reporting use of home-based education activities................................35 Figure 14: Reading Skills Framework with EGRA Subtask Mapping ...........................................................38 iii List of Tables Table 1: STARS Project Stakeholders............................................................................................................9 Table 2: Non-Zero Performance Indicators Defined at Baseline ................................................................12 Table 3: Target and Actual Sample Numbers..............................................................................................15 Table 4: Early Grade Reading Assessment Subtasks...................................................................................16 Table 5: School-based Surveys and Observation Checklists.......................................................................17 Table 6: Updated Indicator Performance Tracking Table for Non-Zero Baseline Indicators......................22 Table 7: Initial Sound Identification Mean Scores by Sex (Correct out of 10)............................................25 Table 8: Letter Sound Identification Mean Scores by Sex (Correct out of 100) .........................................25 Table 9: Nonword Reading Mean Scores by Sex (Correct out of 50)..........................................................26 Table 10: Oral Reading Fluency Mean Scores by Sex (Correct out of 57)...................................................26 Table 11: Reading Comprehension Mean Scores by Sex (Correct out of 5) ...............................................26 Table 12: Distribution of Correct Reading Comprehension Questions by Sex ...........................................27 Table 13: Listening Comprehension Mean Scores by Sex (Correct out of 5)..............................................27 Table 14: Distribution of Correct Listening Comprehension Questions by Sex..........................................27 Table 15: Instructional Time Lost Due to Teacher Absenteeism ................................................................29 Table 16: Proportion of children meeting minimum acceptable diet threshold........................................32 Table 17: Parent responses to reasons for child absence ..........................................................................33 Table 18: Parent Responses to Homework Support and Literacy Activities Within Past Week.................34 Table 19: Distribution of home-based education activities........................................................................35 Table 20: Sanitation Facilities at Sampled Schools.....................................................................................36 Table 21: Status of Sanitation Facilities at Sampled Schools......................................................................36 Table 22: Handwashing Facilities at Sampled Schools................................................................................36 Table 23: Accessibility of Handwashing Facilities at Sampled Schools.......................................................36 Table 24: Water Sources at Sampled Schools.............................................................................................37 Table 25: Status of Water Source ...............................................................................................................37 iv List of Acronyms Acronym Full Term BMI Body mass index CE1 Cours élémentaire première année, or Grade 3 CP2 Cours préparatoire deuxième année, or Grade 2 CRS Catholic Relief Services – United States Conference of Catholic Bishops EGRA Early Grade Reading Assessment FY Fiscal Year IHfRA Innovative Hub for Research in Africa IYCF Infant and Young Child Feeding assessment MAD Minimum acceptable diet MDD Minimum Dietary Diversity McGovern-Dole McGovern-Dole International Food for Education and Child Nutrition Program PMP Performance Monitoring Plan SO Strategic Objective SOW Statement of Work STARS Santé, Transformation et Apprentissage pour une Réussite Scolaire STS School-to-School International ToC Theory of Change USDA United States Department of Agriculture WHO World Health Organization 1 Executive Summary Project Background and Purpose Catholic Relief Services (CRS) is implementing a McGovern-Dole International Food for Education and Child Nutrition (McGovern-Dole) project, Santé, Transformation et Apprentissage pour une Réussite Scolaire (STARS) 1 , in Togo. Funded by the United States Department of Agriculture (USDA), the project aims to reduce hunger while improving literacy and primary education in Togo’s Savanes and Kara regions. It is designed to achieve these goals by providing school meals, training teachers and school administrators, improving water and sanitation facilities, providing school infrastructure, and building skills and knowledge. CRS began implementation of the STARS project activities in fiscal year (FY) 2020. 2 STARS aims to reach 37,589 primary school students at 138 schools in its first year and expand to 39,000 students in FY24 due to anticipated enrollment increases. The objectives of STARS align with the standard strategic objectives (SO) of the McGovern-Dole Program: • SO 1: Improved literacy of school-aged children; and • SO 2: Increased use of health and dietary practices of school-aged children. This report presents the findings of the STARS baseline evaluation. The evaluation establishes baseline values for all performance indicators, generates data for comparative analysis, and validates project strategies and assumptions. This report elucidates contextual factors that can improve student health and literacy in the Savanes and Kara regions and will enable the McGovern-Dole STARS project team to establish questions to test their theory of change and refine indicator targets. Evaluation Design, Methods, and Limitations The external evaluation of STARS is being conducted over five years. Baseline data collection for the evaluation took place in November 2020, with the midline data collection occurring in spring 2022 and the endline in spring 2024. At each time point, the evaluation is going to use a quantitative approach that includes five data collection tools: • Early Grade Reading Assessment (EGRA) • Student survey • Head Teacher survey • Parent survey • School and classroom observation tool 1 In English: “Health, Transformation and Learning for School Success” 2 CRS received approval from USDA to begin some activities prior to the submission of this baseline report due to lengthy delays in data collection resulting from the global Covid-19 pandemic. 2 School-to-School International (STS) led the baseline evaluation. Data were collected from a sample of 77 schools where the project is intervening across the Savanes and Kara regions. A regional data collection firm, Innovative Hub for Research in Africa (IHfRA), was contracted to manage the fieldwork. IHfRA enumerators administered the EGRA and student survey to 16 randomly selected students enrolled in grade 3 at each school—eight boys and eight girls—using a random number generator application on their tablets. Enumerators collected additional data, using school-based tools with each school’s head teacher and administering surveys at each school with three parents of students who also had a child younger than two. Limitations The following limitations should be considered when reviewing the findings of the STARS baseline evaluation: • Insufficient time for EGRA adaptation workshop and pilot. The baseline data collection utilized an existing French EGRA tool created for use in Djibouti, so the tool was not designed specifically for the Togolese context. While the development of a new EGRA tool through a thorough and local adaptation workshop is best practice, the existing tool was reviewed by STS and CRS Togo prior to baseline and was deemed acceptable. Further details can be found within the report. • Language of the EGRA tool. The instructions and content for the EGRA subtasks were in French and not the most widely spoken local languages of Konkomba (Dankpen), Gourma (Kpendjal), and Ngam-gam (Oti-Sud). Although French is the official language of instruction in Togo, only 8.8 percent of parents report primarily speaking French to their children at home. Based on the results of the listening comprehension task, it is likely that many students struggle with listening comprehension in French and may not have understood the instructions or testing content. • Inherent bias in sampling children present on the day of assessment. Students’ EGRA results may be biased towards students who attend school regularly and may exclude those students who are enrolled but do not regularly attend school. However, the method of randomly sampling on the day of the assessment is preferable to sampling students in advance, as it may create opportunities for school-based actors to manipulate the sample so only high performers participate. • Less reliable sampling of parents with children between the ages of 6 to 23 months. Due to the lack of active STARS activities engaging parents and community members, STS and IHfRA did not have a list of parents from which to sample. Instead, they relied on head teachers to identify and contact parents within the community known to have young children, and, in some cases, parents were incorrectly identified. This issue will be resolved for the midline and endline evaluations, as CRS will have a roster of appropriate parents participating in their activities. • Global COVID-19 pandemic. The emergence of the global COVID-19 pandemic presented unique challenges during the baseline evaluation. o Delay of baseline data collection from March 2020 to November 2020. With the closure of schools in Togo, CRS and STS were required to delay data collection expected to take place in March 2020 until schools reopened in November 2020. 3 o Interruption in schooling for primary school students. Due to the COVID-19 pandemic and resulting school closures, students in Togo lost approximately four months of instructional time from the end of the 2019-2020 academic year and the start of the 2020-2021 academic year. As a result, students experienced an unusual amount of learning loss because of the extended school closures. The baseline data collection has determined students’ learning levels—inclusive of this learning loss—prior to exposure to the intervention. o Remote training. With international travel prohibited, STS trainers could not go to Togo and instead facilitated the training remotely over the Zoom web platform with in-person support from IHfRA and CRS. In addition to real-time facilitation, STS created a suite of training videos about the content and administration protocols of the EGRA, sampling procedures, COVID-19 precautions, and survey administration best practices. o Decision to not measure students’ heights for safety. The height measurement used to calculate students’ body mass index (BMI) was removed from the baseline evaluation to ensure sufficient social distancing between enumerators and the students. Instead, enumerators collected students’ weights with scales while avoiding physical contact with students. They then asked students their birthdates for an alternate calculation of their BMI. Unfortunately, the vast majority of students could not provide their birthdates or ages during data collection, so this topic of analysis could not be included at baseline. Findings and Conclusions When examining the literacy scores of the sampled students, the students’ performance was quite weak. The proportion of students unable to provide a single correct response on each subtask—known as zero scores—was very high. On the initial sound identification subtask, 72 percent of students were not able to respond correctly to even one of the five items. The letter sound identification subtask had the lowest proportion of zero score students, with only 38 percent of students not able to identify at least one letter correctly in one minute. On the nonword reading subtask, 93 percent of students were not able to correctly read a single nonword. When presented with a reading passage, 71 percent of students were not able to read a single word. Linked to the reading passage subtask, the reading comprehension questions also had a high number of zero scores; 96 percent of students were not able to correctly answer a single reading comprehension question. On the listening comprehension subtask, 83 percent of students were unable to answer a single question. Across all subtasks, boys had a lower proportion of zero scores than girls. 4 Figure 1: Percentage of Students Receiving Zero Scores by Sex On average, students correctly responded to 1.35 out of 10 items on the initial sound identification subtask. On the letter sound identification subtask, students correctly identified 5.29 letters within one minute, on average. For nonword reading, on average, students correctly read 0.56 words in one minute. Students read on average at a rate of 1.60 words per minute on the oral reading fluency subtask. On average, students were not able to correctly answer a single reading comprehension question, with the average number of questions correctly answered being only 0.04. On listening comprehension, students were only able to answer 0.28 questions correctly. Figure 2 below presents mean percentages of correct responses for each subtask, disaggregated by sex. 83% 96% 71% 93% 38% 72% 86% 98% 76% 95% 44% 79% 81% 95% 66% 91% 32% 66% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Listening Comprehension Reading Comprehension Oral Reading Fluency Nonword Reading Letter Sound Identification Initial Sound Identification Boys Girls Total 5 Figure 2: Mean EGRA Accuracy Scores by Sex Note: ** indicates a statistical difference of p<0.01, * indicates a statistical difference of p<0.05 Mean scores for each EGRA subtask are presented in more detail in the body of the report, providing a better understanding of students’ reading performance. Statistical significance tests were used to determine the difference in mean scores between boys and girls and are noted where applicable. Recommendations STS proposes the following recommendations for CRS for both project implementation, as well as things to consider for the midline and endline evaluations. Implementation Recommendations • Examine existing student and teacher French language abilities. Overall student performance, particularly on listening comprehension, indicates that students have a limited ability to understand spoken French. The project may want to consider undertaking more targeted research into the reasons for this gap in comprehension. • Examine gender constraints within target communities. Girls’ underperformance compared with boys deserves further exploration and may warrant a specific focus within the project to address the underlying causes of these gender disparities. • Consider seasonality when defining rations within nutrition activities, as well as during program monitoring. The baseline evaluation findings show a higher percentage than expected of children from the ages of 6 to 23 months who met the minimum acceptable diet (MAD) requirements. The 16.6% 5.8% 1.4% 3.3% 1.2% 6.0% 10.2% 4.8% 0.9% 2.2% 0.6% 5.0% 13.5% 5.3% 1.1% 2.8% 0.8% 5.6% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% Initial Sound Identification** (10 items) Letter Sound Identification (100 items) Nonword Reading (50 items) Oral Reading Fluency* (57 items) Reading Comprehension (5 items) Listening Comprehension (5 items) Boys Girls Total 6 fluctuations in access to quality nutrition due to the seasons should be considered when defining rations for students and pregnant and lactating women, as well as when interpreting data collected during program monitoring. Recommendations for Midline Evaluation • Revise the EGRA tool to align with current best practices and associated benchmarks for tracking reading improvement. The baseline administration used an EGRA developed for use in Djibouti, rather than the local Togolese context. Additionally, generic benchmarks for reading comprehension were used due to a lack of Togo-specific benchmarks set by the Ministry of Primary and Secondary Education. A revised and equated EGRA, as well as country-specific reading benchmarks, would result in a more nuanced understanding of student reading proficiency. • Consider seasonality when interpreting MAD results for the midline and endline evaluations. Under the original timeline for the baseline evaluation, data collection was planned for the end of the academic year. With delays due to COVID-19, data collection took place in a different season than planned. Should the midline and endline evaluations occur at the end of the school year, this seasonal difference should be considered when interpreting the results for the MAD indicator. • Modify existing survey items, indicators, or definitions to allow for greater accuracy during data collection. CRS should review existing indicators and definitions within their Performance Monitoring Plan to identify any areas for clarification or refinement. STS should make corresponding changes to the tools to reflect more nuanced definitions and indicators. Specifically, reviewing indicators related to school absences, as well as teacher and administrator behavior, are recommended. 7 1.Introduction and Purpose 1.1. Project Context The Republic of Togo is located in West Africa and is home to approximately 7.9 million people, with 47.9 percent of the population under 18. 3 Due to political upheaval in the 1990s, Togo’s diplomatic and economic ties with much of the world were severed. Diplomatic ties were restored in the mid-2000s, but the impact of political isolation has been lasting. While the poverty rate has decreased in recent years, economic growth has not been equitable across the rural-urban divide. Within Togo’s agriculturally dependent economy, 69 percent of rural households lived below the poverty line in 2015.4 The disparities between the urban and rural populations are also evident in education. In 2017, out-of-school children of primary school age came mainly from rural areas (88.1 percent), compared to 11.9 percent from urban areas. These out-of-school children were primarily located in the northern regions (27.9 percent in the Savanes and 27.0 percent in Kara), were mainly from the lowest-income families, and most are girls (53 percent). Girls from low-income families have an 89 percent probability of entering primary school but only a 60 percent chance of completing it.5 Furthermore, according to studies by the Conférence des Ministres de l’Education des Etats et Gouvernements de la Francophonie (CONFEMEN) in 2014 and 2019, more than 75 percent of grade 2 students are not at an acceptable reading level. 6,7 The rural-urban divide is particularly stark when examining health indicators and access to appropriate water and sanitation facilities. In 2019, UNICEF reported 89.1 percent of urban households had access to improved water sources, while only 48.4 percent of rural households had such access. The divide was even grimmer for improved sanitation facilities, with 28.6 percent of urban households reporting improved sanitation facilities, compared to 7.4 percent of rural households.8 3 United Nations, Department of Economic and Social Affairs, Population Division (2019). World Population Prospects 2019, Online Edition. Rev. 1. 4 https://www.worldbank.org/en/country/togo/overview 5 Analyse du secteur de l’éducation de la République togolaise, Des défis pour un enseignement de qualité pour tous, République togolaise, UNICEF, IIPE-Pôle de Dakar - UNESCO, 2019. 6 PASEC 2014 Performances du Système Éducatif Togolais. Programme d’Analyse des Systèmes Éducatifs de la CONFEMEN, 2015. 7 PASEC 2019 Qualité des Systèmes Éducatifs en Afrique Subsaharienne Francophone. Programme d’Analyse des Systèmes Éducatifs de la CONFEMEN, 2020. 8 WHO/UNICEF Joint Monitoring Programme for Water Supply, Sanitation and Hygiene (JMP). Last update: June 2019. https://data.unicef.org/resources/dataset/sowc-2019-statistical-tables/ Figure 3: Map of CRS Togo Intervention Prefectures 8 1.2. Project Description Catholic Relief Services (CRS) is implementing the new Santé, Transformation et Apprentissage pour une Réussite Scolaire (STARS) 9 project in the Republic of Togo. STARS is funded by the United States Department of Agriculture’s (USDA) McGovern-Dole International Food for Education and Child Nutrition (McGovern-Dole) program, which strives to reduce hunger and improve literacy and primary education. McGovern-Dole projects worldwide provide school meals, teacher training, and other support activities to boost school enrollment and academic performance.10 STARS is a five-year program running from fiscal year (FY) 2020 through FY2024. Through this $20 million project, CRS aims to reach 37,589 primary school students at 138 schools in its first year and expand to 39,000 students by its final year with anticipated enrollment increases. CRS is providing academic and nutritional support to communities in Togo’s northern Savanes and Kara regions—specifically in the Kpendjal and Oti-Sud prefectures of Savanes and the Dankpen prefecture of Kara. The program seeks to achieve the following objectives: • Improve literacy outcomes by strengthening school systems and community support; • Improve the quality of literacy instruction by building the capacity of teachers and administrators and providing sufficient literacy materials; • Improve student attentiveness and attendance by providing daily school lunches and ensuring a safe school environment; • Improve health and dietary practices of targeted beneficiaries by increasing awareness of nutrition, health, and hygiene behaviors combined with water and sanitation infrastructure improvements; and • Increase the capacity of the government and other key actors to improve school feeding, health, and nutrition and prioritize literacy in education. CRS is aiming to work alongside various partners and stakeholders throughout the life of the STARS project, as shown in Table 1. In addition to community members and local and national government stakeholders, CRS’s leadership and implementing team is expecting to coordinate with other actors such as the World Food Program for school feeding and high-level policy influence; UNICEF for school governance, teacher training, WASH, and protection activities; and FHI360 for de-worming activities. This collaboration is going to ensure a better impact of the interventions on school communities. All stakeholders will be surveyed or qualitatively interviewed for the midterm and final evaluations, with only key stakeholders surveyed for the baseline study, including students, parents, and school administrators. Findings will be shared with all stakeholders, either through dissemination workshops, webinars, or written reports. 9 In English: “Health, Transformation and Learning for School Success” 10 United States Department of Agriculture, “McGovern-Dole Food for Education Program,” accessed January 20, 2021, https://www.fas.usda.gov/programs/mcgovern-dole-food-education-program. 9 Table 1: STARS Project Stakeholders Students Community leaders Parents Ministry of Primary and Secondary Education Teachers Ministry of Grassroots Development School administrators Ministry of Health and Social Protection Food preparers National Federation of Parents School Management Committee members Inter-ministerial committee members Parent-Teacher Association members UNICEF and partners Savings and Internal Lending Community members World Food Program Lead mothers World Bank Child Promotion Agents USDA Community Health Workers 1.3. Results Framework Theory of Change In the implementation of STARS, CRS is using several field-tested and evidence-based approaches, including 1) a school feeding strategy guided by the five standards of successful school feeding programs—policy, design and implementation, financial capacity, institutional coordination, and community participation; 2) its extensive experience improving the literacy of school-age children; 3) its proven experiences in facilitating access and use of health services at the community level in relation to child illness prevention, nutrition, and dietary practices; 4) its signature Savings and Internal Lending (SILC) program to strengthen assets and access to finances to cover basic fee services, like health and education; and 5) leveraging its extensive experiences in improving WASH infrastructure, access, and use. Evidence includes secondary research as well as primary data from CRS’s M&E reports, stakeholder consultations, and analyses of progress, field assessments, and successes and lessons learned from prior McGovern-Dole investments in other countries. 10 OVERALLL SCHOOL FEEDING LITERACY INSTRUCTION HEALTH & ATTENDANCE Figure 4: STARS Theory of Change Critical Assumptions The following critical assumptions influence the STARS theory of change: • Security will remain stable in project areas. Dankpen prefecture in Kara borders Ghana and experiences patterns of displaced persons, a result of border tensions. Additionally, the northern border region is adjacent to Burkina Faso, where extremist groups have carried out attacks; however, this has not affected Togo. CRS will monitor developments through its monitoring systems and alert USDA of any real or potential impact on project implementation. CRS McGovern-Dole programs in Mali and Burkina Faso face similar security concerns. CRS Togo will apply learning from their experiences to the Togo context. • UNICEF will implement continued sanitation activities and new WASH, protection, school governance, and community engagement activities. If UNICEF does not meet expectations, CRS will target schools to ensure no gaps in activity coverage. Strategic Objectives The STARS project centers around the two USDA McGovern-Dole strategic objectives (SOs): • SO 1: School-aged children in the Savanes and Kara regions have improved literacy; and • THEN children in Savanes and Kara regions will attend school regularly, thrive and learn, as evidenced by assessment results IF the school system is strengthened and delivers quality literacy instruction, IF communities and parents support their child’s education and invest in health, IF children benefit from safe and nutritious meals and IF schools provide a safe and stimulating learning environment, • THEN schools systems will be strengthened and contribute to sustain the access to a quality meal for each pupil. IF national and local government authorities coordinate their actions toward their vision to expand school’s canteens nationwide, IF policy and regulatory frameworks are strengthened with clear roles set out for the management of school feeding, IF school governance structures at the community level, such as the SMC and APEs, and community members hold government officials and school administrators accountable to improve schools governance, • THEN the school system will deliver quality literacy instruction and student literacy will improve. IF the educational system prioritizes literacy improvement, IF teachers and school administrators improve competency and commitment to teach literacy and to be accountable to parents, IF schools have access to appropriate teaching and reading materials and IF communities sustain literacy activities both in and out of the classroom, • THEN children will remain in good health and will attend school regularly. IF students have access to improved water and sanitation infrastructures, IF household members have increased awareness, and improved practice, of key nutrition, health and hygiene behaviors, IF parents are supportive of student’s education and empowered to pay related costs, IF children consume improved diets at home and at school, IF schools provide a safe and enabling environment, 11 • SO 2: Communities in the Savanes and Kara regions have increased use of improved health, nutrition, and dietary practices. Both SOs are being supported as outlined in the STARS Project Results Framework (Annex C). Under the project’s first SO, STARS is gearing up to implement several school-based activities to improve school-aged children’s literacy in 138 intervention schools. CRS recognizes teachers’ critical role in students’ learning and is planning to focus on literacy training for teachers, school directors, and inspectors. These efforts are going to be further bolstered by the provision of quality teaching materials for use in the classroom. As the heart of the McGovern-Dole program, daily school lunches are going to be provided through community-operated canteens at all intervention schools to encourage students’ attendance and attentiveness. Food preparers and school administrators are going to receive training on proper food preparation, storage, and sanitation practices. The project’s second SO seeks to increase the use of health, nutrition, and dietary practices by promoting health, nutrition, and personal hygiene initiatives within the schools and communities. As such, CRS is planning to improve school water and sanitation facilities, enabling students to put proper health behaviors into practice. The project is going to build and repair gender-segregated latrines in accordance with national standards, and new wells are expected to be built at schools currently without access to water. CRS is also preparing to distribute take-home rations to pregnant and lactating women and children under two years of age who participate in CRS’s community-based maternal and child nutrition activities. To achieve these ambitious goals and promote local and national sustainability, the STARS team is consistently planning to work alongside local communities, organization partners, and Government of Togo ministries, departments, and agencies, including the Ministries of Education, Health, Agriculture, and Grassroots Development. 1.4. Purpose of the Evaluation CRS contracted School-to-School International (STS) as the independent external evaluator for the STARS project. In addition to the baseline evaluation conducted in November 2020 outlined in this reported, the project’s evaluation plan also includes a midline evaluation in spring 2022 and an endline evaluation in spring 2024. The purpose of the baseline evaluation is to establish baseline values, define targets for the STARS project performance indicators, generate data for comparative analysis, and help CRS validate the project’s strategies and assumptions. Evidence from this report elucidates contextual factors for improving student health and literacy in the Savanes and Kara regions, enabling CRS to make evidence￾based decisions in their programming to maximize the effectiveness, relevance, efficiency, sustainability, and impact over the life of the project. Furthermore, findings from this series of evaluations, particularly those from the midline and endline evaluations, will contribute to the McGovern-Dole Learning Agenda to inform current and future McGovern-Dole projects around the world and contribute to the 12 knowledge base around the relevance, effectiveness, efficiency, impact, and sustainability of school feeding programs. The two McGovern-Dole Learning Agenda questions that will be addressed throughout the evaluation are: • Question 4 in the Learning Agenda’s Health Evidence Gaps section: “What systems of community health care governance are the most effective at sustaining the delivery of health interventions through school meal programs?” • Question 5 in the Learning Agenda’s Education/Literacy Evidence Gaps section: “What are the differences in educational outcomes from school meal programs between malnourished or undernourished children and those who are not?” 2.Evaluation Design and Methodology 2.1. Evaluation Design The STARS project’s baseline evaluation is a non-experimental quantitative evaluation that establishes baseline values and targets for the project’s performance indicators and provides information for evidence-based decision-making regarding the design and assumptions of the STARS project. The baseline evaluation also establishes a point of reference for comparison at later evaluation timepoints. Because the baseline’s focus is to report data for all non-zero baseline indicators, there are no explicit research questions. Research questions regarding the project’s effectiveness and other areas of interest will be established before the midterm and final evaluations. The STARS performance monitoring plan requires that most performance indicators be set to zero for the baseline evaluation. However, the twelve performance indicators in Table 2 have non-zero values to be established at baseline. Table 2: Non-Zero Performance Indicators Defined at Baseline STARS Indicator No. Activity Performance Indicator Standard USDA or CRS Custom Indicator 1 Raising awareness on importance of education (Activity 12) Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text Standard #1 8 Provide school meals (Activity 11) Percent of students in target schools identified as attentive during class/instruction Custom 9 Take home rations (Activity 14) Average student attendance rate in USDA supported classrooms/schools Standard #2 15 Building/ Rehab: Latrines (Activity 2) Number of schools with improved sanitation facilities Standard #28 13 STARS Indicator No. Activity Performance Indicator Standard USDA or CRS Custom Indicator 16 Building/ Rehab: Wells and water stations/ systems (Activity 4) Number of schools using an improved water source Standard #27 19 Promote teacher attendance (Activity 10) Percent of instructional time lost due to teacher absenteeism USAID Education Proposed 23 Training: Teachers (Activity 18) Percent of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Custom 27 Training: School admins (Activity 17) Percent of school officials in target schools who demonstrate use of new and quality supervision and leadership techniques or tools Custom 28 Take home rations (Activity 14) Percent of children 6–23 months receiving a minimum acceptable diet FFP #BL12 30 Raise awareness on health, nutrition, and WASH (Activity 12) Percent of parents who state their children had health-related school absences in the previous month Custom 34 Establish activities to promote literacy (Activity 7) Percent of caregivers who report spending time on literacy activities with their school-age children in the previous week Custom 36 Establish activities to promote literacy (Activity 7) Percent of community members who promote early childhood practices and support their children’s education Custom To establish baseline values for these indicators, STS and CRS developed a suite of quantitative and direct observation tools for various stakeholders to provide broad perspectives for the project— students, head teachers, and parents. STS collected data on students’ enrollment, attendance rates, literacy, and attentiveness levels. Similarly, STS collected data on teachers’ attendance and use of teaching practices. No qualitative tools were included at baseline as they did not contribute directly to the non-zero baseline indicators; however, additional qualitative tools will be added for the midline and endline evaluations to address all indicators and research questions. Evaluation Timeline Shifts Under the original terms of reference, the baseline evaluation was planned for the end of the 2019-2020 academic year with grade 2 students (cours préparatoire 2, CP2) in the spring of 2020. However, the COVID-19 pandemic interrupted the baseline evaluation after STS completed initial activities—tool development and enumerator training—in March 2020. With school closures across Togo in April 2020, data collection was paused until the situation stabilized and schools could reopen. 14 After months of disruption, baseline evaluation activities were able to resume in October 2020 at the start of the 2020-2021 academic year. This delay required conducting a second round of enumerator training due to the eight-month gap between the original STS training in Lomé in March 2020 and the new data collection timeline of November 2020. Due to COVID-19 and the revised data collection timeline, school closures also warranted a shift in the target sample to grade 3 students (cours élémentaire 1, CE1). While Indicator #1 measures the “percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade-level text,” the baseline evaluation assessed students at the start of CE1 as a proxy for students at the end of CP2 because their exposure to CE1 instruction was minimal at the time of the evaluation. Assessing students at the start of a new academic year as a proxy measure for student learning levels at the end of the prior academic year is common among education evaluations. Further, COVID-19-related school closures in April 2020 meant that students entering CE1 in the 2020-21 school year had not been exposed to the full CP2 curriculum by the start of the new school year. Ethical Considerations The CRS Togo team reviewed the study tools before the beginning of data collection to ensure that the study adhered to applicable ethical rules and societal norms. STS and its data collection partner trained all enumerators on child protection policies and procedures. Enumerators obtained affirmative informed consent from all head teachers and classroom teachers to assess the children in their care. All children provided affirmative assent to be assessed and interviewed and could opt out of the assessment or survey at any time. Furthermore, for data privacy concerns, data collected electronically were stored on a secure, password￾protected server, which only STS can access. Respondents were assigned a randomly generated identification code, so no names were recorded in the datasets that included respondents’ answers. 2.2. Sampling methods A two-stage cluster sampling approach was used for the baseline evaluation. Sample sizes were calculated using Equations (6), (19), and (22) for clustered continuous, non-clustered binary, and clustered binary outcomes, respectively, in McConnell and Vera-Hernandez, using the standard 80 percent power and 5 percent significance level.11 First, 77 schools were randomly selected from the list of 138 intervention schools to serve as clusters. Within each selected school, enumerators sampled the following units for surveys or observations: • One head teacher or assistant head teacher; • One classroom between grades 1 and 5 to be observed for a classroom observation; and • Three parents of students who also have a child under the age of two.12 For the second stage of sampling, enumerators followed a specific procedure to randomly select 16 students—eight boys and eight girls—from those present in the CE1 classroom at each school on the day 11 McConnell, Brendon, and Marcos Vera-Hernandez. 2015. Going beyond simple sample size calculations: a practitioner's guide. Institute for Fiscal Studies. 12 Sampled parents were identified and invited by the head teacher. For the midline and endline evaluations, parents will be selected from active participants in STARS activities to ensure they meet the sampling requirements. 15 of the visit to participate in the evaluation. This was in excess of the minimum target sample size of 15 students per school to allow for an equal number of boys and girls per school. If a school had more than one class of CE1, enumerators randomly selected one classroom to identify the 16 students. The target sample size of 77 schools covered just over half of the 138 intervention schools. The sample was drawn to be generalizable at the project level. The target and achieved sample numbers are reflected in Table 3 below. Table 3: Target and Actual Sample Numbers Group Minimum Target sample Actual sample Response rate Schools 77 77 100.0% Head Teachers 77 77 100.0% CE1 Students 1,155 1,157 100.2% Classroom observation 77 77 100.0% Parents 212 228 107.5% Caregivers of children ages 6-23 months 178 153 86.0%13 In addition to the sample, STS created a list of replacement schools in case of unforeseen challenges. For each closed or inaccessible school, the study team selected a comparable school from the list of replacement schools to visit. The evaluation team documented and tracked these replacements throughout data collection to ensure their appropriateness. 2.3. Data Collection Methods Data Collection Tools The STARS baseline evaluation utilized data collection tools adapted from comparable contexts. The tools include an Early Grade Reading Assessment (EGRA); a CRS-developed standard student survey and a classroom observation tool used across CRS McGovern-Dole projects; and newly developed surveys for head teachers and parents. STS and the CRS Togo team reviewed the tools and made specific revisions before data collection to ensure survey tools were responsive to the STARS performance monitoring plan and were culturally appropriate. EGRA STS administered a baseline EGRA to students at the start of CE1 to measure their core early grade reading skills. Due to the baseline evaluation’s compressed timeline, STS and CRS were unable to conduct the weeklong EGRA local adaptation workshop to create an EGRA tool specifically for the Togolese context. The adaptation workshop typically brings together local education professionals to examine the national curriculum and create grade-appropriate tool items aligned with the curriculum. A 13 Only 86.0 percent of caregivers with children between the ages of 6 and 23 months were reached during baseline data collection. This is due to the limitation of relying on the head teachers of sampled schools to identify and invite appropriate parents. In 75 cases, the age of the parents’ children did not fall within the appropriate age range and thus the parents were excluded from the sample for the indicator on the minimum acceptable diet. See the Limitations section for more details. 16 small pilot data collection then follows this process to validate the tools. Without this adaptation process, CRS and STS chose to use an existing French-language EGRA tool that STS had adapted for use in Djibouti—another African country where French is the language of instruction in primary school. The assessment contained six subtasks—initial sound identification, letter sound identification, nonword reading, oral reading fluency, reading comprehension, and listening comprehension. Table 4 provides a summary of the subtasks. Table 4: Early Grade Reading Assessment Subtasks Subtask Core Reading Skill Subtask Description Initial Sound Identification Phonemic awareness The enumerator said 10 common words out loud and asked students to identify the first letter of each word. Letter Sound Identification Alphabet knowledge The enumerator presented students with a grid of 100 letters, or groups of letters, in both uppercase and lowercase in a random order and asked them to say the sound of as many letters as they could in one minute. Nonword reading Decoding The enumerator presented students with a grid of 50 simple nonsense words. The enumerator asked students to make letter-sound correspondences by the reading the nonsense words. Oral reading fluency Decoding and reading fluency The enumerator asked students to read a short, grade-appropriate story of 57 words in one minute with accuracy and little effort. Reading comprehension Reading comprehension The enumerator asked students as many as five questions, including four literal questions and one inferential question, about the passage read in the previous subtask. Listening comprehension Listening comprehension and oral language The enumerator read aloud a short story of 38 words and asked students five questions, including four literal questions and one inferential question, about the story. Enumerators administered the EGRA to 16 CE1 students at each school on tablets using Tangerine®, an electronic data collection software. Following the EGRA subtasks, enumerators administered a short survey to these same students, as outlined in Table 5 below. 17 School-based Surveys and Observation Tools For a comprehensive picture of a sampled school’s environment, enumerators collected data from three survey tools and a classroom observation tool at each school. The content of these surveys is described in Table 5. Table 5: School-based Surveys and Observation Checklists Tool Types of information collected Student Survey Students’ feelings about school; their teachers’ use of quality teaching practices; educational support at home; available water and sanitation resources at school and home; and home socioeconomic factors. Head Teacher Survey Enrollment and attendance data; teacher attendance and support information; school administration tools; teaching and learning materials available; and school water, sanitation, and nutrition resources. Parent Survey Household demographics; child school absences; knowledge of and use of nutrition, health, and sanitation practices; educational support at home; and dietary practices for children under two years. Classroom Observation Presence and use of teaching and learning materials in the classroom; use of quality teaching practices within an observed lesson; evidence of student attentiveness; and the school’s physical attributes, including sanitation facilities, water sources, and food preparation and storage areas. The CRS global education team developed the student survey and the classroom observation tool for use across all their McGovern-Dole projects. STS added a few questions to these tools to address the required performance indicators but kept the core tools consistent. STS developed the parent and head teacher surveys with input from the STARS project team to align with the performance indicators and adapted several questions from similar tools from CRS’s McGovern-Dole projects in both Benin and Burkina Faso. Recruitment and Training of Enumerators STS contracted a West African firm, Innovative Hub for Research in Africa (IHfRA), to support and supervise the baseline data collection. IHfRA recruited 33 enumerators from Lomé, Togo, for the enumerator training from March 6 to 10, 2020. Three STS staff members traveled to Lomé to conduct the enumerator training, alongside representatives from CRS. However, with the outbreak of the COVID￾19 pandemic in March 2020, the data collection was postponed until the next academic year, with the enumerator training conducted for a second time due to the delay. 18 From November 16 to 21, 2020, STS and IHfRA trained 34 participants on the evaluation tools and protocols. The six-day training held in Kara, Togo, covered the contents of the EGRA tool and school￾based surveys; administration protocols for the data collection software and use of tablets; ethical considerations; and the responsibilities of enumerators and supervisors during data collection. The training included one day of field testing in a nearby non-intervention school near Kara. Due to COVID￾19 travel restrictions, STS staff facilitated the training remotely via a mix of pre-recorded videos and live facilitation over the Zoom web platform. IHfRA trainers present at the training site helped manage the agenda, facilitate practice sessions, and answer questions. Field Tests of Data Collection Tools During both enumerator trainings—the initial one in Lomé in March 2020 and the second in Kara in November 2020—enumerators visited a nearby school for one day to field test the data collection tools in a school setting. In each instance, all the survey and observation tools were tested. The benefits of this activity were two-fold—it enabled enumerators to practice the administration of the tools in a real￾life setting while also enabling the evaluation team to identify potential challenges and solutions. The need to hire enumerators with the appropriate local language fluency was a lesson learned during the first school visit in Lomé and was applied to hiring enumerators for the second round of training in Kara. Minor refinements to the tools’ wording or instructions were also made from lessons learned during both field tests. School-based Data Collection The baseline data collection was conducted in the Savanes and Kara districts from November 23 to December 2, 2020. Ten teams of three—consisting of two enumerators who administered the EGRA and student survey and one enumerator who conducted the school-based surveys—visited one school per day. Within each team, one enumerator was designated as the supervisor responsible for introducing the teams to the school and conducting the classroom and student sampling for each team. Data Monitoring and Quality Assurance Throughout data collection, IHfRA closely supervised enumerators to ensure data quality. IHfRA had three field coordinators—one per prefecture—to supervise three to four teams apiece. These field coordinators visited multiple schools in person to conduct on-site spot checks and troubleshoot any issues teams encountered in the field. Additionally, STS’s Senior Data Associate monitored the incoming data daily by checking results uploaded to the server for completeness. Communication with the enumerator teams was maintained through a WhatsApp© group comprised of team supervisors, IHfRA, and STS; this allowed for broader communication and faster responsiveness when issues arose in the field. An additional means of data quality control was inter-rater reliability (IRR) measures during data collection.14 Per standard EGRA practice, IRR was conducted with 10 percent of the sampled students. 14 Inter-rater reliability is the degree of agreement between two enumerators who are assessing the same student independently. It allows the data collection monitors to identify and resolve problems within the enumerator teams during data collection to improve the quality of the data collected. 19 Each day, the two EGRA enumerators assessed the first student together—one enumerator acting as the assessor and one as the observer—and STS compared their results for alignment. IHfRA’s staff ensured enumerator teams followed data collection procedures and submitted a field report that logged any discrepancies in the number and type of data collected prescribed in the target sample. STS cross-referenced these reports against the uploaded data. Disposition codes were applied to categorize any issues that emerged during the data collection process. These coding and flagging procedures helped to ensure the nuanced contexts of data collection at the school level were sufficiently cataloged and considered during the data cleaning, analysis, and reporting process. 2.4. Data Analysis Methods Sample Weighting The analysis used sampling weights to produce more representative estimates in the sample of students. Although random sampling does not acknowledge that some students have a lower probability of being selected when they represent smaller subgroups within the population, sampling weights enable analysts to account for these differences in probabilities. STS computed the weights using background data available from each school in the sample populations, including the number of CE1 classrooms at the school and the number of students in each classroom. STS collected this information via the head teacher survey. Weights were applied when analyzing the EGRA and survey results. STS used a combined school and student weight for all students and applied the school weight to all school-based surveys. Generation of Findings In December 2020, STS generated the following descriptive statistics using the baseline data: • Mean scores: Average percentage of items answered correctly on a given subtask • Zero scores: Proportion of students who were unable to answer a single item correctly on a given subtask • Proportions: Proportion of respondents who replied in a specific way to an item • Means: Average score on survey items Analysts computed inferential statistics on subtask mean scores to determine differences in performance between girls and boys. Where detected, statistically significant differences are noted in the findings. 2.5. Evaluation Limitations The following limitations should be considered when reviewing the findings of the STARS baseline evaluation: • Insufficient time for EGRA adaptation workshop and pilot. The baseline data collection utilized an existing French EGRA tool that had been adapted in Djibouti, so the tool was not created specifically for the Togolese context. While the development of a new EGRA tool through a thorough and local adaptation workshop is best practice, the existing tool was reviewed by STS 20 and CRS Togo prior to baseline and was deemed acceptable. This may be improved for the midline and endline evaluations should CRS desire a full adaptation and weighting process prior to the midline evaluation. Additionally, since the evaluation timeline did not leave enough time for a pilot of the tools, STS was unable to examine the extent of any potential social desirability bias inherent in the tools, as well as their cultural relevance and appropriateness. STS believes no additional bias was introduced due to the training and recruitment of the enumerators. Outside enumerators had no inherent interest in the outcomes of the study. • Language of the EGRA tool. The instructions and content for the EGRA subtasks were in French. However, based on the listening comprehension task results, it is likely that many students struggle with listening comprehension in French and may not have understood the instructions or testing content. This lack of comprehension may have been further exacerbated by the enumerators wearing masks during the assessment as a precaution against the spread of COVID￾19, but there is insufficient data to substantiate that hypothesis. • Inherent bias in sampling children present on the day of assessment. Students’ EGRA results may be biased towards students who attend school regularly and may exclude those students who are enrolled but do not regularly attend school. However, the method of randomly sampling on the day of the assessment is preferable to sampling students in advance, as it may create opportunities for school-based actors to manipulate the sample to have only high performers participate. This sampling approach will remain the same for future assessments, and therefore the comparison across timepoints will be valid. • Less reliable sampling of parents with children between the ages of 6 to 23 months. Without active STARS activities being implemented with parents of children in this age range, STS and IHfRA did not have a list of parents from which to sample. Instead, the baseline study relied on head teachers to identify and contact parents within the community known to have young children. In 75 of the 228 parent surveys, the parents’ child was not in the appropriate age range, and thus the parent could not complete the portion of the survey on the Minimum Acceptable Diet (MAD). Their responses to other portions of the parent survey were retained. This issue will be resolved for the midline and endline evaluations because CRS will have a roster of appropriate parents participating in their activities. • Global COVID-19 pandemic. The emergence of the global COVID-19 pandemic presented unique challenges during the baseline evaluation. o Delay of baseline data collection from March 2020 to November 2020. With the closure of schools in Togo, CRS and STS were required to delay data collection expected to occur in March 2020 until schools reopened in November 2020. o Interruption in schooling for primary school students. Due to the COVID-19 pandemic and resulting school closures, students in Togo lost approximately four months of instructional time between the end of the 2019-2020 academic year and the start of the 2020-2021 academic year. As a result, students experienced an unusual amount of learning loss because of the extended school closures. The baseline data collection determined students’ learning levels, including this learning loss, before exposure to the intervention. 21 o Remote training. With international travel prohibited, trainers from STS could not travel to Togo for the second enumerator training. As a result, STS facilitated the training remotely over the Zoom web platform with in-person support from IHfRA and CRS. Enumerators were divided into two rooms to adhere to social distancing guidelines and were required to wear masks at all times. In addition to live facilitation, STS created a suite of training videos on the content and administration protocols of the EGRA, sampling procedures, COVID-19 precautions, and survey administration best practices. o Decision to not measure students’ heights for safety. Before the COVID-19 outbreak, STS and CRS intended to collect both the heights and weights of all assessed children to calculate their body mass index (BMI) and examine the relationship between students’ BMI and their literacy outcomes. However, the height measurement was removed from the baseline to allow for sufficient social distancing between the enumerators and the students; measuring children’s heights would have required disregarding social distancing guidelines. Instead, enumerators collected weights using scales but avoiding physical contact with students. They then asked students their ages and birthdates for an alternate calculation of their BMI. Unfortunately, the vast majority of students could not provide their birthdates or ages during data collection, so this topic of analysis could not be included at baseline. As a result, it is anticipated that STS and CRS will return to the original plan of collecting heights and weights measurements at midline and endline to allow for the analysis for a special study linking students’ BMI to their EGRA performance. This plan will allow for analysis at each timepoint, as well as across two timepoints rather than the originally intended three timepoints. 3.Findings Baseline Performance Indicators The STARS performance monitoring plan requires that most performance indicators be set to zero for the baseline evaluation. Indicators that are not set to zero are spelled out below. The values in Table 6 below represent data from both STS’s external baseline evaluation and CRS’s internal monitoring data. Census data provided by CRS from all 138 intervention schools are presented in shaded boxes, while the non-shaded boxes show evaluation data collected only from the 77 schools sampled for the baseline evaluation. Based upon these baseline findings, select indicator targets will be adjusted during the baseline amendment process in consultation with USDA. 22 Table 6: Updated Indicator Performance Tracking Table for Non-Zero Baseline Indicators STARS Indicator No. Indicator Name Indicator No. Target Baseline (2020) Male Female Total 1 Percentage of students who, by the end of two grades of schooling, demonstrate that they can read and understand the meaning of grade-level text MCGOVERN￾DOLE 1 21% 0.0% 0.0% 0.0% 8 Percentage of students in target schools identified as attentive during class/instruction CRS Custom 60% 59.7% 9 Average student attendance rate in USDA supported classrooms/schools McGovern￾Dole 2 93% 81.3% 79.0% 80.2% 15 Number of schools with improved sanitation facilities McGovern￾Dole 28 66 57 16 Number of schools using an improved water source McGovern￾Dole 27 90 70 19 Percentage of instructional time lost due to teacher absenteeism USAID Proposed 52% 9.1% 23 Percentage of teachers/educators/ teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance CRS Custom 0% 23.4% 27 Percentage of school officials in target schools who demonstrate use of new and quality supervision and leadership techniques or tools CRS Custom 10% 6.5% 28 Percentage of children 6–23 months receiving a minimum acceptable diet FFP #BL12 9.3% 17.0% 23 STARS Indicator No. Indicator Name Indicator No. Target Baseline (2020) Male Female Total 30 Percentage of parents who state their children had health-related school absences in the previous month CRS Custom 30% 15.0% 34 Percentage of caregivers who report spending time on literacy activities with their school-age children in the previous week CRS Custom 42% 15.8% 36 Percentage of community members who promote early childhood practices and support their children’s education CRS Custom 20% 60.1% Strategic Objective 1: School-Age Children in the Savanes and Kara Regions Have Improved Literacy The first Strategic Objective of the STARS project is the improved literacy of school-aged children in the Savanes and Kara regions. Achievement of this SO is measured through the percentage of students who, by the end of two grades of schooling, demonstrate that they can read and understand the meaning of grade-level text (McGovern-Dole Indicator #1). For the baseline analysis, the specified threshold is that a student is able to correctly answer at least three of the five reading comprehension questions correctly, or a 60 percent accuracy score. No student assessed for the 2020 baseline met this threshold. The proportion of students who did not answer a single item correct on each subtask—known as a zero score—are presented in Figure 5. A majority of students received zero scores in five out of the six subtasks. The proportion of students with zero scores was lowest on the letter sound identification subtask (38 percent) and highest on the reading comprehension subtask (96 percent). Across all subtasks, boys had a lower proportion of zero scores than girls. 24 Figure 5: Percentage of Students Receiving Zero Scores by Sex Figure 6 below illustrates mean percentages of correct responses for each subtask, disaggregated by sex. Figure 6: Mean EGRA Accuracy Scores by Sex Note: ** indicates a statistical difference of p<0.01, * indicates a statistical difference of p<0.05 Mean scores for each EGRA subtask are presented in greater detail in the following section, providing a better understanding of students’ reading performance. Statistical significance tests were used to 83% 96% 71% 93% 38% 72% 86% 98% 76% 95% 44% 79% 81% 95% 66% 91% 32% 66% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Listening Comprehension Reading Comprehension Oral Reading Fluency Nonword Reading Letter Sound Identification Initial Sound Identification Boys Girls Total 16.6% 5.8% 1.4% 3.3% 1.2% 6.0% 10.2% 4.8% 0.9% 2.2% 0.6% 5.0% 13.5% 5.3% 1.1% 2.8% 0.8% 5.6% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% Initial Sound Identification** (10 items) Letter Sound Identification (100 items) Nonword Reading (50 items) Oral Reading Fluency* (57 items) Reading Comprehension (5 items) Listening Comprehension (5 items) Boys Girls Total 25 determine the difference in mean scores between boys and girls; statistically significant differences are noted under each table. Initial Sound Identification For the initial sound identification subtask, the enumerator read aloud 10 common words to students, one at a time. The enumerator asked students to say the name of the letter corresponding to the word’s initial sound. The initial sound identification subtask measures students’ awareness of phonemes and their ability to distinguish among multiple phonemes. Baseline results for the initial sound identification subtask are shown in Table 6. Out of 10 possible items, students correctly identified the initial sound of 1.35 items on average. Boys had statistically significantly higher mean scores than girls; boys, on average, correctly responded to half an item more than girls. Table 7: Initial Sound Identification Mean Scores by Sex (Correct out of 10) Gender N Mean Score Percent Correct Standard Error Boys** 593 1.66 16.6% 0.17 Girls 564 1.02 10.2% 0.12 Total 1,157 1.35 13.5% 0.11 Note: ** indicates a statistical difference of p<0.01. Letter Sound Identification In the letter sound identification subtask, the enumerator presented students with a grid of 100 letters in uppercase and lowercase and asked students to say the sound of as many letters as they could in one minute. The letter sound identification subtask measures students’ knowledge of letters of the alphabet and their ability to recognize each letter’s graphemic features. Baseline results for the letter sound identification subtask are presented in Table 7. On average, students named 5.29 letters correctly out of 100. There was no significant difference between girls’ and boys’ scores. Table 8: Letter Sound Identification Mean Scores by Sex (Correct out of 100) Gender N Mean Score Percent Correct Standard Error Boys 593 5.75 5.8% 0.43 Girls 564 4.79 4.8% 0.41 Total 1,157 5.29 5.3% 0.30 Nonword Reading For the nonword reading subtask, the enumerator presented students with a grid of 50 invented words that follow French phonological and spelling rules but are not actual words in the language. The enumerator asked students to read aloud as many nonwords as possible in one minute. Nonword reading measures students’ decoding skills. 26 Baseline results for the nonword reading subtask are displayed in Table 8. Out of 50 items, students correctly read 0.56 invented words on average. There was no significant difference between girls’ and boys’ scores. Table 9: Nonword Reading Mean Scores by Sex (Correct out of 50) Gender N Mean Score Percent Correct Standard Error Boys 593 0.68 1.4% 0.13 Girls 564 0.43 0.9% 0.09 Total 1,157 0.56 1.1% 0.08 Oral Reading Fluency and Reading Comprehension For the oral reading fluency and reading comprehension subtasks, the enumerator presented students with a short story of 57 words and asked students to read as much of the story out loud as they could in one minute. After finishing, the enumerator read aloud as many as five comprehension questions—four direct and one inferential—to students to test their understanding of the story’s content. The number of comprehension questions asked is linked to how many words students were able to read in one minute; in other words, students were not asked questions about parts of the story they did not read. Together, these two subtasks measure decoding, reading fluency, and reading comprehension. Baseline results for the oral reading fluency subtask are presented in Table 9. From a short story of 57 words, students correctly read 1.6 words per minute on average. Boys had statistically significantly higher mean scores than girls; boys, on average, correctly read slightly more than half a word more than girls. Table 10: Oral Reading Fluency Mean Scores by Sex (Correct out of 57) Gender N Mean Score Percent Correct Standard Error Boys* 593 1.90 3.3% 0.24 Girls 564 1.28 2.2% 0.16 Total 1,157 1.60 2.8% 0.15 Note: * indicates a statistical difference of p<0.05 Baseline mean scores for the reading comprehension subtask are presented in Table 10. Overall, students were able to answer 0.04 reading comprehension questions correctly at baseline. Table 11: Reading Comprehension Mean Scores by Sex (Correct out of 5) Gender N Mean Score Percent Correct Standard Error Boys 593 0.06 1.2% 0.01 Girls 564 0.03 0.6% 0.01 Total 1,157 0.04 0.8% 0.01 27 The distribution of students able to answer reading comprehension questions correctly is detailed in Table 11. No students were able to answer the fourth and fifth comprehension questions. Table 12: Distribution of Correct Reading Comprehension Questions by Sex Number of Questions Correct Boys Girls Total 0 95.3% 97.7% 96.4% 1 3.8% 1.5% 2.7% 2 0.9% 0.8% 0.8% 3 0.1% 0.0% 0.0% 4 0.0% 0.0% 0.0% 5 0.0% 0.0% 0.0% Listening Comprehension The listening comprehension subtask consists of a short story of 38 words that the enumerator read aloud to students. The enumerator then asked students five comprehension questions related to the story—four direct and one inferential. Listening comprehension measures students’ overall oral language comprehension and vocabulary. The listening comprehension subtask complements the reading passage and comprehension subtasks as it enables a better understanding of whether comprehension difficulties result from reading skills or bigger issues with comprehension of the language. Baseline results for the listening comprehension subtask are presented in Table 12. Out of a possible five questions, students correctly answered, on average, 0.28 questions. There was no significant difference between girls’ and boys’ scores. The distribution of students able to answer reading comprehension questions correctly is detailed in Table 13. Table 13: Listening Comprehension Mean Scores by Sex (Correct out of 5) Gender N Mean Score Percent Correct Standard Error Boys 593 0.30 6.0% 0.04 Girls 564 0.25 5.0% 0.04 Total 1,157 0.28 5.6% 0.03 Table 14: Distribution of Correct Listening Comprehension Questions by Sex Number of Questions Correct Boys Girls Total 0 80.9% 85.6% 83.2% 1 11.4% 7.7% 9.6% 2 5.5% 4.1% 4.8% 28 Number of Questions Correct Boys Girls Total 3 1.7% 1.7% 1.7% 4 0.6% 0.5% 0.5% 5 0.0% 0.5% 0.2% IR 1.1: Improved Quality of Literacy Instruction Enumerators used classroom observations to measure quality classroom literacy instruction in 77 project schools. Observers observed a classroom lesson for one hour and recorded activities linked to quality instruction. Further details of the observation tool can be found in Annex E. As defined by the CRS standard classroom observation tool, 23.4 percent of teachers observed (n=18) met the threshold, scoring at least five out of nine on the quality instruction index.15 To create the measure, nine scoring items were used: 1. Teacher provided learning opportunities to support literacy skills; 2. Teacher referred to a lesson plan for structuring their literacy teaching; 3. Teacher provided learning opportunities to develop expressive language skills; 4. Teacher spoke in French; 5. Teacher read books to students; 6. Teacher provided learning opportunities to promote fine motor skills; 7. Teacher provided learning opportunities that allow children to engage in gross motor skills activities; 8. Teacher provided learning activities that promote free choice or open play; and 9. Teacher provided learning opportunities that allow children to participate in music/movement activities Annex F includes the frequency tables of individual teacher practices across the classroom observation tool. A histogram displaying the teachers’ composite scores of overall quality literacy instruction is shown in Figure 7. 15 The classroom observations observed both math and literacy activities; only items relevant to literacy were used to calculate the score. In cases where an item was skipped, the item score was treated as zero. Each question was equally weighted. This means that all activities were given a possible score of 1. While some items were treated as a binary yes or no (e.g., “did the instructor speak French?”), a number of questions used ordinal response items, asking the enumerator to rate the quality of an activity. In this case each question received a total possible score of 1, with each rating incrementally increasing in value from 0 (e.g., 1-4 will be transferred to .25, .5, .75, 1 respectively). 29 Figure 7: Histogram of Quality Literacy Instruction Score IR 1.1.1 More Consistent Teacher Attendance Enumerators asked the head teacher at each sampled school (n=77) a series of questions about teacher attendance, including the number of teachers officially in the school records, the number of teachers present the day of baseline data collection, and the average number of hours per school day teachers are estimated to be teaching. These individual questions were used to calculate the percent of instructional time lost due to teacher absenteeism. It is estimated that, across 76 schools,16 224 hours of teaching time were lost due to teacher absenteeism, or 9.3 percent. Table 15: Instructional Time Lost Due to Teacher Absenteeism Valid Schools 76 Teachers Enrolled (total hours) 2,416 Teachers Present (total hours) 2,192 Estimated Hours Lost 224 Estimated Percentage Lost 9.3% IR 1.1.5 Increased Skills and Knowledge of School Administrators Enumerators asked the head teacher at each sampled school (n=77) questions about the school management tools at the school. These tools included a record of daily teacher attendance, a teacher task list, visual teaching aids and teaching materials, an inventory book, and school records. Out of 10 possible items, a head teacher was considered to be using quality supervision techniques and tools if all 10 items were observed by or shown to the enumerator. Five head teachers (6.5 percent) met this threshold at the 77 sampled schools, as shown in Figure 8. 16 In cases where records of teaching time were abnormally high (over 13 hours, as high as 63 hours), average time responses were reverted to the mean. In cases where any one of the records were missing, the case was dropped (this only affected a single record). 1 7 14 20 17 11 4 2 1 0 0 5 10 15 20 25 0 1 2 3 4 5 6 7 8 9 Number of Teahers Number of quality teaching practices demonstrated during lesson threshold 30 Figure 8: Proportion of School Officials Using All 10 Quality Supervision Tools Note: n=77 IR. 1.2: Improved attentiveness As part of classroom observations, enumerators rated students’ level of engagement during the lesson in one of four categories: 1. Few children (25 percent or less) are engaged for most of the observation; 2. Some children (26 percent to 50 percent) are engaged for most of the observation; 3. Most children (51 percent to 75 percent) are engaged for most of the observation; and 4. Almost all of the children (76 percent to 100 percent) are engaged for most of the observation If a majority of students were engaged for most of the observation—categories 3 and 4—the classroom was considered “attentive.” Nearly three of five classrooms—or 59.7 percent (n=46)—met the threshold as being “attentive.”17 Figure 9 displays the breakdown of schools meeting the attentiveness threshold, while the distribution of classroom ratings is shown in Figure 10. 17 While the same number of classrooms met the threshold of being attentive as practiced quality literacy instruction, there did not seem to be a relationship between the two factors. Below Threshold 93.5% Meets Threshold 6.5% 31 Figure 9: Proportion of Attentive Classrooms Observed Note: n=77 Figure 10: Proportion of Student Attentiveness Levels in Observed Classrooms Note: n=77 IR 1.2.1: Reduced Short-Term Hunger Enumerators asked parents a series of questions adapted from the World Health Organization’s (WHO) Infant and Young Child Feeding (IYCF) assessment18 to identify the percentage of children, aged 6-23 months, receiving a minimum acceptable diet (MAD). This composite indicator, displayed in full in Figure 11, comprises the Minimum Dietary Diversity (MDD) and Minimum Meal Frequency questions. It also requires screening questions for child age and breastfeeding status. 18 Indicators for Assessing Infant and Young Child Feeding Practices: Conclusions of a Consensus Meeting Held 6-8 November 2007 in Washington D.C., USA. World Health Organization (WHO), 2008. Attentive 59.7% Not Attentive 40.3% Attentive Not Attentive 13.0% 27.3% 37.7% 22.1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Observed Attentiveness Levels Few students (25% or less) are engaged Some students (26-50%) are engaged Most students (51-75%) are engaged Almost all students (76-100%) are engaged 32 Figure 11: Calculation of minimum acceptable diet indicator (WHO) Among those who were eligible, 17.0 percent of 6- to 23-month-old children met the MAD threshold. Table 15 provides a full breakdown of the number and overall percentage of children that met the MAD criteria. It is important to note that this survey was conducted in November 2020, so seasonality may have been a factor in these findings. Table 16: Proportion of children meeting minimum acceptable diet threshold Meets MAD Criteria N Percent (Overall) Percent (Among Eligible) No 127 55.7% 83.0% Yes 26 11.4% 17.0% Ineligible19 75 32.9% Total 228 100.0% IR 1.3: Improved Student Attendance Enrollment figures for each school were collected during the head teacher survey, while attendance numbers were recorded during a physical headcount during the classroom and school observations. These measures of student attendance and enrollment were used to determine the average student attendance rate in project schools. According to CRS’s census baseline data, 33,334 students are enrolled in the primary grades at their 138 intervention schools. The average attendance rate across all 138 schools on the days of their school visits was 80.2 percent, with 79.0 percent of girls in attendance and 81.3 percent of boys in attendance. At the 77 sampled schools at baseline, 84.7 percent of students were present on average. This generally held across gender, with 85.7 percent of boys and 83.5 percent of girls attending the day of the classroom observations. Figure 12 displays the total number of recorded enrolled students and students in attendance on the day of the data collection at the 77 sampled schools only. 19 Ineligible cases were those included any of those cases that did meet the following criteria: did not report age of the child, outside the age criteria, skipped the question of if the child was ever breastfed. The high number of ineligible cases are due to inaccurate sampling through head teachers, rather than project rosters because community activities have not yet commenced. This will be resolved for the midline and endline evaluations. 33 Figure 12: Total number of students present and enrolled at sampled schools by gender IR 1.3.2: Reduced Health-Related Absences In the parent survey, administered to three parents at each school, respondents were asked about student absences over the past month and the cause of the absences. About one in five parents—or 20.2 percent—stated that at least one of their children missed school in the past month. As shown in Table 16, 15.0 percent of all parents responding stated that their child (or children) missed school over the past month due to illness. Table 17: Parent responses to reasons for child absence Have any of your children missed school in the past month? N Percent Did they miss school because of an illness? N Percent (overall) Percent (sub question) No/No response 174 76.3% Yes 46 20.2% No 13 5.9% 28.3% Yes 33 15.0% 71.7% Don’t know/No response 8 3.5% Total 228 - 18397 9646 8751 15574 8269 7305 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 Overall Boys Girls Present Enrolled 34 IR 1.3.5: Increased Community Understanding of the Benefits of Education Percentage of caregivers spending time on literacy activities with their children in the previous week Enumerators asked parents and caregivers whether they supported their children’s learning and engaged in literacy activities at home.20 STS first examined the percentage of caregivers who reported spending time on literacy activities with their school-age children in the previous week. Of the 228 parents surveyed, 76 reported helping their children with homework in the past week, with 36 parents (15.8 percent) engaging in at least one literacy activity with their child in the past week. Of the 76 parents who helped their child with homework in the past week, 28 parents reported having helped their child read letters, 16 parents helped their child read words, and 10 parents helped their child read a text. Table 18: Parent Responses to Homework Support and Literacy Activities Within Past Week Did you help your children with their homework in the last week? N Percent Parents engaged in at least 1 literacy activity in past week If yes, for which types of activities? N Percent (overall) Percent (sub question) No 144 63.2% Yes 76 33.3% 36 (15.8%) Read letters 28 12.3% 36.8% Read words 16 7.0% 21.1% Read texts 10 4.4% 13.2% Don’t know/No response 8 3.5% Total 228 - Percentage of community members who promote early childhood practices and support their children’s education However, for the broader indicator of the percentage of community members who promote early childhood practices and support their children’s education, STS looked across the entire population of parents and caregivers, for which most parents reported participating in broader at-home education activities beyond the past week. This included the following four activities: 1. Telling stories to children; 2. Having children read aloud to parents; 3. Asking children what they learned in school; and 4. Helping children with their homework or having another family member help with homework About three of five respondents—or 60.1 percent—reported having participated in three or more education activities with their child or children at home, as shown in Table 18 and Figure 13. 20 If the respondent answered “don’t know” to all questions, it was not included in analysis. 35 Table 19: Distribution of home-based education activities Number of home education activities N Percentage % of parents reporting at least 3 home-based education activities 0 23 10.1% 1 30 13.2% 2 38 16.7% 3 55 24.1% 60.1% 4 82 36.0% Note: n=228 Figure 13: Proportion of parents reporting use of home-based education activities Note: n=228 parents Strategic Objective 2: Communities in the Savanes and Kara Regions have increased use of improved health, nutrition, and dietary practices While CRS Togo has not yet begun implementing the STARS project activities in support of Strategic Objective 2, the baseline evaluation did examine the current state of the water and sanitation facilities at sampled schools. These findings are shown in the following tables. IR 2.4: Number of schools with improved sanitation facilities For sanitation facilities, CRS’s census baseline monitoring data shows that 57 of the 138 intervention schools have improved sanitation facilities. Findings on sanitation facilities at only the 77 sampled schools are presented in Table 20 and Table 21. Out of the 77 schools observed, 48 schools had some available sanitation facility, with 93.8 percent of available sanitation facilities functional on the day of the visit. However, only 10 schools met the definition of improved sanitation facilities, or 13 percent of the 77 sampled schools. 10.1% 13.2% 16.7% 24.1% 36.0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Parents 0 practices 1 practice 2 practices 3 practices 4 practices 36 Table 20: Sanitation Facilities at Sampled Schools Frequency Percent No toilets available (only in the bush or in the fields) 29 37.7% The toilets are pit latrines or buckets 38 49.4% The toilets are composting toilets 10 13.0% Total 77 - Table 21: Status of Sanitation Facilities at Sampled Schools Frequency Percent Not functional 3 6.3% Functional 45 93.8% Total 48 - Fifty-three of the 77 sampled schools, or 68.8 percent, had a handwashing station at the school. Table 22: Handwashing Facilities at Sampled Schools Frequency Percent No handwashing station at the school 24 31.2% Shared basin or bucket (handwashing is done in water; water does not flow or is not poured) 8 10.4% Hand pouring system with used water separated from water to clean hands but without soap 17 22.1% There is running water OR a hand pour system (with the wastewater separated from the clean water for washing hands) AND soap 28 36.4% Total 77 - Of the 53 schools with a handwashing station at the school, most handwashing stations (81.1 percent) were deemed accessible to both the youngest students and students with disabilities. Table 23: Accessibility of Handwashing Facilities at Sampled Schools Frequency Percent Not accessible to the youngest children or children with disabilities 4 7.6% Accessible to the youngest children OR children with disabilities 6 11.3% Accessible to the youngest children AND children with disabilities 43 81.1% Total 53 - 37 IR 2.5: Number of schools using an improved water source According to CRS’s census baseline monitoring data of all intervention schools, 70 of the 138 schools use an improved water source. While handwashing stations were present at the majority of schools, drinking water was not. Only 32 of the sampled schools (41.6 percent) had access to drinking water, and only 26 were from an improved water source (33.8 percent). Of those 32 schools with access to sources of drinking water, 24 (75 percent) were functional on the day of the school visit. Table 24: Water Sources at Sampled Schools Frequency Percent No water available at school. Water, if present, is provided by parents, children, or staff 45 58.4% Available water is: Unprotected inground well / spring, untreated rainwater, surface water 6 7.8% Available water is a cart with a small tank / drum or a protected spring 0 0.0% The available source of sanitary water is running water, a public tap, treated rainwater, a protected dug well, or bottled water 26 33.8% Total 77 - Table 25: Status of Water Source Frequency Percent Not Functional 8 25.0% Functional 24 75.0% Total 32 - 4.Conclusions The findings of this study will serve as the baseline for two future evaluations. By comparing the results of future evaluations to this baseline study, the STARS project’s impact on students’ progression in their fundamental reading skills will be examined, as measured by the EGRA subtasks. Using the Southwest Educational Development Laboratory’s (SEDL) Cognitive Framework for Reading, it is possible to map EGRA subtasks to reading skills as follows:21 21 Sebastian Wren, The Cognitive Foundations of Learning to Read: A Framework. Southwest Educational Development Laboratory, 2001. https://sedl.org/reading/framework/framework.pdf 38 Figure 14: Reading Skills Framework with EGRA Subtask Mapping Understanding or Meaning Making Mechanics of Reading • Listening Comprehension • Initial Sound Identification • Letter Sound Identification • Nonword Reading • Oral reading fluency Reading Comprehension • Reading Comprehension A total of 1,157 CE1 students participated in the EGRA during the baseline evaluation. The EGRA was administered in French, which is the official language of instruction in Togo. To examine students’ basic understanding or meaning-making abilities in French, students completed a listening comprehension subtask. For this subtask, consisting of five questions about a story read aloud in French, students were only able to answer 0.28 questions correctly, which shows the CE1 student population has a very limited ability in understanding the French language. Four EGRA subtasks speak to students’ abilities within the mechanics of reading. Students must master these necessary building blocks to progress to reading comprehension. Literacy and reading instruction in the early grades—including those grades targeted by the STARS project—often focus predominantly on these skills. On average, students correctly responded to 1.35 out of 10 items on the initial sound identification subtask. On the letter sound identification subtask, students correctly identified 5.29 letters out of 100 in one minute, on average. For nonword reading, on average, students correctly read 0.56 words out of 50 in one minute. Students read on average at a rate of 1.60 words per minute on the 39 oral reading fluency subtask. Grade 3 students in the baseline sample have considerable opportunity to improve their skills in these areas, especially when considering the large proportion of zero scores associated with these subtasks. The final subtask, reading comprehension, speaks to students’ ability to utilize the mechanics of reading, demonstrate fluency, and understand the passage’s meaning. It is the most advanced EGRA subtask, as it measures the ultimate goal of literacy—comprehension. Unsurprisingly, Grade 3 students in the evaluation struggled the most with this subtask. On average, students were not able to correctly answer a single reading comprehension question, with the average number of questions correctly answered being only 0.04. The proportion of students unable to provide a single correct response on each subtask was often high. On the initial sound identification subtask, 72 percent of students were not able to correctly respond to even one of the five items. The letter sound identification subtask had the lowest proportion of students with a zero score, with only 38 percent of students not being able to identify at least one letter in one minute correctly. On the nonword reading subtask, 93 percent of students were not able to correctly read a single nonword. When presented with a reading passage, 71 percent of students were not able to read a single word. Linked to the reading passage subtask, the reading comprehension questions also had a high number of zero scores, as 96 percent of students were not able to correctly answer a single reading comprehension question. On listening comprehension, 83 percent of students were unable to answer a single question correctly. 5.Recommendations STS proposes the following recommendations to CRS for both project implementation, as well as things to consider for the midline and endline evaluations. Implementation Recommendations • Examine existing student and teacher French language abilities. Overall student performance, particularly on listening comprehension, indicates that students have a limited ability to understand spoken French. The project may want to consider undertaking more targeted research into the reasons for this gap in comprehension. Specifically, these efforts may mean a deeper investment in coaching for basic skills for literacy instruction for early grade teachers, whose French-language proficiency was not addressed in this baseline data collection. Improving teachers’ French abilities may be a necessary step to ensuring they can confidently teach students to read in French. Comprehensive discussions with the Ministry of Education about the curriculum design may also be beneficial. • Examine gender constraints within target communities. Girls’ underperformance compared with boys deserves further exploration and may warrant a specific focus within the project to address the underlying causes of these gender disparities. 40 • Consider seasonality when defining rations within nutrition activities, as well as during program monitoring. The baseline evaluation findings showed a higher percentage than expected of children between the ages of 6 and 23 months who met the MAD requirements. This unexpected finding may be due to the season in which the data collection was conducted—November 2020. The fluctuations in access to quality nutrition due to seasonality should be considered when defining rations for students and pregnant and lactating women, as well as when interpreting data collected during program monitoring. Recommendations for Midline Evaluation • Revise the EGRA tool to align with current best practices and associated benchmarks for tracking reading improvement. The baseline administration used an EGRA originally developed for use in Djibouti and was not created for the local Togolese context. Additionally, generic benchmarks for reading comprehension were used due to a lack of Togo-specific benchmarks. A revised and equated EGRA, as well as country-specific reading benchmarks, would result in more nuanced understanding of student reading proficiency. • Consider seasonality when interpreting MAD results for the midline and endline evaluations. Under the original timeline for the baseline evaluation, data collection was planned for the end of the academic year. With the delays due to COVID-19, data collection took place in a different season than planned. Should the midline and endline evaluations occur at the end of the school year, this seasonal difference should be considered when interpreting the results for the MAD indicator. • Modify existing survey items, indicators, or definitions to allow for greater accuracy during data collection. CRS should review existing indicators and definitions within their Performance Monitoring Plan to identify any areas for clarification or refinement. STS should make corresponding changes to the tools to reflect more nuanced definitions and indicators. 41 Annexes Annex A: Bibliography McConnell, Brendon, and Marcos Vera-Hernandez. Going beyond simple sample size calculations: a practitioner’s guide. Institute for Fiscal Studies. 2015. UNICEF. Analyse du secteur de l’éducation de la République togolaise, Des défis pour un enseignement de qualité pour tous, République togolaise, IIPE-Pôle de Dakar - UNESCO, 2019. United Nations, Department of Economic and Social Affairs, Population Division. World Population Prospects 2019, Online Edition. Rev. 1, 2019. United States Department of Agriculture. “McGovern-Dole Food for Education Program,” accessed January 20, 2021, https://www.fas.usda.gov/programs/mcgovern-dole-food-education-program WHO/UNICEF. Joint Monitoring Programme for Water Supply, Sanitation and Hygiene (JMP). Last update: June 2019. https://data.unicef.org/resources/dataset/sowc-2019-statistical-tables/ World Bank. Togo Country Overview. Last update: November 2020. https://www.worldbank.org/en/country/togo/overview Wren, Sebastian. The Cognitive Foundations of Learning to Read: A Framework. Southwest Educational Development Laboratory, 2001. https://sedl.org/reading/framework/framework.pdf 42 Annex B: Updated Indicator Performance Tracking Table No. Results framework statement RF Activity Performance Indicator Standard or CRS Custom Baseline 1 School-Age Children in the Savanes and Kara Regions Have Improved Literacy (SO 1) SO1 Raising awareness on importance of education (Activity 12) Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text Standard #1 0% 2 School-Age Children in the Savanes and Kara Regions Have Improved Literacy (SO 1) SO1 Provide school meals (Activity 11) Number of individuals benefiting indirectly from USDA-funded interventions Standard #31 0 3 Communities in the Savanes and Kara Regions Have Increased Use of Improved Health, Nutrition and Dietary Practices (SO 2) IR 2.1 Raise awareness on health, nutrition and WASH (Activity 12) Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance Standard #19 0% 4 Communities in the Savanes and Kara Regions Have Increased Use of Improved Health, Nutrition and Dietary Practices (SO 2) SO2 Training: Food preparation and storage practices (Activity 15) Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance Standard #20 0% 5 Communities in the Savanes and Kara Regions Have Increased Use of Improved Health, Nutrition and Dietary Practices (SO 2) SO2 Provide school meals (Activity 11) Number of individuals participating in USDA food security programs Standard #30 0 6 Communities in the Savanes and Kara Regions Have Increased Use of Improved Health, Nutrition and Dietary Practices (SO 2) SO2 Provide school meals (Activity 11) Number of schools reached as a result of USDA assistance Standard #32 0 7 Improved Quality of Literacy Instruction (IR 1.1) IR 1.1 Training: Teachers (Activity 18) Percent of teachers providing quality classroom instruction with USG support USAID Education Proposed 0% 8 IR 1.2 Improved Attentiveness IR 1.2 Provide school meals (Activity 11) Percent of students in target schools identified as attentive during class/instruction Custom 59.7% 43 No. Results framework statement RF Activity Performance Indicator Standard or CRS Custom Baseline 9 Improved Student Attendance (IR 1.3) IR 1.3 Take home rations (Activity 14) Average student attendance rate in USDA supported classrooms/schools Standard #2 80.2% 10 Increased Knowledge of Safe Food Prep and Storage Practices (IR 2.2) IR 2.2 Training: Food preparation and storage practices (Activity 15) Number of individuals trained in safe food preparation and storage as a result of USDA assistance Standard #22 0 11 Improved Knowledge of Health and Hygiene Practices (IR 2.1) 2.1 Raise awareness on health, nutrition and WASH (Activity 12) Number of individuals trained in child health and nutrition as a result of USDA assistance Standard #23 0 12 Increased Knowledge of Nutrition (IR 2.3) IR 2.3 Raise awareness on health, nutrition and WASH (Activity 12) Number of children under five (0-59 months) reached with nutrition-specific interventions through USDA-supported programs Standard #24 0 13 Increased Knowledge of Nutrition (IR 2.3) IR 2.3 Raise awareness on health, nutrition and WASH (Activity 12) Number of pregnant women reached with nutrition-specific interventions through USDA-supported programs Standard #26 0 14 Increased Access to Clean Water and Sanitation Services (IR 2.4) IR 2.4 Raise awareness on health, nutrition and WASH (Activity 12) Number of children under two (0-23 months) reached with community-level nutrition interventions through USDA￾supported programs Standard #25 0 15 Increased Access to Clean Water and Sanitation Services (IR 2.4) IR 2.4 Building/ Rehab: Latrines (Activity 2) Number of schools with improved sanitation facilities Standard #28 57 16 Increased Access to Clean Water and Sanitation Services (IR 2.4) IR 2.5 Building/ Rehab: Wells and water stations/ systems (Activity 4) Number of schools using an improved water source Standard #27 70 44 No. Results framework statement RF Activity Performance Indicator Standard or CRS Custom Baseline 17 Increased Access to Clean Water and Sanitation Services (IR 2.4) IR 2.5 Building/ Rehab: Latrines (Activity 2) Percent of health and nutrition infrastructure, constructed as a result of USDA assistance, maintained by communities/local authorities Custom 0% 18 Increased Access to Requisite Food Prep and Storage Tools and Equipment (IR 2.6) IR 2.6 Building/ Rehab: Kitchens (Activity 1) Number of Schools receiving energy saving stoves Custom 0 19 More Consistent Teacher Attendance (Sub-IR 1.1.1) Sub-IR 1.1.1 Promote teacher attendance (Activity 10) Percent of instructional time lost due to teacher absenteeism USAID Education Proposed 9.3% 20 More Consistent Teacher Attendance (Sub-IR 1.1.1) Sub-IR 1.1.1 Promote teacher attendance (Activity 10) Number of schools implementing the use of school score cards Custom 0% 21 Better Access to School Supplies and Materials (Sub-IR 1.1.2) Sub-IR 1.1.2 Distribution School supplies and materials (Activity 6) Number of teaching and learning materials provided as a result of USDA assistance Standard #3 0 22 Increased Skills and Knowledge of Teachers (Sub-IR 1.1.4) Sub-IR 1.1.4 Training: Teachers (Activity 18) Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Standard #4 0% 23 Increased Skills and Knowledge of Teachers (Sub-IR 1.1.4) Sub-IR 1.1.4 Training: Teachers (Activity 18) Percentage of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Custom 23.4% 24 Increased Skills and Knowledge of Teachers (Sub-IR 1.1.4) Sub-IR 1.1.4 Training: Teachers (Activity 18) Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance Standard #5 0 25 Increased Skills and Knowledge of School Administrators (Sub-IR 1.1.5) Sub-IR 1.1.5 Training: School admins (Activity 17) Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance Standard #6 0% 45 No. Results framework statement RF Activity Performance Indicator Standard or CRS Custom Baseline 26 Increased Skills and Knowledge of School Administrators (Sub-IR 1.1.5) Sub-IR 1.1.5 Training: School admins (Activity 17) Number of school administrators and officials trained or certified as a result of USDA assistance Standard #7 0 27 Increased Skills and Knowledge of School Administrators (Sub-IR 1.1.5) Sub-IR 1.1.5 Training: School admins (Activity 17) Percent of school officials in target schools who demonstrate use of new and quality supervision and leadership techniques or tools Custom 6.5% 28 Reduced Short-Term Hunger (Sub-IR 1.2.1) Sub-IR 1.2.1 Take home rations (Activity 14) Percent of children 6–23 months receiving a minimum acceptable diet FFP #BL12 17.0% 29 Increased Economic and Cultural Incentives (Sub-IR 1.3.1) Sub-IR 1.3.1 Provide school meals (Activity 11) Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance Standard #17 0 30 Reduced Health-Related Absences (Sub-IR 1.3.2) Sub-IR 1.3.2 Raise awareness on health, nutrition and WASH (Activity 12) Percent of parents who state their children had health-related school absences in the previous month Custom 15.0% 31 Improved School Infrastructure (Sub-IR 1.3.3) Sub-IR 1.3.3 Building/ Rehab: Kitchens (Activity 1) Number of educational facilities (i.e. school buildings, classrooms, improved water sources, and latrines) rehabilitated/constructed as a result of USDA assistance [Kitchens, cook areas] Standard #8 0 32 Increased Student Enrollment (Sub-IR 1.3.4) Sub-IR 1.3.4 Raising awareness on importance of education (Activity 13) Number of students enrolled in school receiving USDA assistance Standard #9 0% 33 Increased Student Enrollment (Sub-IR 1.3.4) Sub-IR 1.3.4 Raising awareness on importance of education (Activity 13) Number of schools that held an enrollment campaign. Custom 0 34 Increased Community Understanding of the Benefits of Education (Sub-IR 1.3.5) Sub-IR 1.3.5 Establish activities to promote literacy (Activity 7) Percent of caregivers who report spending time on literacy activities with their school-age children in the previous week Custom 15.8% 46 No. Results framework statement RF Activity Performance Indicator Standard or CRS Custom Baseline 35 Increased Community Understanding of the Benefits of Education (Sub-IR 1.3.5) Sub-IR 1.3.5 Raising awareness on importance of education (Activity 13) Number School Management Committee (SMC) and Parent Teacher Association (APE) members, and Mother Leaders trained on activities to promote literacy Custom 0 36 Increased Community Understanding of the Benefits of Education (Sub-IR 1.3.5) Sub-IR 1.3.5 Establish activities to promote literacy (Activity 7) Percent of community members who promote early childhood practices and support their children’s education Custom 60.1% 37 Increased Access to Food (Output 1.2.1.1, 1.3.1.1) Output 1.2.1.1, 1.3.1.1 Take home rations (Activity 14) Quantity of take-home rations provided (in metric tons) as a result of USDA assistance Standard #14 0 38 Increased Access to Food (Output 1.2.1.1, 1.3.1.1) Output 1.2.1.1, 1.3.1.1 Take home rations (Activity 14) Number of individuals receiving take-home rations as a result of USDA assistance Standard #15 0 39 Increased Access to Food (Output 1.2.1.1, 1.3.1.1) Output 1.2.1.1, 1.3.1.1 Provide school meals (Activity 11) Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance Standard #16 0 40 Increased Access to Food (Output 1.2.1.1, 1.3.1.1) Output 1.2.1.1, 1.3.1.1 Provide school meals (Activity 11) Number of social assistance beneficiaries participating in productive safety net as a result of USDA assistance Standard #18 0 41 Increased Access to Food (Output 1.2.1.1, 1.3.1.1) Output 1.2.1.1, 1.3.1.1 Form savings and lending groups (Activity 9) Number of individuals participating in group-based savings, micro-finance or lending programs with USDA assistance FFPr Standard #6 0 47 No. Results framework statement RF Activity Performance Indicator Standard or CRS Custom Baseline 42 Increased Capacity of Government Institutions (FR 1.4.1) FR 1.4.1 Capacity Building: Local, regional, national level (Activity 5) Number of members of the interministerial steering committee conducting monitoring visits to targeted schools Custom 0 43 Increased Capacity of Government Institutions (FR 1.4.1) Increased Capacity of Government Institutions (FR 2.7.1) FR 1.4.1/ 2.7.1 Capacity Building: Local, regional, national level (Activity 5) Number of Early Grade Reading Assessments (EGRAs) administered by inspectors, pedagogical advisors, school administrators, and teachers in target schools Custom 0 44 Improved Policy and Regulatory Framework (FR 1.4.2) Improved Policy and Regulatory Framework (FR 2.7.2) FR 1.4.2/ 2.7.2 Capacity Building: Local, regional, national level (Activity 5) Number of policies, regulations, or administrative procedures in each of the following stages of development as a result of USDA assistance Standard #10 0 45 Increased Government Support (FR 1.4.3) Increased Government Support (FR 2.7.3) FR 1.4.3/ 2.7.3 Capacity Building: Local, regional, national level (Activity 5) Value of new USG commitments, and new public and private sector investments leveraged by USDA to support food security and nutrition [Host Government amount] Standard #11 0 46 Increased Engagement of Local Organizations and Community Groups (FR 1.4.4) FR 1.4.4 Training: Parent￾Teacher Associations (Activity 16) Number of Parent Teacher Associations (APE) or similar school governance structure supported as a result of USDA assistance Standard #13 0 47 Increased Engagement of Local Organizations and Community Groups (FR 1.4.4) Increased Access to Food (Output 1.2.1.1, 1.3.1.1) FR 1.4.4/ Output 1.2.1.1, 1.3.1.1 Form savings and lending groups (Activity 9) Number of public private partnerships formed as a result of USDA assistance Standard #12 0 48 Annex C: Results Framework for STARS Project Strategic Objective 1 (SO1) 49 Strategic Objective 2 (SO2) 50 Annex D: Terms of Reference for the Evaluation TERMS OF REFERENCE (TOR) Baseline, Midterm and Final Evaluation Republic of Togo McGovern Dole FY20-FY24 1. Purpose and Overview: The purpose of these Terms of Reference (TOR) is to outline the conditions and responsibilities of the external evaluator who will undertake the baseline, midterm evaluation and final evaluation of the Santé, Transformation et Apprentissage pour une Réussite (STARS)22 project, a USDA-funded McGovern-Dole International Food for Education project in the Republic of Togo. Please note these ToR and its annexes are subject to donor approval, and thus may change before contract signing. Note these ToR rely heavily on Annex 1. Evaluation Plan for the STARS project; specific relevant sections are outlined below. The external evaluator should be very familiar with Annex 1, and Annex 2. Indicator Performance Tracking Table (IPTT), in addition to the USDA’s Food Assistance Indicators and Definitions and its Monitoring and Evaluation Policy. Finally, the external evaluator should also be very familiar with Annex 5, the project’s Performance Monitoring Plan (PMP). Retention of the external evaluator to proceed with the midterm and/or final evaluation is dependent upon their satisfactory performance on the preceding work products. Any changes to evaluator team composition from on evaluation to the next would need to be approved. CRS would relaunch the selection process for the midterm review and/ or final evaluation where the external evaluator does not meet expectations. Please note that all evaluation reports will be reviewed in line with Annex 3. Checklist for Evaluating USDA Evaluation Reports (CRS internal). 2. Project Background: Section 2 of Annex 1 provides an overview of the STARS project. 3. Evaluation Purpose, Scope, Approach, and Methodology: Please note that Section 3 of Annex 1 provides an overview of evaluation activities including stakeholders, anticipated data collection tools, the STARS Results Framework, and sample size requirements. Section 8 of Annex 1 describes special studies for which the external evaluator will be responsible Information in this section, and in Annex 1, outline the standards expected of the external evaluator during data collection and analysis. Justified deviations from these standards, after consultation with CRS, are possible. Data Collection Tools. As the McGovern-Dole program is new to Togo, Togo-specific data collection tools to be used by external evaluators have not yet been developed. The selected external evaluator will need to work with CRS to develop these tools, keeping in mind the project’s PMP. CRS, as an agency, is attempting to standardize tools used in its education sector projects and had developed a Classroom Observation tool and Student Survey (see Annex 4. CRS Standard Tools), and CRS would welcome feedback from the external evaluator on these tools. Some of the content in these tools are likely good proxies for measuring a few of the project’s IPTT indicators. In addition, CRS can share tools used in evaluation in its seven ongoing McGovern-Dole awards. 22 In English: “Health, Transformation and Learning for School Success” 51 Use random samples and document any sample bias due to non-random sampling. Representative samples should always be selected randomly, ideally from a list or using a random walk, etc. However, often due to resource constraints, sample selection bias does occur. This frequently happens due to security constraints that prevent study teams from reaching an off-limits area or when the rosters from which individuals or clusters are randomly selected are outdated, and it would prove too costly or impossible to locate those randomly selected. In this case, in the limitations section of the evaluation report, describe any sources of bias as best as possible. For example, if students are not present in school the day of evaluation, how do absent students differ from those present? Does a t-test of means show that the proportion of key groups (gender, ethnicity, geographic area)23 in the sample is the same as those that were not included? If not, how might the sample be biased? How else might students not present that day be different? Might they not perform as well on literacy tests, etc. because they might frequently miss school? Check for statistical differences in outcome-level indicators over time. The mid-term and final evaluations should, at minimum, check for statistical differences between baseline and respective report values. This will can be via a t-test; however, a preferred general specification would be: 𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑖𝑖𝑖𝑖𝑖𝑖 = 𝐼𝐼𝐼𝐼 𝐼𝐼 𝐼𝐼 + 𝑀𝑀𝑀𝑀𝑀𝑀 + 𝐹𝐹 𝐹𝐹𝐹𝐹 + 𝐹𝐹 𝐹𝐹𝐹𝐹 𝑖𝑖 + 𝑆𝑆 + 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 where • 𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑖𝑖𝑖𝑖𝑖𝑖 is the outcome indicator of interest for individual i at time t (baseline, midterm, or final) in strata s; • 𝑀𝑀𝑀𝑀𝑀𝑀 is a binary variable taking the value 1 if the data was collected during the midterm evaluation, and zero otherwise; • 𝐹𝐹 𝐹𝐹𝐹𝐹 is a binary variable taking the value 1 if the data was collected during the final evaluation, and zero otherwise (only relevant at final evaluation); • 𝐹𝐹 𝐹𝐹𝐹𝐹 𝑖𝑖 is a binary variable taking the value 1 if individual i is female, and zero otherwise; • 𝑆𝑆 is a vector of binary variables for each stratum (excluding one to avoid the dummy variable trap); • 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖is the error-term that should be clustered at the cluster-level during analysis. Ideally, a table with each indicator of interest could be presented per row, with the coefficient (or marginal value when using probit/ logit models) and standard errors for the midterm, final, and female indicators in columns. It is not necessary to present marginal values per stratum. The specification can be adapted if the outcome indicator is not at the individual level, not stratified, or not clustered. Sample weights. Sample weights should always be used when providing unconditional descriptive statistics (means or totals) for the underlying population. However, results from regression analyses, would ideally report unweighted and weighted results, and where there are differences, include a discussion of the underlying reasons. For example, observations from a school that has 90 second-graders vs. 30 will carry 3 times the weight; if there are heterogenous project effects for large vs. small schools (e.g. larger schools have a higher teacher/ student ratio; perhaps this lack of student attention results in poorer educational outcomes, etc.) then the conditional means might be different for weighted vs. unweighted analyses (Solon, Haider, and Wooldridge 2015). 23 The analyst may not have much information about students not present. However, based on student names and school locations, they might at least have this information. 52 Clustered or stratified samples and regression analysis. When reporting weighted conditional means from regression analyses, weighted values should use the appropriate weighted counterpart (e.g. weighted least squares, weighted maximum likelihood, etc.). Additionally, because observations within a cluster are likely correlated, standard errors should always be clustered at the cluster-level (Cameron and Miller 2015). Statistical packages have functions for this; the appropriate function will vary depending on the method of analysis. Control for any sample stratification in regression analyses by using binary variables for each stratum (excluding one to avoid the dummy variable trap). Population Proportional to Size (PPS) cluster selection may not appropriate. PPS is a quantitative sample selection methodology commonly used to account for the size of clusters when selecting them in the first stage of evaluation studies, in which every person in every cluster has an equal probability of being selected into the sample. If, in the second stage, a simple random sample is used to select each individual among all individuals in the cluster, then the sample is “self-weighting” and no sample weights need be applied at the analysis stage. Analysts of data collected via a PPS-selected sample should understand that if the sample was stratified, or if a simple random sample was not used in the second stage, then the sample is not self-weighting and sample weights must be used. At the analysis stage, the Hansen-Hurwitz or Horvitz-Thompson estimators should be used to estimate the sample mean, and variance in any regression models (Hansen and Hurwitz 1942, Horvitz and Thompson 1952). When using PPS, the measure of size should be accurate, otherwise it will over- or underestimate the sample variance, as compared to simple random selection of clusters (Thomsen, Tesfu, and Binder 1986), despite using the estimators described above. Even if baseline measures of size are accurate, if using a repeated cross￾section (schools are commonly maintained across all three evaluation points) when evaluating in the same clusters at midterm or final evaluation and the “size” of the clusters changes notably over time (likely to occur, as we expect enrollment to increase as a result of project activities), the same issue of mis-estimating the sample variance will occur. For all these reasons, using PPS is likely too complex and not appropriate for these evaluations, and therefore not recommended. In lieu of PPS, clusters and individuals can be selected via a random sample, and sample weights used in analysis. Project indicators. Only the fifteen indicators with non-zero baseline values in Table 1 are collected during the baseline study. All individual-level data must be disaggregated by gender. 53 Table 1. STARS Project Indicators Performance Indicator USDA Standard/ CRS Custom Baseline 1. Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text a Standard #1 21% 2. Number of individuals benefiting indirectly from USDA-funded interventions b Standard #31 0 3. Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance c Standard #19 0 4. Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance c Standard #20 0 5. Number of individuals participating in USDA food security programs b Standard #30 0 6. Number of schools reached as a result of USDA assistance b Standard #32 0 7. Percent of teachers providing quality classroom instruction with USG support c USAID Ed Supp-10 0% 8. Percent of students in target schools identified as attentive during class/instruction c Custom 60% 9. Average student attendance rate in USDA supported classrooms/schools c Standard #2 93% 10. Number of individuals trained in safe food preparation and storage as a result of USDA assistance b Standard #22 0 11. Number of individuals trained in child health and nutrition as a result of USDA assistance b Standard #23 0 12. Number of children under five (0-59 months) reached with nutrition-specific interventions through USDA-supported programs b Standard #24 0 13. Number of pregnant women reached with nutrition-specific interventions through USDA-supported programs b Standard #26 0 14. Number of children under two (0-23 months) reached with community-level nutrition interventions through USDA-supported programs b Standard #25 0 15. Number of schools with improved sanitation facilities c Standard #28 66 16. Number of schools using an improved water source c Standard #27 90 17. Percent of health and nutrition infrastructure, constructed as a result of USDA assistance, maintained by communities/local authorities c Custom 0% 18. Number of Schools receiving energy saving stoves b Custom 0 19. Percent of instructional time lost due to teacher absenteeism c USAID Ed Supp-11 52% 20. Number of schools implementing the use of school score cards c Custom 0 21. Number of teaching and learning materials provided as a result of USDA assistance b Standard #3 0 22. Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance c Standard #4 0 23. Percentage of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance c Custom 20% 24. Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance b Standard #5 0 25. Number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance c Standard #6 0 26. Number of school administrators and officials trained or certified as a result of USDA assistance b Standard #7 0 27. Percent of school officials in target schools who demonstrate use of new and quality supervision and leadership techniques or tools c Custom 10% 54 Performance Indicator USDA Standard/ CRS Custom Baseline 28. Percent of children 6–23 months receiving a minimum acceptable diet c, d FFP #BL12 9% 29. Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance b Standard #17 0 30. Percent of parents who state their children had health-related school absences in the previous month c Custom 30% 31. Number of educational facilities (i.e. school buildings, classrooms, improved water sources, and latrines) rehabilitated/constructed as a result of USDA assistance b Standard #8 0 32. Number of students enrolled in school receiving USDA assistance c Standard #9 0 33. Number of schools that held an enrollment campaign b Custom 0 34. Percent of caregivers who report spending time on literacy activities with their school-age children in the previous week c Custom 42% 35. Number School Management Committee (SMC) and Parent Teacher Association (APE) members, and Mother Leaders trained on activities to promote literacy b Custom 0 36. Percent of community members who practice promoted early childhood practices and support their children’s education c Custom 20% 37. Quantity of take-home rations provided (in metric tons) as a result of USDA assistance b Standard #14 0 38. Number of individuals receiving take-home rations as a result of USDA assistanceb Standard #15 0 39. Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistanceb Standard #16 0 40. Number of social assistance beneficiaries participating in productive safety net as a result of USDA assistance b Standard #18 0 41. Number of individuals participating in group-based savings, micro-finance or lending programs with USDA assistance b, e FFPr Standard #6 0 42. Number of members of the interministerial steering committee conducting monitoring visits to targeted schools b Custom 0 43. Number of Early Grade Reading Assessments (EGRAs) administered by inspectors, pedagogical advisors, school administrators, and teachers in target schoolsb Custom 0 44. Number of policies, regulations, or administrative procedures in each of the following stages of development as a result of USDA assistance b Standard #10 0 45. Value of new USG commitments, and new public and private sector investments leveraged by USDA to support food security and nutrition b Standard #11 0 46. Number of Parent Teacher Associations (APE) or similar school governance structure supported as a result of USDA assistance b Standard #13 0 47. Number of public private partnerships formed as a result of USDA assistance b Standard #12 0 a Collected by only external evaluator b Collected only by CRS; triangulated by external evaluator c Collected by external evaluator; triangulated with CRS annual report data d USAID Food for Peace standard indicator e USDA Food for Progress standard indicator 4. Deliverables: The evaluator is expected to follow American Evaluation Association’s Guiding Principles for Evaluators (http://www.eval.org/p/cm/ld/fid=51). Dependent upon participants in the evaluation, the evaluator should specify steps that will be taken to ensure informed consent, confidentiality, and protection of minors. The evaluator should specify steps taken to safeguard data collected and data management procedures to be used in the evaluation. There will be a data rights clause in the signed contract, and the external evaluator should obtain permission from CRS before sharing the final evaluation report with any external party, including posting it to their organization’s website. 55 All deliverables should be completed in English (and data collection tools must also be in French), be free of typos or grammatical errors, and be a polished document ready for submission to USDA. This means the document contains no factual errors or inaccuracies and citations are properly used. Deliverables for baseline, midterm, and final include the following: • Work plan (including evaluator responsibilities for identifying, interviewing, contracting, training and overseeing a balanced team of male and female enumerators and enumerator supervisors). • Sampling plan, including if the sample sizes will differ from Annex 1, approved by CRS. • Instruments, data collection manual, and training materials for enumerators (i.e., focus group guides, key informant interview guide, observation checklist), approved by CRS. • Quality Assurance Plan (including training of enumerators and weekly check-ins during data collection, approved by CRS. • Conduct interview with USDA (it is expected USDA will facilitate this exercise by providing the contact person and the means of interview) • Data sets with accompanying codebook/data dictionary (original paper and/or electronic as well as final, clean electronic data sets with syntax).  If the evaluator provides .dta, .do, .sps, or .sav files, they must also provide open source file versions (.txt, .csv, .doc, etc.)  If part of a longitudinal design, an identifier file that links respondent PII with ID numbers in the data file(s)  Deidentified transcripts of selected interviews and focus groups and/or data files of coded sections of text from interviews and focus groups • Draft Report with one round of edits from CRS and another subsequent round from USDA • Final Report with the following sections:  Executive summary 2 to 3 pages (including brief introduction of program evaluated, key evaluation questions, findings, and conclusions);  Background;  Evaluation questions;  Evaluation design including assumptions and limitations;  Methodology;  Findings;  Conclusions, lessons learned and effective practices (if any), and  Recommendations (should be clear, concise, relevant, specific and practical, following directly from findings and conclusions established in report);  Annex with original scope of work (marked for redaction from final web version);  Annex with final data collection instruments;  Annex with description of team members’ qualifications and their positionality;  Annex with additional methodological discussion/ robustness checks as needed;  Annex with updated IPTT. • Final reports must not contain any propriety or personally identifiable information (PII). PII is any information that directly or indirectly identifies an individual. This information can be used on its own or with other information to identify, contact or locate a single person, or to identify an individual in a specific situation. This may include, for example, a name, national ID number, address, birthplace, etc. PII includes both direct and indirect identifiers that, when taken together, could allow for identification of an individual (such as a village name, gender, age, name, and/ or facial image).” 56  In addition, final reports should not allow for the identification of individual schools or communities. Any list of schools or communities provided should be included as in the report annex, so that it can be easily removed before submitting to USDA for external sharing. • Final reports must be compliant with Section 508 of the United States Access Board which requires that information and services are accessible to persons with disability. (See https//section 508.gov/create). • A two to four-page outward-facing summary document, with easily accessible graphics, highlighting the project’s key successes, for sharing with a larger audience • Presentation of final evaluation to stakeholders. This can occur before or after report submission to USDA, as long as any key feedback is incorporated into the final version of the report (that USDA posts to the Development Experience Clearinghouse). This can be done via an additional annex, if the report is in its final stages before this presentation is conducted. • A webinar of key findings and lessons learned for CRS globally and USDA (if requested). In addition, at baseline only, a 10-page preliminary report, suitable for presentation to USDA, 6 weeks after the end of data collection. The report will only contain: • An IPTT for the indicators with non-zero baseline values, including relevant disaggregates; • Enough information about the methodology to engender confidence in the data quality. This should include a list of the data collection tools, number and gender of people interviewed, any information about stratification, and any data limitations. Whenever possible, the preliminary report should simply refer to the approved ToR and/ or Evaluation Plan, rather than incorporate the information; • Annex with description of team members’ qualifications and their positionality. 5. Items provided to the external evaluator by CRS: • Use of CRS CommCare software license, if desired. Evaluator is free to use their preferred data collection platform. • Tablets for data collection. • Scales and stadiometers for anthropometric data collection as described in Special Study 3. • All Annexes to this ToR. 6. Main Evaluation Questions and Timetables: Sections 4 – 6 of Annex 1 outlines the timelines of the baseline, midterm, and final evaluations and present anticipated evaluation questions. 7. Evaluator Qualifications: Team must have the following qualifications a) Advanced Degree in social sciences with strong knowledge of statistics/ demography; b) Knowledge and experience in survey and sampling design; c) Experience managing complex and multi-sectoral evaluations; d) Knowledge of performance evaluations, especially in the education sector; e) Knowledge of the education sector; basic education in the development context; school feeding programs especially in West Africa, preferably Togo; f) Demonstrated experience in conducting evaluation surveys of similar nature, preferably for USDA-funded projects; g) Good verbal and written communication skills in English and French; h) Willingness to work in remote areas without electricity and running water. 8. Evaluation team, management and coordination: Section 9 of Annex 1 broadly describes evaluation management. In addition, please see Table 2 below 57 Table 2. Evaluation team members Team Member CRS Staff or hired independently by the evaluation firm Main Roles and Responsibilities External evaluator Hired independently Preside over the conduct of the entire evaluation, from methodology and tool development to training in the use of the tool to field testing, data collection, entry and analysis and report writing. Enumerators/data collectors Hired independently by the evaluation firm Receive training and undertake data collection in the field. Data Collection Supervisors Hired independently by the evaluation firm Receive training in data collection and supervise data collectors daily for the duration of the data collection exercise. Data entry clerks Hired independently by the evaluation firm Receive training in data entry and enter data collected from the field. Data Entry Supervisors Hired independently by the evaluation firm Receive training in data entry and supervise data entry clerks throughout the data entry exercise. CRS Togo Country Manager, CRS Benin/ Togo MEAL Coordinator CRS Staff Supports the entire evaluation process ensuring compliance on the part of the evaluation firm CRS MEAL Advisors in Central Africa and Baltimore CRS Staff Supports the entire evaluation process ensuring compliance on the part of the evaluation firm. 9. Structure of Proposal and Submission Guidelines CRS will publish a request for bids (financial and technical proposals) for the conduct of the baseline, midterm and final evaluation of the STARS project to both domestically and internationally. Applicants should meet the qualifications stipulated in these ToR. The bid evaluation process will be managed by the Togo CRS Procurement Officer and the Central Africa Regional Technical Advisor (RTA) for MEAL and will follow the standard rules and procedures for the competitive and transparent procurement of consultancy services. The successful evaluator would be contracted to execute the baseline, midterm and final evaluation. However, retention of the evaluator to proceed with the midterm and/or final evaluation will depend on satisfactory performance of the baseline evaluation. CRS will re-launch the selection process for the midterm final evaluation where the baseline consultant(s) does not meet expectations. Key criteria that will be considered during the bid evaluation process will include the following: 1. Bidders must submit a technical proposal including a detailed description of the study design and methodology for the baseline. 2. Bidders must submit a detailed financial proposal for the baseline, midline, and final evaluation, and special studies, not exceeding $450,000 for the three data collection points. a. Please list a separate line item for Special Study 3 in Annex 1. 3. Bidders should submit a detailed work plan showing clearly how they wish to accomplish the study. 4. Profile of the bidders including relevant knowledge and experience to undertake the assignment 5. Bidders should have stated their relevant qualification and demonstrate relevant experience in the project area and experience in evaluating education programs. 58 6. Delivery timeline The proposal should contain no more than a total of 25 pages of which; technical proposal 20 pages and financial proposal 5 pages. See table 9 below. Table 3: Proposal layout and number of pages Proposal content layout Maximum pages Technical Proposal 20 Expression of interest 1 Table of content 1 Introduction and background 1 ½ Qualification and profile of team members 2 ½ Evaluation methodology 5 Evaluation questions 2 ½ Work plan and deliverables 2 ½ Technical reference of the firm 4 Financial Proposal 5 Summary 1 Detailed budget 3 Budget explanatory notes 1 Total 25 Sealed bids must be delivered in electronic and/or hard copy to: The CRS-Togo Office 01 BP 173 Hedzanawoe-Derriere Sito Aeroport Lomé, Togo Email: togo@global.crs.org The proposals must be submitted no later 23 October 2019 at midnight GMT. Bids for multiple awards. CRS currently also has an open bid for its newly awarded McGovern-Dole project in Guinea-Bissau and understands that some bidders may be interested in bidding for both contracts. The process is run separately in each country program. Applying for both contracts is acceptable, but country programs do consult each other in these processes. Thus, please note the following: 1) Given that timelines overlap, evaluators should clearly demonstrate they have the bandwidth to produce quality evaluations for both countries, either through expected LOE for overlapping staff members; different staff over specified dates; or the use of different study teams altogether. 2) Evaluators that are currently slated to conduct midterm or final evaluations for other CRS country programs during overlapping timeframes should also include clarity around point 1) above. Table 4. List of Annexes (attached as separate documents) Annex Number Document 1 STARS Evaluation Plan (Budget Information Redacted) 2 STARS Indicator Performance Tracking Table 3 CRS Report Review Template for USDA Evaluations 4 CRS Standard Tools 5 STARS Performance Monitoring Plan (PMP) Annex E: Data collection instruments EGRA – Letter Sound Identification b S un on V i m E ou e T r e oi m ê au P J en D O M z A C k R g L N S f a e y t U j an B d E v G ein eu c F B s I p Y K T R s A Z L o u c ei E in U qu V r é ai un s A b ain i ç e a è P gn L n u N n f ui L t E S I g L O D o ch EGRA – Nonword Reading abi tur gassolle autin bo glin ébale intour nari dère nal éna lon miède noque lanne carsun pouge toubête trond valle oupon tissonde movi mau oli jil aro nayo onda pému sarte cani dai norchant chotre enti souner gouma ravre rour tal fu dumolle brache rassan leul zein lagi doile flosse fape vur EGRA – Oral Reading Fluency Ali finit de balayer sa maison. Il a faim. Ali va au marché où il achète trois mangues. En rentrant chez lui, il tombe dans un trou. Ali laisse tomber les mangues. Elles roulent vers des chèvres. Les animaux commencent à manger les fruits. Ensuite, leurs visages deviennent oranges. Ali rit parce que les chèvres sont amusantes. Student Survey Variable Name Prompt Options SS_SLE_Trajet 1. En allant à et en rentrant de l’école, est￾ce que tu te sens: 1 - “pas en sécurité ?” 2 - “un peu en sécurité ? “ 3 - “en sécurité ? “ 4 - “très en sécurité ? “ 888 - “Refuse de répondre/Pas de réponse” SS_SLE_Ecole 2. À l’école, est-ce que tu te sens: 1 - “pas en sécurité ?” 2 - “un peu en sécurité ? “ 3 - “en sécurité ? “ 4 - “très en sécurité ? “ 888 - “Refuse de répondre/Pas de réponse” SS_SLE_Bienvenue 3. Est-ce que tu te sens bien à l’école ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_ECTM_PositiveGirl 4. Tes enseignants racontent-ils des histoires positives sur les personnages féminins, tels que les filles qui sont des leaders ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_ECTM_PostiveBoy 5. Tes enseignants racontent-ils des histoires positives sur les personnages de garçons, tels que les garçons qui sont des leaders ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_ECTM_Communaut e 6. Est-ce que tes devoirs te demandent d’interagir avec ta communauté ? (interviewer les membres de ta communauté, écrire des histoires sur la maison, mesurer le terrain agricole de ta famille pour les mathématiques, etc.) 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_ECTM_Vie 7. Ce que tu apprends à l’école aides-tu dans ta vie quotidienne ? 1 - “Ça ne t’aide pas” 2 - “Ça t’aide un peu” 3 - “Ça t’aide pas mal” 4 - “Ça t’aide beaucoup” 888 - “Refuse de répondre/Pas de réponse” SS_CCP_Groupe 8. Est-ce que tu travailles en petits groupes ou en paires pendant les cours ? 1 - “Rarement” 2 - “Parfois” Variable Name Prompt Options 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_CCP_Questions 9. Est-ce que tes enseignants t’encouragent à poser des questions à l’école ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_CCP_Pratique 10. As-tu le temps de pratiquer de nouveaux concepts en classe ? (au-delà de simplement écouter l’enseignant / copier des notes.) 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_SG_Question 11. Tes parents ou tuteurs t’interrogent-ils sur tes devoirs ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_SG_Lecture 12. Est-ce que quelqu’un dans ton ménage lit pour ou avec toi ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_SG_Performance 13. Tes parents / tuteurs ont-ils parlé à tes enseignants sur ta performance à l’école ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_SG_Langue 14. Est-ce que tes parents / tuteurs parlent français ? 1 - “Oui” 0 - “Non” 888 - “Refuse de répondre/Pas de réponse” SS_ST_aide 15. Est-ce que tes enseignants t’aident à mieux réussir à l’école ? 1 - “Les Enseignants ne t’aident pas” 2 - “Les Enseignants t’aident parfois” 3 - “Les Enseignants t’aident la plupart du temps” Variable Name Prompt Options 4 - “Les Enseignants t’aident tout le temps” 888 - “Refuse de répondre/Pas de réponse” SS_SG_aidentautres 16. Lorsqu’un élève en classe éprouve des difficultés ou prend du retard, est-ce que tes enseignants essaient de l’aider ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” latrine_confirm Ton école a-t-elle des latrines ou toilettes ? 1 - “Oui” 0 - “Non” SS_WASH_Toilettepou rFille 17. Est-ce que les toilettes / latrines pour filles de ton école sont accessibles pendant la journée scolaire ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_WASH_Toilettepou rGarcon 18. Est-ce que les toilettes / latrines pour garçons de ton école sont accessibles pendant la journée scolaire ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_WASH_LavageToile tteFille 19. Les filles aident-elles à nettoyer les toilettes / latrines de ton école ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_WASH_LavageToile tteGarcon 20. Les garçons aident-ils à nettoyer les toilettes / latrines de ton école ? 1 - “Rarement” 2 - “Parfois” 3 - “La plupart du temps” 4 - “Presque toujours” 888 - “Refuse de répondre/Pas de réponse” SS_WASH_AccesToilet te 21. Les toilettes / latrines de ton école sont-elles accessibles aux plus jeunes et aux handicapés ? 0 - “NON accessible aux plus jeunes ou aux handicapés” 1 - “Accessible aux plus jeunes OU aux handicapés” 2 - “Accessible aux plus jeunes ET aux handicapés” 888 - “Refuse de répondre/Pas de réponse” Variable Name Prompt Options Q_22 22. Combien de personnes vivent dans ta maison, y compris toi ? Q_23 23. Chez toi, y a-t-il une latrine ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_24 24. Chez toi, y a-t-il des livres ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_25 25. Chez toi, y a-t-il une source de courant ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_26 26. Y a-t-il le téléphone chez toi (fixe ou mobile) ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_27 27. Chez toi, y a-t-il une télévision ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_28 28. Chez toi, y a-t-il un vélo ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_29 29. Chez toi, y a-t-il une moto ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_30 30. Chez toi, y a-t-il une voiture ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_31 31. Chez toi, y a-t-il des poules/pintades ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” Variable Name Prompt Options 888 - “Refuse de répondre/Pas de réponse” Q_32 32. Chez toi, y a-t-il des chèvres ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_33 33. Chez toi, y a-t-il des vaches ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_34 34. Chez toi, y a-t-il un jardin ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_35 35. Y a-t-il d’autres enfants dans votre famille qui ne vont pas à l’école, mais qui sont assez âgés ? 0 - “Non” 1 - “Oui” 777 - “Ne sait pas” 888 - “Refuse de répondre/Pas de réponse” Q_36 36. Quel genre de travail fait ton père ? 0 - “Sans emploi” 1 - “Ménagère” 2 - “Travail agricole” 3 - “Propriétaire foncier” 4 - “Journalier(ière)” 5 - “Marchand(e)” 6 - “Travailleur(euse) de bureau” 7 - “Artisan(e)” 8 - “Retraité(e)” 777 - “Ne sait pas/ Pas de réponse” 555 - “Autre” Q_36_other Si autre, préciser Q_37 37. Quel genre de travail fait ta mère ? 0 - “Sans emploi” 1 - “Ménagère” 2 - “Travail agricole” 3 - “Propriétaire foncier” 4 - “Journalier(ière)” 5 - “Marchand(e)” 6 - “Travailleur(euse) de bureau” 7 - “Artisan(e)” 8 - “Retraité(e)” 777 - “Ne sait pas/ Pas de réponse” 555 - “Autre” Variable Name Prompt Options Q_37_other Si autre, préciser Measures_consent Maintenant, je souhaiterai mesurer ton poids avec cet instrument [montrez la balance]. Tu n’es pas obligé de participer si tu ne le veux pas. As-tu des questions? Peut-on mesurer ton poids? 1 - “Oui” 0 - “Non” Weight Poids de l’élève (en kilos) School Director Survey Variable Name Prompt Options director Êtes-vous le directeur/la directrice de l’école ? 1 - “Oui” 0 - “Non” director_other Quel est le rôle du répondant à l’école ? sex Le répondant est-il de sexe masculin ou féminin ? 1 - “Masculin” 0 - “Féminin Years_Teacher Depuis combien d’années êtes-vous dans l’enseignement ? Years_School Depuis combien d’années êtes-vous affecté(e) à cette école ? Years_Director Depuis combien d’années travaillez-vous en tant que directeur ? Q_1 1. Quelles classes avez-vous au sein de votre école ? 0 - “Maternelle” 1 - “CP1” 2 - “CP2” 3 - “CE1” 4 - “CE2” 5 - “CM1” 6 - “CM2” 555 - “Autre(s) “ Q_1_other Si autre, précisez. Q_2 2. L’école a-t-elle des classes combinées ? 1 - “Oui” 0 - “Non” Q_3_enroll enroll_1_m Nombre de garçons inscrits en CP1 enroll_1_f Nombre de filles inscrites en CP1 enroll_2_m Nombre de garçons inscrits en CP2 enroll_2_f Nombre de filles inscrites en CP2 enroll_3_m Nombre de garçons inscrits en CE1 enroll_3_f Nombre de filles inscrites en CE1 enroll_4_m Nombre de garçons inscrits en CE2 enroll_4_f Nombre de filles inscrites en CE2 enroll_5_m Nombre de garçons inscrits en CM1 Variable Name Prompt Options enroll_5_f Nombre de filles inscrites en CM1 enroll_6_m Nombre de garçons inscrits en CM2 enroll_6_f Nombre de filles inscrites en CM2 Q_4_attend attend_1_m Nombre de garçons présents en CP1 attend_1_f Nombre de filles présentes en CP1 attend_2_m Nombre de garçons présents en CP2 attend_2_f Nombre de filles présentes en CP2 attend_3_m Nombre de garçons présents en CE1 attend_3_f Nombre de filles présentes en CE1 attend_4_m Nombre de garçons présents en CE2 attend_4_f Nombre de filles présentes en CE2 attend_5_m Nombre de garçons présents en CM1 attend_5_f Nombre de filles présentes en CM1 attend_6_m Nombre de garçons présents en CM2 attend_6_f Nombre de filles présentes en CM2 teachers_total 5. Combien d’enseignants avez-vous dans cette école ? teachers_m A. Nombre d’enseignants de sexe masculins ? teachers_f B. Nombre d’enseignants de sexe féminin ? teach_attend_total 6. Combien d’enseignants sont présent(e)s aujourd’hui ? teach_attend_m A. Nombre d’enseignants de sexe masculin présents aujourd’hui ? teach_attend_f B. Nombre d’enseignants de sexe féminin présentes aujourd’hui ? teach_log 7. L’école dispose-t-elle d’un système d’enregistrement de la fréquentation quotidienne des enseignants, tel qu’un agenda quotidien ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” teach_time 8. En moyenne, combien d’heures par jour d’école les enseignants doivent-ils enseigner ? teach_house 9. Un logement est offert à vos enseignants ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” bureau_obs_1 a. Tableau de bord présence des enseignants 1 - “Oui” 0 - “Non” bureau_obs_2 b. La liste des taches des enseignants 1 - “Oui” 0 - “Non” bureau_obs_3 c. Supports visuels d’enseignement 1 - “Oui” Variable Name Prompt Options 0 - “Non” bureau_obs_4 d. Matériels didactiques 1 - “Oui” 0 - “Non” bureau_1 a. Livre inventaire 1 - “Oui” 0 - “Non” bureau_2 b. Dossiers scolaires 1 - “Oui” 0 - “Non” bureau_3 c. Journal de bord 1 - “Oui” 0 - “Non” bureau_4 d. Livre d’or 1 - “Oui” 0 - “Non” bureau_5 e. Comptes rendus de Conseils de classe/réunions pedagogiques 1 - “Oui” 0 - “Non” bureau_6 f. Cahier de présence des enseignants 1 - “Oui” 0 - “Non” textbooks 12. La classe de CP2 a-t-elle des manuels de lecture ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” textbooks_share 13. Dans les classes de CP2, combien d’élèves se partage un manuel de scolaire ? 1 - “1 enfant par manuel” 2 - “2 enfants par manuel” 3 - “3 enfants par manuel” 4 - “4 enfants par manuel” 5 - “5 et plus enfants par manuel” 888 - “Ne sait pas / Pas de réponse” textbook_storage 14. Où sont stockés les manuels ? 1 - “Dans le bureau du directeur” 2 - “En classe dans un placard verrouillé” 3 - “En classe sur une étagère ouverte” 4 - “Aux bureaux des élèves” 888 - “Ne sait pas / Pas de réponse” 555 - “Autre” textbook_storage_oth er Si autre, veuillez préciser. kitchen 15. Votre école a-t-elle une cantine fonctionnelle ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” APE 16. Votre école a-t-elle une APE ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” APE_why Pourquoi pas? 1 - “Les parents n’ont pas les moyens (argent)” Variable Name Prompt Options 2 - “Les parents n’ont pas le temps” 3 - “Les parents ne sont pas intéressés” 4 - “L’école ne souhaite pas avoir d’APE.” 888 - “Ne sait pas / Pas de réponse” 555 - “Autre” APE_why_other Si autre, veuillez préciser. APE_active 17. Est-t-elle active c’est à dire l’APE organise des réunions et tient des procès￾verbaux ? 1 - “Très active” 2 - “Modérément active” 3 - “Pas du tout active” 888 - “Ne sait pas / Pas de réponse “ APE_inactive_why Si pas du tout active, pourquoi pas? 1 - “Les parents n’ont pas les moyens (argent)” 2 - “Les parents n’ont pas le temps” 3 - “Les parents ne sont pas intéressés” 4 - “L’école ne souhaite pas avoir d’APE” 888 - “Ne sait pas / Pas de réponse” 555 - “Autre” APE_inactive_why_oth er Si autre, veuillez préciser. parentengage_school 18. Comment encouragez-vous l’engagement des parents à l’école ? 1 - “Réunion d’information via APE” 2 - “Activités de sensibilisation” 3 - “Rencontres avec le directeur de l’école” 4 - “Rencontres avec les enseignants” 0 - “Je ne fais rien.” 888 - “Ne sait pas / Pas de réponse” 555 - “Autre” parentengage_school_ other Si autre, veuillez préciser. parentengage_home 19. Comment encouragez-vous l’engagement des parents à la maison? 1 - “Réunion d’information via APE” 2 - “Activités de sensibilisation” Variable Name Prompt Options 3 - “Rencontres avec le directeur de l’école” 4 - “Rencontres avec les enseignants” 0 - “Je ne fais rien.” 888 - “Ne sait pas / Pas de réponse” 555 - “Autre” parentengage_home_ other Si autre, veuillez préciser. water_access 20. Votre école dispose-t-elle d’un accès à de l’eau ? 1 - “Oui, dans l’école” 2 - “Oui, à proximité de l’école” 3 - “Oui, mais loin de l’école” 0 - “Non” 888 - “Ne sait pas / Pas de réponse” water_type 21. Quel est le type du point d’eau ? 1 - “L’eau, si elle est présente, est apportée par les parents, les enfants, ou le personnel.” 2 - “Puits / source creusé non protégé, eau de pluie non traitée, eau de surface” 3 - “Chariot avec un petit réservoir/tambour, ou une source protégée.” 4 - “Eau courante, robinet public, eau de pluie traitée, puits creusé protégé ou eau en bouteille.” 555 - “Autre” 888 - “Ne sait pas/ Pas de réponse” water_type_other Si autre, veuillez préciser. water_drink 22. L’eau de l’école est-elle potable ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” water_function 23. La source d’eau est-elle fonctionnelle aujourd’hui ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” water_nofunction Si non, pourquoi pas? 1 - “La source est cassée.” 2 - “La source s’est tarie.” 555 - “Autre” water_nofunction_oth er Si autre, veuillez préciser. lat_access 24. Vos élèves ont-ils accès à des latrines à l’école ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” Variable Name Prompt Options lat_type 25. Quel type de latrines l’école a-t-elle ? 1 - “Latrines à fosse améliorées ventilées” 2 - “Toilettes à compostage” 3 - “Latrines à fosse avec dalle” 4 - “Rincer ou verser / rincer les installations” 5 - “Latrines à fosse” 888 - “Ne sait pas / Pas de réponse” 555 - “Autre” lat_type_other Si autre, veuillez préciser. lat_function 26. Les latrines sont-elles fonctionnelles ? 1 - “Oui, vraiment” 2 - “Oui, plus ou moins” 3 - “Non, pas vraiment” 4 - “Non, pas du tout” 888 - “Ne sait pas / Pas de réponse” lat_suff 27. Le nombre de latrines est-il suffisant ? 1 - “Oui, vraiment” 2 - “Oui, plus ou moins” 3 - “Non, pas vraiment” 4 - “Non, pas du tout” 888 - “Ne sait pas / Pas de réponse” lat_girls 28. Les filles ont-elles leurs propres latrines ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” lat_teachers 29. Existent-t-ils des latines réservées uniquement pour les enseignants ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” wash_access 30. Existent-ils des systèmes de lavage de mains à côté des latrines ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” wash_soap 31. Existe-il du savon permanament au niveau du dispositif de lavage des mains ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” wash_water 32. Existe-il de l’eau en permanence dans le dispositif de lavage des mains ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas/Pas de réponse” Classroom & School Observations Classroom Portion Variable Name Prompt Options Class 2. Quelle classe observez-vous aujourd’hui? 0 - “Maternelle” 1 - “CP1” Variable Name Prompt Options 2 - “CP2” 3 - “CE1” 4 - “CE2” 5 - “CM1” 6 - “CM2” Class_enroll 3. Combien d’élèves sont inscrits dans la classe que vous observez aujourd’hui ? CO_Inscr_Garcons 3a. Nombre total de garçons inscrits dans la classe qui sera observée CO_Inscr_Filles 3b. Nombre total de filles inscrites dans la classe qui sera observée CO_Presents_Garcons 4. Nombre de garçons présents [Demandez à tous les garçons de se lever et de les comptez les] CO_Presentes_Filles 5. Nombre de filles présentes [Demandez à toutes les filles de se lever et de les comptez les] CO_Presents_Adultsqu itravaillent 6. Nombre d’enseignants / assistants d’enseignement / autres adultes présents dans la classe et travaillant avec des enfants? [Entrez le nombre] CO_ECTM_Math 7a. Possibilités d’apprentissage pour soutenir le développement des compétences en mathématiques (sens des nombres, temps, formes, couleurs, séquence, taille) 1 - “Aucune activité mathématique n’est observée.” 2 - “L’enseignant enseigne les concepts mathématiques UNIQUEMENT en: • Activités répétitives. Les exemples incluent la réponse de groupe à des questions fermées (comme compter jusqu’à dix); enfants individuels utilisant un pointeur pour nommer des nombres; écrire ou copier des nombres” 3 - “L’enseignant enseigne les concepts mathématiques en utilisant UNE des stratégies suivantes: • Les enfants explorent et jouent avec des objets concrets pour apprendre le concept • Les enfants ont le choix sur la façon de mener une activité Variable Name Prompt Options • L’enseignant engage les enfants dans la discussion et utilise parfois des questions ouvertes • L’enseignant relie la leçon aux expériences de la vie réelle ou de tous les jours” 4 - “L’enseignant enseigne les concepts mathématiques en utilisant DEUX OU PLUSIEURS des stratégies suivantes: • Les enfants explorent et jouent avec des objets concrets pour apprendre le concept • Les enfants ont le choix sur la façon de mener une activité • L’enseignant engage les enfants dans la discussion et utilise parfois des questions ouvertes • L’enseignant relie la leçon aux expériences de la vie réelle ou de tous les jours” CO_ECTM_PlanMath 7b. Vérifiez si l’enseignant se réfère à un plan de cours pour structurer son enseignement des mathématiques 1 - “Oui” 0 - “Non” CO_ECTM_Alphabetisa tion 8a. Possibilités d’apprentissage pour soutenir le développement des compétences en alphabétisation (identification des lettres, phonétique). 1 - “Aucune activité d’alphabétisation n’est observée.” 2 - “L’enseignant enseigne les concepts d’alphabétisation UNIQUEMENT en: • Activités répétitives. Les exemples incluent la réponse du groupe à des questions fermées (telles que chanter l’alphabet, répéter les sons des lettres); enfants individuels utilisant un pointeur pour nommer des lettres; écrire ou copier des lettres” 3 - “L’enseignant enseigne les concepts d’alphabétisation en utilisant UNE des stratégies suivantes: • Les enfants explorent et jouent avec des objets concrets pour apprendre le concept • Les enfants ont le choix sur la façon de mener une activité Variable Name Prompt Options • L’enseignant engage les enfants dans la discussion et utilise parfois des questions ouvertes • L’enseignant relie la leçon aux expériences de la vie réelle ou de tous les jours” 4 - “L’enseignant enseigne les concepts d’alphabétisation en utilisant DEUX OU PLUSIEURS des stratégies suivantes: • Les enfants explorent et jouent avec des objets concrets pour apprendre le concept • Les enfants ont le choix sur la façon de mener une activité • L’enseignant engage les enfants dans la discussion et utilise parfois des questions ouvertes • L’enseignant relie la leçon aux expériences de la vie réelle ou de tous les jours” CO_ECTM_PlanAlphab etisation 8b. Vérifiez si l’enseignant se réfère à un plan de cours pour structurer son enseignement de l’alphabétisation. 1 - “Oui” 0 - “Non” CO_ECTM_LangageExp 9a. Possibilités d’apprentissage pour développer des compétences linguistiques expressives. Ce sont des conversations qui ont lieu entre les enseignants et les enfants tout au long des observations. Les conversations peuvent avoir lieu pendant les leçons, ou entre les leçons (lors du passage d’une activité à une autre; pendant le jeu libre, etc.) 1 - “Les enfants ne sont jamais ou rarement invités à raconter une histoire, à décrire des événements ou des objets, ou à répondre à des questions tout au long de l’observation.” 2 - “L’enseignant encourage les compétences linguistiques expressives UNIQUEMENT en: • Activités répétitives. Les exemples incluent la réponse de groupe à des questions fermées (comme demander aux enfants de répéter une histoire ou des phrases mot par mot); chaque enfant utilise un pointeur pour répéter des mots ou des phrases; réponses individuelles à des questions par cœur ou fermées.” 3 - “L’enseignant encourage les compétences linguistiques expressives en utilisant UNE Variable Name Prompt Options activité d’échange verbal, telle que: • Demander aux enfants de décrire des objets (par exemple, couleur, forme, taille, fonction) ou des images; • Encourager les enfants à raconter des histoires ou à décrire des événements; • Raconter une histoire et poser aux enfants deux ou plusieurs questions ouvertes sur l’histoire • Répéter et étendre ce que dit l’enfant, et inclure un vocabulaire plus avancé • Utiliser des histoires ou des discussions pour encourager un vocabulaire qui établit des liens avec la vie et les expériences des enfants.” 4 - “L’enseignant encourage les compétences linguistiques expressives en utilisant DEUX OU PLUSIEURS activités d’échange verbal, telles que: • Demander aux enfants de décrire des objets (par exemple, couleur, forme, taille, fonction) ou des images; • Encourager les enfants à raconter des histoires ou à décrire des événements; • Raconter une histoire et poser aux enfants deux ou plusieurs questions ouvertes sur l’histoire • Répéter et étendre ce que dit l’enfant, et inclure un vocabulaire plus avancé • Utiliser des histoires ou des discussions pour encourager un vocabulaire qui établit des liens avec la vie et les expériences des enfants” CO_ECTM_LangueParl ee 9b. Vérifiez si l’enseignant parle en français. 1 - “Oui” 0 - “Non” Variable Name Prompt Options CO_ECTM_Livre 10. Lecture de livres pour aider les enfants à écouter et à parler 1 - “Pour le developpment des touts-petits – CP1 et maternelle – l’enseignant: • Ne lit pas les livres aux enfants OU • Lit des livres qui ne sont pas adaptés à l’âge (c.-à-d. Des textes ou des manuels scolaires pour les enfants plus âgés ou les adultes; des textes religieux pour les adultes; ou des livres sans images). Pour les classes des plus âgés – CP2 ou plus – les elèves: • Ne lisent pas le texte OU • Lisent des textes qui ne conviennent pas à leur âge (c.-à-d. De textes ou des manuels scolaires pour les jeunes enfants; des livres d’images).” , 2 - “Pour le développement des tout-petits – CP1 et maternelle, l’enseignant: • Lit à la classe sans discussion OU • Lit à la classe sans aucune question sur la lecture. Pour les classes des plus âgés – CP2 ou plus – l’enseignant: • Ne discute pas de la lecture OU • Ne pose pas de questions sur la lecture.” 3 - “L’enseignant discute de la lecture avec la classe en utilisant UNE des stratégies suivantes: • Pose des questions élémentaires aux enfants ou des questions fermées sur ce qui s’est passé • Encourage les enfants à discuter de la lecture à travers des questions ouvertes • Parle du vocabulaire appris dans le livre • Relie la lecture aux expériences ou au contexte des enfants • Les enfants jouent avec des objets ou font une activité liée à la lecture” Variable Name Prompt Options 4 - “L’enseignant discute de la lecture avec la classe en utilisant DEUX OU PLUSIEURS des stratégies suivantes: • Pose des questions élémentaires aux enfants ou des questions fermées sur ce qui s’est passé • Encourage les enfants à discuter de la lecture à travers des questions ouvertes • Parle du vocabulaire appris dans le livre • Relie la lecture aux expériences ou au contexte des enfants • Les enfants jouent avec des objets ou font une activité liée à la lecture” CO_ECTM_MotricFine 11. Opportunités d’apprentissage pour promouvoir la motricité fine: Ecriture, Dessin/coloriage, Collecte de petits objets, Mettre en ordre des petits objets, Tissage, Enfiler des perles. 1 - “Aucune activité motricité fine n’est observée.” 2 - “ L’enseignant enseigne la motricité fine UNIQUEMENT par l’utilisation : • Des activités qui ne sont PAS adaptées au développement de l’enfant (c’est-à-dire qu’elles sont trop difficiles ou trop faciles à comprendre ou à faire pour la plupart des enfants, par exemple utiliser des crayons pour tracer des lignes avant de commencer avec des crayons ou des marqueurs). 3 - “ L’enseignant enseigne la motricité fine en utilisant des activités adaptées au développement MAIS : • Les activités sont axées sur l’accomplissement de la tâche définie par l’enseignant plutôt que sur le développement de sa motricité fine. • Les activités se concentrent sur le produit, et non sur le processus. • Les activités ne sont pas dirigées par les enfants ; les enfants n’ont pas le choix de ce qu’ils doivent Variable Name Prompt Options faire ou de la manière dont ils doivent utiliser les matériaux.” 4 - “L’enseignant enseigne la motricité fine en utilisant des activités adaptées au développement ET: • Des activités orientées vers les enfants et axées sur le processus plutôt que sur un objectif • Des activités qui permettent aux enfants d’explorer les matériaux et la façon dont ils peuvent être manipulés de manière ludique. 5 - “N’est pas applicable” CO_ECTM_MotriGloba le 12. Des possibilités d’apprentissage qui permettent aux enfants de s’adonner à des activités de motricité globale: La course, L’étirement, La danse, Les Jeux de balle, Jeux de chasse. 1 - “Aucune activité motricité brute n’est observée. 2 - “Moins de 10 minutes d’activité motricité globale sont observées ou seuls quelques enfants y participent.” 3 - “Moins de 20 minutes d’activité motricité globale sont observées OU moins de la moitié des enfants y participent.” 4 - “La plupart des enfants pratiquent au moins 20 minutes d’activité motricité globale” CO_ECTM_JeuLibre 13. Activités d’apprentissage qui favorisent le choix libre ou le jeu ouvert: Explorez les centres d’activités en classe, Jeux autogérés en petits groupes, Le jeu peut être à l’intérieur ou à l’extérieur de la salle de classe 1 - “Aucune activité de choix libre / jeu ouvert n’est observée.” 2 - “L’enseignant choisit le lieu ou comment les enfants joueront avec le matériel OU l’enseignant propose un choix limité d’activités ET les enfants doivent jouer avec le matériel d’une manière prescrite.” 3 - “Les enfants ont UNE occasion de choisir leur propre activité, où et comment ils jouent avec les matériaux MAIS l’enseignant n’interagit pas pour ajouter au jeu des enfants ou prolonger l’apprentissage” 4 - “Les enfants ont UNE ou plusieurs occasions de choisir leur propre activité et où et comment ils jouent avec du matériel ET Variable Name Prompt Options l’enseignant interagit pour ajouter au jeu des enfants ou prolonger l’apprentissage.” CO_ECTM_Mouvemen t 14. Possibilités d’apprentissage qui permettent aux enfants de participer à des activités de musique / mouvement: Chanter des chansons, Danse, Jouer et être acteur, Chansons / danses de groupe, ensemble ou à tour de rôle, Comptines, Clips musicaux éducatifs. 1 - “Aucune activité de musique / mouvement n’est observée.” 4 - “Au moins une activité de musique ou de mouvement s’est produite pendant l’observation.” CO_CCP_Attentive 15. Les enfants sont engagés tout au long de l’observation. Les exemples d’engagement incluent faire attention, regarder l’enseignant, se concentrer sur la leçon ou le travail, participer aux activités. 1 - “Peu d’enfants (25% ou moins) sont engagés pour la plupart de l’observation” 2 - “Certains enfants (26% à 50%) sont engagés pour la plupart de l’observation” 3 - “La plupart des enfants (51% à 75%) sont engagés pour la plupart de l’observation” 4 - “Presque tous des enfants (76% à 100%) sont engagés pour la plupart de l’observation” CO_CCP_Groupe 16. Groupes. Les types de regroupement incluent: Groupe entier (classe entière), Petits groupes (trois ou plus), Paires (deux élèves) travaillant ensemble, Elèves travaillant seuls. 1 - “Un type de regroupement est utilisé tout au long de l’observation.” 2 - “Deux types de regroupement sont utilisés tout au long de l’observation” 3 - “Trois types de regroupement sont utilisés tout au long de l’observation” 4 - “Les quatre groupes sont formés tout au long de l’observation” CO_ST_Individuel 17. L’enseignant donne des instructions individualisées aux enfants 1 - “Enseignant : • Ne montre AUCUNE prise de conscience que certains enfants ont des besoins et des capacités différents (l’enseignant utilise une approche «taille unique» où tous les enfants font le même travail et reçoivent la même instruction et le même soutien, ignore l’enfant qui se débat, ne fait aucune adaptation pour les enfants avec besoins spéciaux)” Variable Name Prompt Options 2 - “Enseignant : •Montre occasionnellement une prise de conscience des besoins individuels des enfants en vérifiant la compréhension des concepts et en fournissant un soutien minimal.” 3 - “Enseignant: • Recherche les enfants qui éprouvent des difficultés et leur apporte de l’aide (avec ou sans demande d’aide spécifique) OU • Recherche les enfants qui ne sont pas mis au défi et leur propose des activités ou des questions appropriées au développement pour les maintenir engagés.” 4 - “Enseignant: • Recherche les enfants qui éprouvent des difficultés et leur apporte de l’aide (avec ou sans demande d’aide spécifique) ET • Recherche les enfants qui ne sont pas mis au défi et leur propose des activités ou des questions appropriées au développement pour les maintenir engagés” CO_TLM_Ecrire 18. Instrument d’écriture (crayons, stylos, crayons, craie) 1 - “Aucun matériel présent” 2 - “Matériaux présents MAIS les enfants ne les utilisent pas” 4 - “Le matériel est présent ET les enfants les utilisent” CO_TLM_Jouets 19. Jouets éducatifs ou matériel mathématique (capsules de bouteille, dés, eau, perles, roches, boulier, matériaux utilisés pour compter ou trier, puzzles, jeux) 1 - “Aucun matériel présent” 2 - “Matériaux présents MAIS les enfants ne les utilisent pas” 4 - “Le matériel est présent ET les enfants les utilisent” CO_TLM_Texte 20. Textes (livres avec images (jeunes), texte, etc., y compris ceux rédigés par l’enseignant) 1 - “Aucun matériel présent” 2 - “Matériaux présents MAIS les enfants ne les utilisent pas” 4 - “Le matériel est présent ET les enfants les utilisent” CO_TLM_LivreInstructi on_francais 21a. Nombre de manuels scolaires de français 1 - “25% ou moins des élèves actuels (Rapport 1: 4)” Variable Name Prompt Options 2 - “26 à 50% des élèves actuels (Rapport 1: 2)” 3 - “51 à 75% des élèves actuels (Rapport 3: 4)” 4 - “76 à 100% des élèves actuels (Rapport 1: 1)” CO_TLM_LivreInstructi on_math 21b. Nombre de manuels scolaires de mathématiques 1 - “1- 25% ou moins des élèves actuels (Rapport 1: 4)” 2 - “26 à 50% des élèves actuels (Rapport 1: 2)” 3 - “51 à 75% des élèves actuels (Rapport 3: 4)” 4 - “76 à 100% des élèves actuels (Rapport 1: 1)” School Portion Variable Name Prompt Options attendcount_1_m Nombre de garçons présents en CP1 attendcount_1_f Nombre de filles présentes en CP1 attendcount_2_m Nombre de garçons présents en CP2 attendcount_2_f Nombre de filles présentes en CP2 attendcount_3_m Nombre de garçons présents en CE1 attendcount_3_f Nombre de filles présentes en CE1 attendcount_4_m Nombre de garçons présents en CE2 attendcount_4_f Nombre de filles présentes en CE2 attendcount_5_m Nombre de garçons présents en CM1 attendcount_5_f Nombre de filles présentes en CM1 attendcount_6_m Nombre de garçons présents en CM2 attendcount_6_f Nombre de filles présentes en CM2 Q1 L’école dispose-t-elle d’une cantine ? 1 - “Oui” 0 - “Non” Q2 La cantine est-elle bien équipée ? 4 - “Oui, très bien” 3 - “Oui, plutôt” 2 - “Assez bien” 1 - “Pas vraiment” 0 - “Non, pas du tout” Q3 La cantine est-elle propre ? 4 - “Oui, très propre” 3 - “Oui, plutôt” 2 - “Assez propre” 1 - “Pas vraiment” 0 - “Non, pas du tout” Q4 L’école dispose-t-elle d’un magasin ? 1 - “Oui” Variable Name Prompt Options 2 - “Oui, mais non-accessible fermé” 0 - “Non” Q5 Le magasin est-il propre ? 4 - “Oui, très propre” 3 - “Oui, plutôt” 2 - “Assez propre” 1 - “Pas vraiment” 0 - “Non, pas du tout” Q6 Le magasin est-il bien rangé ? 4 - “Oui, très bien” 3 - “Oui, plutôt” 2 - “Assez bien” 1 - “Pas vraiment” 0 - “Non, pas du tout”“ CO_WASH_Engage Eau potable 1 - “Pas d’eau disponible à l’école. L’eau, si elle est présente, est apportée par les parents, les enfants, ou le personnel.” 2 - “L’eau disponible est : Puits/source creusée non protégée, eau de pluie non traitée, eau de surface.” 3 - “L’eau disponible est un chariot avec un petit réservoir/tambour ou une source protégée.” 4 - “La source d’eau sanitaire disponible est l’eau courante, le robinet public, l’eau de pluie traitée, le puits creusé protégé ou l’eau en bouteille.” CO_WASH_EauFonctio nne Vérifier si la source est fonctionnelle aujourd’hui 1 - “Oui” 0 - “Non” CO_WASH_LavageMai n Installations pour le lavage des mains 1 - “Pas de station de lavage des mains à l’école.” 2 - “Bassin ou seau partagé (le lavage des mains se fait dans l’eau, l’eau ne coule pas ou n’est pas versée).” 3 - “Système à verser à la main avec de l’eau usée séparée de l’eau pour se nettoyer les mains mais sans savon.” 4 - “Il existe de l’eau courante OU un système à verser à la main (avec l’eau usée séparée de l’eau propre pour se nettoyer les mains) ET du savon.” Variable Name Prompt Options CO_WASH_AccesLavag eMain Accessibilité aux installations de lavage des mains 1 - “NON accessible aux plus jeunes ou aux handicapés.” 3 - “Accessible aux plus jeunes OU aux handicapés. 4 - “Accessible ET aux plus jeunes et aux handicapés.” CO_WASH_Toilettes Toilettes 1 - “Pas de toilettes disponibles (uniquement en brousse ou dans les champs).” 3 - “Les toilettes sont des latrines à fosse ou des seaux.” 4 - “Les toilettes sont des toilettes à compostage.” CO_WASH_ToiletteOu verte Vérifiez si les toilettes sont ouvertes/utilisées par les élèves aujourd’hui 1 - “Oui” 0 - “Non” CO_WASH_EtatToilett e Etat des Toilettes • Les toilettes sont propres • Les toilettes sont séparées par sexe • Il y au minimum une cabine pour 50 garçons et une cabine pour 25 filles • Les toilettes sont accessibles aux plus jeunes enfants • Les toilettes sont accessibles aux enfants handicapés • Il y a une cabine, avec l’eau, pour la gestion de l’hygiène menstruelle pour les filles et une pour les enseignants 1 - “Aucune condition n’est remplie.” 2 - “Une condition est remplie.” 3 - “Deux conditions sont remplies.” 4 - “Trois ou plus conditions sont remplies.” CO_WASH_PratiqueLa vageMain Pratiques de lavage des mains (Pendant la pause recréation, observez si les enfants se lavent les mains avant de manger ou après avoir utilisé les latrines. Utilisez la feuille de comptage dans le formulaire vierge d’observation de la classe pour vos notes et vos calculs.) 1 - “Les enfants ne se lavent pas les mains ou seuls quelques enfants se lavent les mains (25 % ou moins).” 2 - “Le lavage des mains est sporadique (26 à 50 %) OU plus de 50% des enfants se lavent les mains, mais sans savon ni cendre.” 3 - “51 à 75 % des enfants se lavent les mains avec du savon ou de la cendre. Il existe un système ou un processus de soutien au lavage des mains (l’enseignant supervise, encourage, fait partie de la routine, etc.)” 4 - “Presque tous les enfants (76 % à 100%) se lavent les mains avec du savon ou de la cendre. Il existe Variable Name Prompt Options un système ou un processus de soutien au lavage des mains (l’enseignant supervise, encourage, fait partie de la routine, etc.)” Parent Survey Variable Name Prompt Options SEX Le répondant est-il de sexe masculin ou féminin ? 1 - “féminin” 0 - “masculin” AGE Quel âge avez-vous ? LANGUAGE Parlez-vous couramment le français ? 1 - “Oui” 0 - “Non” 777 - “Ne sait pas / pas de réponse” Q_1 1. Combien de personnes vivent avec vous, y compris vous-même? Par example, les gens qui mange ensemble. Q_2 2. Combien de filles avez-vous ? Q_3 3. Combien de vos filles sont inscrites dans cette école ? Q_4 4. En quelles classes sont-elles ? 0 - “Maternelle” 1 - “CP1” 2 - “CP2” 3 - “CE1” 4 - “CE2” 5 - “CM1” 6 - “CM2” 777 - “Ne sait pas / Pas de réponse” Q_5 5. Combien de garçons avez-vous ? Q_6 6. Combien de vos garçons sont inscrits dans cette école ? Q_7 7. En quelles classes sont-ils ? 0 - “Maternelle” 1 - “CP1” 2 - “CP2” 3 - “CE1” 4 - “CE2” 5 - “CM1” 6 - “CM2” 777 - “Ne sait pas / Pas de réponse” Q_8 8. L’un de vos enfants a-t-il manqué l’école au cours du dernier mois? 1 - “Oui” 0 - “Non” 777 - “Ne sait pas / pas de réponse” Q_9 9. Si oui, pourquoi ont-ils manqué l’école? 1 - “Maladie” 2 - “Travail à la maison” Variable Name Prompt Options 3 - “Est allé(e) chercher de l’eau” 4 - “Travaux agricoles” 5 - “Surveillance du bétail” 6 - “Pas d’argent pour les frais de scolarité” 7 - “L’enfant ne voulait pas y aller” 555 - “Autre” 777 - “Ne sait pas / Pas de réponse” Q_9_other Si autre, préciser Q_10 10. Quelles langues parlez-vous principalement à la maison ? 1 - “Français” 2 - “Kabye” 3 - “Gourma” 4 - “Ngam-gam” 5 - “Tchokossi” 6 - “Konkomba” 7 - “Bassar” 555 - “Autre” 888 - “Pas de réponse” Q_10_other Si autre, préciser Q_11 11. Quel est le niveau de scolarité le plus élevé que vous avez atteint ? 0 - “Aucun” 1 - “Primaire” 2 - “Secondaire” 3 - “Lycée” 4 - “Université” 5 - “Diplôme” 555 - “Autre” 888 - “Pas de réponse” Q_11_other Si autre, préciser Q_12 12. Quelle est votre profession principale ? 0 - “Sans emploi” 1 - “Ménagère” 2 - “Travail agricole” 3 - “Propriétaire foncier” 4 - “Journalier(ière)” 5 - “Marchand(e)” 6 - “Travailleur(euse) de bureau” 7 - “Artisan(e)” 8 - “Retraité(e)” 555 - “Autre” 888 - “Pas de réponse” Q_12_other Si autre, préciser Q_13 13. Se laver les mains avant de manger peut permettre d’éviter la diarrhée. 1 - “Vrai” 0 - “Faux” 777 - “Ne sait pas / Pas de réponse” Q_14 14. Marcher pieds nus peut causer des maladies. 1 - “Vrai” 0 - “Faux” Variable Name Prompt Options 777 - “Ne sait pas / Pas de réponse” Q_15 15. Il n’y a aucun moyen de prévenir la mort d’un enfant à cause de la diarrhée. 1 - “Vrai” 0 - “Faux” 777 - “Ne sait pas / Pas de réponse” Q_16 16. On se lave les mains avec du savon pour retirer les microbes et éviter qu’ils se retrouvent sur la nourriture. 1 - “Vrai” 0 - “Faux” 777 - “Ne sait pas / Pas de réponse” Q_17 17. Une alimentation constituée uniquement de riz et d’œuf est équilibrée. 1 - “Vrai” 0 - “Faux” 777 - “Ne sait pas / Pas de réponse” Q_18 18. Il est suffisant de rincer le bidon qui contient l’eau à boire avec de l’eau pour qu’il soit propre. 1 - “Vrai” 0 - “Faux” 777 - “Ne sait pas / Pas de réponse” Q_19 19. Le meilleur moyen d’éviter les maladies est de se laver les mains avec de l’eau et du savon avant de manger et après être allé aux toilettes. 1 - “Vrai” 0 - “Faux” 777 - “Ne sait pas / Pas de réponse” Q_20 20. Pour améliorer la qualité de l’eau de boisson, on peut ajouter un peu d’eau de javel/chlor. 1 - “Vrai” 0 - “Faux” 777 - “Ne sait pas / Pas de réponse” Q_21 21. On se brosse les dents uniquement pour que notre bouche sente bon. 1 - “Vrai” 0 - “Faux” 777 - “Ne sait pas / Pas de réponse” Q_22 22. L’alimentation équilibrée est importante pour assurer la bonne santé des enfants. 1 - “Vrai” 0 - “Faux” 777 - “Ne sait pas / Pas de réponse” Q_23 23. Selon vous, qu’est ce qui constitue une alimentation équilibrée ? 1 - “manger des céréales” 2 - “manger des tubercules” 3 - “manger des proteines (viande, poisson, oeuf)” 4 - “manger des legumineuses (Haricot, Niébé, soja,…)” 5 - “manger des aliments contenant des vitamines” 6 - “manger des fruits” 777 - “Ne sait pas / Pas de réponse” 555 - “Autre” Q_23_other Si autre, préciser Q_24 24. Avez-vous ces aliments dans vos repas quotidiens ? 1 - “Oui, toujours” 2 - “Oui, La plupart du temps” 3 - “Non, Rarement” 4 - “Non, Jamais” 777 - “Ne sait pas / Pas de réponse” Q_25 25. Pour quelles raisons ne mettez-vous pas systématiquement ces aliments dans vos repas ? 1 - “Je ne connais pas les règles” 2 - “Cela ne m’intéresse pas” Variable Name Prompt Options 3 - “Ma famille n’a pas les moyens d’acheter certains aliments” 4 - “Nous n’avons accès aux fruits et légumes quand cela n’est pas la saison” 5 - “La priorité c’est d’avoir le ventre plein” 6 - “Cela prend trop de temps” 7 - “J’oublie /Je n’y pense pas” 777 - “Ne sait pas / Pas de réponse” 555 - “Autre” Q_25_other Si autre, préciser Q_26 26. Quand est-ce que vous vous lavez les mains? 1 - “Après avoir utilisé les toilettes” 2 - “Avant de manger” 3 - “Après avoir lavé les enfants/et les couches culottes” 4 - “Après le nettoyage des latrines” 5 - “Après le nettoyage de pot” 6 - “Avant la préparation du repas” 7 - “Après le repas” 8 - “Après avoir travaillé dans les champs” 9 - “Jamais” 555 - “Autre” 777 - “Ne sait pas / Pas de réponse” Q_26_other Si autre, préciser Q_27 27. Qu’est-ce que vous utilisez pour vous laver les mains ? 1 - “Savon” 2 - “Liquide vaisselle” 3 - “Cendre” 4 - “Feuilles de citron” 0 - “Ne se lave pas les mains” 555 - “Autre” 777 - “Ne sait pas / Pas de réponse” Q_27_other Si autre, préciser Q_28 28. Vous-même (ou l’autre parent) racontez￾vous des histoires à vos enfants ? 1 - “Oui” 0 - “Non” 777 - “Ne sait pas / pas de réponse” Q_29 29. Avec quelle fréquence ? 4 - “Tous les jours” 3 - “2 à 3 fois par semaine” 2 - “1 fois par semaine” 1 - “Quelque fois par mois” 777 - “Ne sait pas / Pas de réponse” Q_30 30. Est-ce que vos enfants vous lisent à haute voix à la maison ? 1 - “Oui” 0 - “Non” 777 - “Ne sait pas / pas de réponse” Variable Name Prompt Options Q_31 31. Si oui, avec quelle fréquence ? 4 - “Tous les jours” 3 - “2 à 3 fois par semaine” 2 - “1 fois par semaine” 1 - “Quelque fois par mois” 777 - “Ne sait pas / Pas de réponse” Q_32 32. Quand vos enfants rentrent de l’école, leur demandez-vous ce qu’ils ont appris ? 1 - “Oui” 0 - “Non” 777 - “Ne sait pas / pas de réponse” Q_33 33. Avez-vous aidé vos enfants avec leurs devoirs dans la semaine passée ? 1 - “Oui” 0 - “Non” 777 - “Ne sait pas / pas de réponse” Q_34 34. Pour quels types d’activités ? 1 - “Lire des lettres” 2 - “Lire des mots” 3 - “Lire un texte” 4 - “Mathématiques” 5 - “Faire réciter les leçons” 777 - “Ne sait pas / Pas de réponse” 555 - “Autre” Q_34_other Si autre, préciser Q_35 35. Quelqu’un d’autre dans votre famille les aide-t-il à faire leurs devoirs? 1 - “Oui” 0 - “Non” 777 - “Ne sait pas / pas de réponse” Q_36 36. Qui ? 1 - “Père” 2 - “Mère” 3 - “Frère/Soeur” 4 - “Grand-parent” 777 - “Ne sait pas / Pas de réponse” 555 - “Autre” Q_36_other Si autre, préciser Q_37 37. Êtes-vous la principale personne qui s’occupe d’un enfant âgé de 6 mois à 23 mois ? 1 - “Oui, mère” 2 - “Oui, père” 3 - “Oui, mère et père ensemble” 0 - “Non” Q_38 38. Quelle est la date de naissance de cet enfant ? Q_38_verify VÉRIFIER LA DATE DE NAISSANCE FOURNIE : L’enfant a-t-il/ elle entre 6 et 23 mois ? 1 - “Oui” 0 - “Non” Q_39 39. Quelle est son nom ? Q_40 40. Cet enfant, (NOM), est-il de sexe masculin ou féminin ? 1 - “Masculin” 0 - “Féminin” Q_41 41. Est-ce que (NOM) n’a jamais été nourri(e) au sein ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas / pas de réponse” Variable Name Prompt Options Q_42 42. Est-ce que (NOM) a été nourri(e) au sein hier, dans la journée ou la nuit ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas / pas de réponse” Q_43 43. Combien de fois est-ce que (NOM) a mangé hier des aliments solides, semi-solides ou mous autres que des liquides, dans la journée ou la nuit ? Q_44 A-t-on donné à (NOM) du/de (LIQUIDE DE LA LISTE) ? Q_44_a a. Eau ? 1 - “Oui” 0 - “Non” 888 - “Ne sait pas / pas de réponse” Q_44_b b. Préparations pour nourrissons, telle que France lait ? Q_44_c c. Lait en boîte, en poudre ou lait frais d’origine animale ? Q_44_d d. Jus ou boisson dérivée de jus ? Q_44_e e. Bouillon clair ? Q_44_f f. Yaourt ? Q_44_g g. Bouillie d’avoine diluée ? Q_45 Hier, durant la journée ou la nuit, est-ce que (NOM) a bu ou mangé du/de la/des (ALIMENTS DU GROUPE) ? Q_45_a a. Bouillie d’avoine, pain, riz, pâtes ou autres aliments dérivés de céréales 1 - “Oui” 0 - “Non” 888 - “Ne sait pas / pas de réponse” Q_45_b b. Potiron, carottes, courge ou patates douces à chair jaune ou orange Q_45_c c. Pommes de terre à chair blanche, ignames à chair blanche, manioc ou autres tubercules Q_45_d d. Tous légumes à feuilles vert foncé Q_45_e e. Mangues mûres, papayes mûres, néré, ronier, pastèque, ou orange? Q_45_f f. Autres fruits ou légumes Q_45_g g. Foie, rognon, cœur ou autres abats Q_45_h h. Viandes telles que bœuf, porc, agneau, chèvre, poulet ou canard Q_45_i i. Œufs Q_45_j j. Poisson frais ou séché, crustacés ou fruits de mer Q_45_k k. Plats ou aliments contenant des haricots, pois, lentilles, noix ou graines Q_45_l l. Fromage, yaourt ou autre produit laitier Q_45_m m. Huile, graisse ou beurre ou tout aliment en contenant Q_45_n n. Tous aliments sucrés tels que chocolats, bonbons, friandises, pâtisseries, gâteaux ou biscuits Variable Name Prompt Options Q_45_o o. Condiments aromatiques tels que piments, épices, herbes ou poudres de poisson Q_45_p p. Larves, escargots ou insectes Q_45_q q. Aliments préparés avec de l’huile de palme rouge, de la noix de palme rouge ou de la pulpe de noix de palme rouge Annex F: Key Survey Frequency Tables Classroom Observation – Teaching Practices Table F. 1. Learning opportunities to support the development of literacy skills CO_ECTM_Alphabetisation Frequency Percent No literacy lesson observed. 29 37.7% The teacher teaches literacy concepts ONLY in: • Repetitive activities. Examples include group response to closed-ended questions (such as singing the alphabet, repeating letter sounds); individual children using a pointer to name letters; write or copy letters 20 26.0% The teacher teaches literacy concepts using ONE of the following strategies: • Children explore and play with concrete objects to learn the concept • Children have a choice of how to carry out an activity • The teacher engages the children in discussion and sometimes uses open-ended questions • Teacher relates lesson to real-life or everyday experiences 12 15.6% The teacher teaches literacy concepts using TWO OR MORE of the following strategies: • Children explore and play with concrete objects to learn the concept • Children have a choice of how to conduct an activity • The teacher initiates children in the discussion and sometimes uses open-ended questions • The teacher relates the lesson to real-life or everyday experiences 16 20.8% Total 77 - Table F. 2. Teacher referred to a lesson plan for structuring their literacy CO_ECTM_PlanAlphabetisation Frequency Percent No 10 13.0% Yes 38 49.4% SKIPPED 29 37.7% CO_ECTM_PlanAlphabetisation Frequency Percent Total 77 - Table F. 3. Learning opportunities to develop expressive language skills. CO_ECTM_LangageExp Frequency Percent Children are never or rarely asked to tell a story, describe events or objects, or answer questions throughout the observation. 25 32.5% The teacher encourages expressive language skills ONLY by: • Repetitive activities. Examples include group response to closed-ended questions (such as asking children to repeat a story or sentences word by word); each child uses a pointer to repeat words or phrases; individual responses to rote or closed questions. 23 29.9% The teacher encourages expressive language skills using ONE verbal exchange activity, such as: • Asking children to describe objects (eg color, shape, size, function) or pictures; • Encourage children to tell stories or describe events; • Tell a story and ask the children two or more open-ended questions about the story • Repeat and expand on what the child is saying, and include more advanced vocabulary • Use stories or discussions to encourage vocabulary that makes connections with the lives and experiences of children. 15 19.5% The teacher encourages expressive language skills by using TWO OR MORE verbal exchange activities, such as: • Asking children to describe objects (eg color, shape, size, function) or pictures; • Encourage children to tell stories or describe events; • Tell a story and ask the children two or more open-ended questions about the story • Repeat and expand on what the child is saying, and include more advanced vocabulary • Use stories or discussions to encourage vocabulary that makes connections with children’s lives and experiences 14 18.2% Total 77 - Table F. 4. The teacher speaks in French during class. CO_ECTM_LangueParlee Frequency Percent No 3 3.9% Yes 74 96.1% Total 77 - Table F. 5. Reading books to help children listen and speak CO_ECTM_Livre Frequency Percent For toddler development - CP1 and Kindergarten - the teacher: • Does not read books to children OR • Reads books that are not age appropriate (ie texts or textbooks for older children or adults; religious texts for adults; or books without pictures). //For older classes - CP2 or higher - students: • Do not read the text OR • Read texts that are not suitable for their age (ie texts or textbooks for children young children; picture books). 23 29.9% For toddler development - CP1 and Kindergarten, the teacher: • Reads to class without discussion OR • Reads to class without any questions about reading. // For older classes - CP2 or higher - the teacher: • Does not discuss reading OR • Does not ask questions about reading. 20 26.0% The teacher discusses reading with the class using ONE of the following strategies: • Asks children basic or closed-ended questions about what happened • Encourages children to discuss reading through open-ended questions • Talks about vocabulary learned in the book • Relates reading to children’s experiences or context • Children play with objects or do some activity related to reading 22 28.6% The teacher discusses reading with the class using TWO OR MORE of the following strategies: • Asks children basic or closed-ended questions about what happened • Encourages children to discuss reading through questions open-ended • Talks vocabulary learned in book • Relates reading to children’s experiences or context • Children play with objects or do some activity related to reading 12 15.6% Total 77 - Table F. 6. Learning opportunities to promote fine motor skills: Writing, Drawing / coloring CO_ECTM_MotricFine Frequency Percent No fine motor activity is observed 60 77.9% The teacher teaches fine motor skills ONLY through the use of: 0 0.0% CO_ECTM_MotricFine Frequency Percent • Activities that are NOT appropriate for the child’s development (that is, they are too difficult or too easy to understand or do for most children e.g. use pencils to draw lines before starting with pencils or markers) The teacher teaches fine motor skills using developmentally appropriate activities BUT: • Activities focus on accomplishing the task defined by the teacher rather than developing fine motor skills. • Activities focus on the product, not the process. • Activities are not led by children; children do not have a choice of what to do or how to use the materials. 4 5.2% The teacher teaches fine motor skills using developmentally appropriate activities AND: • Child-oriented and process-oriented rather than goal-oriented activities • Activities that allow children to explore the materials and how they can be handled in a fun way. 10 13.0% Not applicable 3 3.9% Total 77 - Table F. 7. Learning opportunities that allow children to engage in gross motor skills CO_ECTM_MotriGlobale Frequency Percent No gross motor activity is observed. 64 83.1% Less than 10 minutes of gross motor activity are observed or only a few children participate. Less than 20 minutes of gross motor activity are observed OR less than half of the children participate. 5 6.5% Less than 20 minutes of gross motor activity are observed OR less than half of the children participate. Most children get at least 20 minutes of gross motor activity 8 10.4% Most children practice at least 20 minutes of gross motor activity 0 0.0% Total 77 - Table F. 8. Learning activities that promote free choice or open play CO_ECTM_JeuLibre Frequency Percent No free choice / open play activity is observed. 69 89.6% The teacher chooses where or how the children will play with the materials OR the teacher offers a limited choice of activities AND the children must play with the materials in a prescribed manner. 2 2.6% Children have ONE opportunity to choose their own activity, where and how they play with the materials BUT the teacher does not interact to add to children’s play or extend learning 2 2.6% Children have ONE or more opportunities to choose their own activity and where and how they play with materials AND the teacher interacts to add to children’s play or extend learning. 4 5.2% CO_ECTM_JeuLibre Frequency Percent Total 77 - Table F. 9. Learning opportunities that allow children to participate in music / movement activities CO_ECTM_Mouvement Frequency Percent No music / movement activity is observed. 43 55.8% At least one music or movement activity occurred during the observation. 34 44.2% Total 77 - School Director Survey – Teacher Attendance Table F. 10. On average, how many hours per school day are teachers scheduled to be teaching? teach_time Frequency Percent 6 hours 55 71.4% 7 hours 5 6.5% 8 hours 9 11.7% 11 hours 6 7.8% 13 hours 1 1.3% 63 hours 1 1.3% Total 77 - Table F. 11. How many teachers do you have at this school? teachers_total Frequency Percent 2 teachers 1 1.3% 3 teachers 22 28.6% 4 teachers 11 14.3% 5 teachers 15 19.5% 6 teachers 12 15.6% 7 teachers 10 13.0% 8 teachers 5 6.5% 11 teachers 1 1.3% Total 77 - Table F. 12. How many teachers are in attendance today? teach_attend_total Frequency Percent 1 teacher 1 1.3% 2 teachers 6 7.8% 3 teachers 23 29.9% 4 teachers 8 10.4% 5 teachers 15 19.5% teach_attend_total Frequency Percent 6 teachers 15 19.5% 7 teachers 4 5.2% 8 teachers 3 3.9% 11 teachers 1 1.3% BLANK 1 1.3% Total 77 - School Director Survey – Management Tools Present Table F. 13. Observed in the head teacher’s office: a. teacher attendance board bureau_obs_1 Frequency Percent Not seen 44 57.1% Seen 33 42.9% Total 77 - Table F. 14. Observed in the head teacher’s office: b. teacher task list bureau_obs_2 Frequency Percent Not seen 47 61.0% Seen 30 39.0% Total 77 - Table F. 15. Observed in the head teacher’s office: c. visual teaching supports bureau_obs_3 Frequency Percent Not seen 49 63.6% Seen 28 36.4% Total 77 - Table F. 16. Observed in the head teacher’s office: d. teaching materials bureau_obs_4 Frequency Percent Not seen 32 41.6% Seen 45 58.4% Total 77 - Table F. 17. Observed in the head teacher’s office: e. inventory book bureau_1 Frequency Percent Not seen 33 42.9% Seen 44 57.1% Total 77 - Table F. 18. Observed in the head teacher’s office: f. school records bureau_2 Frequency Percent Not seen 33 42.9% Seen 44 57.1% Total 77 - Table F. 19. Observed in the head teacher’s office: g. visitor logbook bureau_3 Frequency Percent Not seen 25 32.5% Seen 52 67.5% Total 77 - Table F. 20. Observed in the head teacher’s office: h. gold book bureau_4 Frequency Percent Not seen 45 58.4% Seen 32 41.6% Total 77 - Table F. 21. Observed in the head teacher’s office: i. Reports of Class Councils / Educational Meetings bureau_5 Frequency Percent Not seen 16 20.8% Seen 61 79.2% Total 77 - Table F. 22. Observed in the head teacher’s office: j. teacher attendance logbook bureau_6 Frequency Percent Not seen 15 19.5% Seen 62 80.5% Total 77 - Annex G: Description of team members’ qualifications Crystal Chiglinsky Ms. Crystal Chiglinsky brings over seven years’ experience providing technical support to health and education projects and evaluations funded by USAID, USDA, DFID, and other donors. Her expertise includes programmatic, budgetary, logistical, and contractual support, as well as monitoring of enumerator training, data collection, and data entry. She develops and manages electronic data capture tools; creates enumerator training materials; supervises data collection including arranging logistics and procurement, training and monitoring enumerators and quality control officers, reviewing accuracy of data as data collection is ongoing in the field, and recommending midcourse corrections as needed to ensure high quality of data; and drafts research and program reports for donors and education officials. Ms. Chiglinsky has managed numerous Early Grade Reading Assessments (EGRAs) as well as both qualitative and quantitative surveys targeting a variety of key education stakeholders including teachers, parents, students, Ministry of Education officials, and community members. Ms. Chiglinsky has planned and supervised the trainings in Mali, Syria, and Togo. She has also supported data collection in several countries, including Pakistan, Niger, and Sierra Leone. Prior to her work in education, Ms. Chiglinsky managed projects at Partners In Health, where her portfolio included projects in child nutrition, maternal health, and nursing education in Haiti, Rwanda, and Malawi. In particular, she worked closely with clinicians to improve the nutrition outcomes for the tens of thousands of children within the catchment area of PIH’s 12 hospitals and clinics in central Haiti. She has also worked with local partners on short-term health projects in Ghana and Kenya. Ms. Chiglinsky holds a Bachelor of Science in Neuroscience from the College of William and Mary and a Master of Science in Human Rights, with a focus on development studies, from University College Dublin. She is fluent in English with an intermediate knowledge of French. Candace Debnam Ms. Debnam oversees STS’s global operations and business development activities, which includes programmatic work in 15 countries. She has over a decade of experience across the non-profit sector including managing large development contracts and grants for education, health, agriculture, energy, and community engagement projects overseas. She has experience in implementing development strategies, as well as organizing new initiatives for projects funded by USAID, MCC, and DFID as well as working with multilateral funding organizations. Debnam serves as the co-chair of the executive board of directors for the Basic Education Coalition—a group of leading US-based organizations and academic institutions working together to promote global peace and prosperity through education; there she plays a central role in convening and coordinating the international education development community. Prior to joining STS, Debnam supported a variety of health, research, and education initiatives at IntraHealth, SNV, FHI 360, and AED. Ms. Debnam received her master’s degree in management from University College Dublin’s Smurfit School of Business and her undergraduate degree in English and political science from the University of North Carolina-Chapel Hill. Randy Tarnowski Mr. Randy Tarnowski is an international education researcher with a diverse range of experiences in international program management and evaluation. Since joining School-to-School International, Randy has played critical roles in the evaluation of USAID and DFiD education projects in Ethiopia, Afghanistan, Morocco, Mali, the Democratic Republic of Congo, and Tajikistan. He is trained in quantitative and qualitative research methods and data visualization, having applied this training with the Foundation for Students Rising Above, as well as with Harvard’s Research Schools International on a mixed-methods project studying the relationship between social networks and growth mindset among UK high school students. Randy later served as Program Manager for WorldTeach and as a Teaching Fellow for the Center for Asia Leadership, where he managed teacher quality and education capacity building programs in over 17 countries. Annex H: Evaluation Public Disclosure Form